[00:05:07] <dschwen>	 hm, I'll try adding the rule directly to default
[00:06:25] <dschwen>	 yeah, that actually works
[00:12:25] <quentinv57>	 Hi, could anyone help me to get an account on tools, please ?
[00:13:20] <Cyberpower678>	 petan, ^
[00:14:26] <Cyberpower678>	 !toolsrequest
[00:14:26] <wm-bot>	 https://wikitech.wikimedia.org/wiki/Special:FormEdit/Tools_Access_Request
[00:27:14] <\pi{r^2}>	 quentinv57: hi
[00:27:32] <quentinv57>	 hey, I'm about to leave
[01:19:39] <\pi{r^2}>	 hi
[01:19:50] <\pi{r^2}>	 anyone here who knows about SQL?
[01:42:44] <halfak>	 \pi{r^2}, still want to talk SQL?
[01:42:54] <\pi{r^2}>	 Yes!
[01:43:36] <\pi{r^2}>	 Earlier hoo told me a faster way to query multiple wikis was to use unions
[01:43:45] <\pi{r^2}>	 I would like to see an example of this.
[01:43:58] <\pi{r^2}>	 (I want to run the same query on all wikis)
[01:44:59] <halfak>	 Hmm...  That's not something I'm aware of.  One of the problems is that many wikis are on different servers.  
[01:45:13] <halfak>	 However, there may be something special to labs that lets you run cross server queries. 
[01:45:51] <halfak>	 Honestly, if I wanted to run a query on all wikis, I'd automate my interaction with MySQL with python or a Makefile depending on the level of complexity. 
[01:46:46] <halfak>	 I should specify that I know exactly what UNION is and does, but I'm not aware of it being able to cross servers. 
[01:50:44] <halfak>	 \pi{r^2}, I can help you hack together something that will query all wikis if you like. 
[01:51:16] <Reedy>	 No, labs won't let you do cross servers
[01:51:24] <Reedy>	 Like on toolserver you could do servers on a cluster..
[01:51:38] <halfak>	 Thanks Reedy. 
[01:51:49] <Reedy>	 but combining output from multiple databases isn't exactly difficult
[01:51:56] <halfak>	 Agreed. 
[01:52:04] <Reedy>	 same query run against 850+ wikis etc
[01:52:48] <halfak>	 Reedy, do you know if there's a machine readable index of dbnames to servers we could reference?
[01:53:08] <Reedy>	 https://noc.wikimedia.org/conf/
[01:53:14] <Reedy>	 s[1-7].dblist
[01:53:47] <halfak>	 cool.  That should work nicely. 
[01:54:03] <\pi{r^2}>	 SELECT dbname FROM meta_p.wiki GROUP BY slice;
[01:54:52] <\pi{r^2}>	 s/GROUP/ORDER/
[01:54:54] <halfak>	 Even better. :) 
[01:55:08] <\pi{r^2}>	 halfak: I'm a PHP and SQL n00b
[01:55:39] <Reedy>	 \pi{r^2}: that will only give you a list of dbnames
[01:55:43] <Reedy>	 You won't know where they are ;)
[01:55:57] <\pi{r^2}>	 then SELECT dbname, slice
[01:56:04] <\pi{r^2}>	 Reedy: is that correct?
[01:56:27] <Reedy>	 Yeah, something like that so you can find the mapping
[01:56:40] <\pi{r^2}>	 This is my ugly php code http://tools.wmflabs.org/pirsquared/iw.php.txt
[01:56:46] <\pi{r^2}>	 I'm trying to speed up the SQL part
[01:57:48] <halfak>	 Tabs are important 
[01:58:01] <\pi{r^2}>	 I'm completely new to programming.
[01:58:05] <\pi{r^2}>	 I started yesterday.
[01:58:24] <halfak>	 :)  Then you're already doing pretty well. 
[01:59:18] <halfak>	 If you give me a sec, I'll throw some tabs in this doc to show you what I'm talking about. 
[01:59:22] <\pi{r^2}>	 okay
[01:59:27] <halfak>	 s/doc/file
[01:59:46] <\pi{r^2}>	 How do you edit text files on tools? Emacs, nano, vi/vim, ed?
[01:59:57] <\pi{r^2}>	 How do you even use ed?
[02:06:33] <\pi{r^2}>	 Or do you have to type echo "contents of file" > filename to edit files...?
[02:07:25] <halfak>	 Oh sorry.  I was scanning the code.  I use Jedit.  It's a moderately popular editor.   Emacs is substantially more popular. 
[02:07:50] <halfak>	 lol @ > to file
[02:07:55] <halfak>	 I could just use sed
[02:08:10] <\pi{r^2}>	 true
[02:08:44] <halfak>	 So, it looks like you have to hit the API in order to get a list of namespaces for every wiki on every request. 
[02:08:53] <halfak>	 Am I reading that right?
[02:09:00] <\pi{r^2}>	 Yes.
[02:09:49] <\pi{r^2}>	 https://bugzilla.wikimedia.org/show_bug.cgi?id=48625 WONTFIX
[02:10:15] <\pi{r^2}>	 and I'm not particularly keen on creating my own namespace database and updating it every day
[02:11:00] <halfak>	 Yeah.  That's not a nice clean solution. 
[02:11:01] <\pi{r^2}>	 I lied when I said I'm new to programming, but I've never written anything useful in PHP (or any server-side script web thing) before.
[02:11:23] <\pi{r^2}>	 and I used SQL for the first time a week ago or less
[02:11:43] <halfak>	 If you were using WSGI, I'd show you how to store an in-memory cache, but that would require a switch to python. 
[02:11:55] <Reedy>	 Namespaces don't change every day
[02:12:17] <Reedy>	 It'd be unlikely that they'd change on more than one day a week too
[02:12:31] <\pi{r^2}>	 I have used python many times before, but I thought PHP might be better for this task.
[02:12:32] <\pi{r^2}>	 No?
[02:12:48] <Reedy>	 Some aliases and such might
[02:12:53] <halfak>	 Oh.  IMHO, PHP is not the right tool for any task. :) 
[02:13:06] <Reedy>	 Works fine for Wikipedia ;)
[02:13:12] <\pi{r^2}>	 Reedy: why was the bug WONTFIX'd?
[02:13:26] <halfak>	 "works"
[02:13:46] <halfak>	 Actually, you're right.  Does work.  So do rocks, but we have hammers now and they are better. 
[02:13:55] <Reedy>	 "You /have/ an alternative:  use one line of code at the beginning of your
[02:13:56] <Reedy>	 script to fetch the namespaces info, and look the name up when outputting it."
[02:14:03] * halfak  regrets starting this battle already

[02:14:10] <Reedy>	 PHP sucks
[02:14:14] <\pi{r^2}>	 I'm doing something like that.
[02:14:24] <halfak>	 Reedy, that won't work.  He needs namespaces for all the wikis. 
[02:14:30] <\pi{r^2}>	 but only for the wikis I need. Sometimes that means all wikis, sometimes only a few.
[02:14:38] <\pi{r^2}>	 depending on the $_REQUEST thing
[02:14:51] <Reedy>	 One line of code wrapped in a loop?
[02:14:55] <Reedy>	 congrats. 4 lines of code
[02:15:00] <halfak>	 That's basically what he has. 
[02:15:03] <Reedy>	 hard query is hard
[02:15:24] <\pi{r^2}>	 tautology is tautology
[02:15:28] <halfak>	 \pi{r^2}, http://pastebin.com/mhzuv3kP
[02:16:09] <halfak>	 It helps me think when I tab the code, you're welcome to copy-paste -- or not. 
[02:16:22] <\pi{r^2}>	 Thanks. Do you think it's worth switching to python at this point?
[02:16:35] <\pi{r^2}>	 It doesn't have prepared statements.
[02:16:43] <halfak>	 Yes it does. 
[02:16:56] <halfak>	 Well... it has it to the extent that PHP does
[02:17:02] <halfak>	 However, I'm not sure it is worthwhile. 
[02:17:41] <\pi{r^2}>	 I'm much more experienced with python. I didn't know PHP was so hated. :O
[02:17:58] <Reedy>	 you'll find haters of most languages
[02:18:12] <Reedy>	 for many many different reasons
[02:18:52] <\pi{r^2}>	 halfak: so... any immediate suggestions to speed it up?
[02:19:09] <\pi{r^2}>	 Apparently using UNION on each slice makes it faster
[02:19:12] <halfak>	 Nothing jumping out at me.  What sort of runtimes are you getting for common queries?
[02:19:20] <\pi{r^2}>	 Minutes.
[02:19:35] <halfak>	 How many DBs are commonly hit for each query?
[02:19:50] <\pi{r^2}>	 By default, all (700-800 I think).
[02:19:58] * halfak  whistles

[02:20:04] <\pi{r^2}>	 e.g. http://tools.wmflabs.org/pirsquared/iw.php?iw=wiki takes a few minutes
[02:20:14] <\pi{r^2}>	 but http://tools.wmflabs.org/pirsquared/iw.php?iw=wiki&wiki=enwiki,metawiki is much faster
[02:20:26] <Reedy>	 "My code is fast it's just hte 800 database queries..."
[02:20:34] <\pi{r^2}>	 * http://tools.wmflabs.org/pirsquared/iw.php?iw=wiki&wikis=enwiki,metawiki
[02:20:38] <halfak>	 Could you take out the API call and try it for me?
[02:20:40] <\pi{r^2}>	 I forgot the url param.
[02:20:45] <halfak>	 I bet the DB side of things is super fast. 
[02:21:04] <\pi{r^2}>	 It's very fast when you only use one or two wikis
[02:21:38] <\pi{r^2}>	 but the whole point is to tell if interwikis are used and where, to make maintenance of the interwiki map easier
[02:23:15] <\pi{r^2}>	 [[m:interwiki map]], I'm the most active editor
[02:25:00] <Reedy>	 That's not necesserily a good thing ;)
[02:26:47] <\pi{r^2}>	 Why?
[02:27:04] <\pi{r^2}>	 halfak: http://tools.wmflabs.org/pirsquared/iw.php?wikis=&iw=wiki&hideclosed=on is still slow, new code http://tools.wmflabs.org/pirsquared/iw2.php?wikis=&iw=wiki&hideclosed=.txt
[02:27:13] <\pi{r^2}>	 * code = http://tools.wmflabs.org/pirsquared/iw2.php.txt
[02:27:28] <halfak>	 How slow?
[02:27:57] <\pi{r^2}>	 it's also broken :S
[02:28:07] <halfak>	 That's OK.  This is a test.  
[02:29:13] <\pi{r^2}>	 Okay, I put the old one at iw.php and the new one at iw2.php
[02:29:42] <halfak>	 If you wouldn't mind running the test and timing it, I'd be greatfull as I'm multitasking some other stuff. 
[02:29:50] <\pi{r^2}>	 How do you time it?
[02:30:02] <halfak>	 What I really want to know is this: What proportion of the execution time is accounted for by the API call. 
[02:30:05] <\pi{r^2}>	 Stopwatch or is there another way?
[02:30:21] <halfak>	 A watch is fine.  
[02:30:30] <halfak>	 Otherwise, you could use datetime. 
[02:30:37] <halfak>	 Let me remember how that works in PHP
[02:31:03] <halfak>	 http://us3.php.net/manual/en/function.mktime.php
[02:31:34] <halfak>	 Woops.  Looks like you want http://us3.php.net/manual/en/function.time.php
[02:31:42] <halfak>	 But this only goes down to seconds. 
[02:31:52] <halfak>	 It would be nice to have more precision, but it will work for us now.  
[02:31:56] <halfak>	 Here's what you do:
[02:31:59] <halfak>	 $start = time()
[02:32:06] <halfak>	 code does stuff
[02:32:17] <halfak>	 echo time - $start
[02:32:29] <halfak>	 s/time/time()
[02:32:49] <halfak>	 That will print the number of second the "code does stuff" took. 
[02:34:20] <halfak>	 Ahh there it is! http://us1.php.net/microtime
[02:34:37] <halfak>	 That will give us microseconds of precision to work with. 
[02:35:02] <\pi{r^2}>	 halfak: I added it to the bottom of both iw.php and iw2.php
[02:35:17] * \pi{r^2}  goes now, bbl

[02:35:28] <\pi{r^2}>	 I would test http://tools.wmflabs.org/pirsquared/iw2.php?iw=wiki vs http://tools.wmflabs.org/pirsquared/iw.php?iw=wiki
[02:36:16] <halfak>	 \pi{r^2}, Ping me later if you want to keep hacking.  I'm happy to help :) 
[02:39:46] <\pi{r^2}>	 old: 256
[02:39:53] <\pi{r^2}>	 (nice round number)
[02:40:01] <\pi{r^2}>	 new: 142
[02:40:18] <\pi{r^2}>	 It might be in seconds.
[02:44:03] <\pi{r^2}>	 More samples times for old: 144, 145
[02:44:15] <\pi{r^2}>	 More *for now
[02:44:31] <\pi{r^2}>	 For old: 257, 258
[02:44:37] <\pi{r^2}>	 ping halfak 
[02:44:58] <\pi{r^2}>	 but I'm really gone now
[02:45:23] <halfak>	 Hmmm  That's not the benefit I expected. 
[02:45:41] <halfak>	 I really thought that the API calls were consuming most of the time.  
[02:45:56] <halfak>	 Reedy: are you still around?
[02:47:02] <\pi{r^2}>	 halfak: they're consuming a considerable portion of the time...
[02:47:20] <halfak>	 yeah.  I was hoping it was more like 9/10ths of the times. 
[02:49:09] <\pi{r^2}>	 So...suggestions?
[02:51:02] <halfak>	 Nothing is jumping out at me.  
[02:51:19] <halfak>	 Can you describe the use case of this script to me quick? 
[02:51:27] <halfak>	 What is the user trying to do?
[03:11:15] <halfak>	 Gotta run.  Have a good one folks. 
[03:24:37] <hedonil>	 Ryan_Lane: Are there any ldap issues on toollabs?. no groupnames show up, login permission denied , startet ~ 5 minutes ago
[03:25:10] <Ryan_Lane>	 yes, ldap issues
[03:25:14] <Ryan_Lane>	 Coren|Travel, andrewbogott_afk: ^^
[03:25:28] <hedonil>	 Ryan_Lane: can this be fixed?
[03:25:39] <Ryan_Lane>	 yes
[03:26:08] <Ryan_Lane>	 trying to run a command for it now
[03:26:18] <Ryan_Lane>	 hedonil: try now
[03:26:19] <hedonil>	 Ryan_Lane: good ;)
[03:26:22] <Ryan_Lane>	 you may need to log out and back in
[03:26:38] * hedonil  tries

[03:26:54] <hedonil>	 Ryan_Lane: yep! thx.
[03:27:59] <hedonil>	 Ryan_Lane: slow as heck, but at least works ;)
[03:28:17] <Ryan_Lane>	 yeah, the slow part is what I'm trying to fix which caused the issue
[03:35:52] <Ryan_Lane>	 well, I think I found it now
[03:41:59] <Ryan_Lane>	 hedonil: should be fast now
[03:42:05] <Ryan_Lane>	 are you getting ldap errors again?
[03:42:17] * hedonil  checks

[03:42:39] <Ryan_Lane>	 yeah. that broke things again
[03:42:48] <Ryan_Lane>	 one sec.
[03:42:50] <hedonil>	 Ryan_Lane: yeah, seems back to normal
[03:47:20] <hedonil>	 Ryan_Lane: but some troubles accessing my tools files with my user account - access denied
[03:48:01] <Ryan_Lane>	 it's due to the command I ran
[03:48:09] <Ryan_Lane>	 it corrupted nslcd's cache
[03:48:29] <hedonil>	 Ryan_Lane: what can I do?
[03:48:33] <Ryan_Lane>	 let me try to restart it again
[03:48:36] <Ryan_Lane>	 any idea which node?
[03:48:48] <hedonil>	 tools-login
[03:49:06] <Ryan_Lane>	 hm. I think it should be working now
[03:49:13] <Ryan_Lane>	 it's possible you'll need to log out and back in
[03:49:18] * hedonil  checks

[03:49:43] <hedonil>	 permission denied
[03:49:57] <Ryan_Lane>	 log out and back in
[03:50:05] <hedonil>	 'k
[03:50:44] <hedonil>	 no, still error
[03:51:29] <Ryan_Lane>	 type: id
[03:51:41] <hedonil>	 yes
[03:51:49] <Ryan_Lane>	 what's the output?
[03:51:59] <hedonil>	 uid=3710(hedonil) gid=500(wikidev) groups=500(wikidev),50062(project-bastion),50380(project-tools),50769(local-wikiviewstats),50903(local-newwebtest)
[03:52:04] <Ryan_Lane>	 ok
[03:52:10] <Ryan_Lane>	 what file are you trying to access?
[03:52:25] <hedonil>	 /data/project/wikiviewstats/public_html/dev/index.php
[03:52:57] <Ryan_Lane>	 I'm going to su to you to troubleshoot
[03:53:27] <hedonil>	 shall i try again?
[03:53:35] <Ryan_Lane>	 yes
[03:53:48] <Ryan_Lane>	 it seems to be working to me
[03:54:13] <hedonil>	 opened that file with nano, write permission denied
[03:54:36] <Ryan_Lane>	 hm. indeed
[03:54:53] <Ryan_Lane>	 likely nfs cache
[03:55:08] <Ryan_Lane>	 not sure there's much I can do there
[03:55:10] <Ryan_Lane>	 Coren|Travel: ^^
[03:55:13] <Ryan_Lane>	 he may be able to help you
[03:55:36] <hedonil>	 seems I ave to go to bed now ;)
[03:55:42] <Ryan_Lane>	 cache will timeout eventually
[03:55:45] <Ryan_Lane>	 I'm not sure how long it is
[11:23:37] <stephankn>	 Hi, I'm currently on Toolserver and thought about having a look into migrating to Labs. I work on OSM integration. I found a lot of information about wikipedia databases, but nothing about OSM. Could someone give me some links with more documentation regarding OSM on Labs?
[11:38:09] <MaxSem>	 stephankn, still being worked on
[14:33:29] <apper>	 was something changed regarding group rights? I'm not able to delete or add files to folders with write permission even if I'm member of the group...
[14:34:40] <scfc_de>	 apper: Not that I know of.  Which directory do you mean?
[14:34:56] <apper>	 several ones of several tools ;)
[14:35:18] <apper>	 [03:47:20] <hedonil>	 Ryan_Lane: but some troubles accessing my tools files with my user account - access denied
[14:35:28] <apper>	 that sounds like my problem ;)
[14:36:13] <scfc_de>	 Just one example, please? :-)
[14:36:31] <apper>	  /data/project/persondata/public_html
[14:36:31] <hedonil>	 apper: yeah some ldap issues since ~ 4 am
[14:36:35] <kenrick95>	 Hi, I'm Kenrick, I have some trouble uploading things on my labs project account; previously my method was: uploading files trhough winSCP, and then take ownership with PuTTY. Now  whenever I tried to upload anything on winSCP I got permission error. Can anyone help me?
[14:36:51] <apper>	 hehe, another one ;)
[14:37:00] <hedonil>	 apper: Ryan_Lane tried to fix it, but didn't succed
[14:37:11] <apper>	 good to know that it's not my fault ;)
[14:37:15] <apper>	 this time ;)
[14:37:43] <apper>	 so is there a workaround?
[14:37:50] <giftpflanze>	 hm, do we have that problem since days and it is still not solved?
[14:38:03] <hedonil>	 apper: I chmod'ed my files to 666, so I can work
[14:38:23] <apper>	 hedonil: hm, okay
[14:39:33] <scfc_de>	 apper: The problem seems to be that "getent group apper" is empty.  It should list all of your tools instead.
[14:40:14] <hedonil>	 scfc_de: (04:48:10) Ryan_Lane: it corrupted nslcd's cache
[14:40:27] <apper>	 "groups apper" works fine ;)
[14:40:28] <scfc_de>	 Okay, I'll read through the logs first :-).
[14:40:56] <apper>	 "id" is correct, too
[14:42:39] <apper>	 knowing nothing about nslcd I would think: clear the cache ;)
[14:45:20] <hoo>	 wikidata replication looks broken
[14:47:02] <hedonil>	 hoo: Some major troubles yesterday, maybe related to that.  http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/20140102.txt
[14:47:08] <scfc_de>	 I've restarted nslcd on tools-login ("sudo service nslcd restart"), but that didn't help ("sudo sudo -u apper touch /data/project/persondata/test" => "Keine Berechtigung").  The permissions will be checked on the NFS server, and I can't access that (only Coren & Co. have).
[14:47:57] <apper>	 scfc_de: thanks for trying
[14:48:04] <scfc_de>	 andrewbogott: Do you have access to the NFS server?
[14:48:18] <andrewbogott>	 I do, but I don't immediately know what would help.  Poking about...
[14:48:22] <andrewbogott>	 (Unless you have a good idea?)
[14:48:22] <hoo>	 hedonil: mh... it's stuck at 20140103141808
[14:48:53] <andrewbogott>	 I can certainly restart nslcd
[14:49:20] <scfc_de>	 andrewbogott: No.  Ryan_Lane suggested in the logs (at 03:48Z today), that a command he ran corrupted nslcd's cache, but he claimed that it would heal itself.
[14:49:31] <scfc_de>	 (No = I don't have a good idea :-).)
[14:50:47] <hedonil>	 hoo: preferably file a bz. 
[14:51:12] <andrewbogott>	 scfc_de: any better?
[14:52:26] <scfc_de>	 andrewbogott: No, "sudo sudo -u apper touch /data/project/persondata/test" still fails on tools-login.
[14:52:47] <scfc_de>	 Do you know which command Ryan ran?
[14:52:55] <scfc_de>	 (Or what else he did?)
[14:53:23] <andrewbogott>	 I know some of what he did but not the bit that would've broken the cache
[14:53:57] <giftpflanze>	 scfc_de: completely unrelated, i still wait upon resolution of https://bugzilla.wikimedia.org/show_bug.cgi?id=58949. are you/do you feel competent to create the grid node?
[14:54:31] <giftpflanze>	 Coren|Travel seems to be busy
[14:55:15] <scfc_de>	 giftpflanze: a) Coren's right on this one, a dedicated instance is better :-).  b) I can't set that up.
[14:55:16] <andrewbogott>	 scfc_de: I want to understand that command… what is the relationship between 'apper' and 'persondata'?
[14:55:35] <giftpflanze>	 scfc_de: ok, thank you
[14:55:48] <giftpflanze>	 maybe we should reassign it then?
[14:55:55] <scfc_de>	 User "apper" is (should) be a member of the tool persondata ("local-persondata") and thus be able to write to that directory.
[14:56:16] <andrewbogott>	 but apper isn't you, it's… someone else?
[14:56:22] <andrewbogott>	 Why are you able to sudo -u apper?
[14:56:38] <apper>	 it's me ;)
[14:57:08] <scfc_de>	 andrewbogott: I'm able to "sudo sudo -u apper" :-).
[14:57:10] <andrewbogott>	 then why not just 'sudo touch <etc>'?
[14:57:33] <scfc_de>	 The problem isn't that root can't touch the file, but a user of a tool's group.
[14:57:37] <andrewbogott>	 ah, ok, I see what you're saying.
[14:57:40] * andrewbogott  tries it

[14:58:02] <scfc_de>	 giftpflanze: Done.
[14:58:13] <giftpflanze>	 uh, what?
[14:58:22] <scfc_de>	 giftpflanze: The bug reassignment.
[14:58:27] <giftpflanze>	 ah, ok
[14:58:29] <giftpflanze>	 :)
[14:58:32] <andrewbogott>	 so you'd expect me to be able to 'touch /data/project/<tool I am in>/foo.txt' as well
[14:58:38] * andrewbogott  doesn't ever do things this way

[14:59:06] <scfc_de>	 Yes, that should be possible.  That's why the tools' directories have +s set as well, I believe.
[15:01:16] <apper>	 when not doing it this way: how to transfer files? upload to own folder and then transfer it using the tool account? I don't really know, what's the best practice... 
[15:01:50] <apper>	 uploading it using the user account and then taking it using "take" was the easiest way
[15:03:49] <scfc_de`>	 apper: That should work as a workaround.
[15:04:54] <apper>	 scfc_de: I was wondering how andrewbogott is doing uploads, if he doesn't need that right - maybe there is a way I'm not aware of ;)
[15:05:28] <andrewbogott>	 I mostly haven't been doing uploads -- the work I've done in toollabs has been via gerrit.
[15:06:06] <apper>	 ah, thanks
[15:06:15] <scfc_de`>	 andrewbogott: Do you have an idea how to debug this further?
[15:06:24] <andrewbogott>	 not really, but I'm trying :)
[15:10:47] <anomie>	 andrewbogott, scfc_de: We've seen this before. NFS isn't honoring the supplemental groups again for some reason; if you use 'sg' to change your gid to the proper group, then the touch works.
[15:11:36] <scfc_de>	 anomie: Do you remember the solution then? :-)
[15:11:46] <hedonil>	 andrewbogott: are things fixed?
[15:12:12] <anomie>	 scfc_de: I never knew it. And Coren|Travel is travelling.
[15:12:18] <hedonil>	 andrewbogott: still doesn't work for me
[15:16:16] <hedonil>	 scfc_de: andrewbogott: what you need is a BRB labeled "clear cache & make everything ok"
[15:16:42] <hedonil>	 scfc_de: andrewbogott: at least some docs, where this button can be found
[15:17:17] <hoo>	 really no one around to have a look at the DB?
[15:17:53] <andrewbogott>	 scfc_de, try now?
[15:18:35] <hedonil>	 andrewbogott: seems to work now
[15:18:39] <andrewbogott>	 huh
[15:18:49] <scfc_de>	 andrewbogott: Seems to work.  What did you do? :-)
[15:19:02] <apper>	 andrewbogott: works for me now
[15:19:09] <hoo>	 andrewbogott: Do you have any insight into the MariaDB replication?
[15:19:12] <apper>	 andrewbogott, scfc_de: thanks for fixing!
[15:19:19] <andrewbogott>	 I purged things on the backup nfs server.  Which makes me think that we're confused about which is the primary and which is the backup :)
[15:19:29] <apper>	 andrewbogott: lol
[15:19:32] <andrewbogott>	 hoo, I really don't.  Lemme see if sean is up
[15:19:55] <giftpflanze>	 hm, for me, it is still broken
[15:20:11] <andrewbogott>	 hoo: Looks like he's sleeping.  He's in Australia…  I'd advise waiting here for Coren or else send an email to the labs list.
[15:20:17] <scfc_de>	 giftpflanze: Yes, for my primary tool it works, but for the rest not.  *Argl*.
[15:20:57] <scfc_de>	 "touch /data/project/dbreps/test" => "Keine Berechtigung".
[15:20:57] <hoo>	 andrewbogott: mh.. thanks
[15:21:13] <apper>	 andrewbogott, hoo: maybe open a bug in bugzilla and adding sean as cc?
[15:22:48] <andrewbogott>	 scfc_de: OK, so, /now/ what's an example of something I can do to reproduce the problem?
[15:23:28] <scfc_de>	 sudo to scfc, "touch /data/project/dbreps/test" fails, "touch /data/project/wikilint/test" succeeds.
[15:24:01] <andrewbogott>	 what's the difference between those two projects?
[15:24:47] <scfc_de>	 I'm a member of both ("groups"), and the directories are both "drwxrwsr-x", so I don't see a difference?
[15:25:19] <hoo>	 apper: Is this tools specific or rather infrastructure?
[15:25:28] <andrewbogott>	 Ah, ok, you said something about 'primary' vs 'other' and I thought that was an actual config diff
[15:25:36] <andrewbogott>	 hoo:  tools
[15:25:42] <hoo>	 txh
[15:27:38] <giftpflanze>	 touching files works, but scp doesn't
[15:28:11] <hoo>	 andrewbogott: https://bugzilla.wikimedia.org/show_bug.cgi?id=59267 didn't really know who to CC
[15:30:24] <scfc_de>	 giftpflanze, andrewbogott: Both work for me: "touch /data/project/{dbreps,wikilint}/test" on tools-login, as well as "scp /bin/true tools-login.wmflabs.org:/data/project/wikilint/" externally.
[15:31:37] <andrewbogott>	 scfc_de: you still can't touch touch /data/project/dbreps/test?
[15:33:38] <scfc_de>	 andrewbogott: No, everything's fine now (touch'ing and scp'ing from external).
[15:33:46] <giftpflanze>	 ok, scp works here, too
[15:33:47] <andrewbogott>	 welp
[15:34:06] <andrewbogott>	 Maybe it really was a cache issue and I just happened to be standing here when the clock ran out
[15:34:55] <scfc_de>	 Hmmm.  Rather unlikely :-).  You didn't do anything special?
[15:35:54] <andrewbogott>	 I restarted nslcd
[15:36:01] <andrewbogott>	 But that was a while ago :)
[15:44:14] <scfc_de>	 andrewbogott: The move to eqiad should provide ample opportunity to document the whole NFS setup :-).
[15:44:59] <andrewbogott>	 yep
[15:45:30] <Cyberpower678>	 What's going on with the centralauth db?
[16:19:05] <PiRSquared>	 halfak: around?
[16:19:12] <PiRSquared>	 and not busy?
[16:20:00] <halfak>	 PiRSquared, I was just about to get up from my desk.  :( 
[16:46:22] <apper>	 are there possibilities to do an EXPLAIN on a replicated database? I get "ERROR 1345 (HY000): EXPLAIN/SHOW can not be issued; lacking privileges for underlying table" -> I think it's because the tables are only views to other tables, but sometimes it would be useful to analyze the request behavior
[16:47:11] <hedonil>	 apper: afaik there's already a bz for that
[16:47:49] <apper>	 hedonil: okay, so no workaround known for this?
[16:47:57] <apper>	 hedonil: thanks for the info :)
[16:51:56] <hedonil>	 apper: As you pointed out, in _p there are only views and in "main database"  SELECT is not granted to common users  - which is required for EXPLAIN
[16:52:43] <apper>	 hedonil: thanks
[16:56:24] <apper>	 hedonil: I will watch https://bugzilla.wikimedia.org/show_bug.cgi?id=48875 :)
[16:56:52] <hedonil>	 apper:thx. I will add some suggestion
[16:57:04] <hedonil>	  I would suggest to have create a replica-db with fake data but identical structure and indeces
[16:58:38] <hedonil>	 so you can check - and not just guess - if you are torturing the db with brutal full table scans ;)
[16:59:05] <apper>	 hmm... I don't know how MariaDB works, but does the way a query is served only depends on the structure? Maybe the data is also important
[16:59:38] <hedonil>	 apper: as I wrote: with fake data
[17:00:31] <apper>	 hmm... but the fake data has to be similar to the original data, but of course, some fake data would be sufficient for most things
[17:01:00] <apper>	 fortunately the database layout doc for mediawiki is relatively good ;)
[17:01:49] <hedonil>	 apper: yeah, it's not the optimal solution, but due to security concerns - better than now (=nothing)
[17:02:35] <apper>	 yep
[17:41:19] <Cyberpower678>	 anomie, is the s7 cluster broken on labs?
[17:42:35] <anomie>	 Cyberpower678: Doesn't seem to be, but I have no way to really tell.
[17:43:11] <Cyberpower678>	 meta-p, centralauth, the seem to be corrupted.
[17:43:35] <anomie>	 Cyberpower678: How so?
[17:43:49] <PiRSquared>	 What is broken?
[17:43:58] <Cyberpower678>	 WorkBench can't seem to retrieve anything originating from s7.
[17:44:09] <anomie>	 Cyberpower678: Works for me from the command line mysql client.
[17:44:26] <Cyberpower678>	 Strange
[19:12:57] <chippy>	 I'm setting 
[19:14:17] <chippy>	 im setting up postgres - and thinking about puppet and configurations - where is the best place to put the data files? locally, with occasional backups to /data/project, or just in /data/project
[19:14:58] <chippy>	 or in /mnt ?
[19:18:48] <scfc_de>	 chippy: I would keep the default settings, but as / has usually only 10 GB and the extra space resides in /mnt, move big directories to /mnt and set up symlinks.
[19:31:08] <chippy>	 scfc_de, thanks :) Would it be safe in the case of say another datacentre migration?
[19:33:39] <scfc_de>	 chippy: I think all (:-)) data center migrations will be totally transparent to the instances.  Maybe FQDNs will change (.pmtpa.wmflabs -> .eqiad.wmflabs), but I don't know what the plans for that are.
[19:34:59] <valhallasw>	 scfc_de: famous last words ;-)
[19:36:35] <chippy>	 scfc_de, ahh i see :-)
[19:38:46] <chippy>	 is /mnt /dev/vdb fast / on the same hardware?
[19:42:17] <scfc_de>	 chippy: I think all instance-local disks are identically provided.
[19:44:34] <chippy>	 thanks again :)
[22:02:48] <huh>	 Hi, who can get bugzilla for their tool?
[22:03:09] <huh>	 quentinv57, john lewis, and I would like it for local-irc-wmt
[22:03:41] <huh>	 BTW, hi halfak 
[22:04:20] <valhallasw>	 huh: you can request it with a bug under wikimedia/bugzilla
[22:04:20] <halfak>	 Hi huh.  :P  
[22:04:34] <halfak>	 I've got to run in just a minute. 
[22:04:36] <huh>	 So file a bug to be able to file bugs?
[22:04:38] <huh>	 Okay.
[22:05:47] <valhallasw>	 huh: a bug to get a component, yes
[22:05:56] <huh>	 quentinv57: ^
[22:09:31] <huh>	 thanks @ quentinv57 
[22:10:50] <quentinv57>	 huh, https://bugzilla.wikimedia.org/show_bug.cgi?id=59619
[22:10:57] * huh  saw

[22:13:51] <huh>	 Cyberpower678: do you have access to irc-wmt ?
[22:15:07] <Cyberpower678>	 huh, not at the moment, just the webtools.  But I haven't asked for access to wmt yet.  I want to get the webtools squared out first.
[22:16:18] <huh>	 #wmt
[23:57:42] <Coren|airport>	 This has been the crappiest flights ever.
[23:57:59] <Coren|airport>	 Delays, cancellations, delays, Yeay!
[23:58:29] <Coren|airport>	 Thank $deity for lounge access.