[00:48:00] New patchset: Tim Landscheidt; "Make job output files private by default." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67644 [00:54:58] New review: Tim Landscheidt; "Just noticed: I'll still have to test the cases where -e points to a directory and -stderr or not -s..." [labs/toollabs] (master) C: -1; - https://gerrit.wikimedia.org/r/67644 [01:47:51] Hm.. the "quotas" field is new I think, not sure if it is meant to be working already. [01:47:55] If it is, it seems broken [01:47:55] https://wikitech.wikimedia.org/w/index.php?title=Special:NovaProject&action=displayquotas&projectname=cvn [01:47:59] it says 0 everywhere [01:48:16] eventough "Floating IPs: 0/1" should be 1/1 and "Instances: 0/10 " should be 2/10 [01:57:16] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 0 minutes) [02:10:40] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 13 minutes) [02:24:01] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 26 minutes) [02:37:29] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 40 minutes) [02:37:46] * wolfgang42 eeks [02:38:12] I just started a nano that took up 76% of the memory on tools-login... [02:38:30] before I killed it, since it was still eating memory. [02:50:50] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 53 minutes) [03:04:14] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 67 minutes) [03:17:38] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 80 minutes) [03:31:06] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 93 minutes) [03:44:38] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 107 minutes) [03:58:06] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 120 minutes) [04:11:30] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 134 minutes) [04:24:55] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 147 minutes) [04:38:27] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 161 minutes) [04:51:52] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 174 minutes) [05:05:20] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 188 minutes) [05:18:48] Warning: There is 1 user waiting for shell: Seahorseruler (waiting 201 minutes) [05:32:12] Warning: There are 2 users waiting for shell, displaying last 2: Seahorseruler (waiting 215 minutes) Ruakh (waiting 9 minutes) [05:45:33] Warning: There are 2 users waiting for shell, displaying last 2: Seahorseruler (waiting 228 minutes) Ruakh (waiting 23 minutes) [05:58:53] Warning: There are 2 users waiting for shell, displaying last 2: Seahorseruler (waiting 241 minutes) Ruakh (waiting 36 minutes) [06:12:21] Warning: There are 2 users waiting for shell, displaying last 2: Seahorseruler (waiting 255 minutes) Ruakh (waiting 50 minutes) [06:37:47] zz_SickPanda: https://github.com/plausibility/flask-nsa [07:46:52] on the tool labs where is the Zend framework ? [08:02:20] Oren_Bochman hmm I don't think we have it [08:02:24] can install that on -dev if u need [08:02:35] please install it [08:03:59] Oren_Bochman done [08:04:07] tools-dev [08:25:20] !log tools petrb: fixing missing packages on exec nodes [08:25:22] Logged the message, Master [08:33:12] petan: I can't seem to subscribe to labs-l [08:33:26] :( [08:33:33] SickPanda unfortunately I am no admin of that list... you need to wait for Ryan [08:33:40] alright [08:33:49] he's the boss :P [08:34:00] and he's american [08:34:08] so he will wake up when you go sleep :P [08:44:34] !log tools petrb: updated /usr/local/bin/logsplitter on webserver-01 in order to fix become afcbot 49383 [08:44:35] Logged the message, Master [08:45:12] omg [08:45:25] !log tools petrb: updated /usr/local/bin/logsplitter on webserver-01 in order to fix become afcbot 49383 [08:45:27] Logged the message, Master [08:45:30] o.O [08:45:43] !log tools petrb: updated /usr/local/bin/logsplitter on webserver-01 in order to fix !b 49383 [08:45:44] Logged the message, Master [08:45:56] I just learned new thing... [08:49:26] to run my tool off tool-dev what url do i need ? [09:02:45] addshore ping [09:03:31] Pong [09:03:55] I created a collector of global access (to any page on tools project) I would like to create some statistics from that [09:04:00] like some graph [09:04:14] I can create a script which could get some data from it [09:04:15] Linky ? :-) [09:04:21] there is no link [09:04:32] it's just a file similar to ~/access.log [09:04:39] I want to create it [09:04:42] but I don't know how [09:04:52] I have the data but I don't know how to create graph from it [09:05:00] I would like to create several graphs [09:05:07] So you made some log to count hits per tool? :p [09:05:11] access in total (whole tools) per day [09:05:17] Kk [09:05:25] access in total (per tools) per day [09:05:41] get requests in total per day [09:05:44] post requests in total per day [09:05:45] etc [09:05:56] Where is the file? :-) maybe ganglia can handle it? [09:06:02] how should I store these data so that it is easy to convert he output to graph? [09:06:09] what is that stathat thing you work on [09:06:10] If not maybe me and Lego should work on our stats project ;p [09:06:27] Stathat isn"t free any more :( [09:06:35] I can do something like "file per statistic" [09:06:44] with space [09:06:51] but maybe there is a better format? [09:07:11] csv wold be better [09:07:21] ofcourse it can be csv [09:07:28] then you can use R [09:07:39] with coma coma [09:07:40] :P [09:07:55] I'll have a look later today :) there are already scripts to read from Apache log format, and I'm sure one could be tweaked [09:08:00] addshore if I give you data, can you create graphs? :P [09:08:16] creating a script from the logs is pretty simple... [09:08:22] I'll have a. Look later, currently eating breakfast with some friends ;p [09:08:28] ok [09:08:33] just with coma [09:10:00] where is the file [09:11:41] Oren_Bochman it is in /data/project/.system/logs/apache2 but it's not accessible now, because I need to ask Coren|Sleep first if people can read it or not (may be some legal issue - it may contain some private data?) [09:12:01] format is: [09:12:20] make me a version with 6 lines [09:12:22] tools-webserver-02: 127.0.0.1 - - [10/Jun/2013:09:10:47 +0000] "GET xxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 200 191 [09:12:27] ok [09:13:39] I'll try to parse it with R [09:14:13] /shared/global_access.log [09:14:33] it's just as access.log but prefixed with server [09:14:47] so that we can monitor how utilized each apache server is [09:26:16] well I can make a barplot of the http code v.s. frequency [09:28:44] is xxx a url [09:29:00] yes [09:29:09] but I don't need to parse that [09:29:27] I only want to collect information of how many requests per hour / day were made on each webserver [09:29:36] they are all the same ? for each file ? [09:29:40] in order to monitor the load and number of visits [09:29:53] instead of xxxxxxxx there is some string with no spaces [09:30:15] like/cluebot/cb.php?user=Mdadel&ns=rfefsgf [09:30:27] ok [09:30:30] this string can be ignored [09:31:31] I just want to collect some interesting statistics, mostly the number of visits per hour / day / month in total as well per server [09:31:38] so that we can see how much tool labs is utilized / being used [09:44:54] New patchset: Tim Landscheidt; "Make job output files private by default." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67644 [09:47:02] !toolsadmin is https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Admin [09:47:02] Key was added [09:51:14] New review: Tim Landscheidt; "Developing this way sucks. I'll need to think about introducing autotools or something similar in a..." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67644 [09:57:52] @notify Coren [09:57:52] I'll let you know when I see Coren around here [11:17:45] addshore ping [11:18:08] addshore what do you think of that webserver statistics thing? [11:18:15] is it worth of working on? [11:18:23] do u want to help me set it up [11:18:28] it requires lot of php coding :P [11:19:29] @notify anomie [11:19:29] I'll let you know when I see anomie around here [11:31:47] Petan I TT hink its a great idea but I can't do anything right now :-) [11:32:04] Maybe tomorrow [11:32:06] :-) [11:36:46] petan: Could you install the package "devscripts" on tools-dev if it doesn't collide with existing packages, please? [11:51:41] scfc_de [11:51:42] done [11:54:45] petan: Thanks. [12:05:59] petan I got it to work in R - I'm trying to convert it it into a function [12:06:12] Oren_Bochman I created public version [12:06:20] /data/project/.system/logs/public_apache2 [12:06:35] there you have actual data, refreshed every hour [12:27:42] hm... I set up experimental awstats... http://tools.wmflabs.org/awstats/cgi-bin/awstats.pl doesn't show much but it can give you some information about how many pages on tools project are being visited [12:43:29] ok [12:43:34] still fixing a bug [13:03:55] anomie is that memcache server working? [13:05:08] petan: Haven't tried it yet, I need another package installed first. libcrypt-gcrypt-perl [13:05:18] ok [13:05:34] !log tools petrb: installing libcrypt-gcrypt-perl [13:05:35] Logged the message, Master [13:06:34] anomie done [13:08:12] petan: did you do it on tools-login too? [13:08:34] ha! I forgot... [13:08:48] done [13:08:56] * anomie cheated, ran the updater for http://tools.wmflabs.org/anomiebot/packages.html to check [13:09:35] anomie: did you notice I synced all the nodes so there are almost no different packages? :P [13:09:46] just kernels and boot loaders [13:09:49] petan: Yes, I did. Good! [13:12:59] petan: Ok. FYI, I'll probably not be able to actually try it out until after 22:00 UTC or so, unless I finish my other work so I can justify "helping test Tool Labs" as "work" ;) [13:13:16] np [13:13:17] * anomie has very blurry lines between "work" and "not work" sometimes [13:14:20] ok it looks better [13:52:26] Is the source code for Edit Counter available? [13:55:19] FutureTense hm? [13:55:30] if it's a project on labs then yes [13:55:52] http://tools.wmflabs.org/xtools/pcount/ [13:56:10] ok, then ask one of the project maintainers to provide the source code [13:56:19] I don't have all these links in my head :) [13:56:33] but every project must be opensource for it to live on wmf labs [13:57:01] thats what I'm looking for [13:57:10] is a project maintainer [13:57:19] https://tools.wmflabs.org/ [13:57:45] Cyberpower678 ^ [13:59:02] Cyberpower678: you rang? [13:59:08] i mean I rang :) [14:06:15] hello does anybody knows how i can set this on my wiki on tools? http://www.mediawiki.org/wiki/Manual:$wgImageMagickConvertCommand [14:06:32] for enabling thumbnails [14:07:08] petan: ^ [14:07:51] just follow that manual [14:08:07] petan: where it is? [14:08:22] http://www.mediawiki.org/wiki/Manual:$wgImageMagickConvertCommand [14:09:13] petan: I know I want to know address of ImageMagik is different in tool or not [14:11:42] * Coren waves. [14:12:57] maybe problem is in http://www.mediawiki.org/wiki/Manual:$wgMaxShellMemory [14:13:02] i raised it [14:15:38] hi Coren I have lot of things to you [14:15:51] petan: No doubt. :-) [14:15:54] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Admin [14:15:57] this is first one [14:16:03] I don't know if we have some admin docs so far or not [14:16:10] if not I started one, if yes we can merge it :> [14:16:22] there are things that needs to be documented [14:16:58] second thing is that I created /data/project/.system/logs for global logs (for now of apache server only) so that we can use them to generate some statistics [14:17:11] I needed to change that logsplitter script - is that in some repository? [14:17:15] I couldn't find it in puppet [14:17:27] petan: Nope. I'll be adding stuff to that doc [14:17:41] also I wanted to know if there is some privacy policy preventing these logs from being accessible by public? [14:17:47] I wasn't sure so I created 2 folders [14:17:55] 1 is private for members of local-admin only [14:17:55] petan: It's not in any repo, right now. I'd probably be a good idea to stuff it with the others. [14:18:12] the second one is public, where the "potentialy sensitive" information is filtered out [14:18:31] petan: Yes; strictly speaking, only tool maintainers should be able to see how their tools is getting used because we can't presume that tool usage doesn't disclose information. [14:18:32] I noticed that these access.log files are not readable by world [14:18:39] ok [14:18:51] well as you can notice these logs only contain information like [14:19:09] tools-webserver-01: 127.0.0.1 - - [10/Jun/2013:09:28:24 +0000] "GET xxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 403 101 [14:19:10] tools-webserver-02: 127.0.0.1 - - [10/Jun/2013:09:28:24 +0000] "GET xxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 200 193 [14:19:20] petan: This is why the mess with error logs -- It's not possible to "tag" error log lines so that logsplitter could work. :-( [14:19:29] hmm [14:19:31] anyway [14:19:42] this allows us to create some nice statistics of how the webservers are being used [14:19:45] Yes, that's okay (removing the GET value); it pretty much anonymizes the whole thing correctly. [14:19:57] I didn't create any cool gui for now, but I set up awstats for now [14:20:04] http://tools.wmflabs.org/awstats/cgi-bin/awstats.pl [14:20:16] so we can finally see how many people actually use tool labs so far :> [14:20:35] average 4k hits per our, more than I expected [14:20:39] * hour [14:21:39] Indeed. [14:22:00] I am curious if this will grow next days [14:22:15] still more and more people are joining the projects and moving the tools [14:22:28] it will be interesting once the url's on wikipedia will switch from toolserver to tool labs [14:22:48] hopefully web servers won't die under the load [14:23:53] this is interesting http://tools.wmflabs.org/awstats/cgi-bin/awstats.pl?framename=mainright#errors [14:24:01] 403 Forbidden 7,963 hits [14:24:21] in last few hours someone hit almost 8k of 403 errors o.O [14:26:49] Petan, its a shame awstats can't tell the difference between tool dirs :( [14:27:42] And lol at 8k errors! [14:29:17] addshore the awstats don't see the tool dirs ;) [14:29:30] Why not ? :( :p [14:29:34] Privacy? [14:29:34] that is one of private information which is filtered out... [14:29:41] :/ [14:29:46] addshore: /data/project/.system/logs/public_apache2/global_access.log [14:29:53] this is where the awstats get the information from [14:30:05] Not at a computer so can't look :p [14:30:32] Would it not be okay to just anon the rest of the URL but keep the first directory? [14:32:36] addshore: maybe... :o [14:32:42] Coren needs to decide... [14:33:05] addshore when u arrive to berlin [14:33:29] question: I'm testing a change to the role::solr::geodata::labs role. I've got a labs machine running as a local puppetmaster. how do I assign that role to myself? [14:33:36] petan: Keeping the first path component should be okay. [14:34:07] there is a section in the docs titled "How do I get my puppet classes to display in labsconsole ? " but I can't find what it wants me to click [14:34:07] manybubbles: "Manage puppet groups" in the project, and add it as an option. [14:34:26] manybubbles: Once you do so, you can use "configure" on the instance to add it. [14:34:50] Coren: is the link to configure the option "Add group"? [14:35:29] got it! [14:37:31] manybubbles: You need to add a group (arbitrary grouping) and add the class in the group. [14:37:32] Petan, I should be there my the 1st of July :-) [14:38:08] Coren: yup - I just saw that. I've updated the docs at https://wikitech.wikimedia.org/wiki/Help:Self-hosted_puppetmaster#FAQ to help folks that don't already have a group. [14:40:41] addshore ok if you create a regex for it, we can have it [14:40:50] :P [14:40:58] not that easy :P [14:42:22] Create a regex for the log? [14:42:35] Again, can't look right now but will when I get home ;) [14:42:36] regex which matches the part of log which needs to be removed [14:42:48] right now it's obvious [14:42:55] but if you want to keep the tool name it's not [14:43:29] Petan, pm :-) [14:43:44] where [14:43:51] didn't get any :o [15:04:33] Coren done [15:04:47] check if doesn't leak any private data: vi /data/project/.system/logs/public_apache2/global_access.log [15:04:48] :P [15:04:52] but I couldn't find any [15:04:54] petan: Done... what? Ah. Checking. [15:05:07] tools-webserver-02: 127.0.0.1 - - [10/Jun/2013:09:28:24 +0000] "GET /cluebot HTTP/1.1" 200 186 [15:05:32] this actually let us create a chart of most popular tools as well [15:06:00] petan: Not sure if I want to do that; it's editcountitis for devs. :-) [15:06:19] what is editcountitis [15:10:59] http://tools.wmflabs.org/awstats/cgi-bin/awstats.pl?framename=mainright&output=urldetail [15:11:03] here we go :D [15:13:13] petan: haha, now watch tool authors artificially inflate their views...*that's* editcountitis :p [15:13:25] hmm [15:13:42] nobody beats cluebot [15:44:53] Coren: I've been so stuck in my progress due to the double table issue :( [15:45:08] "Double table issue"? [15:45:17] the revision vs revision_userindex [15:45:33] actually a rails issue though [15:45:52] can't find a way to combine to tables in a single model optimally [15:45:57] two* [15:46:02] AzaToth: Like I said, the rule is simple. If you have a where clause on rev_user or rev_user_text, use revision_userindex, else use revison (this includes joins, obviously) [15:46:48] yea, that's simple, the issue I have is I'm too stupid to figure out how to do it in rails without exposing the fact of two tables outside the model [15:47:07] have even posted a question at http://stackoverflow.com/questions/17002809/handling-two-almost-identical-tables-as-one-model [15:48:30] I am haveing weird problems [15:48:45] I can't use become [15:49:08] I get sudo: sorry, a password is required to run sudo [15:50:57] also I am having issues with ssh from one host to another [15:51:44] I'm no expert with mysql, but wouldn't a join of all colums minus user colums and all columns without rows where rev_deleted & 4? suffice? [15:51:54] Oren_Bochman: using ssh agent? [15:52:16] ussing putty + pagent [15:52:40] it stopped working recently [15:53:05] Oren_Bochman: poke Coren until he answers ツ [15:54:26] I bettwr hit the Gym [15:54:35] I wrote a bunch of R code [15:58:07] Oren_Bochman: I'll need a bit more information before I can help -- can you tell me the exact sequence that fails? [16:32:20] Oren_Bochman, regarding your sudo issues: what project & username? [16:32:42] Oh, maybe Coren fixed already? [16:44:11] the filesystem on the mobile 'staging' labs instance apperas to have been switched to read-only: [16:44:12] awjrichards@staging:~/var/bingle$ cat /dev/null > bingle.err [16:44:12] -bash: bingle.err: Read-only file system [16:44:17] anyone know anything about that? [16:44:47] Coren: ^ gluster issues again? [16:44:50] awjr reboot it [16:45:05] ah yes, the classic 'turn it off and on again' [16:50:32] awjr I would happily fix it... but I just can't :/ you can either wait for someone who is approved to do that, or reboot it and have it fixed [16:51:03] i just rebooted :) [16:51:05] thanks petan [16:51:24] awjr, did reboot fix the problem? [16:51:30] looks like no :() [16:51:37] andrewbogott, petan ^ [16:51:45] awjrichards@staging:~$ touch foo [16:51:45] touch: cannot touch `foo': Read-only file system [16:51:51] What project? And is this $home or /data/project? [16:52:00] mobile project, $home [16:52:41] awjr: How's that? Better? [16:52:49] yes andrewbogott :) [16:52:52] what was it? [16:53:23] I don't specifically, some kind of sync failure between the two gluster servers. [16:53:32] *know specifically [16:53:40] I just gave it a nudge [16:54:17] awjr: It switches into readonly when it can't be confident of its mirroring. [16:54:19] ah well, thanks andrewbogott :) [16:54:23] roger [16:54:30] And sometimes is not great about switching back :( [16:54:51] yeah, IIRC if you get 'readonly' errors it is always gluster tat needs a nudge [17:50:11] Coren: do we have commons joining yet? [17:50:35] Betacommand: I'm about done with S7; the joins with commons and wikidata are next on my TODO [18:14:56] Coren: is ssh://gerrit.wikimedia.org:29418/labs/private.git a honeypot repo? [18:15:16] in meeting atm [18:15:18] k [18:16:16] I can't see why I should have access to your private keys [18:16:35] AzaToth, it shadows the real private repo. So that dependencies and such can resolve to the bogus keys/passwords in that repo [18:17:16] andrewbogott: I see, so the labs-puppet-key, even though it's not 0 bytes, is a bogus key? [18:17:59] Some stuff is live there… e.g. things that a labs instance needs to know to come up is presumed pretty much public. [18:18:07] and the star.wmflabs.key [18:18:27] ok [18:18:43] You can write to the labs list if you want an enumeration of the security concerns there, but -- the sort answer is that you're supposed to be able to see it. [18:18:45] *short [18:20:05] andrewbogott: I just thought there was a error in some system somewhere [18:20:21] Nope, but thanks for being vigilant! [18:21:03] np [18:21:19] andrewbogott: But it's meant to not be publicly viewable, isn't it? [18:21:26] (For anonymous users.) [18:21:41] when reading the changelog, it seems to be a mix of bogus usernames/passwords and real passwords [18:21:55] scfc_de: I'm logged in on labs as a normal user [18:22:03] Hm… I'm not sure. I don't think it much matters but you can check w/Ryan when he shows up [18:22:21] AzaToth: I mean logged in Gerrit. [18:22:40] andrewbogott: Well, it wasn't last time I checked, and that's probably not bad. [18:22:42] scfc_de: yea, I meant that as well [18:23:01] haven't tried looked at it in gerrit logged out [18:23:07] I just cloned it [18:23:46] andrewbogott: I see you added bogus passwords, but ariel seems to have added a real password [18:24:59] Ah, yeah, but it's labs-specific... [18:25:27] I haven't thought about this hard, obviously, but basically anything that lands on a labs instance is not-especially-private. [18:25:29] * AzaToth scratches heads [18:26:14] I will make a note to write some docs about security rings w/in labs to clarify who should know what. [18:26:20] ok [18:27:23] difficult to know what's on a labs instance or not [18:33:15] Warning: There is 1 user waiting for shell: Stefan Kühn (waiting 0 minutes) [18:46:48] Warning: There is 1 user waiting for shell: Stefan Kühn (waiting 13 minutes) [18:46:49] Warning: There is 1 user waiting for access to tools project: Stefan Kühn (waiting 4 minutes) [19:00:12] Warning: There is 1 user waiting for shell: Stefan Kühn (waiting 27 minutes) [19:00:13] Warning: There is 1 user waiting for access to tools project: Stefan Kühn (waiting 17 minutes) [19:07:10] !tr Stefan_Kühn [19:07:10] request page: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Stefan_K%c3%bchn?action=edit talk page: https://wikitech.wikimedia.org/wiki/User_talk:Stefan_K%c3%bchn?action=edit§ion=new&preload=Template:ToolsGranted link: https://wikitech.wikimedia.org/w/index.php?title=Special:NovaProject&action=addmember&projectname=tools [19:08:27] !tr Stefan_Kühn [19:08:27] request page: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Stefan_K%c3%bchn?action=edit talk page: https://wikitech.wikimedia.org/wiki/User_talk:Stefan_K%c3%bchn?action=edit§ion=new&preload=Template:ToolsGranted link: https://wikitech.wikimedia.org/w/index.php?title=Special:NovaProject&action=addmember&projectname=tools [19:08:36] !rq Stefan_Kühn [19:08:36] https://wikitech.wikimedia.org/wiki/Shell_Request/Stefan_K%c3%bchn?action=edit https://wikitech.wikimedia.org/wiki/User_talk:Stefan_K%c3%bchn?action=edit§ion=new&preload=Template:ShellGranted https://wikitech.wikimedia.org/wiki/Special:UserRights/Stefan_K%c3%bchn [19:09:08] New patchset: AzaToth; "Make proper debian packages" [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67859 [19:09:53] wtf [19:10:25] I got "Your change was committed before the commit hook was installed" but gerrit accepted the commit anyways [19:11:11] it probably amended in the change-id [19:11:39] AzaToth, that's just git-review adding a change-id to the commit message before submitting. [19:11:39] AzaToth git review does it [19:12:10] ah, it says Amending, I thought it said "amend" [19:12:27] Coren [19:12:34] https://gerrit.wikimedia.org/r/67826 ... [19:12:39] merge wait [19:14:00] petan: Merged. [19:14:03] ty [19:18:04] New review: Petrb; "are you sure we want this in /bin ? I am no fan of /usr/local as well, but this should be at least i..." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67859 [19:18:59] hi valhallasw [19:49:07] is petrb someone here? [19:49:41] petan: loha [19:50:00] petan: it installs in /usr/bin or /usr/sbin [19:50:28] petan: http://paste.debian.net/9554/ [19:51:04] petan: there's a lack of copyright assignments though, and lintian complains a lot of the man pages :-P [19:51:35] petan: http://paste.debian.net/9555/ [19:52:30] also I see license for one manpage is MIT and the other copy is CC-BY-SA [19:52:59] http://paste.debian.net/9556/ [19:54:05] petan: the install is defined in https://gerrit.wikimedia.org/r/#/c/67859/1/debian/jobutils.install and https://gerrit.wikimedia.org/r/#/c/67859/1/debian/tools.install [20:00:00] * AzaToth wonders whom he is talking to [20:01:55] AzaToth: labs-l might be better :/ [20:01:57] sorry [20:02:08] sumanah: :-P [20:02:31] * sumanah goes to train someone in person, sorry [20:02:36] Coren: ^ petan ^ [20:02:47] hi [20:03:21] hai [20:03:26] ok [20:03:39] i cant +2 :\ [20:04:28] just wanted you to know [20:04:54] New review: coren; "Petr is correct; if this is made into a package, then it should no longer be in /usr/local but /bin ..." [labs/toollabs] (master) C: -1; - https://gerrit.wikimedia.org/r/67859 [20:05:07] can someone explain me why some users have no registration_date in the user-table? [20:05:16] AzaToth: I'd +2 if you push stuff to /usr [20:05:40] Coren: http://paste.debian.net/9554/ [20:06:01] Coren: but perhaps we should talk about the copyright first [20:06:19] lbenedix: Those that have been created "earlier" than, IIRC, 1.3 [20:06:38] there's no blody license specified anywhere except a small "under MIT/CC-BY-SA" in one of the man pages [20:06:41] AzaToth: Oh, right, I keep forgetting to add those things. Everything I write for the labs is MIT [20:07:10] mmmmhh... so It's not possible to get the registration-date for them? [20:07:26] and I don't know if petan is the "maintainer" or if we should have a generic maintainer [20:07:28] AzaToth: I'd CC-0 if it made sense (but Legal prefers attribution) [20:07:49] lbenedix: The UI uses the date of the earliest contribution. [20:08:07] good tip! [20:08:44] AzaToth: Strictly speaking, I wrote most of this, Petr wrote the man pages (mostly C&P from the usage), and nobody cares very much. :-) [20:09:01] Coren: hehe [20:09:21] yea, lintian complained a lot of petr's man pages ツ [20:09:43] The debian-maintainer right now should probably be me (mpelletier@wikimedia.org), at least while I work at the WMF. [20:09:50] ok [20:09:51] the missing registration-date has nothing to do with the user_id... [20:10:47] ISC > MIT [20:10:48] Lemme +2 this, then I'll do the © boilerplate before I build the package and put in in repo. [20:10:55] Coren: mpelletier or mark? [20:10:59] ok [20:11:42] it builds two packages by the way, jobutils and tools [20:11:53] Works for me. [20:12:20] paravoid: Huh. Would you believe I never actually noticed the ISC stuff wasn't just MIT? [20:12:21] Coren: I meant marc [20:12:32] paravoid: I agree. ISC it is. [20:12:37] ymmv :) [20:12:42] I prefer ISC since it's simpler [20:12:55] what is ISC? [20:13:09] Islamic World Science Citation Center(ISC)? [20:13:27] New review: coren; "Nevermind (about /bin)" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67859 [20:13:27] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67859 [20:13:29] Internet Scrabble Club? [20:14:58] Coren: You might want to update debian/copyright as well then [20:15:10] AzaToth: Doing so now; also adding a couple dependencies. [20:15:11] when you are adding Internet Scrabble Club license [20:16:02] FutureTense, ? [20:16:09] can anyone tell me what license you are talking about? [20:16:18] petan, ? [20:16:58] AzaToth: http://en.wikipedia.org/wiki/ISC_license [20:17:07] oh [20:17:23] simplified 2-clause BSD basically [20:17:49] some of the users with the missing registration date have MIN(rev_timestamp) in 2007 [20:18:23] Coren: the FSF recommends MIT/Expat over ISC [20:18:34] but again, ymmv :) [20:19:13] paravoid: That seems like a compelling argument in favour or using the ISC license. [20:19:44] lbenedix: That's because the account is old, but didn't have a recent contrib. [20:31:26] petan: Mind if I relicense your manpages CC-0? [20:31:28] AzaToth: Before I rebase my changes, is toollabs now stable? [20:32:02] scfc_de: I'm about to push a changeset for licenscing, etc. [20:32:21] Coren: Okay, then I'll wait till the dust settles. [20:32:25] scfc_de: stable is a pretty ambigious question [20:32:34] Should only be a few minutes. [20:33:53] AzaToth: I mean path changes, etc., anything that Gerrit/Git will probably have a hard time doing theirselfes :-). [20:33:53] paravoid: {{source}} [20:34:05] hm? [20:34:06] scfc_de: git doesn't care about paths [20:34:13] Coren: the FSF recommends MIT/Expat over ISC [20:34:22] http://www.gnu.org/licenses/license-list.html#ISC [20:35:29] okai [20:36:18] AzaToth: Yes, I noticed. It's just the FSF being silly. I prefer the terser verbiage of the ISC license. [20:36:46] New patchset: Tim Landscheidt; "Fix typo in jsub." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67288 [20:38:07] New patchset: Tim Landscheidt; "Fix warnings." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67643 [20:39:18] New patchset: Tim Landscheidt; "Make job output files private by default." [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67644 [20:39:43] AzaToth: It seems you're right, rebasing went smoothly. [20:43:25] New patchset: coren; "Tweaks to the packaging" [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67884 [20:43:46] AzaToth: Care to take quick look, see if I broke something? [20:43:58] Coren: where is /usr/local/bin/job comming from? [20:44:05] I never saw it in the git [20:44:20] AzaToth: Forgotten one. It should have been there to begin with, it just got overlooked. :-) [20:44:31] heh [20:44:47] I noticed it was missing when I tried to add the boilerplate to it. :-P [20:45:05] ah, I see it's there now ツ [20:46:02] Coren: are you marc or mpelletier@wikimedia.org? [20:46:11] License: IRC [20:46:13] Heh. [20:46:16] * Coren amends. [20:46:33] I made a comment on that but never saved [20:46:36] ツ [20:46:52] License: found in IRC log from 1998 [20:46:52] mpelletier. marc@ is muscle memory. [20:47:51] New patchset: coren; "Tweaks to the packaging" [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67884 [20:47:51] Coren: I'm sure you could get that if you ask :) [20:47:53] Coren: you might want to update/change debian/changelog [20:48:21] Meh. It's still the original packaging. :-) [20:48:22] AzaToth: we have a bunch of people trying to do packaging in ops, you should help [20:49:09] New review: AzaToth; "might want to update debian/copyright before comitting" [labs/toollabs] (master) C: 1; - https://gerrit.wikimedia.org/r/67884 [20:49:56] paravoid: it's easy ツ dh_make && rm -f pkg/debian/*.ex [20:50:23] AzaToth: I'm paravoid@debian.org jfyi [20:50:30] oh [20:50:48] I never went DD [20:50:52] I saw [20:51:30] I'm just saying, we could always use more people with packaging skills [20:51:35] ok [20:51:43] +1000 [20:51:47] are there many packages that are fubar atm? [20:51:53] gerrit [20:52:38] well, offcourse I could help out [20:52:52] \o/ awesome. thanks :) [20:53:35] I'm sure we have a number of others too [20:53:36] paravoid: Do you know any intro for people with an RPM background? debs seem much more "fragile", i. e. you must pay attention to your working directory & Co. Any "good practices", etc.? [20:56:01] I just got the following notice: Did you know that your home isn't protected by default? [20:56:01] If you don't want to allow others reading it type [20:56:01] chmod 700 ~ [20:56:01] But when I ls -l ~ I get perms of -rw------- [20:56:03] AzaToth, Coren: With the move to /usr/bin, we need to update the help pages and probably add some symlinks so that not everything is broken. [20:56:15] Is the DYK wrong? If so, who do I tell? [20:56:20] no sqlite support from the php installed on tool labs? [20:56:24] AzaToth: "might want to update debian/copyright before comitting" ... update with? [20:56:48] wolfgang42: The tips don't check your actual permissions. [20:57:09] But, the permissions are the default. I didn't change them. [20:57:50] scfc_de: I can see no reference to /usr/local in the help pages except the hint to add them to the path (which won't break things) [20:57:59] Coren: oops, I forgot [20:58:16] need to change jsub to use /usb/bin/job [20:58:20] usr [20:58:39] Ah, true. [20:59:07] and you want to increase the version over 1.0.0 as current jobutils has that version [20:59:09] wolfgang42: "Did you know that you can update these tips on https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Tips ?" So just be bold :-). [20:59:44] scfc_de: Ah, thanks :) I guess I'll go do that [20:59:47] AzaToth: It's in jstop as well. [21:00:29] ok [21:00:53] scfc_de: I noticed [21:01:24] Ryan_Lane: did you mean that "gerrit" needs to be packaged, or that there are code in gerrit that needs to be packaged? [21:01:31] scfc_de: ??! The tip isn't there... [21:01:43] AzaToth: we have a package for gerrit, but it's terrible [21:01:47] ok [21:01:50] I never did a proper package for it [21:03:26] https://gerrit.wikimedia.org/r/operations/debs/gerrit? [21:04:17] wolfgang42: Then the tip that was displayed to me was lying :-). It looks as they are stored in /data/project/.system/tips; I don't know what/who syncs them with the wikipage, probably petan. [21:05:00] AzaToth: I've never seen this, but feel free to add me ("Faidon") as a reviewer to this or other packaging changes [21:05:10] AzaToth: I'll review & merge for you [21:05:31] scfc_de: It seems that the tips file has the stuff from the tips wiki, plus a couple extra tips at the end... [21:05:32] paravoid: for gerrit? [21:05:53] for gerrit or anything else packaging-related you provide patches for :) [21:06:52] k [21:07:10] Ryan_Lane: was that the package you had made? [21:07:31] yep. it's basically just a wrapper for the war file [21:07:51] fatal: Not a valid object name upstream/2.6-rc0-322-geeed497 [21:08:05] ok. boarding time…. [21:08:08] I don't see a upstream branch on gerrit as well ヾ [21:08:22] Ryan_Lane: opendj too, right? [21:08:23] it's a custom war built from jenkins [21:08:29] paravoid: opendj is built properly [21:08:30] ok [21:08:32] oh [21:08:49] and we're waiting on an upstream package for that anyway [21:08:59] next release will have one [21:09:43] ok. time to go [21:09:51] * paravoid doesn't trust upstream packages [21:10:19] AzaToth: suffice to say, if you want to also upload some of our stuff to Debian, I'd be more than happy to sponsor you [21:10:43] I see [21:11:01] ty [21:11:06] I'm guessing you're a DM for your packages? [21:11:40] nope, just maintainer [21:12:03] had so much work when DM was made I didn't had time to focus on debian packages then [21:13:35] New patchset: coren; "Tweaks to the packaging" [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67884 [21:14:46] AzaToth: Howzzat look to you? [21:16:18] Coren: +2 [21:16:49] New review: coren; "AzaToth approves. Good enough for me. :-)" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67884 [21:16:50] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67884 [21:18:38] ツ [21:34:12] Coren: tools-db's database is on Gluster at the moment? [21:34:58] scfc_de: Yeah, it's mostly moot because nobody should be using the global storage on it (because nobody should be logging in to that box anyways); but it needs a reboot to switch. [21:37:07] Coren: Kolossos found that importing one of the files needed for Templatetiger took 10 hours on tools-db, while he thinks it took about 2 - 3 on Toolserver. (Also, the prospect of Gluster corrupting the database isn't that nice :-).) [21:38:07] The queries itself seem to work pretty fine: "select count(*) from dewiki;" took 16 minutes on the Toolserver, on Tools just 1.3. [21:38:10] scfc_de: The database isn't on gluster either, no worries there. But it's not a very well-tuned database in the first place -- it was meant as a temporary hack. [21:38:24] Coren: So the database is on NFS? [21:38:37] scfc_de: Oh, wait, that's on the *replica* databases or on tools-db? [21:38:42] tools-db. [21:38:46] scfc_de: No, the database is on a "local" disk. [21:38:59] I. e. Gluster? [21:39:54] i.e.: "local" to the instance, which is a VM on local virtual storage. No gluster involved at any point. [21:40:12] What's the disk space? [21:42:16] 170 GByte total (from https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools)? [21:43:00] scfc_de: 158G for the DB [21:43:16] scfc_de: But a DB on virtual hardware is always... teh painz. [21:43:38] I could have looked it up myself: http://ganglia.wmflabs.org/latest/?c=tools&h=tools-db => 119.78 free. [21:44:27] AzaToth: Your packages are now even puppetized on the cluster. All hail the package master. :-) [21:44:48] * Coren uses the opportunity to do more code review. [21:45:21] Coren: As Templatetiger will load at least 30 GByte more text files (don't know how that translates to DB space on MariaDB), that should probably be wtched. [21:45:31] AzaToth: Thank you very much, BTW. [21:45:56] no probs [21:46:18] I'm trying to figure out how gerrit though using "buck" was a splendid idea [21:47:10] http://icinga.wmflabs.org/icinga/ is down. [21:53:06] Coren: remember to remove all old copies from local [21:53:23] New review: coren; "LGM" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67644 [21:53:24] dunno if puppet will do that for you [21:53:49] AzaToth: It won't; I'm going to be replacing them with symlinks. [21:54:58] Puppet can do symlinks, too :-). [21:55:18] New review: coren; "LGM" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67643 [21:56:04] New review: coren; "How did that not get noticed earlier?" [labs/toollabs] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/67288 [21:56:04] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67288 [21:56:18] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67643 [21:56:27] Change merged: coren; [labs/toollabs] (master) - https://gerrit.wikimedia.org/r/67644 [21:58:32] Coren: no need to have symlinks there afaik [21:59:05] AzaToth: I expect not, but we don't know who may have hardcoded /usr/local/bin/j* in crontabs or scripts. :-( [22:00:29] Coren: For your Copious Free Time To Do List™: Replace the symlinks with wrappers that log the caller :-). [22:00:59] (Well, actually, no -- if PATH has /usr/local/bin with higher priority, you get too many false positives.) [22:01:51] (Test $0? That should work.) [22:03:46] scfc_de: err: /Stage[main]/Gridengine::Master::Monitoring/File[/usr/local/sbin/grid-ganglia-report]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///files/gridengine/grid-ganglia-report at /etc/puppet/modules/gridengine/manifests/master.pp:17 [22:03:52] scfc_de: FYI. [22:05:53] That's from the puppet run? [22:07:40] scfc_de: Yep. [22:10:53] So basically "/files/" => "/modules/" is needed? [22:18:13] Coren: https://gerrit.wikimedia.org/r/67899 [23:22:59] !log cvn Added seahorseruler to cvn access group. [23:23:01] Logged the message, Master