[00:08:37] legoktm: Around? Some local-afch-updater grid jobs are failing because "can't open output file "/afch-updater.out": Permission denied". Perhaps $HMOE/afch-updater.out instead of $HOME/...? [00:09:22] yeah i noticed that last night, busy right now, ill fix it in an hour or so [00:20:24] legoktm: No problem, it just prevents your jobs from running :-). [00:55:03] What's up with tools-login [00:55:05] Connection to 208.80.153.224 timed out while waiting to read [00:55:11] can't ssh into it [00:55:21] Coren: Ryan_Lane [00:56:00] ssh'ing into non-toollabs instances via bastion works fine [01:56:55] Krinkle: I can't log into tools-login.wmflabs.org either, but in a session on tools-dev from before, I can access the filesystem, so NFS seems to be fine. [01:57:56] scfc_de: seems to be stuck at the keys phase. [02:00:04] YuviPanda: Are the directories beneath /public/keys/$USER supposed to be non-empty? [02:00:17] I *think* not, but I'm not sure [02:03:02] Then what's their purpose? :-) [02:03:50] Ah, okay, now I see some files. [02:04:17] /public/keys/scfc/.ssh/authorized_keys, /public/keys/scfc/.bash_logout, /public/keys/scfc/.bashrc, /public/keys/scfc/.profile [02:04:28] Still can't log into tools-login, though. [02:04:30] yeah [02:04:36] still stuck at that part [02:06:36] I can log into tools-webproxy.pmtpa.wmflabs (as member of local-admin). [02:07:13] Krinkle: And tools-dev.wmflabs.org works perfectly. [02:07:34] ➜ ~ ssh -v tools-login.pmtpa.wmflabs [02:07:36] doesn't work [02:08:27] http://ganglia.wmflabs.org/latest/?r=hour&cs=&ce=&m=load_one&s=by+name&c=tools&h=tools-login&host_regex=&max_graphs=0&tab=m&vn=&sh=1&z=small&hc=4 looks like tools-login is down. [02:09:18] petan's terminator daemon killed gmond last time, perhaps this time something more vital? [02:09:39] Anybody having a session to tools-login? Otherwise I would reboot it. [02:10:06] hehe, that's possible [02:10:08] restart [02:11:43] !log tools Reboot tools-login, was not responsive [02:11:45] Logged the message, Master [02:12:20] scfc_de: should be down [02:14:36] Login to tools-login works again. [02:14:59] :) [02:16:26] Jul 29 00:55:16 tools-login terminatord: System is out of memory, only 104464384 bytes remaining, killing random process [02:17:06] "killing random process" -- I really think we should reconsider using standard, proven Linux OOM killer :-). [02:18:07] Oh, apparently the Linux one killed another process even before. [02:18:38] 'random' process?! [02:18:54] x.x why is it even running with that much permissions there.... [02:18:54] gah [02:19:13] Jul 29 00:53:29 tools-login kernel: [888046.683106] puppet invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0 [02:19:24] (That's Linux.) [02:19:50] how much was uptime, btw? [02:20:58] Jul 29 00:53:51 tools-login kernel: [888046.697824] Out of memory: Kill process 13686 (perl) score 106 or sacrifice child [02:21:09] Jul 29 00:53:51 tools-login kernel: [888046.698539] Killed process 16148 (sh) total-vm:4400kB, anon-rss:96kB, file-rss:508kB [02:21:20] Very descriptive: "perl", "sh". [02:21:28] Uptime, let's see ... [02:22:39] syslog.10.gz:Jul 18 18:13:00 tools-login kernel: imklog 5.8.6, log source = /proc/kmsg started. [02:22:57] So -- 11 days. [02:23:11] not bad [02:23:14] Or rather 10.something. [02:23:19] or bad, depending on how you look at it :) [02:23:31] also I just got an 'add two numbers' program done on INTERCAL! [02:23:32] \o/ [02:24:28] terminatord killed gmond before: "Jul 29 00:51:07 tools-login terminatord: Most preferred process has score 29 : 633 gmond with usage of memory: 6776KB with priority 19 owned by user (id: 999) killing now" [02:25:49] YuviPanda: Let's rewrite memcached in INTERCAL! :-) [02:26:05] scfc_de: :D sure why not... [02:26:24] scfc_de: let's also rewrite ssh while we're at it. It's clearly old and hence shitty and useless and not flexible enough [02:27:00] scfc_de: though to be fair I wish there was a mosh deamon of sorts so we could run it in a more efficient way [02:27:54] Waiting Jobs: 14 Running Jobs: 89 [02:28:58] YuviPanda: Isn't a mosh daemon installed? I remember Coren doing that. [02:29:07] there's a mosh deamon? [02:29:10] I wasn't aware of that [02:29:14] if so then yah, that's good [02:29:39] But I bet it's not in INTERCAL :-). [02:29:51] yet [02:30:04] terminatord is very polite, though: "Your job gmond (633 using 999 bytes of memory) on server tools-login was killed, because the system didn't have enough of free operating memory (only 6776 bytes of memory was remaining in moment) and Your process was one of most resource expensive jobs. This action was done automatically to prevent whole server from dying, and is no way Your fault. System administrators were notified about this [02:30:04] problem and will resolve the issue soon. I am sorry and I wish you a pleasant day. Your terminator daemon" [02:30:36] user friendly, yeah. [02:30:44] scfc_de: https://github.com/yuvipanda/fuckin-angry-intercal [02:30:59] it's... not angry enough yet, mostly because apparently I suck at trying to be angry [02:33:22] YuviPanda: Ts, it's not even part of gcc? :-) [02:33:36] gcc is also old and hence shitty [02:36:53] And tools-login is stuck again?! WTF?! [02:37:17] Ah, okay, just a simple pause. *Argl*, failures cause anxiety :-). [02:40:22] !log tools Restarted toolwatcher on tools-login. [02:40:24] Logged the message, Master [02:48:53] Utilization heatmap is red [03:10:32] (03PS1) 10Yuvipanda: Use argparse to parse parameters for registering stream clients [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76461 [03:10:33] (03PS1) 10Yuvipanda: Add method to list subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76462 [03:10:52] (03CR) 10Yuvipanda: [C: 032 V: 032] Use argparse to parse parameters for registering stream clients [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76461 (owner: 10Yuvipanda) [03:11:27] (03CR) 10Yuvipanda: [C: 032 V: 032] Add method to list subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76462 (owner: 10Yuvipanda) [03:19:22] (03PS1) 10Yuvipanda: Rename to subscriptions.py, which is more accurate [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76464 [03:26:45] (Just before going to bed after a nice weekend of not being around): I need to rebuild tools-login with actual resources rather than the currently skimpy values. [03:27:17] +1 [03:27:24] good night, Coren [03:33:33] (03PS1) 10Yuvipanda: Add delete command to be clear and delete subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76465 [03:33:34] (03PS1) 10Yuvipanda: Perform saving when adding new subscriptions atomically [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76466 [03:34:01] (03CR) 10Yuvipanda: [C: 032 V: 032] Add delete command to be clear and delete subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76465 (owner: 10Yuvipanda) [03:35:49] (03PS2) 10Yuvipanda: Rename to subscriptions.py, which is more accurate [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76464 [03:36:15] (03PS3) 10Yuvipanda: Rename to subscriptions.py, which is more accurate [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76464 [03:36:22] (03CR) 10Yuvipanda: [C: 032 V: 032] Rename to subscriptions.py, which is more accurate [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76464 (owner: 10Yuvipanda) [03:36:37] (03PS2) 10Yuvipanda: Add delete command to be clear and delete subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76465 [03:36:46] (03CR) 10Yuvipanda: [C: 032 V: 032] Add delete command to be clear and delete subscriptions [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76465 (owner: 10Yuvipanda) [03:37:00] (03PS2) 10Yuvipanda: Perform saving when adding new subscriptions atomically [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76466 [03:37:08] (03CR) 10Yuvipanda: [C: 032 V: 032] Perform saving when adding new subscriptions atomically [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76466 (owner: 10Yuvipanda) [03:37:14] woooo spam :) [03:39:10] YuviPanda: first time I saw grrrit-wm speak [03:39:17] :) [03:39:25] it's active in other channels a lot [03:39:30] very few repositories under labs/* for now [07:40:15] Coren: huh? [07:43:48] petan: what's the difference between tools-login and tools-dev? [08:03:43] zhuyifei1999 -login is for bot operation, -dev is for building etc [08:03:48] -dev has more libraries and tools [08:06:18] kjshddkjasjdk [08:06:23] screen -S sessionname -X eval 'stuff "dein befehl mit parametern"\015' [08:06:28] why dos not work [08:06:29] :O [08:06:39] screen -S sessionname -X eval 'stuff "/join #test"\015' [08:06:39] :O [08:07:56] ups. wrong channel. sorrry. [08:25:46] YuviPanda ping [08:25:59] YuviPanda how I retrieve stuff from redis from terminal? [10:39:12] !screenfix [10:39:13] script /dev/null [10:58:20] petan: you can't from our current setup. you'd need to run redis-cli, which is installed only on tools-mc [11:04:50] YuviPanda: isn't there some other terminal client? [11:05:07] i just do python followed by import redis, then connect and do stuff [11:05:18] mhm [11:05:42] what python library [11:05:46] redis? [11:05:53] https://redis-py.readthedocs.org/en/latest/ [11:05:54] ? [11:06:09] yeah, it's already installed clusterwide [11:07:18] YuviPanda: did you see dispatcher-cli (in /data/project/dispatcher/bin) it already works, sort of [11:07:23] you can create new queues using it [11:07:31] and insert /some/ definitions [11:07:34] no, haven't yet [11:07:45] spent some time adding a proper client to my gerrit-to-redis thing [11:07:49] https://wikitech.wikimedia.org/wiki/Bot_Dispatcher#Example_usage [11:08:07] you still haven't told me how you'll handle security for the subscriber [11:08:27] Host: /data/project/dispatcher/hostname and Port /data/project/dispatcher/port [11:08:30] will have to be world readable [11:08:42] if everyone is supposed to be able to execute Script is @ /data/project/dispatcher/bin/dispatcher-cli [11:09:59] petan: so *anyone* can communicate with the deamon. [11:10:04] and do whatever [11:10:33] this is why the gerrit-to-redis subscribe.py is not publicly available to everyone [11:11:48] YuviPanda: loha [11:11:52] hello! [11:12:00] AzaToth: I'm going to 'kill' the 'testing' queue soon [11:12:03] is that okay? [11:12:09] fine for me atm [11:12:17] alright, let me know if you want it back [11:12:20] did you look at the two patchsets left? [11:12:29] there are patchsets left? :| [11:12:31] the draft and the jshint one [11:12:32] not in my dashboard. let me look [11:12:44] looking now [11:12:52] ah, I never added you are reviewer [11:12:54] soz [11:13:44] (03CR) 10Yuvipanda: [C: 032 V: 032] "Thank you!" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76194 (owner: 10AzaToth) [11:14:16] (03PS2) 10Yuvipanda: adding published draft hook [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76193 (owner: 10AzaToth) [11:14:42] AzaToth: looks good to me, but did you test it? [11:15:09] the draft one, nope [11:15:20] AzaToth: meh, let's just do it :) [11:15:26] heh [11:15:46] it's pretty much the same as patchset [11:15:47] (03CR) 10Yuvipanda: [C: 032 V: 032] "Let's test it live!" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76193 (owner: 10AzaToth) [11:16:00] heh [11:16:02] (03PS2) 10Yuvipanda: Fixing jshint hints [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76194 (owner: 10AzaToth) [11:16:15] (03CR) 10Yuvipanda: [C: 032 V: 032] Fixing jshint hints [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76194 (owner: 10AzaToth) [11:16:21] AzaToth: done. want to restart it? [11:16:25] k [11:16:40] hashar: heya! Are there docs on how I can enable jenkins-bot for a tool? [11:16:47] specifically I'd want jshint, and want it to be voting :) [11:17:15] YuviPanda: it's in integration-jenkins-bot-config [11:17:25] integration/jenkins-bot-config? [11:17:26] let me clone [11:17:57] yup [11:18:31] https://git.wikimedia.org/project/integration [11:18:32] no such thing [11:19:20] integration/jenkins-job-builder-config [11:19:59] YuviPanda: sure we get some doc [11:20:42] (03PD1) 10AzaToth: test [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76487 [11:20:47] YuviPanda: works [11:20:55] AzaToth: \o/ thank you ;) [11:21:24] as you notice, it didn't post when I uploaded the draft [11:21:24] AzaToth: I'm thinking that the only thing left to do is, uh, to have a 'ping' mechanism [11:21:34] only when I clicked on the publish draft button [11:21:37] AzaToth: drafts are supposed to be private, no? [11:21:38] so makes sense [11:21:39] !ping [11:21:39] *POOF* "Wadda need?" *POOF* "Wadda need?" *POOF* "Wadda need?" [11:21:45] !ping del [11:21:45] YuviPanda: I wrote two tutorials at https://www.mediawiki.org/wiki/Continuous_integration/Tutorial [11:21:45] Successfully removed ping [11:21:48] probably need one more [11:21:48] !ping is pong [11:21:49] Key was added [11:22:09] hashar: ah, looking [11:22:21] should probably write one to add jslint [11:22:40] hashar: yup! [11:23:16] AzaToth: doing something like 'grrrit-wm: ping' should respond with something like 'Working...' [11:23:19] so we know the bot is up [11:23:35] nothing more, we don't want to become yet another massive trawling can-do-everything bot [11:23:48] hashar: regarding jenkins, I saw you made some work on the zuul last week [11:24:01] I see [11:24:13] AzaToth: yup got a bunch of patch to get merged in ops repo [11:24:24] (03Abandoned) 10AzaToth: test [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/76487 (owner: 10AzaToth) [11:24:33] I am not going to enable them this week though since I am heading in vacation by the end of the week :( [11:24:34] nice [11:24:44] not nice [11:24:55] well, nice for you [11:24:59] YuviPanda: on which project do you want to add jslint? And will you be there for the next couple hours? [11:25:20] AzaToth: it is a rather disrupting change so I would prefer being there whenever it screw up heeh [11:25:25] hehe [11:25:36] hashar: labs/tools/grrrit [11:25:36] hashar: I will be sortof around, just woke up so need to brush, eat, etc [11:25:42] merge it friday evening [11:26:01] YuviPanda: I read it as "I just woke up in a bush" [11:26:06] then I will get phone calls! [11:26:17] turn phone off [11:26:18] hehe [11:27:24] petan: your tool mediawiki-mc seem to be broken with "[41dddf8e] 2013-07-29 11:24:54: Fatal exception of type MWException" [11:27:49] zhuyifei1999 / [11:27:52] what you talk about [11:28:02] http://tools.wmflabs.org/mediawiki-mc/w/ [11:28:03] what is mediawiki-mc tool [11:28:05] ah that [11:28:09] ignore that [11:28:15] it's just a test wiki for me [11:28:21] purposefully broken [11:29:44] petan: did you see my comments about authentication? [11:29:53] where [11:31:42] YuviPanda: going out to grab a snack, will be back in less than half an hour [11:31:49] hashar: alright! [11:32:02] 16:38 YuviPanda: you still haven't told me how you'll handle security for the subscriber [11:32:05] 16:38 YuviPanda: Host: /data/project/dispatcher/hostname and Port /data/project/dispatcher/port [11:32:10] 16:38 YuviPanda: will have to be world readable [11:32:13] 16:38 YuviPanda: will have to be world readable [11:32:17] 16:38 YuviPanda: if everyone is supposed to be able to execute Script is @ /data/project/dispatcher/bin/dispatcher-cli [11:32:21] 16:39 YuviPanda: petan: so *anyone* can communicate with the deamon. [11:32:25] 16:40 YuviPanda: and do whatever [11:32:28] 16:40 YuviPanda: this is why the gerrit-to-redis subscribe.py is not publicly available to everyone [11:32:31] petan: ^ [11:37:51] * YuviPanda considers writing soemthing in Go [11:39:56] YuviPanda, hashar I created a picture for easy understanding: https://wikitech.wikimedia.org/wiki/Bot_Dispatcher [11:39:59] :O [11:40:18] petan: I've read that, that doesn't mention the security issue I asked about [11:40:36] ok the picture was added 1 minute ago :P [11:40:45] anyway YuviPanda, did you read the example usage? [11:40:58] when you create a subscription, you will be granted a security token [11:41:08] you need to provide this token in order to modify the subscription [11:41:27] yes anyone can communicate with the daemon [11:41:40] but it will not allow anyone to do anything without proper permissions [11:41:47] in future we can replace this with keystone [11:41:52] but for that it needs to exist [11:41:59] petan: hmm, that actually makes sense. I had thought the token was for something else. [11:42:05] nope [11:42:13] you can try it yourself it already works [11:42:29] just ln -s /data/project/dispatcher/bin/dispatcher-cli ~/bin/dispatcher-cli [11:43:02] will do in a bit, petan [11:43:03] when you type dispatcher-cli --subscribe yuvipanda it will create a redis queue yuvipanda and give you secret token to manipulate it [11:43:15] wait, the redis queue name is going to be 'yuvipanda'? [11:43:27] it's going to be whatever you call it [11:43:39] you can call it mysupersecretqueuenooneeverfind [11:43:39] hmm, right. [11:43:57] re [11:46:39] hashar: lol [11:46:48] hashar: I didn't notice, but you probably didn't notice either [11:47:04] Antoine Musso 1:04 PM (42 minutes ago) [11:47:05] to me [11:47:17] did you send it to me only? because I clicked reply to all [11:47:23] and there was your e-mail only [11:51:29] petan: I understood your architecture [11:51:45] from picture? :) [11:51:46] but I disagree on using the IRC feed [11:51:49] no from before [11:51:56] I disagree as well [11:52:04] but unfortunatelly there is only 1 option: use IRC feed [11:52:17] then you are on your own [11:52:23] and nobody is going to support that [11:52:38] but... don't you see that? [11:52:39] using IRC Feed is dumb [11:52:41] really [11:52:44] it's not that I like irc feed [11:52:48] hashar: i think that's okay, really. Nobody has supported it for a long time and there exists no alternatives. [11:52:50] it's only one option that WMF provides [11:52:57] WMF doesn't give us any alternative [11:52:59] hashar: and anyone attempting to write an alternative is going to be bikeshedded to death [11:53:03] what am I supposed to use hashar [11:53:19] hashar: and hence there is no interest. See two large wikitech-l threads about this over the last 2 years. [11:53:23] what I told you earlier, a json feed [11:53:31] there is NO json feed [11:53:34] where would I get it? [11:53:37] work with Vitaly and Krenair to get MediaWiki core to support JSON feeding of Rcent Change [11:53:41] that is better investment of your time [11:53:47] hashar: once there is JSON feed I will happily use it [11:53:56] but in this moment there is only irc [11:53:59] so I have to use that [11:54:07] you are talking about distant future [11:54:10] this way you will more more have to care about parsing horrible IRC formatting [11:54:11] hashar: also, you should no better than to try to change petan's mind :) [11:54:18] *know [11:54:42] but you guys make no sense, you tell me not to use A but you don't tell me what I should use instead, because JSON feed is sci-fi today [11:54:42] hashar, unfortunately the current JSON lines are too long to go on to IRC [11:54:42] petan: and luckily we have a patch that does the JSON feed [11:54:53] is it merged/ [11:54:56] live on production? [11:55:00] how do I attach to it [11:55:19] should I copy paste from our discussion earlier ? [11:55:27] no [11:55:34] it's not merged [11:55:41] I will use it as soon as it's live [11:55:49] MediaWiki -- (json) ---> ZeroMq --- (subscriber) ---> Redis [11:55:50] done [11:55:52] now I /have/ (not I prefer to) use irc feed [11:55:58] the json part is being handled in core as we speak [11:56:14] the subscriber is a few lines of code just like your dispatcher [11:56:48] though it will receiver the properly formatted message which will make it way more easier for you to maintain [11:56:50] ok I like the idea, in picture as you can see is written "dispatcher parses from irc or anything better source if it exist" [11:56:51] so there is a bit of investement [11:56:57] but that is well worth it in the long run [11:57:04] it doesn't exist right now, but as soon as it's deployed and working I will happily switch to it [11:57:33] so you are going to invest time on an horrible interface instead of helping get the new json feed setup? [11:57:48] that sounds to me like a vast of your time [11:57:48] how could I help? I don't work for wmf... I have no access here [11:58:24] you could help Krenair and Victor which have a patch for MW core that generate JSON [11:58:30] you can try it out report issues [11:58:39] and help them test the code [11:58:43] that would be a nice first thing [11:58:47] I can only work on wmflabs [11:58:56] we can create a project there to test it [11:59:04] or maybe we could enable that on beta? [11:59:06] the faster the code is merged in the faster it will be deployed in production and usable [11:59:16] yeah exactly [11:59:20] hmm [11:59:26] and have it send the UDP json message to MaxSem event logging instance we have in beta [11:59:31] ok so what is needed to install it? is there some documentation? [11:59:38] from there you can work with them to write a Zero MQueue subscriber [11:59:57] just like you would write your own dispatcher, though you will have to use a Zero MQueue interface instead [12:00:25] whenever you land with something worthwhile in labs, we can copy paste the code in prod and blam. [12:00:27] generalized [12:00:30] I still think that this ZMQ thingie is not able to do what dispatcher does [12:00:33] supported by the whole staff 24/24 7/7 [12:00:43] and you no more have to take care about the json feed. It would just be granted [12:00:54] how does the ZMQ filter the RC items from JSON feed? [12:01:01] and since EventLogging / ZMq is well supported by WMF staff, you end up with a better service than the irc feed. [12:01:40] petan: it would not filter them, just like the IRC feed is unfiltered. [12:01:40] ah ok [12:01:43] so your subscriber (or "dispatcher") still have to handle the filtering [12:01:48] ok [12:01:52] it is just that you are using a much more reliable source [12:02:01] and which is trivial to parse (json !! :D ) [12:02:03] sorry [12:02:08] I like the whole concept [12:02:09] well, so let's deploy it... to beta [12:02:11] it is nice [12:02:21] or maybe we should create another picture lol [12:02:25] I just disagree on the source of the events which is probably going to cause you headache in the long term. [12:02:28] because I am started being confused [12:02:47] btw this is cool thing: https://www.gliffy.com/ is there any free alternative? [12:02:56] I created the picture in that XD [12:06:08] Krenair: so what is current status of JSON support in MediaWiki, also how is it going to send the stuff to ZMQ? Is that an extension or core stuff? [12:06:24] PATCH_TO_REVIEW [12:06:43] I have no idea about ZMQ I don't think I'd heard of it until today [12:07:02] It could be core, but it'll likely be extension or even just in the wiki's config [12:08:11] I think extension is best idea [12:08:24] core should only contains most necessary stuff for it to work [12:08:55] also keep in mind 3rd parties use mediawiki and they don't need this [12:09:15] Krenair: can you link me to that "patch_to_review" [12:10:10] https://gerrit.wikimedia.org/r/#/c/52922/10 [12:14:55] petan: I don't know anything better than gliffy.com besides Microsoft Visio :D [12:15:12] maybe it's worth that $4.90 [12:15:14] :P [12:43:07] [02dispatcher-labs] 07benapetr pushed 031 commit to 03master [+0/-0/±4] 13http://git.io/OtJt2w [12:43:08] [02dispatcher-labs] 07benapetr 039f061c2 - lot of clean up and some comments too [12:43:12] :0 [12:43:14] it works [12:45:07] Not-002? [12:45:16] that's some github bot [12:45:20] not my work I just use it :P [12:46:24] It was Not-001 on tm-irc [12:54:53] https://www.youtube.com/watch?v=RQkdB49hBTc LOLZ [13:05:05] (03PS1) 10Yuvipanda: Add requirements.txt [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76495 [13:05:06] (03PS1) 10Yuvipanda: Ensure that queues do not get too long [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76496 [13:06:18] (03CR) 10Yuvipanda: [C: 032 V: 032] Add requirements.txt [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76495 (owner: 10Yuvipanda) [13:06:20] (03CR) 10Yuvipanda: [C: 032 V: 032] Ensure that queues do not get too long [labs/tools/gerrit-to-redis] - 10https://gerrit.wikimedia.org/r/76496 (owner: 10Yuvipanda) [13:07:28] [02dispatcher-labs] 07benapetr pushed 031 commit to 03master [+1/-0/±5] 13http://git.io/xr7JsA [13:07:29] [02dispatcher-labs] 07benapetr 031715925 - lot more cleanup and file I forgot to push [13:09:26] NFS is stuck again [13:09:27] meh [13:09:52] Coren: ^ [13:09:53] o.o [13:09:56] &trustadd test test [13:09:56] Unknown user level! [13:10:01] &trustadd test trusted [13:10:03] indeed [13:10:05] it is [13:10:09] * Coren grumbles. [13:10:11] bota is stuck too :P [13:10:15] Successfuly added test [13:10:20] it's back o/ [13:10:28] I need to schedule a downtime this week to switch controllers. [13:10:35] Coren: anything from Dell support? [13:11:09] YuviPanda: Not yet. Their linux support is teh suck because it's their own engineers being volunteers for the most part. [13:11:23] sigh [13:11:23] ok [13:11:31] They don't support anything but RHEL officially. [13:11:35] * Coren ponders. [13:11:41] no wonder [13:11:46] Maybe it would help if like, Erik gave them a call. [13:11:57] But then again, ops have a history of odd controller behaviour. [13:12:03] that'll probably help, definitely, yeah. [13:12:11]