[00:25:38] <arturo>	 !log tools (/srv is almost full) aborrero@tools-static-11:/srv/cdnjs$ sudo git gc --aggressive
[00:25:43] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[00:55:20] <arturo>	 !log tools at tools-static-11 the kernel OOM killer stopped git gc at about 20% :-(
[00:55:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[04:54:02] <xaosflux>	 !help  https://wma.wmflabs.org appears to be down
[04:54:02] <wm-bot5>	 xaosflux: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[04:54:21] <xaosflux>	 causing WikiMiniAtlas links to all fail
[04:55:35] <xaosflux>	 user reports with example : https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_(technical)&oldid=822573988#Globe_link_to_the_left_of_title_coordinates_gives_502_Bad_Gateway_error
[05:00:30] <xaosflux>	 T185797 opened
[05:00:30] <stashbot>	 T185797: wma.wmflabs.org is failing with 502 bad gateway - https://phabricator.wikimedia.org/T185797
[05:15:00] <xaosflux>	 have to go chasemp can you get that assigned?
[07:13:32] <andrewbogott>	 !log maps rebooting maps-wma1
[07:13:35] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Maps/SAL
[13:20:55] <chicocvenancio>	 Hey avgas, did you get access to your tool again? 
[13:25:04] <avgas>	 chicocvenancio: yeah, I'm running task just right now but unfortunately I absolutely have to interact with bot so I need to run it on shell
[13:25:51] <avgas>	 as soon as possible I'll try that two solution, I'm pretty curious...
[13:27:38] <avgas>	 mmmh... maybe I could try... chicocvenancio: we said put command in a file and run it like "jsub -N <blabla> <filename>", its correct?
[13:30:47] <chicocvenancio>	 Yes, thats right
[13:31:17] <avgas>	 chicocvenancio: command is "br.txt"
[13:31:27] <avgas>	 submitted jsub br.txt
[13:31:35] <chicocvenancio>	 But I was more referring to gnubotmarco.
[13:32:01] <avgas>	 chicocvenancio: uhm... no, I'm doing this task with IrishBot
[13:32:13] <avgas>	 ok let me see gnubotmarcoo
[13:33:01] <avgas>	 chicocvenancio: ok, it works, thanks!
[13:33:09] <avgas>	 (access to gnubotmarcoo)
[13:33:24] <avgas>	 but jsub on IrishBot not
[13:39:05] <chicocvenancio>	 https://www.irccloud.com/pastebin/B1B2sCO5/
[13:40:40] <chicocvenancio>	 `chmod br.txt` was sufficient to solve it
[13:41:22] <chicocvenancio>	 `jsub br.txt` worked after that
[13:43:29] <chicocvenancio>	 avgas: ^
[13:43:48] <chicocvenancio>	 is it ok that br.txt is running now? or should I kill it?
[13:55:14] <avgas>	 chicocvenancio: its ok
[15:26:12] <xaosflux>	 Hi cloud support, is anyone working on T185797 maps-wma1.maps.eqiad.wmflabs is down (wma.wmflabs.org) ?
[15:26:12] <stashbot>	 T185797: maps-wma1.maps.eqiad.wmflabs is down (wma.wmflabs.org) - https://phabricator.wikimedia.org/T185797
[15:29:10] <paladox>	 xaosflux hi, all of wmf is on allhands so i doin't think anyone will be looking at that until sometime next week.
[15:30:01] <xaosflux>	 thanks paladox, was just tyring to get any sort of expectation - it's really hard to tell if there is SLA's on this type of outage
[15:30:12] <paladox>	 yep
[15:30:23] <paladox>	 xaosflux from andrew's comment, it seems the instance is broken
[15:30:30] <avgas>	 chicocvenancio: how can I move a file from bastion-01 to tools-bastion-03?
[15:31:10] <valhallasw`cloud>	 xaosflux: there's no SLA on most *.wmflabs.org things, as they are almost all volunteer-maintained. In the case of maps.wmflabs.org I'm not entirely certain, but I vaguely remember that it was supposed to be superseded by a production version, maps.wikimedia.org / kartographer
[15:31:15] <xaosflux>	 thanks, I saw that note, but see the ticket isn't claimed
[15:32:06] <xaosflux>	 thanks valhallasw`cloud I'm disabling the enwiki calls to it so it doesn't lead readers to an ugly error message for now
[15:32:19] <valhallasw`cloud>	 avgas: scp bastion:/path/to/file tools-bastion-03:/target/path    from a host that can connect to both
[15:32:44] <valhallasw`cloud>	 avgas: if you use agent forwarding, you should also be able to scp /path/to/file tools-bastion-03:/target/path   on bastion-01
[15:39:58] <chicocvenancio>	 avgas: valhallasw`cloud is correct, but the simplest and safest is download and re-upload 
[15:40:43] <avgas>	 how command to download e upl?
[15:45:53] <avgas>	 ok, got it
[15:49:05] <chicocvenancio>	 scp bastion:/path/to/file /local/path
[15:50:02] <chicocvenancio>	 scp /local/path/file tools-bastion-03:/target/path
[15:50:56] <chicocvenancio>	 if it's large and non-sensitive, using agent forwarding seems sensible
[15:51:20] <chicocvenancio>	 xaosflux: I think I'm the only one from the team not on Allhands
[15:52:59] <chicocvenancio>	 as valhallasw`cloud mentioned, the tools are volunteer maintained we don't really actively support tools beyond basic commands and quick prodding
[15:54:23] <chicocvenancio>	 andrewbogott rebooted the tool 8 hours ago but it was insufficient to bring it up, I'll make a note of that on the task
[15:54:54] <chicocvenancio>	 nvm, he already made that note
[15:55:17] <xaosflux>	 chicocvenancio thanks for the note, its not like its the end of the world; the largest reader impacting sites using this look to be enwiki and dewiki
[15:55:35] <xaosflux>	 easy enough to turn it off there until repaired (hopefully)
[15:55:55] <andrewbogott>	 I'll investigate a bit more this morning… the best scenario would be if someone who actually understands that project could recreate that VM.
[16:03:17] <valhallasw`cloud>	 xaosflux: alternatively, this could be a good time to convert the template to use https://www.mediawiki.org/wiki/Help:Extension:Kartographer
[16:03:46] <valhallasw`cloud>	 although I'm not 100% sure whether that's also deployed on enwiki
[16:04:53] <xaosflux>	 Special:Version onenwiki says it exists at least, would need someone to work on testing etc
[16:06:00] <xaosflux>	 moving to a more supported solution for something reader-facing is the best idea in my opinion
[16:06:28] * chicocvenancio nods
[16:07:48] <andrewbogott>	 yeah, if that's an option that's much better :)
[16:57:03] <andrewbogott>	 chicocvenancio: that VM is fixed now, but the suggestion to move to Kartographer stands :)
[17:01:10] <wikibugs>	 (03CR) 10Zoranzoki21: [C: 031] Add Chicocvenancio's key for Cloud Services [labs/private] - 10https://gerrit.wikimedia.org/r/405376 (https://phabricator.wikimedia.org/T185273) (owner: 10Chico Venancio)
[17:11:57] <wikibugs>	 (03CR) 10Zoranzoki21: "recheck" [labs/icinga2] - 10https://gerrit.wikimedia.org/r/404182 (owner: 10Paladox)
[21:47:42] <wikibugs>	 (03PS2) 10Zoranzoki21: Improve error message of become [labs/toollabs] - 10https://gerrit.wikimedia.org/r/382200 (https://phabricator.wikimedia.org/T149511) (owner: 10Mridubhatnagar)
[23:58:03] <wikibugs>	 (03Draft1) 10Paladox: Remove puppitsement of roles.ini [labs/icinga2] - 10https://gerrit.wikimedia.org/r/406482
[23:58:06] <wikibugs>	 (03PS2) 10Paladox: Remove puppitsement of roles.ini [labs/icinga2] - 10https://gerrit.wikimedia.org/r/406482
[23:58:11] <wikibugs>	 (03CR) 10Paladox: [V: 032 C: 032] Remove puppitsement of roles.ini [labs/icinga2] - 10https://gerrit.wikimedia.org/r/406482 (owner: 10Paladox)