[00:40:46] <shinken-wm>	 PROBLEM - Puppet run on tools-puppetmaster-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[02:36:29] <shinken-wm>	 PROBLEM - SSH on tools-flannel-etcd-01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:24:29] <grrrit-wm>	 (03Draft3) 10Zppix: Add labs/tools/zppixbot [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/317380 
[03:37:56] <wikibugs>	 06Labs, 10Tool-Labs: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736434 (10Zppix)
[03:38:31] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 06Project-Admins: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736447 (10Zppix)
[06:04:49] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Ravikiran.j was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=920488 edit summary: 
[06:05:24] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Kotz was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=920490 edit summary: 
[06:05:49] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/MikeMel was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=920491 edit summary: 
[06:59:06] <shinken-wm>	 PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Not Available - string 'Magnus' not found on 'http://tools.wmflabs.org:80/' - 531 bytes in 0.017 second response time
[07:19:04] <shinken-wm>	 RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 3670 bytes in 0.020 second response time
[11:50:31] <Kotz>	 Hi. Anyone here?
[11:53:08] <Kotz>	 guess not
[12:01:03] <shinken-wm>	 PROBLEM - Puppet staleness on tools-webgrid-generic-1403 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [43200.0]
[12:10:29] <KOTZ_>	 hi?
[12:10:54] <KOTZ_>	 help needed
[12:12:43] <KOTZ_>	 hi?
[12:14:31] <KOTZ_>	 hi
[12:14:36] <KOTZ_>	 can u help me?
[12:33:38] <KOTZ_>	 hi. can  u help me ?
[12:34:03] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 06Project-Admins: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736582 (10Aklapper) Hi @Zppix, unfortunately this report lacks some information. If you have time and can still reproduce the problem, please [[ https://www.mediawiki.org/wiki...
[13:46:24] <lbertolotti>	 Hi
[13:46:36] <Luke081515>	 hi
[13:46:45] <lbertolotti>	 For some reason, pageview statistics stopped working
[14:30:24] <lbertolotti>	 hello?
[14:34:00] <lbertolotti>	 hi
[14:45:50] <shinken-wm>	 PROBLEM - Puppet run on tools-services-02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[15:15:49] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Talfers was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=920681 edit summary: 
[15:25:50] <shinken-wm>	 RECOVERY - Puppet run on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0]
[16:01:02] <lbertolotti>	 anyone?
[16:20:29] <Krenair>	 lbertolotti, hi?
[16:33:37] <addshore>	 Krenair: can you by any chance create labs projects?
[16:35:08] <Krenair>	 probably technically
[16:35:12] <Krenair>	 but...
[16:35:37] <Krenair>	 the proper process is https://phabricator.wikimedia.org/T76375
[16:36:07] <addshore>	 Krenair: indeed, I know, I filed one ;) but of course would like it asap! :)
[16:36:44] <Krenair>	 I thought these started getting approved by weekly labs staff meetings
[16:37:03] <Krenair>	 they come with default non-0 quotas, so
[16:39:01] <Krenair>	 can it wait until tomorrow?
[16:39:52] <addshore>	 It can, but of course, any delay is always annoying ;)
[16:40:02] <Krenair>	 sorry
[16:40:41] <addshore>	 not your fault! classic cross team / org / department / anything delays :D
[16:41:14] <Krenair>	 even if I was labs staff I wouldn't do it right now
[16:41:24] <Krenair>	 though I would be able to attend the meeting, so there's that
[16:53:36] <lbertolotti>	 Krenair, do you know why pageview statistics is not working?
[16:53:44] <Krenair>	 lbertolotti, what url is this?
[16:54:00] <lbertolotti>	 pcik any page on wikipedia
[16:54:08] <lbertolotti>	 for ex:
[16:54:09] <lbertolotti>	 https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&range=latest-20&pages=Shemini_Atzeret
[16:54:31] <Krenair>	 what's not working about that?
[16:54:52] <lbertolotti>	 look at it
[16:55:05] <lbertolotti>	 on day 17 it stopped working
[16:57:17] <Krenair>	 lbertolotti, https://people.wikimedia.org/~krenair/pageviews%20example%202016-10-23.png
[16:57:54] <lbertolotti>	 that's weird
[16:58:02] <Krenair>	 what do you see?
[17:01:08] <lbertolotti>	 https://drive.google.com/file/d/0B7Iw2O2p71kbYTllR2t2SlpNTlU/view?usp=sharing
[17:02:36] <Krenair>	 it still works for me if I switch language to italian
[17:03:12] <Krenair>	 works in firefox
[17:03:27] <lbertolotti>	 It's not a language problem
[17:03:29] <Krenair>	 do you see anything in the developer console? maybe an error message?
[17:03:38] <Krenair>	 evidently
[17:04:35] <lbertolotti>	 what console?
[17:05:16] <Krenair>	 F12 -> Console tab
[17:05:38] <Krenair>	 musikanimal, kaldari
[17:08:03] <lbertolotti>	 this:
[17:08:05] <lbertolotti>	       ___            __ _                     _                             application.js:15:446      | _ \  __ _    / _` |   ___    __ __    (_)     ___   __ __ __  ___    application.js:15:542      |  _/ / _` |   \__, |  / -_)   \ V /    | |    / -_)  \ V  V / (_-<    application.js:15:639     _|_|_  \__,_|   |___/   \___|   _\_/_   _|_|_   \___|   \_/\_/  /__/_   application.js:15:738   _| """ |_|"""""|_|"""""|_|"""
[17:09:43] <Krenair>	 yeah
[17:09:49] <Krenair>	 that's normal
[17:10:08] <lbertolotti>	 also this
[17:10:10] <lbertolotti>	 https://tools.wmflabs.org/pageviews/application.css
[17:12:31] <lbertolotti>	 Any idea?
[17:15:02] <Krenair>	 nope
[17:15:21] <Krenair>	 I might be able to help if I knew why you saw different results
[17:20:41] <lbertolotti>	 Could it be IP or network related?
[17:20:49] <Krenair>	 I doubt it
[17:21:13] <Krenair>	 difficult for me to investigate without being able to reproduce it myself though
[17:23:25] <lbertolotti>	 then why all computers on my network are reproducing what I showed you?
[17:23:35] <lbertolotti>	 One is even a Mac
[17:26:18] <Krenair>	 you should be seeing errors if it failed to fetch resources from the server
[17:32:05] <musikanimal>	 Hey!
[17:32:25] <musikanimal>	 Reading up
[17:51:27] <wikibugs>	 06Labs, 10Beta-Cluster-Infrastructure, 07Puppet: After starting new deployment-prep instance, have to delete /var/lib/puppet/ssl before puppet will function - https://phabricator.wikimedia.org/T148929#2736876 (10AlexMonk-WMF)
[18:20:08] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 06Project-Admins: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736938 (10Zppix) Basically like how there is a subrepo for different extentsions of mediawiki same concept only just as a project like Operations for example on phab But for m...
[18:21:55] <grrrit-wm>	 (03CR) 10Zppix: [C: 031] "Done" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/317380 (owner: 10Zppix)
[18:28:32] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 06Project-Admins: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736946 (10Aklapper) @Zppix: I don't know what a "subproject" is (as no context was provided), I don't know if this is some request to do something on (Tool) Labs servers, or t...
[18:28:53] <musikanimal>	 does Tool Labs use Varnish at all?
[18:30:38] <Krenair>	 don't think so
[18:30:39] <Krenair>	 why?
[18:33:34] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 06Project-Admins: Request for tools lab subproject - https://phabricator.wikimedia.org/T148910#2736947 (10Zppix) @aklapper  Maybe because i got mixed up i meant project and the requested name is to be ZppixBot please! Sorry!
[18:35:18] <musikanimal>	 nvm, sorry
[19:14:53] <Nemo_bis>	 https://joinup.ec.europa.eu/community/eu-fossa/news/study-%E2%80%98open-source-coders-more-aware-security%E2%80%99
[19:15:02] <Nemo_bis>	 wrong window
[21:05:10] <grrrit-wm>	 (03CR) 10Paladox: Add labs/tools/zppixbot (031 comment) [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/317380 (owner: 10Zppix)
[21:09:57] <grrrit-wm>	 (03PS1) 10QChris: DO NOT SUBMIT. Test commit for grrrit. [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/317447 
[21:24:28] <grrrit-wm>	 (03CR) 10QChris: [C: 04-2] "Discussing on IRC with Zppix showed that this Change is meant to" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/317380 (owner: 10Zppix)
[21:59:35] <grrrit-wm>	 (03Abandoned) 10Zppix: DO NOT SUBMIT. Test commit for grrrit. [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/317447 (owner: 10QChris)
[22:01:02] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-102 is DOWN: CRITICAL - Host Unreachable (10.68.21.170)
[22:34:24] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-103 is DOWN: CRITICAL - Host Unreachable (10.68.21.22)
[22:48:07] <shinken-wm>	 PROBLEM - Host secgroup-lag-102 is DOWN: CRITICAL - Host Unreachable (10.68.17.218)