[00:05:49] PROBLEM - Free space - all mounts on integration-castor03 is CRITICAL: (Service Check Timed Out) [00:10:42] RECOVERY - Free space - all mounts on integration-castor03 is OK: OK: All targets OK [00:11:57] PROBLEM - Free space - all mounts on deployment-urldownloader02 is CRITICAL: (Service Check Timed Out) [00:16:48] RECOVERY - Free space - all mounts on deployment-urldownloader02 is OK: OK: All targets OK [00:33:10] PROBLEM - Free space - all mounts on deployment-cache-upload06 is CRITICAL: (Service Check Timed Out) [00:38:03] RECOVERY - Free space - all mounts on deployment-cache-upload06 is OK: OK: All targets OK [00:51:36] PROBLEM - Free space - all mounts on deployment-memc05 is CRITICAL: (Service Check Timed Out) [00:55:39] PROBLEM - Free space - all mounts on integration-agent-stretch-1001 is CRITICAL: (Service Check Timed Out) [00:56:25] RECOVERY - Free space - all mounts on deployment-memc05 is OK: OK: All targets OK [01:00:29] RECOVERY - Free space - all mounts on integration-agent-stretch-1001 is OK: OK: All targets OK [01:07:02] PROBLEM - Free space - all mounts on deployment-mediawiki-07 is CRITICAL: (Service Check Timed Out) [01:08:04] RECOVERY - Free space - all mounts on deployment-chromium01 is OK: OK: All targets OK [01:09:15] PROBLEM - Free space - all mounts on deployment-sca02 is CRITICAL: (Service Check Timed Out) [01:11:00] PROBLEM - Free space - all mounts on deployment-webperf11 is CRITICAL: (Service Check Timed Out) [01:11:54] RECOVERY - Free space - all mounts on deployment-mediawiki-07 is OK: OK: All targets OK [01:15:49] RECOVERY - Free space - all mounts on deployment-webperf11 is OK: OK: All targets OK [01:18:21] PROBLEM - Free space - all mounts on deployment-memc04 is CRITICAL: (Service Check Timed Out) [01:23:12] RECOVERY - Free space - all mounts on deployment-memc04 is OK: OK: All targets OK [01:43:42] PROBLEM - Free space - all mounts on integration-agent-docker-1005 is CRITICAL: (Service Check Timed Out) [01:48:34] RECOVERY - Free space - all mounts on integration-agent-docker-1005 is OK: OK: All targets OK [01:48:56] PROBLEM - Free space - all mounts on integration-agent-docker-1003 is CRITICAL: (Service Check Timed Out) [01:50:06] PROBLEM - Free space - all mounts on deployment-eventstreams-1 is CRITICAL: (Service Check Timed Out) [01:53:46] RECOVERY - Free space - all mounts on integration-agent-docker-1003 is OK: OK: All targets OK [01:54:55] RECOVERY - Free space - all mounts on deployment-eventstreams-1 is OK: OK: All targets OK [02:11:38] PROBLEM - Free space - all mounts on deployment-docker-cxserver01 is CRITICAL: (Service Check Timed Out) [02:16:27] RECOVERY - Free space - all mounts on deployment-docker-cxserver01 is OK: OK: All targets OK [02:34:44] PROBLEM - Free space - all mounts on integration-agent-docker-1005 is CRITICAL: (Service Check Timed Out) [02:36:26] PROBLEM - Free space - all mounts on deployment-ms-be06 is CRITICAL: (Service Check Timed Out) [02:41:15] RECOVERY - Free space - all mounts on deployment-ms-be06 is OK: OK: All targets OK [03:25:23] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: (Service Check Timed Out) [03:30:14] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [03:40:13] PROBLEM - Free space - all mounts on deployment-aqs02 is CRITICAL: (Service Check Timed Out) [03:45:02] RECOVERY - Free space - all mounts on deployment-aqs02 is OK: OK: All targets OK [04:03:06] PROBLEM - Free space - all mounts on deployment-etcd-01 is CRITICAL: (Service Check Timed Out) [04:07:56] RECOVERY - Free space - all mounts on deployment-etcd-01 is OK: OK: All targets OK [04:22:25] PROBLEM - Free space - all mounts on deployment-ms-be06 is CRITICAL: (Service Check Timed Out) [04:26:29] PROBLEM - Free space - all mounts on deployment-puppetmaster04 is CRITICAL: (Service Check Timed Out) [04:27:14] RECOVERY - Free space - all mounts on deployment-ms-be06 is OK: OK: All targets OK [04:28:10] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: (Service Check Timed Out) [04:31:23] RECOVERY - Free space - all mounts on deployment-puppetmaster04 is OK: OK: All targets OK [04:32:58] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [05:05:49] PROBLEM - Free space - all mounts on deployment-puppetdb03 is CRITICAL: (Service Check Timed Out) [05:05:50] PROBLEM - Free space - all mounts on deployment-memc06 is CRITICAL: (Service Check Timed Out) [05:05:59] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: (Service Check Timed Out) [05:06:01] PROBLEM - Free space - all mounts on deployment-restbase01 is CRITICAL: (Service Check Timed Out) [05:06:12] PROBLEM - Free space - all mounts on deployment-aqs02 is CRITICAL: (Service Check Timed Out) [05:06:19] PROBLEM - Free space - all mounts on deployment-ores01 is CRITICAL: (Service Check Timed Out) [05:06:40] PROBLEM - Free space - all mounts on deployment-acme-chief03 is CRITICAL: (Service Check Timed Out) [05:07:11] PROBLEM - Free space - all mounts on deployment-chromium01 is CRITICAL: (Service Check Timed Out) [05:07:11] PROBLEM - Free space - all mounts on integration-agent-pkgbuilder-1001 is CRITICAL: (Service Check Timed Out) [05:07:28] PROBLEM - Free space - all mounts on deployment-acme-chief04 is CRITICAL: (Service Check Timed Out) [05:07:32] PROBLEM - Free space - all mounts on deployment-docker-mathoid01 is CRITICAL: (Service Check Timed Out) [05:07:46] PROBLEM - Free space - all mounts on integration-agent-docker-1004 is CRITICAL: (Service Check Timed Out) [05:07:50] PROBLEM - Free space - all mounts on integration-agent-pkgbuilder-1002 is CRITICAL: (Service Check Timed Out) [05:07:58] PROBLEM - Free space - all mounts on deployment-ms-be05 is CRITICAL: (Service Check Timed Out) [05:08:23] PROBLEM - Free space - all mounts on deployment-cache-text06 is CRITICAL: (Service Check Timed Out) [05:08:23] PROBLEM - Free space - all mounts on deployment-ms-be06 is CRITICAL: (Service Check Timed Out) [05:08:34] PROBLEM - Free space - all mounts on deployment-sca01 is CRITICAL: (Service Check Timed Out) [05:08:34] PROBLEM - Free space - all mounts on deployment-prometheus02 is CRITICAL: (Service Check Timed Out) [05:08:40] PROBLEM - Free space - all mounts on deployment-schema-2 is CRITICAL: (Service Check Timed Out) [05:09:05] PROBLEM - Free space - all mounts on deployment-etcd-01 is CRITICAL: (Service Check Timed Out) [05:09:08] PROBLEM - Free space - all mounts on deployment-restbase03 is CRITICAL: (Service Check Timed Out) [05:09:18] PROBLEM - Free space - all mounts on integration-slave-test-jeena is CRITICAL: (Service Check Timed Out) [05:09:24] PROBLEM - Free space - all mounts on deployment-docker-citoid01 is CRITICAL: (Service Check Timed Out) [05:09:37] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-09 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:09:37] PROBLEM - Free space - all mounts on deployment-echostore01 is CRITICAL: (Service Check Timed Out) [05:09:40] PROBLEM - Free space - all mounts on integration-puppetmaster-02 is CRITICAL: (Service Check Timed Out) [05:09:46] PROBLEM - Free space - all mounts on integration-agent-docker-1002 is CRITICAL: (Service Check Timed Out) [05:09:50] PROBLEM - Free space - all mounts on integration-agent-docker-1010 is CRITICAL: (Service Check Timed Out) [05:09:54] PROBLEM - Free space - all mounts on integration-agent-puppet-docker-1001 is CRITICAL: (Service Check Timed Out) [05:10:14] PROBLEM - Free space - all mounts on deployment-sca02 is CRITICAL: (Service Check Timed Out) [05:10:33] PROBLEM - Free space - all mounts on integration-agent-docker-1009 is CRITICAL: (Service Check Timed Out) [05:10:40] PROBLEM - https://phabricator.wikimedia.org #page on phabricator.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Phabricator [05:10:41] RECOVERY - Free space - all mounts on deployment-memc06 is OK: OK: All targets OK [05:10:41] PROBLEM - Free space - all mounts on integration-agent-docker-1005 is CRITICAL: (Service Check Timed Out) [05:10:51] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:11:12] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:11:38] PROBLEM - Free space - all mounts on deployment-chromium02 is CRITICAL: (Service Check Timed Out) [05:11:47] PROBLEM - Free space - all mounts on deployment-poolcounter06 is CRITICAL: (Service Check Timed Out) [05:11:47] PROBLEM - Free space - all mounts on deployment-mediawiki-09 is CRITICAL: (Service Check Timed Out) [05:11:53] PROBLEM - Free space - all mounts on deployment-kafka-main-2 is CRITICAL: (Service Check Timed Out) [05:11:56] PROBLEM - Free space - all mounts on integration-agent-docker-1003 is CRITICAL: (Service Check Timed Out) [05:11:56] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:12:06] PROBLEM - Free space - all mounts on deployment-memc08 is CRITICAL: (Service Check Timed Out) [05:12:18] PROBLEM - Free space - all mounts on deployment-kafka-jumbo-1 is CRITICAL: (Service Check Timed Out) [05:12:26] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: (Service Check Timed Out) [05:12:29] brennen: greg-g: jrbranaa: liw: longma: marxarelli: thcipriani: twentyafterfour: zeljkof: prod trouble [05:12:31] PROBLEM - Free space - all mounts on integration-agent-docker-1001 is CRITICAL: (Service Check Timed Out) [05:12:54] RECOVERY - Free space - all mounts on deployment-ms-be05 is OK: OK: All targets OK [05:12:54] PROBLEM - Free space - all mounts on deployment-db06 is CRITICAL: (Service Check Timed Out) [05:12:57] PROBLEM - Free space - all mounts on deployment-wdqs01 is CRITICAL: (Service Check Timed Out) [05:13:00] PROBLEM - Free space - all mounts on integration-cumin-02 is CRITICAL: (Service Check Timed Out) [05:13:06] PROBLEM - Free space - all mounts on deployment-mcs01 is CRITICAL: (Service Check Timed Out) [05:13:07] PROBLEM - Free space - all mounts on deployment-ms-fe03 is CRITICAL: (Service Check Timed Out) [05:13:11] rxy: thanks, yeah, SREs are aware [05:13:19] PROBLEM - Free space - all mounts on integration-agent-docker-1014 is CRITICAL: (Service Check Timed Out) [05:13:24] PROBLEM - Free space - all mounts on deployment-mx02 is CRITICAL: (Service Check Timed Out) [05:14:22] PROBLEM - Free space - all mounts on integration-agent-docker-1007 is CRITICAL: (Service Check Timed Out) [05:14:41] PROBLEM - Free space - all mounts on deployment-wikifeeds01 is CRITICAL: (Service Check Timed Out) [05:14:48] PROBLEM - Free space - all mounts on deployment-aqs01 is CRITICAL: (Service Check Timed Out) [05:16:10] PROBLEM - Free space - all mounts on deployment-hadoop-test-2 is CRITICAL: (Service Check Timed Out) [05:16:13] PROBLEM - Free space - all mounts on deployment-eventlog05 is CRITICAL: (Service Check Timed Out) [05:16:28] PROBLEM - Free space - all mounts on deployment-cpjobqueue is CRITICAL: (Service Check Timed Out) [05:16:50] PROBLEM - Free space - all mounts on deployment-memc06 is CRITICAL: (Service Check Timed Out) [05:17:32] PROBLEM - Free space - all mounts on deployment-cumin02 is CRITICAL: (Service Check Timed Out) [05:19:51] PROBLEM - Free space - all mounts on integration-slave-jessie-1002 is CRITICAL: (Service Check Timed Out) [05:21:55] PROBLEM - Free space - all mounts on deployment-imagescaler01 is CRITICAL: (Service Check Timed Out) [05:22:14] PROBLEM - Free space - all mounts on deployment-kafka-main-1 is CRITICAL: (Service Check Timed Out) [05:23:00] PROBLEM - Free space - all mounts on deployment-webperf11 is CRITICAL: (Service Check Timed Out) [05:24:40] PROBLEM - Free space - all mounts on integration-agent-stretch-1001 is CRITICAL: (Service Check Timed Out) [05:24:40] PROBLEM - Free space - all mounts on integration-agent-stretch-1001 is CRITICAL: (Service Check Timed Out) [05:25:16] PROBLEM - Free space - all mounts on integration-agent-docker-1013 is CRITICAL: (Service Check Timed Out) [05:25:32] PROBLEM - Free space - all mounts on deployment-maps05 is CRITICAL: (Service Check Timed Out) [05:26:36] PROBLEM - Free space - all mounts on deployment-zookeeper02 is CRITICAL: (Service Check Timed Out) [05:26:51] PROBLEM - Free space - all mounts on integration-castor03 is CRITICAL: (Service Check Timed Out) [05:27:29] PROBLEM - Free space - all mounts on deployment-puppetmaster04 is CRITICAL: (Service Check Timed Out) [05:27:57] PROBLEM - Free space - all mounts on deployment-imagescaler03 is CRITICAL: (Service Check Timed Out) [05:28:15] RECOVERY - Free space - all mounts on integration-agent-docker-1014 is OK: OK: All targets OK [05:28:39] PROBLEM - Free space - all mounts on deployment-logstash03 is CRITICAL: (Service Check Timed Out) [05:30:33] PROBLEM - Free space - all mounts on deployment-memc05 is CRITICAL: (Service Check Timed Out) [05:31:09] PROBLEM - Free space - all mounts on deployment-cache-upload06 is CRITICAL: (Service Check Timed Out) [05:32:14] RECOVERY - Free space - all mounts on deployment-kafka-jumbo-1 is OK: OK: deployment-prep.deployment-kafka-jumbo-1.diskspace._mnt_kafka.byte_percentfree (No valid datapoints found) [05:32:53] RECOVERY - Free space - all mounts on deployment-db06 is OK: OK: All targets OK [05:34:10] RECOVERY - Free space - all mounts on integration-slave-test-jeena is OK: OK: All targets OK [05:34:28] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-09 is OK: HTTP OK: HTTP/1.1 200 OK - 92448 bytes in 1.047 second response time [05:34:28] RECOVERY - Free space - all mounts on deployment-echostore01 is OK: OK: All targets OK [05:34:29] RECOVERY - Free space - all mounts on integration-puppetmaster-02 is OK: OK: All targets OK [05:34:29] RECOVERY - Free space - all mounts on integration-agent-stretch-1001 is OK: OK: All targets OK [05:34:31] RECOVERY - Free space - all mounts on deployment-wikifeeds01 is OK: OK: All targets OK [05:34:36] RECOVERY - Free space - all mounts on integration-agent-docker-1002 is OK: OK: All targets OK [05:34:43] RECOVERY - Free space - all mounts on integration-agent-docker-1010 is OK: OK: All targets OK [05:34:43] RECOVERY - Free space - all mounts on integration-agent-puppet-docker-1001 is OK: OK: All targets OK [05:34:43] RECOVERY - Free space - all mounts on integration-slave-jessie-1002 is OK: OK: integration.integration-slave-jessie-1002.diskspace._mnt.byte_percentfree (No valid datapoints found) [05:35:06] RECOVERY - Free space - all mounts on integration-agent-docker-1013 is OK: OK: All targets OK [05:35:20] RECOVERY - Free space - all mounts on deployment-maps05 is OK: OK: All targets OK [05:35:23] RECOVERY - Free space - all mounts on integration-agent-docker-1009 is OK: OK: All targets OK [05:35:26] RECOVERY - Free space - all mounts on deployment-memc05 is OK: OK: All targets OK [05:35:41] RECOVERY - Free space - all mounts on deployment-puppetdb03 is OK: OK: All targets OK [05:35:44] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 92744 bytes in 1.052 second response time [05:35:52] RECOVERY - Free space - all mounts on deployment-restbase01 is OK: OK: deployment-prep.deployment-restbase01.diskspace._var_log.byte_percentfree (No valid datapoints found) [05:36:01] RECOVERY - Free space - all mounts on deployment-cache-upload06 is OK: OK: All targets OK [05:36:02] RECOVERY - Free space - all mounts on deployment-aqs02 is OK: OK: All targets OK [05:36:02] RECOVERY - Free space - all mounts on deployment-eventlog05 is OK: OK: deployment-prep.deployment-eventlog05.diskspace._var_lib_mysql.byte_percentfree (No valid datapoints found) [05:36:05] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 52125 bytes in 1.128 second response time [05:36:13] RECOVERY - Free space - all mounts on deployment-ores01 is OK: OK: All targets OK [05:36:29] RECOVERY - Free space - all mounts on deployment-zookeeper02 is OK: OK: All targets OK [05:36:36] RECOVERY - Free space - all mounts on deployment-poolcounter06 is OK: OK: All targets OK [05:36:36] RECOVERY - Free space - all mounts on deployment-mediawiki-09 is OK: OK: All targets OK [05:36:42] RECOVERY - Free space - all mounts on integration-castor03 is OK: OK: All targets OK [05:36:42] RECOVERY - Free space - all mounts on deployment-memc06 is OK: OK: All targets OK [05:36:45] RECOVERY - Free space - all mounts on integration-agent-docker-1003 is OK: OK: All targets OK [05:36:49] RECOVERY - Free space - all mounts on deployment-imagescaler01 is OK: OK: All targets OK [05:36:49] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 92452 bytes in 0.908 second response time [05:36:58] RECOVERY - Free space - all mounts on deployment-memc08 is OK: OK: All targets OK [05:37:04] RECOVERY - Free space - all mounts on deployment-chromium01 is OK: OK: All targets OK [05:37:04] RECOVERY - Free space - all mounts on deployment-kafka-main-1 is OK: OK: All targets OK [05:37:22] RECOVERY - Free space - all mounts on deployment-puppetmaster04 is OK: OK: All targets OK [05:37:24] RECOVERY - Free space - all mounts on integration-agent-docker-1001 is OK: OK: All targets OK [05:37:24] RECOVERY - Free space - all mounts on deployment-docker-mathoid01 is OK: OK: All targets OK [05:37:40] RECOVERY - Free space - all mounts on integration-agent-docker-1004 is OK: OK: All targets OK [05:37:49] RECOVERY - Free space - all mounts on deployment-imagescaler03 is OK: OK: All targets OK [05:37:49] RECOVERY - Free space - all mounts on integration-cumin-02 is OK: OK: All targets OK [05:37:53] RECOVERY - Free space - all mounts on deployment-webperf11 is OK: OK: All targets OK [05:37:59] RECOVERY - Free space - all mounts on deployment-mcs01 is OK: OK: All targets OK [05:37:59] RECOVERY - Free space - all mounts on deployment-ms-fe03 is OK: OK: All targets OK [05:38:15] RECOVERY - Free space - all mounts on deployment-cache-text06 is OK: OK: All targets OK [05:38:30] RECOVERY - Free space - all mounts on deployment-logstash03 is OK: OK: All targets OK [05:38:30] RECOVERY - Free space - all mounts on deployment-schema-2 is OK: OK: All targets OK [06:22:11] 10Phabricator, 10Commons: Review Global Herald H109 - Add #Commons project to #MediaWiki-File-management and #MediaWiki-extensions-GWToolset - https://phabricator.wikimedia.org/T250026 (10Peachey88) [08:13:04] (03Abandoned) 10Awight: [WIP] Bad "pipeline" abstraction [integration/quibble] - 10https://gerrit.wikimedia.org/r/588095 (owner: 10Awight) [09:11:56] (03PS2) 10Awight: Wrap main pipeline in a Sequence [integration/quibble] - 10https://gerrit.wikimedia.org/r/588093 (https://phabricator.wikimedia.org/T249775) [09:11:58] (03PS1) 10Awight: [WIP] Composite commands can be named [integration/quibble] - 10https://gerrit.wikimedia.org/r/588142 [09:12:00] (03PS1) 10Awight: [WIP] Sequence builder [integration/quibble] - 10https://gerrit.wikimedia.org/r/588143 [09:14:02] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Composite commands can be named [integration/quibble] - 10https://gerrit.wikimedia.org/r/588142 (owner: 10Awight) [09:14:24] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Sequence builder [integration/quibble] - 10https://gerrit.wikimedia.org/r/588143 (owner: 10Awight) [10:00:55] (03PS5) 10Majavah: Skip empty sections in train changelogs [tools/release] - 10https://gerrit.wikimedia.org/r/587971 (https://phabricator.wikimedia.org/T249553) [10:10:43] (03PS3) 10Awight: Provide GitClean as a command [integration/quibble] - 10https://gerrit.wikimedia.org/r/588082 [10:10:44] (03PS1) 10Awight: Clone only the target project at first [integration/quibble] - 10https://gerrit.wikimedia.org/r/588146 (https://phabricator.wikimedia.org/T211702) [10:14:42] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Quibble, 10Patch-For-Review: Quibble initialize step should only clone the target repository - https://phabricator.wikimedia.org/T211702 (10awight) I've proposed a patch which doesn't rely on a big pile of refactoring :-... [12:10:49] 10Phabricator, 10Commons: Review Global Herald H109 - Add #Commons project to #MediaWiki-File-management and #MediaWiki-extensions-GWToolset - https://phabricator.wikimedia.org/T250026 (10Aklapper) I found T86444 but no discussion when or how that Herald rule was created or who requested it. :-/ I agree with t... [12:23:28] (03PS15) 10Awight: Replace argparse with docopt [integration/quibble] - 10https://gerrit.wikimedia.org/r/546167 [12:47:09] Last night I created a deployment-restbase03 instance (stretch) to replace deployment-restbase0[12] (jessie). This means we are now at our max VCPU quota. [12:47:32] We'll gain a little bit of space when I delete the old deployment-cache boxes next weekend [12:47:47] But the jessie deadline is two weeks out and we have about 18 instances still running jessie [12:57:22] 10Beta-Cluster-Infrastructure: Migrate deployment-zookeeper02 from jessie to stretch - https://phabricator.wikimedia.org/T222461 (10Aklapper) @elukey: Could you please answer the last comment? Thanks! [12:59:05] 10Gerrit, 10Code-Review-Workgroup, 10Discovery, 10User-DannyS712: Cindy-the-browser-test-bot should use verified, not code review - https://phabricator.wikimedia.org/T249981 (10Reedy) >>! In T249981#6048594, @Aklapper wrote: > I found https://www.mediawiki.org/wiki/Wikimedia_Discovery/BrowserBot and https:... [13:22:21] 10Gerrit, 10Code-Review-Workgroup, 10Discovery, 10User-DannyS712: Cindy-the-browser-test-bot should use verified, not code review - https://phabricator.wikimedia.org/T249981 (10Reedy) Ok, so the docs are/were wrong, it's running on cirrus-integ.search.eqiad.wmflabs (currently) ` [14:17:13] okay... [13:24:41] 10Gerrit, 10Code-Review-Workgroup, 10Discovery, 10User-DannyS712: Cindy-the-browser-test-bot should use verified, not code review - https://phabricator.wikimedia.org/T249981 (10Reedy) I would suggest https://www.mediawiki.org/wiki/Wikimedia_Discovery/BrowserBot and https://wikitech.wikimedia.org/wiki/Cindy... [14:51:52] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:52:15] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:52:56] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:06:31] Reedy: https://phabricator.wikimedia.org/T250037 is Joaquinito01 [15:06:44] Yes, that’s the globally banned user [15:06:54] Link/proof? [15:07:01] I'll disable their account if so [15:07:53] Reedy: they like demanding stuff, love wanting to install Apache Traffic Server, enjoy messing with puppet. If it looks like a Duck, it’s a duck [15:08:10] Name starts similar [15:11:51] https://en.wikipedia.org/wiki/Special:CentralAuth?target=Joaquinito01 [15:11:51] >Reason: Abusing multiple accounts: Joaquinito02 [15:11:53] Well, that's enough for me [15:13:16] Reedy: he got WMF banned for harrasment and impersonation. [15:42:13] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 52110 bytes in 7.940 second response time [15:42:48] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 92446 bytes in 0.996 second response time [15:46:43] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 92752 bytes in 1.119 second response time [15:57:38] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Keyholder, 10Operations, 10Patch-For-Review: Keyholder phab repo duplicate work - https://phabricator.wikimedia.org/T203003 (10hashar) I forgot to check for open changes, thank you for the notification. I guess it is tim... [16:10:07] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Operations: Rebuild helm/helm-diff for buster-wikimedia - https://phabricator.wikimedia.org/T249812 (10hashar) [16:31:44] 10Continuous-Integration-Infrastructure, 10Projects-Cleanup, 10Repository-Admins, 10Wikidata, and 2 others: Archive the WikibaseQualityExternalValidation extension - https://phabricator.wikimedia.org/T204490 (10Physikerwelt) Is there a new project that superseeds the external validation efforts? I would be... [17:53:55] 10Release-Engineering-Team, 10phan, 10Security-Team, 10Documentation: Document that Phan is required for WMF deployed and MW bundled extensions - https://phabricator.wikimedia.org/T250038 (10Reedy) [18:40:04] (03PS1) 10Mainframe98: Add a sniff to detect and fix whitespace in the array indexer brackets [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588162 (https://phabricator.wikimedia.org/T249872) [18:44:23] (03CR) 10jerkins-bot: [V: 04-1] Add a sniff to detect and fix whitespace in the array indexer brackets [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588162 (https://phabricator.wikimedia.org/T249872) (owner: 10Mainframe98) [18:48:26] (03PS2) 10Mainframe98: Add a sniff to detect and fix whitespace in the array indexer brackets [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588162 (https://phabricator.wikimedia.org/T249872) [19:43:30] (03CR) 10DannyS712: Add a sniff to detect and fix whitespace in the array indexer brackets (032 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588162 (https://phabricator.wikimedia.org/T249872) (owner: 10Mainframe98) [19:48:00] 10MediaWiki-Codesniffer, 10User-DannyS712: Update DeprecatedGlobalVariablesSniff re wgVersion deprecation - https://phabricator.wikimedia.org/T250040 (10DannyS712) [19:48:09] (03PS1) 10DannyS712: Update DeprecatedGlobalVariablesSniff for deprecation of $wgVersion [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588167 (https://phabricator.wikimedia.org/T250040) [19:48:36] (03PS2) 10DannyS712: Update DeprecatedGlobalVariablesSniff for deprecation of $wgVersion [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/588167 (https://phabricator.wikimedia.org/T250040) [19:57:53] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:58:13] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:58:56] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:17:43] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 92752 bytes in 1.083 second response time [20:18:06] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 52125 bytes in 1.048 second response time [20:18:47] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 92448 bytes in 1.553 second response time [21:08:55] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:09:13] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:09:58] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:12:25] (03PS2) 10Awight: Rewrite main pipeline as a generator [integration/quibble] - 10https://gerrit.wikimedia.org/r/588092 [21:12:48] (03CR) 10Awight: "PS 2: manual cherry-pick to master" [integration/quibble] - 10https://gerrit.wikimedia.org/r/588092 (owner: 10Awight) [21:23:42] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 92751 bytes in 0.999 second response time [21:24:05] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 52124 bytes in 0.888 second response time [21:24:48] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 92446 bytes in 0.896 second response time [21:37:09] (03PS1) 10Awight: PEP 8: underscore prefix for internal functions [integration/quibble] - 10https://gerrit.wikimedia.org/r/588177 [21:37:51] (03CR) 10jerkins-bot: [V: 04-1] PEP 8: underscore prefix for internal functions [integration/quibble] - 10https://gerrit.wikimedia.org/r/588177 (owner: 10Awight) [21:55:52] 10Continuous-Integration-Infrastructure, 10Projects-Cleanup, 10Repository-Admins, 10Wikidata, and 2 others: Archive the WikibaseQualityExternalValidation extension - https://phabricator.wikimedia.org/T204490 (10Bugreporter) This is planned in https://www.wikidata.org/wiki/Wikidata:Development_plan, but pro... [21:57:03] (03PS12) 10Awight: Parallelism as a command object; parallelize short PHPUnits [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) [21:57:37] (03CR) 10Awight: "PS 12: manual cherry-pick to master" [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) (owner: 10Awight) [21:57:56] (03CR) 10jerkins-bot: [V: 04-1] Parallelism as a command object; parallelize short PHPUnits [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) (owner: 10Awight) [22:01:53] (03PS2) 10Awight: PEP 8: underscore prefix for internal functions [integration/quibble] - 10https://gerrit.wikimedia.org/r/588177 [22:08:32] (03CR) 10DannyS712: Skip empty sections in train changelogs (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/587971 (https://phabricator.wikimedia.org/T249553) (owner: 10Majavah) [22:09:54] (03PS13) 10Awight: Parallelism as a command object; parallelize short PHPUnits [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) [22:11:51] (03CR) 10jerkins-bot: [V: 04-1] Parallelism as a command object; parallelize short PHPUnits [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) (owner: 10Awight) [22:19:56] (03PS14) 10Awight: Parallelism as a command object; parallelize short PHPUnits [integration/quibble] - 10https://gerrit.wikimedia.org/r/587885 (https://phabricator.wikimedia.org/T235449) [22:51:45] (03Abandoned) 10Awight: Clean up logging [integration/quibble] - 10https://gerrit.wikimedia.org/r/546126 (owner: 10Awight) [23:56:05] 10Project-Admins, 10User-DannyS712: Cleaning up #mediawiki-general - https://phabricator.wikimedia.org/T249999 (10DannyS712) @Krinkle #mediawiki-special-pages already exists - should it not? What about splitting blocking (#mediawiki-user-blocking) out of #mediawiki-user-management - a clear component that cov...