[00:17:40] Yippee, build fixed! [00:17:40] Project selenium-Flow » chrome,beta,Linux,BrowserTests build #515: 09FIXED in 1 min 39 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/515/ [03:17:17] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [03:38:16] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:49:19] 10Beta-Cluster-Infrastructure: Beta protection error - https://phabricator.wikimedia.org/T175981#3611683 (10Sau226) [03:50:20] 10Beta-Cluster-Infrastructure: Beta protection error - https://phabricator.wikimedia.org/T175981#3609899 (10Sau226) @Aklapper @Samtar I have updated the task to make it a better bug report as you have requested. Feel free to try to reproduce it. [03:53:27] 10Beta-Cluster-Infrastructure: Beta protection error - https://phabricator.wikimedia.org/T175981#3611688 (10Sau226) [04:02:15] 10Beta-Cluster-Infrastructure: Beta cluster rights clarification - https://phabricator.wikimedia.org/T175917#3611689 (10Sau226) 05Open>03Resolved a:03Sau226 No I was just wondering if those rights could be granted to non WMF accounts. Since this task says you can't grant these rights I now consider this ta... [04:02:28] 10Beta-Cluster-Infrastructure: Beta cluster rights clarification - https://phabricator.wikimedia.org/T175917#3611692 (10Sau226) a:05Sau226>03None [04:09:14] 10Beta-Cluster-Infrastructure: Unable to change beta protection in new tab until reload - https://phabricator.wikimedia.org/T175981#3611694 (10Sau226) [04:17:11] Yippee, build fixed! [04:17:12] Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #518: 09FIXED in 21 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/518/ [04:40:43] doing some live debugging on the beta cluster [04:41:19] we don't have X-Wikimedia-Debug there, do we? [04:43:18] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [04:53:51] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:28:59] Project selenium-Wikibase » chrome,test,Linux,BrowserTests build #485: 04STILL FAILING in 1 hr 48 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=BrowserTests/485/ [06:34:38] 10Deployment-Systems, 10Release-Engineering-Team (Kanban), 10Scap (Scap3-Adoption-Phase1), 10scap2, and 3 others: Deploy jobrunner with scap3 (Trebuchet jobrunner/jobrunner) - https://phabricator.wikimedia.org/T129148#3611702 (10mmodell) itshappening [06:48:26] Project selenium-Wikibase » chrome,beta,Linux,BrowserTests build #485: 04STILL FAILING in 2 hr 8 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/485/ [08:32:00] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:40:49] PROBLEM - Puppet errors on deployment-aqs03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:41:03] PROBLEM - Puppet errors on deployment-aqs02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:47:00] PROBLEM - Free space - all mounts on integration-slave-jessie-1001 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1001.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1001.diskspace._srv.byte_percentfree (<100.00%) [08:53:49] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [10:39:18] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [11:14:18] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [12:02:58] 10MediaWiki-Codesniffer: Add sniff to check for wrong @cover in class comments - https://phabricator.wikimedia.org/T176031#3611736 (10Umherirrender) [12:03:27] (03CR) 10Umherirrender: [C: 031] "@cover can also occur on class comment, but there is no sniff at the moment to check that - have created T176031" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/378268 (owner: 10Jforrester) [12:22:40] Project selenium-GettingStarted » firefox,beta,Linux,BrowserTests build #527: 04FAILURE in 39 sec: https://integration.wikimedia.org/ci/job/selenium-GettingStarted/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/527/ [12:57:54] Anyone around? [12:58:01] I know what's causing those -aqs errors [12:58:06] greg-g? [12:58:32] Reedy? legoktm? [13:04:59] Project selenium-Math » firefox,beta,Linux,BrowserTests build #516: 04FAILURE in 58 sec: https://integration.wikimedia.org/ci/job/selenium-Math/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/516/ [13:33:27] PROBLEM - Puppet errors on castor02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [14:08:25] RECOVERY - Puppet errors on castor02 is OK: OK: Less than 1.00% above the threshold [0.0] [14:33:35] Project selenium-WikiLove » firefox,beta,Linux,BrowserTests build #518: 04FAILURE in 1 min 34 sec: https://integration.wikimedia.org/ci/job/selenium-WikiLove/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/518/ [14:40:18] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [15:15:19] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [15:24:45] Yippee, build fixed! [15:24:45] Project selenium-MobileFrontend » chrome,beta,Linux,BrowserTests build #559: 09FIXED in 2 min 44 sec: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/559/ [15:51:42] (03PS7) 10Umherirrender: Add missing unit test, npm jobs and make tests voting [integration/config] - 10https://gerrit.wikimedia.org/r/376761 [15:52:17] (03CR) 10Umherirrender: "Patch Set 7: Add npm for ExtJSBase" [integration/config] - 10https://gerrit.wikimedia.org/r/376761 (owner: 10Umherirrender) [16:03:46] Project selenium-CentralNotice » firefox,beta,Windows 7,BrowserTests build #520: 04FAILURE in 2 min 45 sec: https://integration.wikimedia.org/ci/job/selenium-CentralNotice/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Windows%207,label=BrowserTests/520/ [16:06:17] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:46:16] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [17:37:28] 10Beta-Cluster-Infrastructure: Beta cluster rights clarification - https://phabricator.wikimedia.org/T175917#3612207 (10Aklapper) 05Resolved>03Invalid (Nothing to fixed / changed, hence [[ https://www.mediawiki.org/wiki/Bug_management/Bug_report_life_cycle | setting task status accordingly ]]) [17:40:00] 10Beta-Cluster-Infrastructure: Beta cluster rights clarification - https://phabricator.wikimedia.org/T175917#3607866 (10Krenair) The rights those groups grant on beta are fine to hand out (within reason) there, but I'd at least change the name of the group before granting it to avoid impersonation concerns [20:14:19] 10MediaWiki-Codesniffer: Add sniff to check for wrong @cover in class comments - https://phabricator.wikimedia.org/T176031#3612329 (10Umherirrender) [20:18:05] PROBLEM - Free space - all mounts on deployment-kafka01 is CRITICAL: CRITICAL: deployment-prep.deployment-kafka01.diskspace.root.byte_percentfree (<100.00%) [21:37:17] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [22:12:17] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [23:38:18] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]