[00:00:19] or just none or clear pixel as long as it doesn't look broken [00:59:05] 10Release-Engineering-Team, 10Page-Previews, 10Reading-Web-Backlog, 10Reading-Web-Kanban-Board: Create bot that automatically rebases and rebuilds patches to master - https://phabricator.wikimedia.org/T167181#3356437 (10Jdlrobson) Boom. https://www.mediawiki.org/wiki/Reading/Web/Pixie.py_rebase_tool It's... [01:34:33] PROBLEM - Puppet staleness on deployment-aqs01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [01:55:29] Project beta-scap-eqiad build #160073: 04FAILURE in 1 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160073/ [02:06:08] Yippee, build fixed! [02:06:09] Project beta-scap-eqiad build #160074: 09FIXED in 2 min 21 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160074/ [04:16:27] Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #425: 04FAILURE in 20 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/425/ [04:52:43] Project selenium-Wikibase » chrome,test,Linux,BrowserTests build #394: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=BrowserTests/394/ [06:50:41] Yippee, build fixed! [06:50:42] Project selenium-Wikibase » chrome,beta,Linux,BrowserTests build #394: 09FIXED in 2 hr 10 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/394/ [07:30:22] 10Beta-Cluster-Infrastructure, 10Wikidata: Wikidata on beta is getting too many edits - https://phabricator.wikimedia.org/T168101#3356601 (10hashar) [07:30:24] 10Beta-Cluster-Infrastructure, 10Browser-Tests-Infrastructure, 10Wikidata: 10.68.21.186 is creating items on beta Wikidata - https://phabricator.wikimedia.org/T168102#3356598 (10hashar) 05Open>03declined `saucelabs-01.integration.eqiad.wmflabs` a host that runs browser tests. [07:33:33] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:37:33] 10Beta-Cluster-Infrastructure, 10Browser-Tests-Infrastructure, 10Wikidata: 10.68.21.186 is creating items on beta Wikidata - https://phabricator.wikimedia.org/T168102#3356606 (10Reedy) Are they supposed to be being made anonymously though? [09:45:28] Project beta-scap-eqiad build #160120: 04FAILURE in 1 min 45 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160120/ [09:51:01] 09:45:27 09:45:27 Check 'Logstash Error rate for deployment-mediawiki04.deployment-prep.eqiad.wmflabs' failed: ERROR: 24% OVER_THRESHOLD (Avg. Error rate: Before: 0.53, After: 7.00, Threshold: 5.28) [09:55:58] Yippee, build fixed! [09:55:59] Project beta-scap-eqiad build #160121: 09FIXED in 2 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160121/ [10:22:26] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [10:56:19] 10Continuous-Integration-Infrastructure, 10Electron-PDFs, 10MediaWiki-Internationalization, 10Reading-Web-Backlog: Phan throws false positive when checking SpecialElectronPDF file - https://phabricator.wikimedia.org/T167995#3356732 (10hashar) @EBernhardson and @Addshore might be able to help since they hav... [10:59:31] PROBLEM - Puppet errors on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [10:59:51] 10Beta-Cluster-Infrastructure, 10Browser-Tests-Infrastructure, 10Wikidata: 10.68.21.186 is creating items on beta Wikidata - https://phabricator.wikimedia.org/T168102#3356734 (10hashar) That would depends on the scenario. I can imagine some scenario attempting to create a property/item as an anonymous user? [12:02:13] (03CR) 10Hashar: [C: 031] Fix indentation [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/359507 (owner: 10Umherirrender) [12:04:00] (03CR) 10Paladox: [C: 031] Fix indentation [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/359507 (owner: 10Umherirrender) [13:04:47] PROBLEM - Puppet errors on deployment-conf03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:09:29] (03CR) 10Hashar: [C: 031] "Looks good! Thank you to have addressed all the nitpicks. Feel free to deploy whenever you feel." [integration/config] - 10https://gerrit.wikimedia.org/r/357741 (https://phabricator.wikimedia.org/T166888) (owner: 10Thcipriani) [15:15:36] Project beta-scap-eqiad build #160153: 04FAILURE in 1 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160153/ [15:25:33] Project beta-scap-eqiad build #160154: 04STILL FAILING in 1 min 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160154/ [15:35:59] Yippee, build fixed! [15:36:00] Project beta-scap-eqiad build #160155: 09FIXED in 2 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160155/ [16:01:42] Project selenium-CentralNotice » firefox,beta,Linux,BrowserTests build #429: 04FAILURE in 41 sec: https://integration.wikimedia.org/ci/job/selenium-CentralNotice/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/429/ [16:45:18] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [17:05:46] PROBLEM - Puppet errors on swift is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:07:17] PROBLEM - Puppet errors on swift-storage-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:25:20] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0] [19:23:16] 10Continuous-Integration-Infrastructure, 10Electron-PDFs, 10MediaWiki-Internationalization, 10Reading-Web-Backlog, 10Patch-For-Review: Phan throws false positive when checking SpecialElectronPDF file - https://phabricator.wikimedia.org/T167995#3357670 (10EBernhardson) The phan config was just not loading... [19:31:55] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [22:26:05] 10Release-Engineering-Team (Kanban), 10Phabricator: EXCEPTION: (AphrontParameterQueryException) Array for %Ls conversion is empty. Query: user.phid - https://phabricator.wikimedia.org/T166984#3358011 (10Paladox) 05Open>03Resolved a:03mmodell [22:55:52] Project beta-scap-eqiad build #160199: 04FAILURE in 2 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160199/ [23:06:11] Project beta-scap-eqiad build #160200: 04STILL FAILING in 2 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160200/ [23:16:12] Yippee, build fixed! [23:16:13] Project beta-scap-eqiad build #160201: 09FIXED in 2 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/160201/ [23:16:20] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [23:56:17] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0]