[00:21:49] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Flow, 10MediaWiki-Database, and 3 others: Beta update.php fails with Can't DROP 'flow_ext_ref_idx_v2'; - https://phabricator.wikimedia.org/T166266#3346720 (10demon) 05Open>03Resolved a:03demon
[01:58:18] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Phabricator (Upstream), 10Upstream: Clicking "Add Existing Panel" and entering `Wxxx` in the field shows no results - https://phabricator.wikimedia.org/T166236#3346832 (10mmodell)
[01:59:32] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Phabricator (Upstream), 10Upstream: Clicking "Add Existing Panel" and entering `Wxxx` in the field shows no results - https://phabricator.wikimedia.org/T166236#3289400 (10mmodell) I submitted the fix (e1850b3c4e4) upstream and I intend to deploy it to production tomorrow.
[03:58:41] <wmf-insecte>	 Yippee, build fixed!
[03:58:42] <wmf-insecte>	 Project selenium-MultimediaViewer » firefox,mediawiki,Linux,BrowserTests build #422: 09FIXED in 2 min 40 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=mediawiki,PLATFORM=Linux,label=BrowserTests/422/
[04:09:38] <wmf-insecte>	 Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #422: 04FAILURE in 13 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/422/
[05:34:21] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Wikidata, 10Story: [Story] Use composer-merge-plugin to include Wikidata components in mediawiki-vendor - https://phabricator.wikimedia.org/T95663#3346975 (10demon) a:05demon>03None
[07:08:37] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10Zuul: Migrate zuul-server behind systemd service - https://phabricator.wikimedia.org/T167845#3347026 (10Marostegui) p:05Triage>03Normal
[07:47:05] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Remove check-voter pipeline from Zuul - https://phabricator.wikimedia.org/T161205#3347055 (10fgiunchedi)
[08:07:48] <wikibugs>	 (03CR) 10Hashar: "Since 'composer test' invokes parallel lint and others, I have used ./vendor/bin/phpunit directly and found the test breaking because my t" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358546 (owner: 10Hashar)
[08:10:26] <wikibugs>	 (03PS2) 10Hashar: Fix WhiteSpace/SpaceAfterClosureSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358567 (https://phabricator.wikimedia.org/T149623)
[08:10:39] <wikibugs>	 (03CR) 10Hashar: [C: 031] "Rebased manually" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358567 (https://phabricator.wikimedia.org/T149623) (owner: 10Hashar)
[08:12:22] <wikibugs>	 (03CR) 10Hashar: "Ah multiprocessing is lovely, that will surely speed up the run :-]  I guess this can wait for a 3.1 or something. I have a similar patch " [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358554 (https://phabricator.wikimedia.org/T158974) (owner: 10Hashar)
[09:17:35] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3347216 (10zeljkofilipin) >>! In T164721#3345389, @zeljkofilipin wrote: > The job ([[ https://integration.wik...
[09:19:44] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3347218 (10zeljkofilipin) Ah, the docs for [[ https://www.mediawiki.org/wiki/Extension:RelatedArticles | Exte...
[09:21:52] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Operations, 10HHVM, 10Patch-For-Review: Switch CI tests back to HHVM 3.18 - https://phabricator.wikimedia.org/T167493#3347232 (10MoritzMuehlenhoff) Can be closed, right?
[09:49:41] <wikibugs>	 10Release-Engineering-Team, 10Page-Previews, 10Reading-Web-Backlog, 10Epic: [EPIC] Generate compiled assets from continuous integration - https://phabricator.wikimedia.org/T158980#3347340 (10Jhernandez) For the record, I sent an email to `wikitech-l` asking how does a build process look like for extensions...
[10:25:50] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Operations, 10HHVM, 10Patch-For-Review: Switch CI tests back to HHVM 3.18 - https://phabricator.wikimedia.org/T167493#3347400 (10hashar) 05Open>03Resolved Indeed nothing reported.  Thank you very much !
[10:27:37] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3347403 (10hashar) Try the mobile URLs:  https://en.m.wikipedia.beta.wmflabs.org/wiki/1 https://en.m.wikipedi...
[10:38:09] <Amir1>	 Just a note, I'm actively hacking mediawiki05 node on apache rules to get https://gerrit.wikimedia.org/r/#/c/357985/4 working
[10:46:30] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3347427 (10zeljkofilipin) @hashar thanks, I have already figured that out :)
[11:22:38] <wikibugs>	 10Beta-Cluster-Infrastructure, 10media-storage, 10Patch-For-Review: deployment-ms-be03.deployment-prep and deployment-ms-be04.deployment-prep have high load / system CPU - https://phabricator.wikimedia.org/T160990#3347467 (10hashar) Gave it a quick try. I have added in /etc/nscd.conf: ``` debug-level 9 logfi...
[11:31:24] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Watching / External), 10MW-1.29-release-notes, 10Patch-For-Review, and 2 others: Update Ruby tests to Selenium 3 - https://phabricator.wikimedia.org/T158074#3347476 (10zeljkofilipin)
[11:33:15] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10JavaScript, 10Patch-For-Review, 10User-zeljkofilipin: Refactor webdriverio tests for mediawiki core so users and pages are created via the api - https://phabricator.wikimedia.org/T167502#3335078 (10zeljkofilipin)
[11:33:46] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10JavaScript, 10Patch-For-Review, 10User-zeljkofilipin: Refactor webdriverio tests for mediawiki core so users and pages are created via the api - https://phabricator.wikimedia.org/T167502#3335078 (10zeljkofilipin) a:03zeljkofilipin
[11:39:55] <wikibugs>	 (03PS1) 10Umherirrender: [webplatform] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/358932
[11:59:51] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3347561 (10zeljkofilipin) Looks like @Jdlrobson's patch is running fine in CI:  - patch: https://gerrit.wikim...
[12:06:31] <wikibugs>	 (03CR) 10Hashar: [C: 032] [webplatform] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/358932 (owner: 10Umherirrender)
[12:07:34] <wikibugs>	 (03Merged) 10jenkins-bot: [webplatform] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/358932 (owner: 10Umherirrender)
[12:23:56] <wikibugs>	 (03PS1) 10Hashar: Unconfigure mediawiki/skins/Donate [integration/config] - 10https://gerrit.wikimedia.org/r/358941 (https://phabricator.wikimedia.org/T124519)
[12:24:57] <hashar>	 !log gerrit: marked mediawiki/skins/Donate has read-only ( https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/skins/Donate ) - T124519
[12:25:01] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[12:25:01] <stashbot>	 T124519: Fully retire MediaWiki "Donate" skin - https://phabricator.wikimedia.org/T124519
[12:30:11] <wikibugs>	 (03CR) 10Hashar: [C: 032] Unconfigure mediawiki/skins/Donate [integration/config] - 10https://gerrit.wikimedia.org/r/358941 (https://phabricator.wikimedia.org/T124519) (owner: 10Hashar)
[12:31:51] <wikibugs>	 (03Merged) 10jenkins-bot: Unconfigure mediawiki/skins/Donate [integration/config] - 10https://gerrit.wikimedia.org/r/358941 (https://phabricator.wikimedia.org/T124519) (owner: 10Hashar)
[12:33:51] <wikibugs>	 (03PS1) 10Hashar: Archive tox log in integration-zuul-layoutdiff [integration/config] - 10https://gerrit.wikimedia.org/r/358946
[12:36:13] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team (Kanban), 10Fundraising-Backlog: Fully retire MediaWiki "Donate" skin - https://phabricator.wikimedia.org/T124519#3347715 (10hashar) I have: * marked the repository read-only in Gerrit * unconfigured it from CI * removed it from the meta...
[12:36:42] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team (Kanban), 10Fundraising-Backlog: Fully retire MediaWiki "Donate" skin - https://phabricator.wikimedia.org/T124519#1958656 (10hashar) a:03hashar
[12:37:04] <wikibugs>	 (03CR) 10Paladox: [C: 031] Archive tox log in integration-zuul-layoutdiff [integration/config] - 10https://gerrit.wikimedia.org/r/358946 (owner: 10Hashar)
[12:39:44] <wikibugs>	 (03CR) 10Hashar: [C: 032] Archive tox log in integration-zuul-layoutdiff [integration/config] - 10https://gerrit.wikimedia.org/r/358946 (owner: 10Hashar)
[12:41:09] <wikibugs>	 (03Merged) 10jenkins-bot: Archive tox log in integration-zuul-layoutdiff [integration/config] - 10https://gerrit.wikimedia.org/r/358946 (owner: 10Hashar)
[12:43:59] <paladox>	 Woo hoo, i found a fix https://gerrit-review.googlesource.com/#/c/110055/ for T161206
[12:44:01] <stashbot>	 T161206: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206
[12:45:48] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3347733 (10Paladox) I created this fix https://gerrit-review.googlesource.com/#/c/110055/ that should fix it. I am unsure if there planni...
[12:47:16] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3124611 (10Paladox) a:03Paladox
[12:52:59] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3347749 (10hashar)
[12:53:21] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3347749 (10hashar) p:05Triage>03Normal
[12:57:23] <wikibugs>	 10Beta-Cluster-Infrastructure, 10media-storage, 10Patch-For-Review: deployment-ms-be03.deployment-prep and deployment-ms-be04.deployment-prep have high load / system CPU - https://phabricator.wikimedia.org/T160990#3347769 (10fgiunchedi) There's potentially other VMs across labs/cloud doing the same expensive...
[13:05:10] <wikibugs>	 (03CR) 10Etonkovidova: [C: 031] WIP Cleanup Ruby jobs [integration/config] - 10https://gerrit.wikimedia.org/r/356413 (https://phabricator.wikimedia.org/T164479) (owner: 10Zfilipin)
[13:06:35] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3347805 (10Paladox) @hashar and @volans ED25519 note that the key may be buggy in gerrit 2.14.1 see this bug, https://bugs.chromium.org/p/gerrit/issues/detail?id=6504 but de...
[13:13:57] <wikibugs>	 10Release-Engineering-Team, 10Operations, 10HHVM: mwlog1001.eqiad.wmnet ‘/srv/mw-log/hhvm.log’ for reading: No such file or directory - https://phabricator.wikimedia.org/T167878#3347823 (10hashar)
[13:16:51] <wikibugs>	 10Release-Engineering-Team, 10Operations, 10HHVM: mwlog1001.eqiad.wmnet ‘/srv/mw-log/hhvm.log’ for reading: No such file or directory - https://phabricator.wikimedia.org/T167878#3347841 (10hashar) Most probably 1fd05f19dce1591f7c63c9cfd579379c7573caaa in puppet by @elukey  ``` commit 1fd05f19dce1591f7c63c9cf...
[13:18:50] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Phabricator (Upstream), 10Upstream: Search does not warn when searching strings too short to index, and fails to find them - https://phabricator.wikimedia.org/T125674#3347844 (10mmodell) 05Open>03Resolved a:03mmodell There is no longer a minimum length.  You can s...
[13:30:20] <wikibugs>	 10Scap: When "scap pull" does a (slow) CDB rebuild, it should tell me that that's what it's doing - https://phabricator.wikimedia.org/T162207#3155641 (10hashar) Still happening eg today:  {P5577}
[13:31:01] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3347897 (10Volans) @Paladox FYI I'm still getting `Invalid SSH Key` when trying to add my key
[13:31:22] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3347899 (10Aklapper) p:05Triage>03Low
[13:32:23] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3347901 (10Paladox) @Volans yep, that will go when we upgrade. Tested on https://gerrit.git.wmflabs.org/r/ when adding ED25519 key.
[13:33:37] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3124611 (10Paladox) Making this a subtask to T156120 since gerrit 2.13 will have no more releases and they did not back port it to the 2....
[13:33:52] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3347907 (10Paladox)
[13:33:54] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[13:37:19] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap: When "scap pull" does a (slow) CDB rebuild, it should tell me that that's what it's doing - https://phabricator.wikimedia.org/T162207#3347923 (10hashar) 05Open>03Resolved a:03hashar Logs are back :-}
[13:37:39] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap: When "scap pull" does a (slow) CDB rebuild, it should tell me that that's what it's doing - https://phabricator.wikimedia.org/T162207#3347927 (10hashar) 05Resolved>03Open a:05hashar>03None Wrong task
[13:38:13] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Operations, 10HHVM, 10Patch-For-Review: mwlog1001.eqiad.wmnet ‘/srv/mw-log/hhvm.log’ for reading: No such file or directory - https://phabricator.wikimedia.org/T167878#3347934 (10hashar) 05Open>03Resolved a:03hashar Fixed, the logs are back on mwlog1001 :}  Than...
[13:55:49] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T167533#3335934 (10Krinkle) Looks like {T167885} was caused by wmf.4 - too bad we didn't see this earlier. Dang alerts didn't go off until today.
[14:33:46] <wmf-insecte>	 Project selenium-WikiLove » firefox,beta,Linux,BrowserTests build #423: 04FAILURE in 1 min 45 sec: https://integration.wikimedia.org/ci/job/selenium-WikiLove/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/423/
[14:43:26] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3348237 (10Pastakhov) Skin is still in development, and it will be released.
[14:54:02] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3348306 (10hashar) Do you happen to have some placeholder code to send?  This way we can setup CI on it properly and when your skin is ready CI would alr...
[14:55:31] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T167536#3336016 (10mmodell)
[14:56:12] <wikibugs>	 (03CR) 10Hashar: [C: 04-1] "Still need a spec to cover the feature :}" [selenium] - 10https://gerrit.wikimedia.org/r/358578 (https://phabricator.wikimedia.org/T167507) (owner: 10Zfilipin)
[15:03:49] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3348375 (10Pastakhov) I have the old path for testing https://gerrit.wikimedia.org/r/#/c/323361/1 it may be merged
[15:04:12] <chasemp>	 twentyafterfour: tx for https://gerrit.wikimedia.org/r/#/c/177181/ I meant to circle back and forgot
[15:04:51] <wikibugs>	 10Release-Engineering-Team, 10Page-Previews, 10Reading-Web-Backlog, 10Reading-Web-Kanban-Board: Create bot that automatically rebases and rebuilds patches to master - https://phabricator.wikimedia.org/T167181#3319993 (10bmansurov) @hashar @zeljkofilipin @greg do you see any problems with the proposed solut...
[15:04:57] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Backlog), 10DBA, 10Operations, 10Patch-For-Review: Better mysql command prompt info for Beta - https://phabricator.wikimedia.org/T157714#3348386 (10hashar) 05Open>03Resolved Solved via https://gerrit.wikimedia.org/r/#/c/336964/ / 35098e1e363e...
[15:13:29] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-PhpTags: Archive/delete mediawiki/skins/Empty - https://phabricator.wikimedia.org/T167874#3348445 (10hashar) 05Open>03Resolved a:03hashar Ah great thank you -:}   So I am going to keep it configured...
[15:19:44] <wikibugs>	 (03PS1) 10Hashar: skins/Empty depends on PhpTags [integration/config] - 10https://gerrit.wikimedia.org/r/358978
[15:21:47] <wikibugs>	 (03CR) 10Hashar: [C: 032] skins/Empty depends on PhpTags [integration/config] - 10https://gerrit.wikimedia.org/r/358978 (owner: 10Hashar)
[15:21:58] <wikibugs>	 (03CR) 10Hashar: [C: 032] "Test code is https://gerrit.wikimedia.org/r/#/c/323361/" [integration/config] - 10https://gerrit.wikimedia.org/r/358978 (owner: 10Hashar)
[15:23:13] <wikibugs>	 (03Merged) 10jenkins-bot: skins/Empty depends on PhpTags [integration/config] - 10https://gerrit.wikimedia.org/r/358978 (owner: 10Hashar)
[15:46:42] <wmf-insecte>	 Yippee, build fixed!
[15:46:42] <wmf-insecte>	 Project selenium-MobileFrontend » chrome,beta,Linux,BrowserTests build #454: 09FIXED in 24 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/454/
[16:10:06] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3348674 (10Aklapper) No, this is not a subtask - it is exactly the other way round (subtasks need to get resolved before parent tasks can...
[16:10:21] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3348676 (10Aklapper)
[16:10:27] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3348675 (10Aklapper)
[16:10:42] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Aklapper)
[16:10:44] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3124611 (10Aklapper)
[16:14:31] <wikibugs>	 10Gerrit, 10Analytics-Tech-community-metrics, 10Upstream: Gerrit patchset 99101 cannot be accessed: "500 Internal server error" - https://phabricator.wikimedia.org/T161206#3348685 (10Paladox) Oh i see. It was confusing which way. But i see now.
[16:15:19] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3348688 (10Paladox)
[16:15:22] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Zuul: Update zuul to upstream master - https://phabricator.wikimedia.org/T158243#3348687 (10Paladox)
[16:15:32] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:15:34] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Zuul: Update zuul to upstream master - https://phabricator.wikimedia.org/T158243#3030929 (10Paladox)
[16:15:49] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:15:51] <wikibugs>	 10Gerrit, 10Upstream: In changeset view, clicking the "Owner" name does not display list of their changesets but of unrelated other user - https://phabricator.wikimedia.org/T146361#3348691 (10Paladox)
[16:15:53] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:15:55] <wikibugs>	 10Gerrit, 10Upstream: In changeset view, clicking the "Owner" name does not display list of their changesets but of unrelated other user - https://phabricator.wikimedia.org/T146361#2658656 (10Paladox)
[16:16:11] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:16:13] <wikibugs>	 10Gerrit, 10Regression, 10Upstream: Gerrit doesn't display committed images - https://phabricator.wikimedia.org/T153085#3348695 (10Paladox)
[16:16:19] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:16:21] <wikibugs>	 10Gerrit, 10Regression, 10Upstream: Gerrit doesn't display committed images - https://phabricator.wikimedia.org/T153085#2869243 (10Paladox)
[16:16:32] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:16:35] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Operations: Make sure replying to emails in gerrit 2.14 works - https://phabricator.wikimedia.org/T158915#3348702 (10Paladox)
[16:16:40] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:16:42] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Operations: Make sure replying to emails in gerrit 2.14 works - https://phabricator.wikimedia.org/T158915#3051483 (10Paladox)
[16:17:06] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964656 (10Paladox)
[16:17:08] <wikibugs>	 10Gerrit, 10Patch-For-Review: Gerrit: Covert Velocity templates to Closure Templates - https://phabricator.wikimedia.org/T158008#3348706 (10Paladox)
[16:17:11] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964669 (10Paladox)
[16:17:15] <wikibugs>	 10Gerrit, 10Patch-For-Review: Gerrit: Covert Velocity templates to Closure Templates - https://phabricator.wikimedia.org/T158008#3023302 (10Paladox)
[16:17:20] <paladox>	 sorry for spam
[16:17:36] <paladox>	 seems phab double notifys when you move from parent to subtask.
[16:17:58] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10Upstream: Convert its-phabricator upstream repo which wmf maintains to bazel - https://phabricator.wikimedia.org/T156024#3348711 (10Paladox)
[16:18:00] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#3348712 (10Paladox)
[16:18:08] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10Upstream: Convert its-phabricator upstream repo which wmf maintains to bazel - https://phabricator.wikimedia.org/T156024#2961872 (10Paladox)
[16:18:08] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964669 (10Paladox)
[16:18:31] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964669 (10Paladox)
[16:18:33] <wikibugs>	 10Gerrit: Gerrit needs a very wide screen or has big horizontal scroll - https://phabricator.wikimedia.org/T38471#3348715 (10Paladox)
[16:18:37] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.1 - https://phabricator.wikimedia.org/T156120#2964669 (10Paladox)
[16:18:39] <wikibugs>	 10Gerrit: Gerrit needs a very wide screen or has big horizontal scroll - https://phabricator.wikimedia.org/T38471#404158 (10Paladox)
[17:20:01] <James_F>	 Ooh, fancy "RELEASE" label on T167893
[17:20:02] <stashbot>	 T167893: MW-1.30.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T167893
[17:20:30] <James_F>	 Though now I can't edit it, which is a bit sad, given it's probably meant to be wmf.*8*.
[17:24:37] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Zuul: Zuul refused to start from contint1001 - https://phabricator.wikimedia.org/T167833#3346309 (10thcipriani) >>! In T167833#3346617, @hashar wrote: > I tried with either: > ``` > systemctl start zuul > /etc/init.d/zuul start >...
[17:34:59] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Jenkins, 10Upstream: Launching Jenkins slave agent fails with "java.io.IOException: Unexpected termination of the channel" - https://phabricator.wikimedia.org/T91697#3348898 (10Krinkle) 05declined>03Open >>! In T91697#13503...
[17:47:55] <wikibugs>	 10Release-Engineering-Team (Watching / External), 10Operations, 10Goal, 10Kubernetes, and 3 others: Prepare and maintain base container images - https://phabricator.wikimedia.org/T162042#3348984 (10MoritzMuehlenhoff) a:05MoritzMuehlenhoff>03None
[17:57:27] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Performance-Team, 10Jenkins, 10Upstream: WebPageTest job fails intemittently with "java.io.IOException: Unexpected termination of the channel" - https://phabricator.wikimedia.org/T166557#3349003 (10Peter) 05Resolved>03Open...
[17:58:21] <wikibugs>	 (03CR) 10Umherirrender: [C: 032] "Resubmit" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358567 (https://phabricator.wikimedia.org/T149623) (owner: 10Hashar)
[18:01:02] <wikibugs>	 (03Merged) 10jenkins-bot: Fix WhiteSpace/SpaceAfterClosureSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/358567 (https://phabricator.wikimedia.org/T149623) (owner: 10Hashar)
[18:01:29] <wikibugs>	 10Release-Engineering-Team, 10Page-Previews, 10Reading-Web-Backlog, 10Reading-Web-Kanban-Board: Create bot that automatically rebases and rebuilds patches to master - https://phabricator.wikimedia.org/T167181#3349015 (10greg) I'll bring it up in our next team weekly (next week).
[18:05:38] <RainbowSprinkles>	 Krinkle: Do you think T167885 is a blocker to wmf.5 continuing?
[18:05:38] <stashbot>	 T167885: Regression: Backend Save Timing raised by about 50% - https://phabricator.wikimedia.org/T167885
[18:07:57] <Krinkle>	 RainbowSprinkles: Unsure. It should've blocked wmf.4
[18:08:04] <Krinkle>	 But we found out too late.
[18:08:18] <RainbowSprinkles>	 Yeah, that's why I asked.
[18:08:24] <Krinkle>	 However, there's a fair chance it's related ops ops/db/services  instead of mw
[18:08:30] <Krinkle>	 But too early to tell
[18:08:36] <RainbowSprinkles>	 Yeah I read the comments
[18:08:38] <RainbowSprinkles>	 Maybe...proceed but we'll watch it closer?
[18:09:20] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Performance-Team, 10Jenkins, 10Upstream: WebPageTest job fails intemittently with "java.io.IOException: Unexpected termination of the channel" - https://phabricator.wikimedia.org/T166557#3349051 (10Krinkle)
[18:09:22] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Jenkins, 10Upstream: Launching Jenkins slave agent fails with "java.io.IOException: Unexpected termination of the channel" - https://phabricator.wikimedia.org/T91697#3349053 (10Krinkle)
[18:09:50] <Krinkle>	 RainbowSprinkles: Indeed.
[18:10:13] <RainbowSprinkles>	 Why did alerts lag? That's concerning
[18:12:22] <wikibugs>	 10Release-Engineering-Team (Kanban), 10MinervaNeue, 10Reading-Web-Backlog, 10Patch-For-Review: Skins cannot run browser tests per commit - https://phabricator.wikimedia.org/T167543#3349061 (10Jdlrobson) Looks good! I can run the entire MobileFrontend browser tests suite now [[ https://gerrit.wikimedia.org/...
[18:12:27] <wikibugs>	 10Release-Engineering-Team (Kanban), 10MinervaNeue, 10Reading-Web-Backlog, 10Patch-For-Review: Skins cannot run browser tests per commit - https://phabricator.wikimedia.org/T167543#3349062 (10Jdlrobson) 05Open>03Resolved
[18:16:18] <wikibugs>	 10Continuous-Integration-Config, 10MinervaNeue: Setup CI on Minerva repo - https://phabricator.wikimedia.org/T166750#3349094 (10Jdlrobson)
[18:23:41] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3349109 (10Jdlrobson) @zeljkofilipin are you including LocalSettings.php? I mentioned this before.. the Local...
[18:24:52] <wikibugs>	 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3349116 (10Jdlrobson) > As to why they do not show up on the Desktop version, really I am clueless. This is b...
[18:41:13] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Watching / External), 10Performance-Team: Create a bot that automatically optimizes PNGs - https://phabricator.wikimedia.org/T127628#3349180 (10Krinkle)
[19:08:19] <paladox>	 thcipriani hi, i've created a zuul systemd script locally. Wondering should we do it like zuul-merger?
[19:08:24] <paladox>	 since i am doing something like
[19:08:36] <paladox>	 https://phabricator.wikimedia.org/P5580
[19:09:41] <paladox>	 that works too
[19:09:46] <paladox>	 with restart and reload :)
[19:10:34] <thcipriani>	 hrm, all those runlevel2.target things I think are targets that the systemd- generator created. I wonder if there's a better, more systemd-like way to do that?
[19:11:37] <paladox>	 I could follow how we do it for phd.
[19:13:31] <wikibugs>	 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-Other, 10Patch-For-Review: Archive AWSSDK and WindowsAzureSDK MediaWiki extensions - https://phabricator.wikimedia.org/T167124#3349370 (10hashar) 05Open>03Resolved
[19:16:23] <paladox>	 thcipriani https://gerrit.wikimedia.org/r/#/c/359016/ :)
[19:17:44] <thcipriani>	 nice
[19:18:24] <wikibugs>	 10MediaWiki-Releasing, 10Release-Engineering-Team (Kanban), 10MW-1.29-release-notes, 10Patch-For-Review: Include release extensions/skins as submodules of core - https://phabricator.wikimedia.org/T137564#3349410 (10demon)
[19:18:32] <wikibugs>	 10MediaWiki-Releasing, 10Release-Engineering-Team (Kanban), 10MW-1.29-release-notes, 10Patch-For-Review: Include release extensions/skins as submodules of core - https://phabricator.wikimedia.org/T137564#2371986 (10demon) NOT VENDOR DEFINITELY NOT VENDOR
[19:18:56] <paladox>	 I've tested it too, with restart, stop, reload, start. Works for me. :)
[19:20:26] <thcipriani>	 cool, lgtm +1'd
[19:21:03] <paladox>	 thanks :)
[19:40:07] <wikibugs>	 (03PS3) 10Chad: After REL1_23 is obsolete, simplify config further [tools/release] - 10https://gerrit.wikimedia.org/r/343591 (owner: 10Reedy)
[19:40:35] <wikibugs>	 (03PS5) 10Chad: Simplify config, remove additions/removal upto REL1_23 [tools/release] - 10https://gerrit.wikimedia.org/r/343589 (owner: 10Reedy)
[19:40:43] <wikibugs>	 (03CR) 10Chad: [C: 032] Simplify config, remove additions/removal upto REL1_23 [tools/release] - 10https://gerrit.wikimedia.org/r/343589 (owner: 10Reedy)
[19:44:28] <wikibugs>	 (03Merged) 10jenkins-bot: Simplify config, remove additions/removal upto REL1_23 [tools/release] - 10https://gerrit.wikimedia.org/r/343589 (owner: 10Reedy)
[19:45:05] <wikibugs>	 (03PS4) 10Chad: After REL1_23 is obsolete, simplify config further [tools/release] - 10https://gerrit.wikimedia.org/r/343591 (owner: 10Reedy)
[19:45:09] <wikibugs>	 (03CR) 10Chad: [C: 032] After REL1_23 is obsolete, simplify config further [tools/release] - 10https://gerrit.wikimedia.org/r/343591 (owner: 10Reedy)
[19:47:37] <wikibugs>	 (03Merged) 10jenkins-bot: After REL1_23 is obsolete, simplify config further [tools/release] - 10https://gerrit.wikimedia.org/r/343591 (owner: 10Reedy)
[20:01:12] <harej>	 Reedy: eeeeeee!!
[20:06:30] <Reedy>	 harej: It's live
[20:06:34] <harej>	 :O
[20:06:42] <harej>	 Amaaazing.
[20:06:43] <harej>	 Cheers!
[20:07:53] <Reedy>	 In theory, enabling it wider can just be done via SWAT
[20:08:25] <harej>	 I imagine so, since it doesn't require database changes or anything like that
[20:08:51] <Reedy>	 Indeed
[20:08:55] <Reedy>	 Plus localisation cache is there now
[20:42:23] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T167533#3349759 (10Jdforrester-WMF)
[20:48:34] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-kafka01 is OK: OK: All targets OK
[20:54:11] <Reedy>	 Now, do I see if I can create atjwiki...
[20:55:51] <Reedy>	 Services aren't using it
[20:59:42] <wikibugs>	 (03PS1) 10Jdlrobson: Remove Cards from CI [integration/config] - 10https://gerrit.wikimedia.org/r/359044 (https://phabricator.wikimedia.org/T167452)
[21:01:04] <jdlrobson>	 @RainbowSprinkles don't suppose you want to do that tomorrow as part of train? ^
[21:01:09] <jdlrobson>	 not sure what preferred practice is?
[21:01:18] <RainbowSprinkles>	 That's not train stuff
[21:01:19] <jdlrobson>	 sorry wrong link
[21:01:23] <RainbowSprinkles>	 :)
[21:01:31] <jdlrobson>	 https://gerrit.wikimedia.org/r/359045 < RainbowSprinkles 
[21:02:03] <RainbowSprinkles>	 I can do it now
[21:02:17] <jdlrobson>	 wait.. you sure?
[21:02:21] <jdlrobson>	 it's not on enwiki yet
[21:02:26] <jdlrobson>	 so there's still a dependency
[21:02:30] <jdlrobson>	 ^ RainbowSprinkles 
[21:02:34] <RainbowSprinkles>	 Ah ok
[21:02:36] <RainbowSprinkles>	 I gotcha
[21:02:51] <RainbowSprinkles>	 Yeah, I'll do that tomorrow after train
[21:02:58] <jdlrobson>	 it can wait till next week too, just figured whatever you prefer
[21:03:06] <jdlrobson>	 seems like a good thing to get rid of things from cluster :)
[21:03:29] <RainbowSprinkles>	 I'm always a fan of getting rid of things from the cluster!
[21:03:31] <RainbowSprinkles>	 :)
[21:03:38] <RainbowSprinkles>	 Did you already drop it from make-wmf-branch for next week?
[21:08:43] <Reedy>	 I'm gonna deploy atjwiki in this two hour gap
[21:31:22] <SMalyshev>	 I'm having trouble with CI for patch adding new dependency to composer.json. Is there anything special one needs to do for this?
[21:31:38] <SMalyshev>	 e.g. this: https://integration.wikimedia.org/ci/job/mediawiki-extensions-hhvm-jessie/15783/console
[21:32:13] <wikibugs>	 10Gerrit, 10Developer-Relations, 10GitHub-Mirrors, 10Repository-Admins, and 2 others: Add CODE_OF_CONDUCT.md to Wikimedia repositories - https://phabricator.wikimedia.org/T165540#3350021 (10matmarex) I think MZMcBride is the only person who has a problem with the CODE_OF_CONDUCT.md in mediawiki/core. Can w...
[21:39:13] <SMalyshev>	 anybody?
[21:40:48] <Reedy>	 SMalyshev: Add it to mediawiki-vendor, and make your patch depend on it?
[21:41:32] <RainbowSprinkles>	 Speaking of vendor, I think I broke REL1_29 with my use-submodules idea
[21:41:37] <SMalyshev>	 Reedy: ah, ok, so CI does not use composer.json but uses vendor repo? Ok, will do
[21:41:39] <SMalyshev>	 thanks
[22:01:43] <SMalyshev>	 Reedy: I've added the new dependency, but after I do composer update, there's a lot of changes, including files like composer/ClassLoader.php. Should I be using specific version of composer?
[22:03:11] <Reedy>	 Latest stable release should be fine; 1.4.2
[22:03:14] <Reedy>	 What version are you using?
[22:03:23] <SMalyshev>	 1.4.2
[22:03:37] <SMalyshev>	 Reedy: see https://gerrit.wikimedia.org/r/#/c/359055/
[22:05:13] <SMalyshev>	 wait maybe my checkout is out of date?
[22:05:27] <Reedy>	 Most of the changes look vaguely sane
[22:07:10] <shinken-wm>	 PROBLEM - Puppet errors on deployment-elastic06 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[22:07:26] <shinken-wm>	 PROBLEM - Puppet errors on deployment-restbase02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:07:30] <shinken-wm>	 PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[22:07:44] <shinken-wm>	 PROBLEM - Puppet errors on saucelabs-03 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[22:07:46] <shinken-wm>	 PROBLEM - Puppet errors on deployment-trending01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:08:04] <shinken-wm>	 PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[22:08:34] <shinken-wm>	 PROBLEM - Puppet errors on deployment-kafka03 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[22:09:09] <shinken-wm>	 PROBLEM - Puppet errors on integration-slave-docker-1000 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[22:09:25] <shinken-wm>	 PROBLEM - Puppet errors on saucelabs-02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[22:09:45] <shinken-wm>	 PROBLEM - Puppet errors on integration-publishing is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[22:09:57] <shinken-wm>	 PROBLEM - Puppet errors on deployment-tmh01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:12:04] <shinken-wm>	 PROBLEM - Puppet errors on deployment-eventlogging04 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:12:50] <shinken-wm>	 PROBLEM - Puppet errors on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:13:06] <paladox>	 errors are happening for me too
[22:13:10] <shinken-wm>	 PROBLEM - Puppet errors on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[22:13:24] <shinken-wm>	 PROBLEM - Puppet errors on deployment-ircd is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:13:26] <shinken-wm>	 PROBLEM - Puppet errors on deployment-stream is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[22:15:08] <SMalyshev>	 Reedy: if the library was used in wikidata, but now I want to add it to main composer - does it need security review?
[22:15:40] <Reedy>	 Probably, yeah. In reality.. Probably not
[22:20:27] <SMalyshev>	 Reedy: ok :)
[22:32:34] <shinken-wm>	 PROBLEM - Puppet errors on deployment-restbase01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[22:33:25] <shinken-wm>	 PROBLEM - Puppet errors on deployment-elastic07 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[22:33:42] <shinken-wm>	 PROBLEM - Puppet errors on deployment-zotero01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:33:44] <shinken-wm>	 PROBLEM - Puppet errors on saucelabs-01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:33:50] <shinken-wm>	 PROBLEM - Puppet errors on deployment-ms-be03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:35:19] <shinken-wm>	 PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[22:39:07] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-docker-1000 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:42:46] <shinken-wm>	 RECOVERY - Puppet errors on saucelabs-03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:43:36] <shinken-wm>	 RECOVERY - Puppet errors on deployment-kafka03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:44:21] <shinken-wm>	 RECOVERY - Puppet errors on saucelabs-02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:44:45] <shinken-wm>	 RECOVERY - Puppet errors on integration-publishing is OK: OK: Less than 1.00% above the threshold [0.0]
[22:47:04] <shinken-wm>	 RECOVERY - Puppet errors on deployment-eventlogging04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:47:08] <shinken-wm>	 RECOVERY - Puppet errors on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:47:30] <shinken-wm>	 RECOVERY - Puppet errors on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:47:48] <shinken-wm>	 RECOVERY - Puppet errors on deployment-trending01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:47:55] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-trusty-1004 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:48:03] <shinken-wm>	 RECOVERY - Puppet errors on deployment-db03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:48:11] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-trusty-1001 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:48:53] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:49:09] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mcs01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:49:13] <shinken-wm>	 RECOVERY - Puppet errors on deployment-aqs03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:50:51] <shinken-wm>	 RECOVERY - Puppet errors on deployment-sca03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:51:07] <shinken-wm>	 RECOVERY - Puppet errors on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:52:06] <shinken-wm>	 RECOVERY - Puppet errors on deployment-db04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:52:24] <shinken-wm>	 RECOVERY - Puppet errors on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:52:44] <shinken-wm>	 RECOVERY - Puppet errors on deployment-etcd-01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:22] <shinken-wm>	 RECOVERY - Puppet errors on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:25] <shinken-wm>	 RECOVERY - Puppet errors on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:25] <shinken-wm>	 RECOVERY - Puppet errors on deployment-ircd is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:25] <shinken-wm>	 RECOVERY - Puppet errors on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:33] <shinken-wm>	 RECOVERY - Puppet errors on integration-puppetmaster01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:45] <shinken-wm>	 RECOVERY - Puppet errors on deployment-elastic05 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:46] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-jessie-android is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:50] <shinken-wm>	 RECOVERY - Puppet errors on deployment-ms-fe02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:54:00] <shinken-wm>	 RECOVERY - Puppet errors on deployment-kafka04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:54:32] <shinken-wm>	 RECOVERY - Puppet errors on deployment-changeprop is OK: OK: Less than 1.00% above the threshold [0.0]
[22:55:00] <shinken-wm>	 RECOVERY - Puppet errors on deployment-tmh01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:55:08] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mx is OK: OK: Less than 1.00% above the threshold [0.0]
[22:55:36] <shinken-wm>	 RECOVERY - Puppet errors on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:56:14] <shinken-wm>	 RECOVERY - Puppet errors on deployment-aqs02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:17] <shinken-wm>	 RECOVERY - Puppet errors on deployment-sentry01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:19] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:33] <shinken-wm>	 RECOVERY - Puppet errors on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:51] <shinken-wm>	 RECOVERY - Puppet errors on deployment-eventlogging03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:58:41] <shinken-wm>	 RECOVERY - Puppet errors on deployment-zotero01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:58:52] <shinken-wm>	 RECOVERY - Puppet errors on deployment-ms-be03 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:58:54] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]
[22:59:18] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:59:50] <shinken-wm>	 RECOVERY - Puppet errors on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:00:09] <shinken-wm>	 RECOVERY - Puppet errors on castor is OK: OK: Less than 1.00% above the threshold [0.0]
[23:00:19] <shinken-wm>	 RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:00:37] <shinken-wm>	 RECOVERY - Puppet errors on jenkinstest is OK: OK: Less than 1.00% above the threshold [0.0]
[23:03:04] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:03:42] <shinken-wm>	 RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:03:56] <shinken-wm>	 RECOVERY - Puppet errors on integration-saltmaster is OK: OK: Less than 1.00% above the threshold [0.0]
[23:04:39] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-trusty-1006 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:15:46] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Operations, 10Patch-For-Review, 10Security-General: setup mwreleases1001.eqiad.wmnet - https://phabricator.wikimedia.org/T164030#3350238 (10Dzahn) We talked about this on IRC and came to the conclusion that we want to use this new VM for not just mediawiki releases bu...
[23:33:34] <RoanKattouw>	 Looks like beta-scap-eqiad is broken?
[23:33:34] <RoanKattouw>	 https://integration.wikimedia.org/ci/job/beta-scap-eqiad/
[23:33:47] <RoanKattouw>	 It hasn't run since 19:33 UTC
[23:34:12] <RoanKattouw>	 ( RainbowSprinkles )
[23:35:02] <RainbowSprinkles>	 Well that's no good
[23:35:09] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T167533#3350288 (10Catrope)
[23:35:27] <Reedy>	 deployment-tin being weird
[23:35:28] <RainbowSprinkles>	 Something about labels
[23:35:30] <Reedy>	 /idle
[23:35:33] <RainbowSprinkles>	 And no available hosts
[23:35:35] <RainbowSprinkles>	 Yeah
[23:51:49] <wikibugs>	 10Release-Engineering-Team (Kanban), 10VisualEditor, 10User-Ryasmeen, 10User-zeljkofilipin: LanguageScreenshotBot trying to edit a non-existent page without signing in - https://phabricator.wikimedia.org/T162454#3350309 (10Xaosflux) Please note, as decided in Wikipedia:Articles for deletion/Screenshot/fodd...
[23:57:34] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T167535#3335992 (10Tgr) https://gerrit.wikimedia.org/r/#/c/359047/ (or some other way of fixing {T167922} in master) is a blocker.
[23:58:23] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Operations, 10Patch-For-Review, 10Security-General: setup releases1001.eqiad.wmnet (was: setup mwreleases1001) - https://phabricator.wikimedia.org/T164030#3350322 (10Dzahn)
[23:58:26] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T167533#3350323 (10greg) from `#wikimedia-releng` ```lang=irc 18:05 <+RainbowSp> Krinkle: Do you think T167885 is a blocker to wmf.5 continuing? 18:05 <  stashbot> T167...