[00:14:00] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [01:04:59] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [02:10:02] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [03:06:34] 10MediaWiki-Codesniffer, 10MediaWiki-Platform-Team: Enforce one class per file in preparation for PSR-4 - https://phabricator.wikimedia.org/T173798#3540108 (10Legoktm) [03:21:53] PROBLEM - Puppet errors on deployment-jobrunner02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [03:32:33] (03PS1) 10Legoktm: Make it easier to figure out which test failed [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/372915 [03:32:34] (03PS1) 10Legoktm: Require only one class/interface/trait per file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/372916 (https://phabricator.wikimedia.org/T173798) [03:33:34] (03PS2) 10Legoktm: Make it easier to figure out which test failed [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/372915 [03:33:36] (03PS2) 10Legoktm: Require only one class/interface/trait per file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/372916 (https://phabricator.wikimedia.org/T173798) [04:01:52] RECOVERY - Puppet errors on deployment-jobrunner02 is OK: OK: Less than 1.00% above the threshold [0.0] [04:13:06] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [04:53:05] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [05:15:49] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [05:35:51] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:02:36] PROBLEM - Puppet errors on integration-r-lang-01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:07:30] (03CR) 10KartikMistry: "@hashar ping. Can you review this?" [integration/config] - 10https://gerrit.wikimedia.org/r/372229 (https://phabricator.wikimedia.org/T172381) (owner: 10KartikMistry) [06:41:28] (03PS3) 10Legoktm: Add "commit-message-validator install" command [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/372424 [06:41:33] (03CR) 10Legoktm: [C: 032] Add "commit-message-validator install" command [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/372424 (owner: 10Legoktm) [06:42:07] (03Merged) 10jenkins-bot: Add "commit-message-validator install" command [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/372424 (owner: 10Legoktm) [06:42:33] RECOVERY - Puppet errors on integration-r-lang-01 is OK: OK: Less than 1.00% above the threshold [0.0] [06:43:02] (03PS2) 10Legoktm: Add apertium-bel-rus and dependencies to CI [integration/config] - 10https://gerrit.wikimedia.org/r/372229 (https://phabricator.wikimedia.org/T172381) (owner: 10KartikMistry) [06:43:08] (03CR) 10Legoktm: [C: 032] Add apertium-bel-rus and dependencies to CI [integration/config] - 10https://gerrit.wikimedia.org/r/372229 (https://phabricator.wikimedia.org/T172381) (owner: 10KartikMistry) [06:44:00] (03Merged) 10jenkins-bot: Add apertium-bel-rus and dependencies to CI [integration/config] - 10https://gerrit.wikimedia.org/r/372229 (https://phabricator.wikimedia.org/T172381) (owner: 10KartikMistry) [06:44:39] !log deploying https://gerrit.wikimedia.org/r/372229 [06:44:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:45:01] kart_: ^ done [06:48:08] (03PS2) 10Legoktm: Fix uprightdiff debian-glue running on master [integration/config] - 10https://gerrit.wikimedia.org/r/370695 [06:48:12] (03CR) 10Legoktm: [C: 032] Fix uprightdiff debian-glue running on master [integration/config] - 10https://gerrit.wikimedia.org/r/370695 (owner: 10Legoktm) [06:49:03] (03Merged) 10jenkins-bot: Fix uprightdiff debian-glue running on master [integration/config] - 10https://gerrit.wikimedia.org/r/370695 (owner: 10Legoktm) [06:49:31] !log deploying https://gerrit.wikimedia.org/r/370695 [06:49:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:49:52] (03CR) 10Legoktm: "recheck" [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:49:59] (03CR) 10jerkins-bot: [V: 04-1] Default PREFIX to /usr [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:50:28] (03CR) 10Legoktm: [C: 032] "I have no idea why jenkins keeps trying to run debian-glue on master." [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:50:32] (03CR) 10Legoktm: [V: 032 C: 032] Default PREFIX to /usr [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:50:35] (03CR) 10jerkins-bot: [V: 04-1] Default PREFIX to /usr [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:50:39] (03CR) 10jerkins-bot: [V: 04-1] Default PREFIX to /usr [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/371201 (owner: 10Legoktm) [06:53:53] (03PS1) 10Legoktm: Remove obsolete fback and linehash code [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372968 [06:54:00] (03CR) 10jerkins-bot: [V: 04-1] Remove obsolete fback and linehash code [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372968 (owner: 10Legoktm) [06:54:02] (03CR) 10Legoktm: [V: 032 C: 032] Remove obsolete fback and linehash code [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372968 (owner: 10Legoktm) [07:13:51] (03PS1) 10Legoktm: Add "make test" target [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372996 (https://phabricator.wikimedia.org/T152740) [07:13:58] (03CR) 10jerkins-bot: [V: 04-1] Add "make test" target [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372996 (https://phabricator.wikimedia.org/T152740) (owner: 10Legoktm) [07:14:02] (03CR) 10Legoktm: [V: 032 C: 032] Add "make test" target [integration/uprightdiff] - 10https://gerrit.wikimedia.org/r/372996 (https://phabricator.wikimedia.org/T152740) (owner: 10Legoktm) [07:17:44] (03CR) 10Legoktm: [C: 032] Address paravoid's review [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/371202 (owner: 10Legoktm) [07:20:05] (03Merged) 10jenkins-bot: Address paravoid's review [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/371202 (owner: 10Legoktm) [07:30:52] (03PS1) 10Legoktm: Merge tag '1.2.0' into debian [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373025 [07:30:54] (03PS1) 10Legoktm: Update version in debian/changelog [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373026 [07:30:56] (03PS1) 10Legoktm: Add debian/watch [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373027 [07:31:08] (03CR) 10jerkins-bot: [V: 04-1] Merge tag '1.2.0' into debian [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373025 (owner: 10Legoktm) [07:34:29] (03CR) 10Legoktm: [V: 032 C: 032] "build is fixed in next commit" [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373025 (owner: 10Legoktm) [07:34:50] (03CR) 10Legoktm: [C: 032] Update version in debian/changelog [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373026 (owner: 10Legoktm) [07:34:54] (03CR) 10Legoktm: [C: 032] Add debian/watch [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373027 (owner: 10Legoktm) [07:37:18] (03Merged) 10jenkins-bot: Update version in debian/changelog [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373026 (owner: 10Legoktm) [07:37:20] (03Merged) 10jenkins-bot: Add debian/watch [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373027 (owner: 10Legoktm) [07:47:49] (03PS1) 10Legoktm: Remove unnecessary override_dh_auto_install [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373028 [07:48:36] (03CR) 10Legoktm: [C: 032] Remove unnecessary override_dh_auto_install [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373028 (owner: 10Legoktm) [07:50:53] (03Merged) 10jenkins-bot: Remove unnecessary override_dh_auto_install [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/373028 (owner: 10Legoktm) [07:55:56] PROBLEM - Puppet errors on deployment-apertium02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [08:11:13] Would anyone mind +2 for a simple backport for https://gerrit.wikimedia.org/r/#/c/372813/? [08:23:50] broken link [08:24:32] ah, question mark at the end. isn't that something for the SimpleChanges maintainer(s)? [08:31:00] RECOVERY - Puppet errors on deployment-apertium02 is OK: OK: Less than 1.00% above the threshold [0.0] [08:39:17] legoktm: thanks! [08:55:49] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [08:58:37] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.30.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T170632#3540399 (10Ladsgroup) [09:17:04] it is? I thought since it was a backport, it could be done by release managers [10:03:19] 10Release-Engineering-Team, 10Page-Previews, 10Readers-Web-Backlog, 10Epic: [EPIC] Generate compiled assets from continuous integration - https://phabricator.wikimedia.org/T158980#3540580 (10Jhernandez) a:05Jhernandez>03None [10:06:00] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [10:09:59] 10Release-Engineering-Team (Kanban), 10RelatedArticles, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking), 10User-zeljkofilipin: Create Jenkins job that runs RelatedArticles Selenium tests daily - https://phabricator.wikimedia.org/T171847#3540599 (10zeljkofilipin) Core job is green ๐Ÿ ๐Ÿ“— ๐Ÿ’š โœ… https://in... [10:19:27] (03PS4) 10Legoktm: Change bootstrap-ci.php to match PHP CodeSniffer 3.0.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/371475 (https://phabricator.wikimedia.org/T172933) (owner: 10Umherirrender) [10:32:25] (03CR) 10Legoktm: [C: 032] "Tested, LGTM. Thanks for working on this :) I suggested an improvement inline if you're interested." (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/371475 (https://phabricator.wikimedia.org/T172933) (owner: 10Umherirrender) [10:33:24] (03Merged) 10jenkins-bot: Change bootstrap-ci.php to match PHP CodeSniffer 3.0.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/371475 (https://phabricator.wikimedia.org/T172933) (owner: 10Umherirrender) [10:41:03] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [11:11:56] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10User-Addshore: Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3540725 (10Addshore) [11:12:03] phuedx: a mobile web question: appending "&mobileaction=toggle_view_mobile" to the url does not open mobile view on beta cluster, unless you also add change "en.wiki..." to "en.m.wiki..." [11:12:11] is that by design? [11:12:48] just appending "&mobileaction=toggle_view_mobile" to the url on my vagrant vm switches to mobile view, so I was surprised by the difference [11:25:54] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [11:39:51] 10Release-Engineering-Team (Kanban), 10RelatedArticles, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking), 10User-zeljkofilipin: Create Jenkins job that runs RelatedArticles Selenium tests daily - https://phabricator.wikimedia.org/T171847#3540822 (10zeljkofilipin) RelatedArticles test job [[ https://i... [12:05:53] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [12:26:17] zeljkof: that might be a bug [12:30:50] hrrm [12:30:55] i'll think about that [12:33:37] PROBLEM - Puppet errors on integration-r-lang-01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [12:54:55] phuedx: no rush, I was just surprised [12:55:13] the patch you merged solved my problem, so I don't really care any more ;) [13:06:16] (03PS1) 10Aude: Update Wikidata to wmf/1.30.0-wmf.15 [tools/release] - 10https://gerrit.wikimedia.org/r/373066 [13:06:26] (03CR) 10Aude: [C: 032] Update Wikidata to wmf/1.30.0-wmf.15 [tools/release] - 10https://gerrit.wikimedia.org/r/373066 (owner: 10Aude) [13:06:58] (03Merged) 10jenkins-bot: Update Wikidata to wmf/1.30.0-wmf.15 [tools/release] - 10https://gerrit.wikimedia.org/r/373066 (owner: 10Aude) [13:13:36] RECOVERY - Puppet errors on integration-r-lang-01 is OK: OK: Less than 1.00% above the threshold [0.0] [13:30:22] 10Release-Engineering-Team (Kanban), 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)), 10Readers-Web-Backlog (Tracking), 10User-zeljkofilipin: Create Jenkins job that runs RelatedArticles Selenium tests daily - https://phabricator.wikimedia.org/T171847#3541087 (10Jdlrobso... [13:32:01] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [14:12:03] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [14:13:53] PROBLEM - Puppet errors on deployment-mediawiki04 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [14:14:05] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [14:14:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Wikidata-Sprint: Trigger Jenkins jobs on patchset uploaded by the user chrisneuroth - https://phabricator.wikimedia.org/T173831#3541231 (10WMDE-leszek) [14:15:11] (03PS1) 10WMDE-leszek: Add Chris Neuroth to the list of users triggering +2 voting checks [integration/config] - 10https://gerrit.wikimedia.org/r/373077 (https://phabricator.wikimedia.org/T173831) [14:15:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Patch-For-Review, 10Wikidata-Sprint: Trigger Jenkins jobs on patchset uploaded by the user chrisneuroth - https://phabricator.wikimedia.org/T173831#3541249 (10WMDE-leszek) p:05Triage>03Normal [14:49:06] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [14:52:31] PROBLEM - Puppet errors on deployment-imagescaler02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [14:53:53] RECOVERY - Puppet errors on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0] [15:28:53] Yippee, build fixed! [15:28:54] Project selenium-MinervaNeue ยป chrome,beta,Linux,BrowserTests build #87: 09FIXED in 19 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/87/ [15:31:14] (03PS1) 10XZise: Use one logic to catch all wrong spellings [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/373092 [15:32:31] RECOVERY - Puppet errors on deployment-imagescaler02 is OK: OK: Less than 1.00% above the threshold [0.0] [15:33:19] (03CR) 10XZise: [C: 04-1] "Could you explain why it does count the line length differently? I know there is I9a775d4, but I don't think that it's related to Python 3" [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/372418 (owner: 10Legoktm) [15:49:52] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Next), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin: WebdriverIO should run Chrome headlessly - https://phabricator.wikimedia.org/T167507#3541741 (10zeljkofilipin) 05Open>03stalled Blocked by {T170032}. [15:51:54] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [15:53:50] 10Release-Engineering-Team (Kanban), 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)), 10Readers-Web-Backlog (Tracking), 10User-zeljkofilipin: Create Jenkins job that runs RelatedArticles Selenium tests daily - https://phabricator.wikimedia.org/T171847#3541752 (10zeljkofi... [16:04:09] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Infrastructure, 10User-zeljkofilipin: Upgrade to Chromium 59 or newer on Debian Jessie in CI - https://phabricator.wikimedia.org/T170032#3541765 (10Jdforrester-WMF) Would it be more natural to "just" switch everything over to Stretch? [16:13:23] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [16:23:16] PROBLEM - Puppet errors on deployment-ores-redis-01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [16:23:30] PROBLEM - Puppet errors on deployment-imagescaler02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:25:11] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:26:53] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [16:29:41] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [16:35:27] PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [16:42:00] (03PS1) 10Zfilipin: WIP Create Jenkins job that runs RelatedArticles Selenium tests daily [integration/config] - 10https://gerrit.wikimedia.org/r/373103 (https://phabricator.wikimedia.org/T171847) [16:42:48] (03CR) 10Zfilipin: "Testing the job here:" [integration/config] - 10https://gerrit.wikimedia.org/r/373103 (https://phabricator.wikimedia.org/T171847) (owner: 10Zfilipin) [16:44:20] (03PS2) 10Zfilipin: WIP Create Jenkins job that runs RelatedArticles Selenium tests daily [integration/config] - 10https://gerrit.wikimedia.org/r/373103 (https://phabricator.wikimedia.org/T171847) [16:50:26] RECOVERY - Puppet errors on deployment-tin is OK: OK: Less than 1.00% above the threshold [0.0] [16:53:21] RECOVERY - Puppet errors on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0] [17:03:16] RECOVERY - Puppet errors on deployment-ores-redis-01 is OK: OK: Less than 1.00% above the threshold [0.0] [17:15:06] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [17:19:56] (03PS3) 10Zfilipin: WIP Create Jenkins job that runs RelatedArticles Selenium tests daily [integration/config] - 10https://gerrit.wikimedia.org/r/373103 (https://phabricator.wikimedia.org/T171847) [17:23:55] 10Release-Engineering-Team (Kanban), 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)), 10Patch-For-Review, and 2 others: Create Jenkins job that runs RelatedArticles Selenium tests daily - https://phabricator.wikimedia.org/T171847#3542088 (10zeljkofilipin) [[ https://integ... [17:50:07] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [17:58:02] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Cloud-VPS, 10Nodepool, and 2 others: figure out if nodepool is overwhelming rabbitmq and/or nova - https://phabricator.wikimedia.org/T170492#3542225 (10Andrew) I've just removed apache and puppetmaster from the lab... [18:04:35] PROBLEM - Puppet errors on integration-r-lang-01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [18:39:35] RECOVERY - Puppet errors on integration-r-lang-01 is OK: OK: Less than 1.00% above the threshold [0.0] [19:12:51] thcipriani: (sorry if wrong channel) if the 3d2png deploy repo gets updated, is there any super-special way we need to update the node_modules as well, or can we do that locally? [19:14:04] marktraceur: eh, that is a question for services and I feel like the answer is "yes there is something super special" and I feel like it has something to do with service-runner and docker. [19:14:10] Ugh [19:14:14] OK, thanks [19:14:45] Found the relevant email... [19:15:54] where all good documentation is [19:16:46] hrm. was just looking through wikitech and couldn't find it :\ [19:18:10] oh, the best documentation is in private emails, duh [19:18:25] my mistake :) [19:30:41] 10Deployment-Systems, 10MediaWiki-extensions-CentralAuth, 10Performance-Team: $wgLocalVirtualHosts should include login.wikimedia.org, wikidata.org and others? - https://phabricator.wikimedia.org/T172357#3542697 (10aaron) Looks like using WikiMap is the best choice IMO. [19:32:53] hrm, is .30-wmf13 in prod or -wmf14? [19:32:56] * brion gets lost backporting patches [19:33:27] brion: https://tools.wmflabs.org/versions/ [19:33:35] excellent [19:33:39] greg-g: thx, bookmarking :D [19:34:14] brion: ftr, wmf.15 is going to group0 right now (scap in progress) [19:34:52] great, i've got a small TMH patch ready for tomorrow for .14 and .15 to switch from ogg to webm playback so we can kill ffmpeg2theora \o/ [19:35:03] weee [19:36:26] bd808: something's weird with https://tools.wmflabs.org/versions/, all timestamps are the same (current time) [19:37:58] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [0.0] [19:43:28] greg-g: can you file a bug for me to look at it? Its a problem parsing the elasticsearch json response, but I don't have time to figure out the fix right now [19:44:23] bd808: totally [19:44:35] Project beta-scap-eqiad build #169783: 04FAILURE in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169783/ [19:54:39] Project beta-scap-eqiad build #169784: 04STILL FAILING in 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169784/ [19:55:49] PROBLEM - Puppet errors on deployment-urldownloader is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [19:56:25] Project beta-scap-eqiad build #169785: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169785/ [19:59:14] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Next), 10MinervaNeue, 10Readers-Web-Backlog, and 6 others: [4 hrs] MinervaNeue browser test are flaking (waiting for {:class=>"mw-notification", :tag_name=>"div"} to become present ) - https://phabricator.wikimedia.org/T170890#3542827 (10Jdlrobson... [20:04:30] > The configuration setting 'wgCaptchaClass' was already set by another extension, and cannot be set again. [20:04:33] Project beta-scap-eqiad build #169786: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169786/ [20:12:59] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:14:22] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:14:32] Project beta-scap-eqiad build #169787: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169787/ [20:20:08] Project beta-update-databases-eqiad build #19384: 04FAILURE in 8.1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/19384/ [20:24:03] legoktm: I think https://gerrit.wikimedia.org/r/#/c/348577/ is what is breaking mergemessagefilelist in beta (and ipso facto scap ^) currently (dunno Florianschmidtwelzow 's irc nick, hence the ping). [20:24:33] Project beta-scap-eqiad build #169788: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169788/ [20:30:48] RECOVERY - Puppet errors on deployment-urldownloader is OK: OK: Less than 1.00% above the threshold [0.0] [20:34:31] Project beta-scap-eqiad build #169789: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169789/ [20:44:33] Project beta-scap-eqiad build #169790: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169790/ [20:45:03] but I'm fixing you scap. I'm fixing you now. [20:49:20] RECOVERY - Puppet errors on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0] [20:51:22] @seen FlorianSW [20:56:13] Yippee, build fixed! [20:56:14] Project beta-scap-eqiad build #169791: 09FIXED in 2 min 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/169791/ [20:57:02] yippee [20:57:11] :) [21:14:52] PROBLEM - Puppet errors on deployment-mediawiki04 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [21:16:06] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [21:21:54] Yippee, build fixed! [21:21:55] Project beta-update-databases-eqiad build #19385: 09FIXED in 1 min 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/19385/ [21:46:07] 10Gerrit, 10LDAP-Access-Requests: +2 in job-related repos for new Android engineers - https://phabricator.wikimedia.org/T173874#3543164 (10greg) [21:50:04] 10Gerrit, 10LDAP-Access-Requests: +2 in job-related repos for new Android engineers - https://phabricator.wikimedia.org/T173874#3542983 (10greg) Other than Yuvi and these new developers, who else should be in https://gerrit.wikimedia.org/r/#/admin/groups/633,members ? (extensions/MobileApp) [21:51:05] 10Gerrit, 10LDAP-Access-Requests: +2 in job-related repos for new Android engineers - https://phabricator.wikimedia.org/T173874#3542983 (10RobH) Ok, since neither of these users have shell access, and are not listed in the admins module, my understanding is the following; * both users must be added to the adm... [21:54:53] RECOVERY - Puppet errors on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0] [21:57:12] 10Gerrit, 10LDAP-Access-Requests: +2 in job-related repos for new Android engineers - https://phabricator.wikimedia.org/T173874#3543186 (10Mholloway) On the topic of shell access, I was separately going to file a request to have both of them added to releasers-mobile (for Bromine access) if that changes the si... [22:02:20] 10Gerrit, 10LDAP-Access-Requests, 10Patch-For-Review: +2 in job-related repos for new Android engineers - https://phabricator.wikimedia.org/T173874#3543230 (10RobH) @Mholloway It does! It means I wouldn't add them via this method, but AFTER their shell access is live. [22:11:47] 10Gerrit, 10LDAP-Access-Requests, 10Operations: Add new users Sharvaniharan and Cooltey to releasers-mobile - https://phabricator.wikimedia.org/T173886#3543236 (10Mholloway) [22:21:05] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [22:21:15] greg-g: https://tools.wmflabs.org/versions/ is all better. Must have been busted like that since the elasticsearch upgrade I did ... last month? [22:33:31] bd808: :) Thanks man [22:33:42] bd808: btw, where should I have filed a task for it? [22:38:17] thcipriani: looking ... [22:40:55] greg-g: ยฏ\_(ใƒ„)_/ยฏ I would have picked https://phabricator.wikimedia.org/tag/tools/ assigned to @bd808 I guess [22:41:31] lacking an obvious bug tracking link that's the catch-all place for bugs against specific tools [22:42:21] thcipriani: thanks for reverting. it looks like the patch work as expected and ConfirmEdit is buggy [22:43:20] legoktm: thanks for looking into it, and yeah, reverted for now, failing on mergeMessageFileList for dewiktionary stacktrace/details on https://phabricator.wikimedia.org/P5904 [22:51:47] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10User-Addshore: Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3543288 (10Legoktm) [22:52:54] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [22:56:45] bd808: /me nods [23:22:51] PROBLEM - Puppet errors on deployment-jobrunner02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [23:23:47] 10Gerrit, 10LDAP-Access-Requests, 10Operations: Add new users Sharvaniharan and Cooltey to releasers-mobile - https://phabricator.wikimedia.org/T173886#3543361 (10RobH) It is typically best to handle each user request as their own task, since there will be steps that each user has to handle on their own. Ho... [23:26:56] PROBLEM - Puppet errors on deployment-apertium02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [23:28:11] 10Gerrit, 10LDAP-Access-Requests, 10Operations: Add new users Sharvaniharan and Cooltey to releasers-mobile - https://phabricator.wikimedia.org/T173886#3543368 (10RobH) p:05Triage>03Normal [23:32:53] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0]