[04:16:35] Project mediawiki-core-doxygen-docker build #16454: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/16454/ [04:20:26] PROBLEM - https://phabricator.wikimedia.org #page on phabricator.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Phabricator [04:22:12] RECOVERY - https://phabricator.wikimedia.org #page on phabricator.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 37204 bytes in 3.608 second response time https://wikitech.wikimedia.org/wiki/Phabricator [04:29:46] PROBLEM - https://phabricator.wikimedia.org #page on phabricator.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Phabricator [04:37:04] RECOVERY - https://phabricator.wikimedia.org #page on phabricator.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 37204 bytes in 0.121 second response time https://wikitech.wikimedia.org/wiki/Phabricator [05:16:07] Yippee, build fixed! [05:16:07] Project mediawiki-core-doxygen-docker build #16455: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/16455/ [05:28:51] 10Deployments, 10Growth-Team, 10WikiLove: Slovenian Wikipedia - WikiLove request for deployment - https://phabricator.wikimedia.org/T254706 (10Mitjas1) [06:45:07] 10Continuous-Integration-Config, 10Operations, 10SRE-tools, 10Patch-For-Review: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494 (10hashar) 05duplicate→03Open Reopening since this is about adding shellcheck on any repository while T254480 is specific to puppet.git and covers... [06:45:37] (03PS1) 10Hashar: docker: use more recent shellcheck [integration/config] - 10https://gerrit.wikimedia.org/r/603175 (https://phabricator.wikimedia.org/T254480) [06:52:45] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Operations, 10Patch-For-Review, and 2 others: Shell/Python/other scripts should not be generated by ERB files; dynamic parts should be a simple ERB config file - https://phabricator.wikimedia.org/T254480 (10hashar) The C... [07:59:13] 10Phabricator, 10Operations, 10Security-Team, 10Traffic: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 (10Fae) Playing with the Tor browser this morning, a work-around could be to for users to keep trying new Tor circuits until they stop getting the Error 500 message. This appe... [08:17:03] 10LibUp, 10User-DannyS712: Library upgrader tests not always being run - https://phabricator.wikimedia.org/T254070 (10Legoktm) >>! In T254070#6199759, @DannyS712 wrote: > Didn't work at https://gerrit.wikimedia.org/r/#/c/wikimedia/textcat/+/602950/ That's on libup. Currently if tries to +2, if that fails, it'... [08:28:37] 10Project-Admins, 10Inuka-Team: Clean up open #New-Readers tasks in Phabricator - https://phabricator.wikimedia.org/T251706 (10AMuigai) 05Open→03Resolved a:03AMuigai [09:23:43] (03CR) 10Jbond: [C: 03+1] "LGTM, thanks" [integration/config] - 10https://gerrit.wikimedia.org/r/603175 (https://phabricator.wikimedia.org/T254480) (owner: 10Hashar) [09:32:01] 10MediaWiki-Codesniffer, 10User-DannyS712: Add a sniff for unneeded __construct functions - https://phabricator.wikimedia.org/T242402 (10thiemowmde) Does it make sense to expand the idea of this sniff to **all** methods that do nothing but calling their own parent? Or is there a reason to ever keep anything li... [09:39:45] 10Gerrit, 10DBA: Get a writable reviewdb clone to test Gerrit upgrade with - https://phabricator.wikimedia.org/T254516 (10Marostegui) Thanks Jaime for taking care of this. Just a clarification, there is a RO gerrit user called: `gerritro` which just SELECT grant. ` GRANT SELECT ON `reviewdb`.* TO 'gerritro'@'1... [09:42:18] 10Gerrit, 10DBA: Get a writable reviewdb clone to test Gerrit upgrade with - https://phabricator.wikimedia.org/T254516 (10Dzahn) Access for the gerritro user was fixed on May 28th in T243800#6171861. And that made it possible to run gerrit on the test server. I think that's an entirely separate request now,... [09:45:31] 10Continuous-Integration-Config, 10Readers-Web-Backlog (Kanbanana-2019-20-Q4), 10Vue.js (Vue.js-Search): Install Jest for unit tests for Vue.js search - https://phabricator.wikimedia.org/T249301 (10ovasileva) p:05Triage→03High [09:45:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Operations, 10Patch-For-Review, and 2 others: Shell/Python/other scripts should not be generated by ERB files; dynamic parts should be a simple ERB config file - https://phabricator.wikimedia.org/T254480 (10jbond) >>! In... [09:57:23] 10MediaWiki-Codesniffer, 10User-DannyS712: Add a sniff for unneeded __construct functions - https://phabricator.wikimedia.org/T242402 (10DannyS712) >>! In T242402#6201001, @thiemowmde wrote: > Does it make sense to expand the idea of this sniff to **all** methods that do nothing but calling their own parent? O... [11:09:26] 10Gerrit, 10DBA: Get a writable reviewdb clone to test Gerrit upgrade with - https://phabricator.wikimedia.org/T254516 (10jcrespo) Thanks, now that I understood the actual requirements (I wasn't aware of the existence of a ro account), I will take care of creating a test upgrade database from a reviewdb snapsh... [11:16:25] 10Deployments, 10Release-Engineering-Team, 10Prod-Kubernetes, 10serviceops: Evaluate cost/benefits of switching to using a programming language to define kubernetes resources - https://phabricator.wikimedia.org/T254739 (10Joe) [11:25:29] 10Phabricator, 10Operations, 10Security-Team, 10Traffic: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 (10jbond) p:05Triage→03Medium [11:43:03] 10phan, 10Patch-For-Review: Upgrade to phan/phan 2.7.2 or later in mediawiki/tools/phan - https://phabricator.wikimedia.org/T250170 (10Daimona) We have to upgrade to at least phan 3.0.2 due to taint-check. [11:43:19] 10phan, 10Patch-For-Review: Upgrade to phan/phan 3.0.2 or later in mediawiki/tools/phan - https://phabricator.wikimedia.org/T250170 (10Daimona) [11:50:02] 10Phabricator, 10EngProd-Virtual-Hackathon: Extract data for burn down charts - https://phabricator.wikimedia.org/T253109 (10dom_walden) 05Open→03Declined [11:54:41] 10phan, 10Lexicographical data, 10Wikidata, 10Wikidata-Campsite, and 2 others: Tests failing for the master branch of WikibaseLexeme - https://phabricator.wikimedia.org/T254701 (10Daimona) 05Open→03Resolved a:03Jakob_WMDE Fixed with https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseLexe... [12:21:30] 10Deployments, 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review: Write script to apply security patches - https://phabricator.wikimedia.org/T247075 (10LarsWirzenius) 05Open→03Resolved [12:21:32] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scap: Automate scap patch - https://phabricator.wikimedia.org/T245185 (10LarsWirzenius) [12:23:50] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Scap: Make a proof of concept set of nested VMs for train dev/test - https://phabricator.wikimedia.org/T253997 (10LarsWirzenius) I have quick-and-dirty prototype that needs to be re-done cleanly and the process documented. [12:49:52] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10Product-Analytics, 10Wikipedia-Android-App-Backlog: MobileWikiAppProtectedEditAttempt: 'protectionStatus' is a required property - https://phabricator.wikimedia.org/T254567 (10Charlotte) @Dbrant - If we are not using these events for... [12:50:31] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10Product-Analytics, 10Wikipedia-Android-App-Backlog: Remove MobileWikiAppProtectedEditAttempt schema from Android app - https://phabricator.wikimedia.org/T254567 (10Charlotte) p:05Triage→03Medium [12:50:47] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10Product-Analytics, 10Wikipedia-Android-App-Backlog (Android-app-release-v2.7.31x-P-Pryanik): Remove MobileWikiAppProtectedEditAttempt schema from Android app - https://phabricator.wikimedia.org/T254567 (10Charlotte) [12:56:10] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Cancel EU SWAT window on Tuesdays when train is run from the EU - https://phabricator.wikimedia.org/T254745 (10LarsWirzenius) [12:56:21] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Cancel EU SWAT window on Tuesdays when train is run from the EU - https://phabricator.wikimedia.org/T254745 (10LarsWirzenius) p:05Triage→03High [15:02:02] (03PS1) 10Thiemo Kreuz (WMDE): [WIP] Add RedundantVarNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603493 [15:04:18] (03PS2) 10Thiemo Kreuz (WMDE): [WIP] Add RedundantVarNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603493 [15:04:23] 10Beta-Cluster-Infrastructure, 10MediaWiki-Core-Testing, 10MediaWiki-User-login-and-signup, 10MediaWiki-extensions-CentralAuth, and 3 others: Login to at least en.wikipedia.beta.wmflabs.org and commons.wikimedia.beta.wmflabs.org sometimes fails with `There seems to... - https://phabricator.wikimedia.org/T243123 [15:05:06] (03PS1) 10Thiemo Kreuz (WMDE): [WIP] Add ClassLevelDocumentationSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603495 [15:06:01] (03PS2) 10Thiemo Kreuz (WMDE): [WIP] Add ClassLevelDocumentationSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603495 [15:10:01] (03PS1) 10Thiemo Kreuz (WMDE): [WIP] Add FullQualifiedClassNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603496 [15:11:58] (03PS2) 10Thiemo Kreuz (WMDE): [WIP] Add FullQualifiedClassNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603496 [15:12:56] Is deployment-logstash2 maybe hung or crashed or something? Is it ok for me to reboot it? [15:20:43] 10Beta-Cluster-Infrastructure, 10MediaWiki-Core-Testing, 10MediaWiki-User-login-and-signup, 10MediaWiki-extensions-CentralAuth, and 3 others: Login to at least en.wikipedia.beta.wmflabs.org and commons.wikimedia.beta.wmflabs.org sometimes fails with `There seems to... - https://phabricator.wikimedia.org/T243123 [15:21:18] (03PS4) 10Jforrester: jjb: [WIP] Rename and simplify all the quibble jobs into one set [integration/config] - 10https://gerrit.wikimedia.org/r/596668 [15:22:02] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add RedundantVarNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603493 (owner: 10Thiemo Kreuz (WMDE)) [15:22:37] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add ClassLevelDocumentationSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603495 (owner: 10Thiemo Kreuz (WMDE)) [15:23:27] (03PS1) 10Jforrester: Add Adham Khatean to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/603513 [15:24:22] (03CR) 10Tchanders: [C: 03+1] Add Adham Khatean to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/603513 (owner: 10Jforrester) [15:24:38] (03CR) 10Jforrester: [C: 03+2] Add Adham Khatean to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/603513 (owner: 10Jforrester) [15:25:44] (03Merged) 10jenkins-bot: Add Adham Khatean to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/603513 (owner: 10Jforrester) [15:26:09] !log Zuul: Add Adham Khatean to CI allow list [15:26:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:26:34] 10Beta-Cluster-Infrastructure, 10Puppet: puppetdb on deployment-puppetdb03 keeps getting OOMKilled - https://phabricator.wikimedia.org/T248041 (10Mholloway) This happened again over the weekend. I've restarted it. [15:29:09] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add FullQualifiedClassNameSniff from WikibaseCodeSniffer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/603496 (owner: 10Thiemo Kreuz (WMDE)) [15:51:50] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Scap: Prepare Scap release 1.15.0 with current changes - https://phabricator.wikimedia.org/T253996 (10LarsWirzenius) That's 3.15.0. [16:04:10] (03CR) 10Awight: [C: 03+2] "There's a comment here saying that setting the environment variable `CI=true` will prevent truncation, https://github.com/pytest-dev/pytes" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602746 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:22:39] 10Release-Engineering-Team-TODO, 10Developer-Advocacy (Apr-Jun 2020), 10Documentation: Wikimedia Technical Talks: April 2020 - Basic of cryptography using OpenPGP and GnuPG - https://phabricator.wikimedia.org/T244813 (10srodlund) 05Open→03Resolved [16:22:53] (03PS1) 10Awight: [squash] mock getcwd [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 [16:23:40] (03CR) 10Awight: [C: 03+2] "> Seems I have to mock the working directory as well. The test fails" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602776 (https://phabricator.wikimedia.org/T211702) (owner: 10Hashar) [16:23:52] (03CR) 10jerkins-bot: [V: 04-1] [squash] mock getcwd [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 (owner: 10Awight) [16:24:19] (03PS1) 10Jforrester: Rename SWAT deploys to backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/603533 [16:25:06] (03CR) 10Awight: [C: 03+2] "Note to other reviewers: I had to remove my `.tox` directory before running locally." [integration/quibble] - 10https://gerrit.wikimedia.org/r/602746 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:25:20] James_F: why backport window? You can have config patches in it? [16:25:27] (03CR) 10Thcipriani: "recheck" [blubber] - 10https://gerrit.wikimedia.org/r/593315 (https://phabricator.wikimedia.org/T248927) (owner: 10DC Slagel) [16:28:16] (03CR) 10Awight: [C: 03+2] "Surprising that we got away with this until now, "assertEquals" has been deprecated for long enough to be included in the python 2to3 fixu" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602747 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:28:38] (03CR) 10Awight: [C: 03+2] tests: no more need to ignore test.py [integration/quibble] - 10https://gerrit.wikimedia.org/r/602743 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:29:03] RhinosF1: https://wikitech.wikimedia.org/wiki/Talk:Backport_windows#Demilitarize_and_rebrand%3F [16:29:46] (03Merged) 10jenkins-bot: tests: no more need to ignore test.py [integration/quibble] - 10https://gerrit.wikimedia.org/r/602743 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:29:48] (03Merged) 10jenkins-bot: tests: singularize assertEquals to avoid warning [integration/quibble] - 10https://gerrit.wikimedia.org/r/602747 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:29:51] Majavah: ah, for me backport window implies no config changes, just backports. [16:29:55] greg-g: ^ [16:30:29] RhinosF1: Tough. [16:30:40] We've spent far too long bikeshedding. [16:31:32] James_F: Backport & Config Window? [16:31:45] Backport & Config Changes Window [16:32:31] (03CR) 10Awight: [C: 03+2] tests: use coverage instead of nose coverage plugin [integration/quibble] - 10https://gerrit.wikimedia.org/r/602752 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:33:22] (03Merged) 10jenkins-bot: tests: use coverage instead of nose coverage plugin [integration/quibble] - 10https://gerrit.wikimedia.org/r/602752 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:33:28] (03Merged) 10jenkins-bot: Migrate from nose to pytest [integration/quibble] - 10https://gerrit.wikimedia.org/r/602746 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [16:33:30] (03CR) 10jerkins-bot: [V: 04-1] High level testing for the execution plan [integration/quibble] - 10https://gerrit.wikimedia.org/r/602776 (https://phabricator.wikimedia.org/T211702) (owner: 10Hashar) [16:35:18] (03PS2) 10Greg Grossmeier: Rename SWAT deploys to backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/603533 (owner: 10Jforrester) [16:36:01] Maybe deploy window? [16:36:12] srly I'd stick with SWAT, what's wrong with it? [16:36:21] I get the reasons [16:36:30] I do not [16:36:37] (03CR) 10Greg Grossmeier: [C: 03+1] Rename SWAT deploys to backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/603533 (owner: 10Jforrester) [16:36:46] (03PS2) 10Awight: [squash] mock getcwd [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 [16:36:49] But I feel like the name excludes Config windows [16:37:06] greg-g: what about "Backport & Config Deployment Window" [16:37:18] Or something that mentions Config [16:37:30] (03CR) 10jerkins-bot: [V: 04-1] [squash] mock getcwd [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 (owner: 10Awight) [16:37:32] Configuration and backports - EU/US/Evening [16:37:35] BCDW will eventually learn to flow I think [16:37:47] Or what hauskatze said sounds good [16:37:52] And CAB is a nice name [16:38:05] CAB is WP:RFA at eswiki :P [16:38:08] nasty place [16:38:12] * hauskatze hides [16:39:05] I never plan an RfA on any Wikipedia [16:39:09] Too evil [16:41:01] (03CR) 10Jforrester: "Leaving to… Tyler?… to merge, as manager." [tools/release] - 10https://gerrit.wikimedia.org/r/603533 (owner: 10Jforrester) [16:41:20] Use the Special:Random naming method [16:43:53] (03PS3) 10Awight: [squash] mock remaining context [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 [16:44:22] (03CR) 10Thcipriani: [C: 03+2] Rename SWAT deploys to backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/603533 (owner: 10Jforrester) [16:44:59] James_F: woops, edit conflict on the phab badge, I liked your description better :) [16:45:00] Thanks, thcipriani. [16:45:23] greg-g: Clearly we need to re-implement Phabricator on top of MediaWiki to get proper edit conflict resolution. ;-) [16:45:25] that's what I get for forgetting my sausages on the burner and being interrupted by the smoke detector ;) [16:45:30] Oops. [16:47:08] LOL [16:47:24] Sausages brulé [16:47:40] What did I burn the other day? [16:48:01] Stash of Benjamins? [16:48:07] Oh I managed to dodgy battered chicken, it wasn't burnt but stuck to the tray [16:48:24] Breaded not batter [16:48:27] That's the word [16:49:10] A lot of breaded stuff came off [16:49:25] "battered chicken" sounded too violent :P [16:50:03] hauskatze: it does [16:50:26] Battered cushion will be what I'm sleeping on by the end of today [16:52:58] (03Merged) 10jenkins-bot: Rename SWAT deploys to backport windows [tools/release] - 10https://gerrit.wikimedia.org/r/603533 (owner: 10Jforrester) [16:56:04] Hi Zppix [16:56:30] Hi? [16:56:41] Zppix: see your bot channel [16:57:43] awight: thank you, deeply. [16:58:00] awight: https://lists.wikimedia.org/pipermail/wikitech-l/2020-June/093479.html [16:58:09] RhinosF1: hauskatze btw: https://lists.wikimedia.org/pipermail/wikitech-l/2020-June/093479.html :) [16:58:26] greg-g: I will reply [16:58:39] oh? I hope I didn't miss something? [16:58:50] greg-g: Right on! I like the self-documenting name, too. [16:59:08] greg-g: backport and config sounds good I think, but SWAT sounded okay to me nonetheless [16:59:53] greg-g: I've dug up a few nice hand-pushed coal hopper photos from commons, if we still want to play with the train metaphor, at least visually. [16:59:59] anyway, instead of SWAT lets have a BACOW [17:00:21] greg-g: we were debating name [17:00:37] how is "BCSW" exactly pronounced? [17:00:48] RhinosF1: We were not. We had decided, and you thought otherwise. That's not a debate. That's dissent. [17:00:50] I see, well, the name change has happened, discussion was on wiki, announcement is out. Thanks for the lp. [17:00:59] Backport And COnfig Window [17:01:00] s/lp/help/ [17:01:07] For there to be debate, it has to involve more than one party. :-) [17:01:20] hauskatze + me [17:01:30] But maybe debate wrong word [17:01:32] See the endlessly toxic "debate me" culture. [17:01:47] I just arrived, I don't care about the name really [17:01:57] I am just allergic to sudden changes :P [17:02:01] haha "putting the prod in production" indeed ;-) [17:02:47] Greg's email says Backport and config [17:02:59] Whereas the patch says Backport only [17:03:13] RhinosF1: Window titles are intentionally as brief as possible. [17:03:54] Ok [17:04:14] If people get confused by the new name, in a month or so we can think about renaming it again. [17:04:21] lol [17:05:07] Sounds like a plan [17:05:09] James_F: :100%: [17:05:33] speaking of confusing changes, can we rename trains to "forwardports"? :P [17:05:49] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10Product-Analytics, 10Wikipedia-Android-App-Backlog (Android-app-release-v2.7.31x-P-Pryanik): Remove MobileWikiAppProtectedEditAttempt schema from Android app - https://phabricator.wikimedia.org/T254567 (10Charlotte) [17:06:09] Majavah: or "panic fests"... [17:07:26] or 'geez I broke the wikis again' [17:09:15] lol [17:09:25] thought SWAT meant Scapping Without A Train [17:09:45] also has other meanings too :) [17:10:40] mutante: :rofl: we missed an entire thread of vagrant possibilities, "riding the boxcars", etc. [17:10:56] * Lucas_WMDE reads backlog for the rename [17:11:10] Lucas_WMDE: https://lists.wikimedia.org/pipermail/wikitech-l/2020-June/093479.html [17:11:25] yes, that’s why I came here :) [17:12:19] someone should probably update the logs link in the topic to https://wm-bot.wmflabs.org/logs/%23wikimedia-releng/ [17:12:28] awight: :) https://www.goodreads.com/en/book/show/104440.Hopping_Freight_Trains_In_America [17:12:43] ok, it looks like nobody has asked this yet – what do we !log now instead of "SWAT: [[gerrit:ID|Subject]]"? [17:12:56] (so that I can update https://phabricator.wikimedia.org/P8845) [17:13:10] Lucas_WMDE: "Backport: foo" [17:13:15] just saying backport seems a bit ambiguous there [17:13:29] Also almost no-one actually follows the log structure we encourage anywayu. [17:13:58] though it's a SAL entry, so a deployment is probably assumed over the other actions described by "backport" [17:14:00] always, or "Config: foo" for config changes? [17:14:18] (meh, my client didn’t scroll down after the ping) [17:14:20] mutante: Thanks! I came across "Boxcar Bertha" in a library once, need to get a hold of that again, it's a beautiful story. [17:14:48] James_F: maybe we need to advertise that script more? it makes it much easier to follow the log structure imo :) [17:15:15] Eh. [17:15:21] awight: once found that at Aardvark books on Church St. S.F. .. the store is gone now :/ [17:15:22] I assume it should be possible to make that script spit out "Backport:" or "Config:" depending on which repo the change targets [17:15:45] Probably. [17:15:47] 10Beta-Cluster-Infrastructure, 10SRE-tools: SSH host key for deployment-imagescaler01.deployment-prep.eqiad.wmflabs changed? - https://phabricator.wikimedia.org/T254786 (10LarsWirzenius) [17:16:47] 10Beta-Cluster-Infrastructure, 10Analytics, 10Analytics-EventLogging, 10Product-Analytics, 10Wikipedia-Android-App-Backlog (Android-app-release-v2.7.31x-P-Pryanik): Remove MobileWikiAppProtectedEditAttempt schema from Android app - https://phabricator.wikimedia.org/T254567 (10Dbrant) a:03Dbrant [17:18:16] Lucas_WMDE: I've also been wondering about the deployment tools in limbo, maybe we should try to make that official and provision to the deployment servers? [17:18:24] 10Gerrit, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10ci-test-error (WMF-deployed Build Failure): Jenkins job failing intermittently due to Gerrit HTTP 502 errors when interacting with repos - https://phabricator.wikimedia.org/T246763 (10Krinkle) Still seen, including r... [17:19:58] Project mediawiki-core-doxygen-docker build #16467: 04FAILURE in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/16467/ [17:20:45] 10Scap: Rename `scap swat` to `scap backport` - https://phabricator.wikimedia.org/T254787 (10greg) [17:22:08] 10Scap: Rename `scap swat` to `scap backport` - https://phabricator.wikimedia.org/T254787 (10LarsWirzenius) a:03LarsWirzenius [17:22:27] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Scap: Rename `scap swat` to `scap backport` - https://phabricator.wikimedia.org/T254787 (10LarsWirzenius) [17:23:43] As a side-effect, TIL that this scap plugin existed! [17:23:59] Yeah, we need to do a better job of scap plugin consolidation. [17:25:49] huh, TIL as well. what’s it do? [17:25:52] * Lucas_WMDE looks for documentation [17:25:59] updated the script at https://phabricator.wikimedia.org/P8845 btw [17:27:59] hm, from the usage examples it sounds like it’s more for Gerrit interaction than doing the actual deployment [17:28:11] Lucas_WMDE: which lead me to rename that project it was tagged in, thanks :) [17:28:26] \o/ [17:31:30] greg-g: Thanks again for being motivated and making this happen--as a person who still has nightmares about LAPD phalanxes, pepper spray and so on, I really appreciate this! Also, topical. [17:32:34] It's funny how you don't realise some of these until it's pointed out :/ [17:36:04] so that means that the backport window I used to remove a “blacklist” from the production config was also the last window to bear the SWAT name [17:36:11] not sure what to make of this [17:36:57] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Scap: Rename `scap swat` to `scap backport` - https://phabricator.wikimedia.org/T254787 (10Jdforrester-WMF) To avoid confusin, we should probably wait for the cut-over from the plugin being in operations/mediawiki-config to being in scap proper. [17:37:27] scap swat ain't used I think [17:41:50] awight: <3 <3 [17:43:56] (03CR) 10Jforrester: [C: 03+1] "Confirmed that this builds as expected. Do SRE want it?" [integration/config] - 10https://gerrit.wikimedia.org/r/603175 (https://phabricator.wikimedia.org/T254480) (owner: 10Hashar) [17:49:34] (03CR) 10Jforrester: [C: 03+2] "Taking Jbond's +1 to mean that. :-)" [integration/config] - 10https://gerrit.wikimedia.org/r/603175 (https://phabricator.wikimedia.org/T254480) (owner: 10Hashar) [17:50:06] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments: 1.35.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T254173 (10DannyS712) Patch going out with the train (hasn't merged yet, but has a +2 and is just waiting for Jenkins): https://gerrit.wikimedia.org... [17:50:31] (03Merged) 10jenkins-bot: docker: use more recent shellcheck [integration/config] - 10https://gerrit.wikimedia.org/r/603175 (https://phabricator.wikimedia.org/T254480) (owner: 10Hashar) [17:53:18] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10MediaWiki-backport-deployments: Cancel EU SWAT window on Tuesdays when train is run from the EU - https://phabricator.wikimedia.org/T254745 (10MarcoAurelio) [17:55:39] !log Docker: Publishing operations-puppet image, now with upgraded shellcheck T254480 [17:55:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:55:41] T254480: Shell/Python/other scripts should not be generated by ERB files; dynamic parts should be a simple ERB config file - https://phabricator.wikimedia.org/T254480 [17:58:46] 10Beta-Cluster-Infrastructure, 10MediaWiki-Core-Testing, 10MediaWiki-User-login-and-signup, 10MediaWiki-extensions-CentralAuth, and 3 others: Login to at least en.wikipedia.beta.wmflabs.org and commons.wikimedia.beta.wmflabs.org sometimes fails with `There seems to... - https://phabricator.wikimedia.org/T243123 [18:14:08] (03PS1) 10Jforrester: jjb: Upgrade puppet jobs to new shellcheck version [integration/config] - 10https://gerrit.wikimedia.org/r/603563 (https://phabricator.wikimedia.org/T254480) [18:16:25] Yippee, build fixed! [18:16:25] Project mediawiki-core-doxygen-docker build #16468: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/16468/ [18:19:15] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/603563 (https://phabricator.wikimedia.org/T254480) (owner: 10Jforrester) [18:20:17] (03Merged) 10jenkins-bot: jjb: Upgrade puppet jobs to new shellcheck version [integration/config] - 10https://gerrit.wikimedia.org/r/603563 (https://phabricator.wikimedia.org/T254480) (owner: 10Jforrester) [18:22:50] Cannot reach logstash-beta [18:22:56] Loads infinitely [18:23:20] same for kibana4 (alias) [18:23:40] Someone mentioned about one of the hosts being seemingly inaccessible [18:23:57] [16:12:56] Is deployment-logstash2 maybe hung or crashed or something? Is it ok for me to reboot it? [18:25:00] I'll reboot it then, and see if that fixes the issue [18:25:12] seems to still be dead, can't SSH to it [18:25:49] thx [18:26:27] Reedy: there are deployment-logstash2 and 3; which one or both? [18:26:42] 3 isn't dead [18:26:48] So I don't see any reason to reboot it [18:26:49] logstash2 is debian-8.0-jessie (deprecated 2015-06-13) [18:26:51] heh [18:26:56] we should reimage that one [18:27:06] seems likely that 03 is meant to replace 2 [18:27:52] !log Deployment-prep: Rebooting deployment-logstash2, looks hung and cannot be reached [18:27:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:27:56] (03CR) 10Jforrester: [C: 03+2] [AdminLinks] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/602830 (owner: 10Umherirrender) [18:28:41] 03 is also running on old software [18:29:11] Yes. [18:29:15] (03Merged) 10jenkins-bot: [AdminLinks] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/602830 (owner: 10Umherirrender) [18:29:34] I'm vaguely remembering that logstash2 has never worked and 03 on the old software is the 'real' one? [18:29:56] !log Zuul: [mediawiki/extensions/AdminLinks] Add phan [18:29:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:30:18] Reedy / andrewbogott -- reboot complete but the web is still not working [18:30:27] this exceeds my competences I guess [18:30:51] the host doesn't seem to be up still anywat [18:31:28] 504 Gateway Time-out [18:31:29] nginx/1.13.6 [18:32:49] maurelio@deployment-logstash2.deployment-prep.eqiad.wmflabs: Permission denied (publickey). [18:32:56] cannot ssh either [18:37:04] James_F may be right - I see I restarted logstash03 once too [18:39:49] 10Phabricator, 10Developer-Advocacy (Jul-Sep 2020), 10Epic: (Semi)automatically close Phabricator tickets with status "stalled" after a while - https://phabricator.wikimedia.org/T252522 (10Ramsey-WMF) [18:44:56] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta issues - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) [18:45:08] ^ [18:46:04] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta issues - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) [18:47:55] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta issues - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) [18:48:26] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) [18:48:34] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) p:05Triage→03High [18:50:20] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10Krenair) It's supposed to be on 03 but I think it got moved back to 2 at some point. What's outdated about 03? It was set up to replace 2 because that was runni... [18:53:45] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10MarcoAurelio) Looked outdated to me: * logstash2 is running on debian-8.0-jessie (deprecated 2015-06-13) * logstash03 is running on debian-9.8-stretch (deprecat... [19:07:34] (03CR) 10Jeena Huneidi: [C: 03+2] Blubber/Blubberoid: mv expansion before verifying [blubber] - 10https://gerrit.wikimedia.org/r/593315 (https://phabricator.wikimedia.org/T248927) (owner: 10DC Slagel) [19:09:41] (03CR) 10jerkins-bot: [V: 04-1] Blubber/Blubberoid: mv expansion before verifying [blubber] - 10https://gerrit.wikimedia.org/r/593315 (https://phabricator.wikimedia.org/T248927) (owner: 10DC Slagel) [19:23:27] (03CR) 10Hashar: "> Surprising that we got away with this until now, "assertEquals" has been deprecated for long enough to be included in the python 2to3 fi" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602747 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [19:28:37] (03CR) 10Hashar: "> There's a comment here saying that setting the environment variable `CI=true` will prevent truncation, https://github.com/pytest-dev/pyt" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602746 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [19:28:46] 10Project-Admins: create subproject? (or tag?) for Analytics-Radar - https://phabricator.wikimedia.org/T254806 (10Nuria) [19:30:12] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Quibble, 10Technical-Debt: Migrate Quibble from nose to pytest - https://phabricator.wikimedia.org/T254610 (10hashar) 05Open→03Resolved a:03hashar The patch series has been reviewed and ap... [19:35:25] 10Beta-Cluster-Infrastructure, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10Krenair) An image is used to create the VM in the first place, once that's done we just keep it updated. If we replaced instances because they were based on imag... [19:41:06] (03CR) 10Hashar: "mocking quibble.is_in_docker() to return true makes it consistent indeed and that sounds like the easiest path." (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/603532 (owner: 10Awight) [19:45:24] 10Gerrit: PHP libraries as Gerrit top-level projects - https://phabricator.wikimedia.org/T125031 (10hashar) [19:46:34] (03CR) 10Awight: [C: 03+2] "> > There's a comment here saying that setting the environment" [integration/quibble] - 10https://gerrit.wikimedia.org/r/602746 (https://phabricator.wikimedia.org/T254610) (owner: 10Hashar) [20:00:13] 10Continuous-Integration-Infrastructure, 10Browser-Tests: Split out all selenium test runs into a buster-based job that runs newer Chromium - https://phabricator.wikimedia.org/T250513 (10Jdforrester-WMF) I'm thinking that rather than adding the buster variants to the already complex naming system for quibble j... [20:01:36] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Rename all our quibble jobs to make some sense - https://phabricator.wikimedia.org/T254814 (10Jdforrester-WMF) [20:01:44] (03PS5) 10Jforrester: jjb: [WIP] Rename and simplify all the quibble jobs into one set [integration/config] - 10https://gerrit.wikimedia.org/r/596668 (https://phabricator.wikimedia.org/T254814) [20:12:16] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release Pipeline: Puppetize Helm init on contint1001 - https://phabricator.wikimedia.org/T254817 (10jeena) [20:25:47] (03CR) 10Jeena Huneidi: [C: 03+2] Blubber/Blubberoid: mv expansion before verifying [blubber] - 10https://gerrit.wikimedia.org/r/593315 (https://phabricator.wikimedia.org/T248927) (owner: 10DC Slagel) [20:27:17] (03PS1) 10Umherirrender: [PageNotice] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603598 [20:28:40] (03Merged) 10jenkins-bot: Blubber/Blubberoid: mv expansion before verifying [blubber] - 10https://gerrit.wikimedia.org/r/593315 (https://phabricator.wikimedia.org/T248927) (owner: 10DC Slagel) [20:31:21] (03CR) 10Jforrester: [C: 03+2] "Nice; see I4778da40e75 for the whole list. ;-)" [integration/config] - 10https://gerrit.wikimedia.org/r/603598 (owner: 10Umherirrender) [20:32:23] (03Merged) 10jenkins-bot: [PageNotice] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603598 (owner: 10Umherirrender) [20:32:50] (03PS1) 10Umherirrender: [RelatedSites] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603601 [20:33:07] !log Zuul: [mediawiki/extensions/PageNotice] Replace seccheck with phan [20:33:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:38:20] (03PS1) 10Umherirrender: [DisableAccount] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603603 [20:40:14] (03CR) 10Jforrester: [C: 03+2] [RelatedSites] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603601 (owner: 10Umherirrender) [20:41:08] (03Merged) 10jenkins-bot: [RelatedSites] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603601 (owner: 10Umherirrender) [20:42:04] !log Zuul: [mediawiki/extensions/RelatedSites] Replace seccheck with phan [20:42:04] (03PS1) 10Umherirrender: [MassEditRegex] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603605 [20:42:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:44:22] (03CR) 10Umherirrender: "Some other extensions have (new) issues and needs some more investigation - work for the next days ;-)" [integration/config] - 10https://gerrit.wikimedia.org/r/603598 (owner: 10Umherirrender) [20:44:51] (03CR) 10Jforrester: [C: 03+2] [DisableAccount] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603603 (owner: 10Umherirrender) [20:45:49] (03Merged) 10jenkins-bot: [DisableAccount] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603603 (owner: 10Umherirrender) [20:46:01] !log Zuul: [mediawiki/extensions/DisableAccount] Replace seccheck with phan [20:46:02] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:46:08] (03CR) 10Jforrester: [C: 03+2] [MassEditRegex] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603605 (owner: 10Umherirrender) [20:47:21] (03Merged) 10jenkins-bot: [MassEditRegex] Replace seccheck by phan job [integration/config] - 10https://gerrit.wikimedia.org/r/603605 (owner: 10Umherirrender) [20:47:24] !log Zuul: [mediawiki/extensions/MassEditRegex] Replace seccheck with phan [20:47:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:47:51] (03PS4) 10Jforrester: layout: Replace extension-seccheck with extension-phan where used [integration/config] - 10https://gerrit.wikimedia.org/r/602482 (https://phabricator.wikimedia.org/T254152) [20:48:43] (03PS5) 10Jforrester: layout: Replace extension-seccheck with extension-phan where used [integration/config] - 10https://gerrit.wikimedia.org/r/602482 (https://phabricator.wikimedia.org/T254152) [20:49:26] (03PS2) 10Jeena Huneidi: Use ExpandIncludesAndCopies in tests [blubber] - 10https://gerrit.wikimedia.org/r/602517 [20:54:31] 10Beta-Cluster-Infrastructure, 10SRE-tools: SSH host key for deployment-imagescaler01.deployment-prep.eqiad.wmflabs changed? - https://phabricator.wikimedia.org/T254786 (10thcipriani) > I'm trying to do a Scap release, and as part of that running cumin on deployment-deploy01, but this fails. Should be running... [21:24:05] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review: Rename all our quibble jobs to make some sense - https://phabricator.wikimedia.org/T254814 (10hashar) [21:24:41] some sense? [21:24:45] not complete sense? [21:26:03] There shall be no sanity under my watch :P [21:29:55] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments: 1.35.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T254173 (10DannyS712) [21:30:16] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments: 1.35.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T254173 (10Urbanecm) >>! In T254173#6197132, @Pchelolo wrote: > Something to be cautious about: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/54... [21:30:27] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) [21:31:15] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10brennen) [21:35:56] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments: 1.35.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T254173 (10Pchelolo) In case reverts need to be done, https://gerrit.wikimedia.org/r/c/mediawiki/core/+/602695 was merged on top of the problematic... [21:37:11] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review: Rename all our quibble jobs to make some sense - https://phabricator.wikimedia.org/T254814 (10Krinkle) [21:51:29] 10Diffusion, 10Gerrit: rPHAB to gerrit mirror stuck at 447032e1d - https://phabricator.wikimedia.org/T213512 (10bd808) 05Open→03Resolved Seems to be magically fixed. [22:06:39] Magical fixes are the best fixes. [22:52:55] 10Beta-Cluster-Infrastructure: SSH host key for deployment-imagescaler01.deployment-prep.eqiad.wmflabs changed? - https://phabricator.wikimedia.org/T254786 (10Volans) Few days ago the global WMCS cumin public key was rotated (see [1] and [2]). This per-se has nothing to do with the cumin installation in deployme... [23:17:02] Project mediawiki-core-doxygen-docker build #16473: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/16473/