[02:35:22] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) @DannyS712 I've added you to the list of users able to access Beta Cluster in WMCS, whic... [02:55:58] !log It appears logstash-beta.wmflabs.org is down/unresponsive [02:55:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [04:34:41] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Update k8s migration tutorial to use codesample template - https://phabricator.wikimedia.org/T253772 (10jeena) 05Open→03Resolved I used the syntaxhighlight tag instead, because I couldn't get the Codesample template to work on some of my yaml text. [04:38:19] 10Gerrit, 10Developer-Advocacy, 10Upstream: Gitiles should add username in git clone links, triggers "Permission denied (publickey)" error - https://phabricator.wikimedia.org/T189182 (10Aklapper) [04:38:24] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Aklapper) [04:38:25] 10Gerrit, 10Upstream: Clone command specified in gittiles doesn't work in most cases - https://phabricator.wikimedia.org/T255453 (10Aklapper) [04:47:24] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE): Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10Aklapper) @Vidhi-Mody: What's the output of `git config --get remote.origin.url` in your local checkout? Which origin branch did you use for cloning the... [05:27:25] 10Gerrit, 10Developer-Advocacy, 10Upstream: Gitiles should add username in git clone links, triggers "Permission denied (publickey)" error - https://phabricator.wikimedia.org/T189182 (10hashar) p:05Triage→03Low [06:05:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Zuul: Zuul deployment fails due to unsupported wheel - https://phabricator.wikimedia.org/T255424 (10hashar) 05Open→03Resolved [06:05:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Zuul, and 2 others: pywikibot get merge rejections due to zuul-merger not being able to update tags - https://phabricator.wikimedia.org/T252310 (10hashar) [06:06:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Zuul, and 2 others: pywikibot get merge rejections due to zuul-merger not being able to update tags - https://phabricator.wikimedia.org/T252310 (10hashar... [06:08:26] hashar: Want to deploy https://gerrit.wikimedia.org/r/c/integration/zuul/+/594527/ too? :-) [06:09:01] (03PS2) 10Jforrester: Squelch ref-replication gerrit warnings [integration/zuul] (patch-queue/debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/594526 [06:09:50] (03PS2) 10Jforrester: Add entry for ref-replication-scheduled event [integration/zuul] (patch-queue/debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/594527 [06:49:58] (03Abandoned) 10Hashar: Port test_integration to pytest [integration/config] - 10https://gerrit.wikimedia.org/r/525476 (owner: 10Legoktm) [06:50:00] (03Abandoned) 10Hashar: Port test_zuul_layout to pytest [integration/config] - 10https://gerrit.wikimedia.org/r/525477 (owner: 10Legoktm) [06:50:03] (03Abandoned) 10Hashar: Port test_zuul_parameters to pytest [integration/config] - 10https://gerrit.wikimedia.org/r/525478 (owner: 10Legoktm) [06:52:53] (03PS2) 10Hashar: puppet-tests: update mtail to 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:53:07] (03CR) 10Hashar: "added a changelog entry" [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:53:09] (03CR) 10jerkins-bot: [V: 04-1] puppet-tests: update mtail to 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:56:27] (03PS3) 10Hashar: puppet-tests: update mtail to 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:56:35] (03CR) 10Hashar: puppet-tests: update mtail to 3.0.0~rc35 (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:56:47] (03CR) 10Hashar: [C: 03+2] puppet-tests: update mtail to 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [06:57:01] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: CRITICAL: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found)deployment-prep.deployment-snapshot01.diskspace.root.byte_percentfree (<10.00%) [06:57:43] (03Merged) 10jenkins-bot: puppet-tests: update mtail to 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/599365 (https://phabricator.wikimedia.org/T251466) (owner: 10Cwhite) [07:01:59] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [07:04:14] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10observability: Extract mtail puppet.git tests to a standalone CI container - https://phabricator.wikimedia.org/T255534 (10hashar) [07:07:01] (03PS1) 10Hashar: jjb: update for mtail 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/605813 (https://phabricator.wikimedia.org/T251466) [07:18:04] liw brennen|afk hey! quick question about this week's train... Looks like the wmf.37 branch is already cut? [07:18:34] can I get this extension update on the train still? https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/605810 [07:20:01] AndyRussG, yes, branch cut happens at 02 UTC automatically now [07:20:15] AndyRussG, so I'm afraid it's too late now [07:21:06] liw: oh hmmm yeah it used to be fine so long as I sent things in by Monday evening (my local time) [07:21:19] liw: so, no way it can still be added, by any chance? [07:21:32] AndyRussG, sorry, no. [07:24:11] (03CR) 10Hashar: [C: 03+2] jjb: update for mtail 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/605813 (https://phabricator.wikimedia.org/T251466) (owner: 10Hashar) [07:25:20] (03Merged) 10jenkins-bot: jjb: update for mtail 3.0.0~rc35 [integration/config] - 10https://gerrit.wikimedia.org/r/605813 (https://phabricator.wikimedia.org/T251466) (owner: 10Hashar) [07:25:35] liw: not even if I do the submodule update myself? [07:27:03] AndyRussG, sorry, no; unless it's an urgen unbreak-now type of issue, it'll have to wait for next week, I fear [07:27:44] liw: ok no worries then. No it's not unbreak-now [07:30:08] thanks anyway and apologies for the bother [07:31:08] no problem [07:53:22] AndyRussG: good news, your patch has an extra week of scrutinity on other developers installations / beta cluster etc [07:53:43] then CentralNotice is an odd beast isn't it deployed in its own window? [07:58:04] No, just cut differently. [08:20:50] (03PS3) 10Hashar: Fail when repository lacks gate-and-submit [integration/config] - 10https://gerrit.wikimedia.org/r/594507 [08:21:09] (03PS1) 10Hashar: zuul: introduce debian-glue job template [integration/config] - 10https://gerrit.wikimedia.org/r/605831 [08:21:11] (03PS1) 10Hashar: zuul: introduce debian-glue-non-voting job template [integration/config] - 10https://gerrit.wikimedia.org/r/605832 [08:21:13] (03PS1) 10Hashar: utils/zuul-layout-diffs.sh: dump invalid layout [integration/config] - 10https://gerrit.wikimedia.org/r/605833 [08:21:21] moare magic [08:21:35] (03CR) 10jerkins-bot: [V: 04-1] Fail when repository lacks gate-and-submit [integration/config] - 10https://gerrit.wikimedia.org/r/594507 (owner: 10Hashar) [08:21:55] (03CR) 10jerkins-bot: [V: 04-1] zuul: introduce debian-glue-non-voting job template [integration/config] - 10https://gerrit.wikimedia.org/r/605832 (owner: 10Hashar) [08:23:15] hashar okok yes it's fine... Yeah CentralNotice is different, this is stuff that's been in master and on the beta cluster for a while. I just merged it from master to the CN deploy branch [08:24:18] anyway apologies again for having been insistent and thanks as always for your work on this stuff [08:25:49] (heading to sleep now but I'll still see any pings in the bsckscroll tomorrowzzzz) [08:27:04] (03PS2) 10Hashar: zuul: introduce debian-glue job template [integration/config] - 10https://gerrit.wikimedia.org/r/605831 [08:27:06] (03PS2) 10Hashar: zuul: introduce debian-glue-non-voting job template [integration/config] - 10https://gerrit.wikimedia.org/r/605832 [08:27:08] (03PS4) 10Hashar: Fail when repository lacks gate-and-submit [integration/config] - 10https://gerrit.wikimedia.org/r/594507 [08:28:12] (03CR) 10jerkins-bot: [V: 04-1] Fail when repository lacks gate-and-submit [integration/config] - 10https://gerrit.wikimedia.org/r/594507 (owner: 10Hashar) [08:43:22] bah [08:48:57] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10LarsWirzenius) Branch cut happened automatically. Merged the change in Gerrit manually (V+1 and C... [08:59:53] hashar: Just ignore repos starting with `operations/` when looking for G&S? [09:00:51] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE), 10User-zeljkofilipin: Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10zeljkofilipin) p:05Triage→03Medium [09:09:37] (03CR) 10Hashar: Replace fabfile.py with a shell script (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/605434 (https://phabricator.wikimedia.org/T236689) (owner: 10Krinkle) [09:09:45] (03CR) 10Hashar: "Looks good, there is some issue with local=$(command) which would shallow any error from command." [integration/config] - 10https://gerrit.wikimedia.org/r/605434 (https://phabricator.wikimedia.org/T236689) (owner: 10Krinkle) [09:11:07] James_F: possibly. I think I wrote that failing patch for a task that asked for jenkins-bot to vote Verified+1 on test [09:11:14] to prevent accidental submit [09:11:32] which means we need gate and submit on all repo to grant the v+2 vote bah [09:11:41] Eh. [09:11:57] there are a few oddities as well. Some repositories have debian-glue in "test" but do not have it in "gate-and-submit" [09:12:04] possibly cause the repo had: [09:12:04] That sounds fine for the mediawiki/ repos, but the operations/ ones are habitually self-reviewed-self-merged by SRE. [09:12:10] test: debian-glue-non-voting [09:12:14] Ack. [09:12:20] and we just dorpped the suffix and forgot to add the gate-and-submit [09:12:23] The templates for that looked nice. [09:12:30] Let's deploy those now, at least? [09:12:44] changing to a template would make sure the job is added to gate and submit as well when turning voting on [09:12:46] yeah [09:12:56] deployable I guess ;] just pending some additional scrutinity hehe [09:13:19] and maybe I will drop a list to a task and ask SRE wetherh some can just be archived [09:13:26] then work with them got get the packaging to work [09:32:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Zuul, and 2 others: pywikibot get merge rejections due to zuul-merger not being able to update tags - https://phabricator.wikimedia.org/T252310 (10Dvorapa... [09:47:09] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE), 10User-zeljkofilipin: Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10zeljkofilipin) @20after4 is phab-deployment hosted on phabricator and mirrored on gerrit? Should @Vidhi-Mody push to phabricator... [09:51:23] lets do them [09:51:45] (03CR) 10Hashar: [C: 03+2] zuul: introduce debian-glue job template [integration/config] - 10https://gerrit.wikimedia.org/r/605831 (owner: 10Hashar) [09:52:01] (03CR) 10Hashar: [C: 03+2] zuul: introduce debian-glue-non-voting job template [integration/config] - 10https://gerrit.wikimedia.org/r/605832 (owner: 10Hashar) [09:52:46] https://www.vice.com/en_us/article/k7qbyv/github-to-remove-masterslave-terminology-from-its-platform [09:52:50] (03Merged) 10jenkins-bot: zuul: introduce debian-glue job template [integration/config] - 10https://gerrit.wikimedia.org/r/605831 (owner: 10Hashar) [09:52:59] I guess it is time to rename all 'master' branches in gerrit 'main' hehe [09:53:12] (03Merged) 10jenkins-bot: zuul: introduce debian-glue-non-voting job template [integration/config] - 10https://gerrit.wikimedia.org/r/605832 (owner: 10Hashar) [09:54:20] though master in French has a different appeal. It is more like maestro , expert of a skill, a reference [09:55:49] I should do a walk-through of my github repos and fix them up [09:55:57] * apergos adds to todo list [09:56:30] I just hope we are not going to do such change. It is going to technically be a pain in the *** [09:56:36] or at least not rush it [09:57:25] I think we could do it in a way that spreads out the work over time to be manageable [10:23:58] !log Reloading Zuul to deploy I31d8c714e2b5c76d0cd823d22fac2f7bbbf583e5 and I95ea11a5224fc52cde4b03e0adf4673d683332f8 [10:24:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:30:18] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Kask functional testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10akosiaris) >>! In T224041#6225405, @jeena wrote: > I created a helm test... [10:39:58] (03PS1) 10Hashar: zuul: add debian glue to g&s for contenttranslation repo [integration/config] - 10https://gerrit.wikimedia.org/r/605866 [10:41:13] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Tgr) How is this a duplicate? The other task is about SSH URLs, not HTTPS ones. [10:46:54] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10observability: Extract mtail puppet.git tests to a standalone CI container - https://phabricator.wikimedia.org/T255534 (10hashar) p:05Triage→03Medium Maybe including `mtail` is less of an issue than `logstash-fi... [11:44:19] (03PS1) 10Lokal Profil: Temporarily allow force push to clean up upstream history from master to allow from clean merges using the upstream branch going forwards [services/wikispeech/pronlex] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/605882 [11:45:10] (03CR) 10Lokal Profil: [C: 03+2] Temporarily allow force push to clean up upstream history from master to allow from clean merges using the upstream branch going forwards [services/wikispeech/pronlex] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/605882 (owner: 10Lokal Profil) [11:47:57] (03CR) 10Lokal Profil: [V: 03+2 C: 03+2] Temporarily allow force push to clean up upstream history from master to allow from clean merges using the upstream branch going forwards [services/wikispeech/pronlex] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/605882 (owner: 10Lokal Profil) [12:05:57] (03PS1) 10Lokal Profil: Revert "Temporarily allow force push to clean up upstream history from master to allow from clean merges using the upstream branch going forwards" [services/wikispeech/pronlex] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/605884 [12:06:21] (03CR) 10Lokal Profil: [V: 03+2 C: 03+2] Revert "Temporarily allow force push to clean up upstream history from master to allow from clean merges using the upstream branch going for [services/wikispeech/pronlex] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/605884 (owner: 10Lokal Profil) [12:20:33] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Aklapper) @Tgr: Feel free to reopen and clarify in the task summaries [12:25:37] 10Release-Engineering-Team, 10Developer-Advocacy, 10Wikimedia-GitHub: Set up Wikimedia as a recipient of GitHub Sponsors? - https://phabricator.wikimedia.org/T253179 (10Aklapper) [13:13:15] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10LarsWirzenius) Group0 promoted to this week's version. [13:31:10] 10phan-taint-check-plugin, 10MediaWiki-extensions-Translate: Phan blocks merge of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/603471 - https://phabricator.wikimedia.org/T255561 (10Nikerabbit) @Daimona This looks like a false positive to me. How to fix? [13:31:29] 10phan-taint-check-plugin, 10MediaWiki-extensions-Translate: Phan blocks merge of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/603471 - https://phabricator.wikimedia.org/T255561 (10Nikerabbit) [13:41:29] (03PS1) 10Zfilipin: Send e-mail notifications to Elena Tonkovidova for selenium-daily-beta(commons)-MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/605910 [13:42:20] (03CR) 10jerkins-bot: [V: 04-1] Send e-mail notifications to Elena Tonkovidova for selenium-daily-beta(commons)-MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/605910 (owner: 10Zfilipin) [13:57:13] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Tgr) 05duplicate→03Open [13:59:09] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Tgr) [13:59:33] fyi: there seems to be something wrong with CI. For example: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/604026 - new PS 2 hours ago, even recheck was commented and no CI results yet [14:00:00] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10Tgr) [14:03:04] (03PS2) 10Zfilipin: Send e-mail notifications to Elena Tonkovidova for selenium(...)MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/605910 [14:04:38] (03CR) 10Zfilipin: "PS2 updates commit message to fix CI." [integration/config] - 10https://gerrit.wikimedia.org/r/605910 (owner: 10Zfilipin) [14:08:36] 10phan-taint-check-plugin, 10MediaWiki-extensions-Translate: Phan blocks merge of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/603471 - https://phabricator.wikimedia.org/T255561 (10Daimona) a:03Daimona I haven't checked deeply, but it might be a false positive. I'm going to suppress it [14:10:10] 10phan-taint-check-plugin, 10MediaWiki-extensions-Translate: Phan blocks merge of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/603471 - https://phabricator.wikimedia.org/T255561 (10Daimona) a:05Daimona→03None Ah, I see this is caused by that very patch, and it's not already broken on m... [14:16:18] (03CR) 10Hashar: [C: 03+1] Send e-mail notifications to Elena Tonkovidova for selenium(...)MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/605910 (owner: 10Zfilipin) [14:38:03] (03CR) 10Hashar: "I would like to avoid hardcoding the configuration in the Jenkins job or in the container. There might be a way to pass the path to logsta" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/602730 (https://phabricator.wikimedia.org/T222826) (owner: 10Cwhite) [14:38:22] (03CR) 10Hashar: [C: 04-1] add filter_scripts volume mount to logstash-filter-verifier job [integration/config] - 10https://gerrit.wikimedia.org/r/602730 (https://phabricator.wikimedia.org/T222826) (owner: 10Cwhite) [14:54:12] (03CR) 10Cwhite: add filter_scripts volume mount to logstash-filter-verifier job (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/602730 (https://phabricator.wikimedia.org/T222826) (owner: 10Cwhite) [15:05:38] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/605910 (owner: 10Zfilipin) [15:06:11] (03CR) 10Jforrester: [C: 03+1] "This is neat." [integration/config] - 10https://gerrit.wikimedia.org/r/605833 (owner: 10Hashar) [15:06:47] (03Merged) 10jenkins-bot: Send e-mail notifications to Elena Tonkovidova for selenium(...)MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/605910 (owner: 10Zfilipin) [15:11:39] (03PS1) 10Hashar: Entry point for a CI fullrun [integration/quibble] - 10https://gerrit.wikimedia.org/r/605939 (https://phabricator.wikimedia.org/T235118) [15:16:13] (03PS2) 10Hashar: Entry point for a CI fullrun [integration/quibble] - 10https://gerrit.wikimedia.org/r/605939 (https://phabricator.wikimedia.org/T235118) [15:18:19] (03PS4) 10Hashar: Integration job for Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/604785 (https://phabricator.wikimedia.org/T235118) [15:22:00] PROBLEM - Parsoid on deployment-parsoid11 is CRITICAL: connect to address 172.16.1.115 and port 8000: Connection refused [15:23:07] (03PS1) 10JMeybohm: Debian glue for operations/debs/chartmuseum [integration/config] - 10https://gerrit.wikimedia.org/r/605945 (https://phabricator.wikimedia.org/T253843) [15:24:35] (03CR) 10Hashar: "Extracted from the WIP job defined at https://gerrit.wikimedia.org/r/#/c/integration/config/+/604785/3/jjb/integration.yaml" [integration/quibble] - 10https://gerrit.wikimedia.org/r/605939 (https://phabricator.wikimedia.org/T235118) (owner: 10Hashar) [15:25:24] (03CR) 10Hashar: "I moved the shell magic to integration/quibble : https://gerrit.wikimedia.org/r/#/c/integration/quibble/+/605939/" [integration/config] - 10https://gerrit.wikimedia.org/r/604785 (https://phabricator.wikimedia.org/T235118) (owner: 10Hashar) [15:27:02] (03CR) 10Hashar: [C: 03+2] Debian glue for operations/debs/chartmuseum [integration/config] - 10https://gerrit.wikimedia.org/r/605945 (https://phabricator.wikimedia.org/T253843) (owner: 10JMeybohm) [15:28:09] (03Merged) 10jenkins-bot: Debian glue for operations/debs/chartmuseum [integration/config] - 10https://gerrit.wikimedia.org/r/605945 (https://phabricator.wikimedia.org/T253843) (owner: 10JMeybohm) [15:30:52] (03CR) 10Hashar: [C: 03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/605945 (https://phabricator.wikimedia.org/T253843) (owner: 10JMeybohm) [15:39:20] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Operations, 10Patch-For-Review: Migrate contint* hosts to Buster - https://phabricator.wikimedia.org/T224591 (10Dzahn) We went into this issue several times b... [16:02:40] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) Note that Logstash-beta is down, which means there has been less visibility in the state... [16:04:13] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) [16:04:30] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) [16:04:37] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) [16:29:01] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10nnikkhoui) Just adding a quick note, that cpt has moved some mediawiki REST endpoints from /v0 namespace into /v1 (http... [16:59:17] 10Phabricator: Rename request - https://phabricator.wikimedia.org/T255588 (10ProcrasinatingReader) [16:59:39] 10Phabricator: Rename request - https://phabricator.wikimedia.org/T255588 (10ProcrasinatingReader) [17:04:08] 10Phabricator: Rename request - https://phabricator.wikimedia.org/T255588 (10ProcrasinatingReader) [17:05:06] 10Phabricator: Rename request - https://phabricator.wikimedia.org/T255588 (10ProcrasinatingReader) [17:05:20] 10Phabricator: Rename request - https://phabricator.wikimedia.org/T255588 (10ProcrasinatingReader) [17:25:34] !log Fiddling on deploy01 with Task 230245 [17:25:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:26:30] I'm getting a readonly warning on beta, is there something going on? [17:26:35] Not consistently though [17:27:29] 10Phabricator: Rename username on Phabricator - https://phabricator.wikimedia.org/T255588 (10Aklapper) a:03Aklapper [17:28:10] Daimona: Like there's excessive replag or something? [17:28:25] "The Wikipedia database is temporarily in read-only mode for the following reason" [17:28:28] 10Phabricator: Rename username on Phabricator - https://phabricator.wikimedia.org/T255588 (10Aklapper) 05Open→03Resolved [17:28:29] When I try to save an edit [17:28:46] (The reason is $1 \o/) [17:29:34] https://en.wikipedia.beta.wmflabs.org/w/api.php?action=query&meta=siteinfo&siprop=dbrepllag&sishowalldb= [17:29:38] No obvious replag... [17:30:36] It comes and it goes [17:35:55] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE), 10User-zeljkofilipin: Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10Vidhi-Mody) >>! In T255471#6226851, @Aklapper wrote: > @Vidhi-Mody: What's the output of `git config --get remote.origin.url` in... [17:40:07] 10phan-taint-check-plugin, 10MediaWiki-extensions-Translate: Phan blocks merge of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/603471 - https://phabricator.wikimedia.org/T255561 (10Umherirrender) You can use Xml class to create wikitext. The Html class is for outputting and thats involved... [17:58:34] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Operations, 10Patch-For-Review: Migrate contint* hosts to Buster - https://phabricator.wikimedia.org/T224591 (10MoritzMuehlenhoff) Definitely, if you need a s... [18:37:27] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Kask functional testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10thcipriani) [18:39:59] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Release Pipeline: Support for deleting images in PipelineLib - https://phabricator.wikimedia.org/T255595 (10thcipriani) [18:41:22] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Release Pipeline: PipelineLib helm test log output - https://phabricator.wikimedia.org/T255596 (10thcipriani) [18:45:27] (03CR) 10Hashar: [C: 03+2] "Yeah and seems to be working properly ;]" [integration/config] - 10https://gerrit.wikimedia.org/r/605833 (owner: 10Hashar) [18:46:27] (03Merged) 10jenkins-bot: utils/zuul-layout-diffs.sh: dump invalid layout [integration/config] - 10https://gerrit.wikimedia.org/r/605833 (owner: 10Hashar) [18:46:56] (03CR) 10Hashar: [C: 03+2] zuul: add debian glue to g&s for contenttranslation repo [integration/config] - 10https://gerrit.wikimedia.org/r/605866 (owner: 10Hashar) [18:47:59] (03Merged) 10jenkins-bot: zuul: add debian glue to g&s for contenttranslation repo [integration/config] - 10https://gerrit.wikimedia.org/r/605866 (owner: 10Hashar) [18:48:58] !log Reloading Zuul for Ibb8b8655f1837ced28783ef44f2b7b1e4a15e9ea [18:49:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:06:59] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Kask functional testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10jeena) >>! In T224041#6227592, @akosiaris wrote: > However, up to now we... [19:07:22] !log Done fiddling with deploy01 [19:07:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:15:32] I am off take care everyone [19:19:58] 10Phabricator, 10Developer-Advocacy: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Jdlrobson) > IMO that's what the Priority field is for. The less important and urgent, the lower the priority should be. Priorities change howe... [19:20:58] 10Phabricator, 10Developer-Advocacy: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Reedy) [19:33:15] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10observability: Logstash-Beta cannot be accessed: 504 Gateway Time-out - https://phabricator.wikimedia.org/T254801 (10herron) 05Open→03Resolved a:03herron After applying `profile::kibana::httpd_proxy` to deployment-logstash03, and playing whac... [19:33:19] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10herron) [19:43:00] 10Phabricator, 10Developer-Advocacy: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Dzahn) > There's no point in caring about a bug from 5 years ago if nobody is around to care about it any more. If someone does care - great - k... [19:44:13] uhm I may have just email bombed the releng mailing list [19:49:33] i saw the subject lines and figured you were just being very active with calendar windows [19:51:53] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) [19:52:08] 10Phabricator, 10Developer-Advocacy: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Jdlrobson) > Imagine reporting a bug, waiting for 5 years and then having a bot automatically close it with the sole reason that it is "stale".... [19:53:33] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) a:05LarsWirzenius→03brennen Group1 is currently on .37. Will watch logs and reassig... [19:56:58] (03CR) 10Kosta Harlan: "> Patch Set 2:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/605939 (https://phabricator.wikimedia.org/T235118) (owner: 10Hashar) [20:04:03] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) [20:14:42] !log livehacking MWExceptionHandler on deploy01 [20:14:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:16:23] if it lets me.. [20:16:39] for numerous things you'll need to sudo [20:19:34] I figured out :) [20:29:22] !log done! [20:29:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:32:30] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops: replace backends for releases.wikimedia.org with buster VMs - https://phabricator.wikimedia.org/T247652 (10Dzahn) [20:32:58] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops: replace backends for releases.wikimedia.org with buster VMs - https://phabricator.wikimedia.org/T247652 (10Dzahn) VMs with buster, releases1002/releases2002 have been created in the subtask. [20:41:18] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) [20:55:04] James_F: is the train running again? If so I have a patch that blocks it continuing [20:55:43] https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikimediaEvents/+/606012/ is the patch [20:58:12] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Jdlrobson) New blocker: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikimediaEvents/+... [21:03:05] !log testing a T230245 fix [21:03:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:03:07] T230245: GenerateFancyCaptchas.php crashes with "FormatJson.php: File not found in" after 1000 iterations - https://phabricator.wikimedia.org/T230245 [21:13:00] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10daniel) >>! In T254175#6228683, @nnikkhoui wrote: > Just adding a quick note, that cpt has moved some mediawiki REST en... [21:14:02] (03PS1) 1020after4: Add a --task option to branch.py [tools/release] - 10https://gerrit.wikimedia.org/r/606014 [21:16:43] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10Krinkle) >>! In T254174#6229840, @Jdlrobson wrote: > New blocker: https://gerrit.wikimedia.org/r/... [21:17:26] (03CR) 10Thcipriani: Add a --task option to branch.py (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/606014 (owner: 1020after4) [21:54:34] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T254174 (10brennen) a:05brennen→03LarsWirzenius Current status: The train is rolled back to group0. As...