[02:13:44] (03PS1) 10Krinkle: Add wmf-config-wg-vars.js [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616639 [02:13:46] (03PS1) 10Krinkle: Update wmf-config-wg-vars.js [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616640 [02:13:48] (03PS1) 10Krinkle: wmf-config-wg-vars.js: Add README [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616641 (https://phabricator.wikimedia.org/T248866) [02:14:02] (03CR) 10Krinkle: [V: 03+2 C: 03+2] Add wmf-config-wg-vars.js [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616639 (owner: 10Krinkle) [02:14:16] (03CR) 10Krinkle: [V: 03+2 C: 03+2] Update wmf-config-wg-vars.js [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616640 (owner: 10Krinkle) [02:14:31] (03CR) 10Krinkle: [V: 03+2 C: 03+2] wmf-config-wg-vars.js: Add README [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616641 (https://phabricator.wikimedia.org/T248866) (owner: 10Krinkle) [02:16:24] (03PS1) 10Krinkle: wmf-config-wg-vars.js: Use the '/deployed' preset for Code search [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616643 (https://phabricator.wikimedia.org/T248866) [02:19:32] (03PS2) 10Krinkle: wmf-config-wg-vars.js: Use the '/deployed' preset for Code search [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616643 (https://phabricator.wikimedia.org/T248866) [02:22:25] (03CR) 10Krinkle: "If you think this can done easily and dependency-free in Python, perhaps you can give me one or two pointers and then I can give that a tr" [tools/code-utils] - 10https://gerrit.wikimedia.org/r/616643 (https://phabricator.wikimedia.org/T248866) (owner: 10Krinkle) [04:42:56] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Joe) Wikimedia is currently on php 7.2, and I expect the move from 7.2 to 7.3 or 7.4 will only happen after we'v... [04:44:18] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Joe) I want to add that, as far as production code goes, we will need to keep support for 7.2 and 7.3 in paralle... [05:04:07] 10Release-Engineering-Team, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Platform Engineering (Icebox): web installer / upgrade doesn't seem to recognize extension.json configuration or overrides to that configuration - https://phabricator.wikimedia.org/T244864 (10Legoktm) This is on purpose. T... [05:10:14] (03CR) 1020after4: [C: 03+2] deploy-promote: assert repo is up-to-date [tools/release] - 10https://gerrit.wikimedia.org/r/616601 (owner: 10Thcipriani) [05:11:55] (03CR) 1020after4: [C: 03+2] "A very nice improvement. Thanks!" [tools/release] - 10https://gerrit.wikimedia.org/r/616600 (owner: 10Thcipriani) [05:12:51] (03Merged) 10jenkins-bot: deploy-promote: usage and explicit error [tools/release] - 10https://gerrit.wikimedia.org/r/616600 (owner: 10Thcipriani) [05:12:56] (03Merged) 10jenkins-bot: deploy-promote: assert repo is up-to-date [tools/release] - 10https://gerrit.wikimedia.org/r/616601 (owner: 10Thcipriani) [05:22:16] 10Gerrit, 10DBA, 10Patch-For-Review: Make sure both `reviewdb-test` (used forgerrit upgrade testing) and `reviewdb` (formerly production) databases get torn down - https://phabricator.wikimedia.org/T255715 (10Marostegui) >>! In T255715#6338856, @Dzahn wrote: > If it hasn't already happened you can remove the... [06:24:28] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<11.11%) [06:25:33] maintenance-disconnect-full-disks build 198052 integration-agent-docker-1011 (/: 18%, /srv: 98%, /var/lib/docker: 79%): OFFLINE due to disk space [06:30:36] maintenance-disconnect-full-disks build 198053 integration-agent-docker-1011 (/: 18%, /srv: 29%, /var/lib/docker: 45%): RECOVERY disk space OK [06:39:27] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [07:35:05] 10Release-Engineering-Team, 10Analytics-Radar, 10Product-Analytics, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10elukey) [07:36:13] 10Release-Engineering-Team, 10Analytics-Radar, 10Product-Analytics, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10elukey) Just completed the creation of the `analytics-product` system user + kerberos keytab, you should now... [08:20:01] Project beta-update-databases-eqiad build #43832: 04FAILURE in 1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43832/ [08:21:36] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Automatic branch cut went OK. Ran the pre-deployment-window steps (... [08:28:27] PROBLEM - Host deployment-changeprop is DOWN: CRITICAL - Host Unreachable (172.16.5.21) [08:28:27] PROBLEM - Host deployment-cpjobqueue is DOWN: CRITICAL - Host Unreachable (172.16.4.124) [08:28:27] PROBLEM - Host deployment-chromium02 is DOWN: CRITICAL - Host Unreachable (172.16.4.14) [09:19:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Selenium quibble jobs have a huge cache overloading the CI ag... - https://phabricator.wikimedia.org/T258972 [09:20:02] Project beta-update-databases-eqiad build #43833: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43833/ [09:34:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Selenium quibble jobs have a huge cache overloading the CI ag... - https://phabricator.wikimedia.org/T258972 [09:41:54] !log Removed bunch of chrome lighthouse cache files on castor instance # T258972 [09:41:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:41:57] T258972: Selenium quibble jobs have a huge cache overloading the CI agents small disk spaces - https://phabricator.wikimedia.org/T258972 [09:45:26] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<33.33%) [09:45:29] maintenance-disconnect-full-disks build 198092 integration-agent-docker-1011 (/: 18%, /srv: 99%, /var/lib/docker: 67%): OFFLINE due to disk space [09:48:13] o/ is there quick link from a gerrit patch set to gitiles? What I do currently Browse/type the project in Filter/click on gittiles, I'd love to find a quicker path to get to gitiles [09:50:34] maintenance-disconnect-full-disks build 198093 integration-agent-docker-1011 (/: 18%, /srv: 83%, /var/lib/docker: 66%): RECOVERY disk space OK [09:51:44] RECOVERY - Free space - all mounts on integration-castor03 is OK: OK: All targets OK [09:53:07] (03CR) 10Jforrester: [C: 03+2] Enable Generic.ControlStructures.DisallowYodaConditions rule [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/615523 (owner: 10Esanders) [09:54:18] (03Merged) 10jenkins-bot: Enable Generic.ControlStructures.DisallowYodaConditions rule [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/615523 (owner: 10Esanders) [10:00:26] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [10:03:56] (03CR) 10Hashar: [C: 04-1] "-1 cause debian-glue is not needed." (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/616177 (https://phabricator.wikimedia.org/T224041) (owner: 10Jeena Huneidi) [10:05:20] (03CR) 10Jforrester: [C: 03+2] Add `extension-javascript-documentation` for GlobalWatchlist [integration/config] - 10https://gerrit.wikimedia.org/r/616503 (owner: 10DannyS712) [10:06:11] thanks James_F [10:06:18] Happy to help. [10:06:23] (03Merged) 10jenkins-bot: Add `extension-javascript-documentation` for GlobalWatchlist [integration/config] - 10https://gerrit.wikimedia.org/r/616503 (owner: 10DannyS712) [10:06:44] !log Zuul: [mediawiki/extensions/GlobalWatchlist] Add extension-javascript-documentation [10:06:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:07:30] when should I be able to expect the docs to be generated? Or do they need another change to be merged to the extension first? [10:07:42] DannyS712: Next merge, yes. [10:07:58] thanks [10:20:02] Project beta-update-databases-eqiad build #43834: 04STILL FAILING in 1.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43834/ [10:24:01] James_F does Zuul need to be restarted, or is that only for whitelist changes [10:24:34] DannyS712: I did it. [10:24:38] DannyS712: ^^^ [10:24:46] Did it not take? [10:25:23] it only logged that you added it, not that it restarted, so I wasn't sure [10:25:48] "It" didn't log anything. I logged that I had refreshed its config. :-P [10:26:25] (We don't actually restart Zuul except in emergencies, just refresh the config; restarting it is lossy for all in-flight CI work and queues.) [10:27:10] oh. Sorry for the misunderstanding (and I didn't look at who typed it and assumed it was log msg bot, sorry) [10:30:01] :-) [10:30:43] Middle-day tea time [11:00:37] PROBLEM - Host deployment-mcs01 is DOWN: CRITICAL - Host Unreachable (172.16.5.64) [11:20:02] Project beta-update-databases-eqiad build #43835: 04STILL FAILING in 2.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43835/ [12:20:01] Project beta-update-databases-eqiad build #43836: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43836/ [13:10:40] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10QChris) It seems we're left now with too many choices to move forward :-) * "lame minimum-effort version" from https://phabricator.wikimedia.org/T257218#6316301 * "slightly more effort version" from h... [13:11:06] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Promoted to group0. [13:11:22] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Filed https://phabricator.wikimedia.org/T259022 but didn't make it... [13:14:59] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) [13:15:20] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Filed {T259023} as a blocker. [13:20:02] Project beta-update-databases-eqiad build #43837: 04STILL FAILING in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43837/ [13:33:04] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Lucas_Werkmeister_WMDE) One yay for me for the old / upstream favicon, which is far more distinct than any of the versions from T257218#6316301. Using the WMF colors is, to me, //not// a benefit, becau... [13:39:12] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) [13:40:07] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Unblocked, because {T259023} doesn't seem to be bad enough to be a... [14:20:01] Project beta-update-databases-eqiad build #43838: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43838/ [14:43:34] (03PS1) 10Hashar: Add tox for integration/gear [integration/config] - 10https://gerrit.wikimedia.org/r/616819 (https://phabricator.wikimedia.org/T258630) [14:44:07] (03CR) 10Hashar: [C: 03+2] Add tox for integration/gear [integration/config] - 10https://gerrit.wikimedia.org/r/616819 (https://phabricator.wikimedia.org/T258630) (owner: 10Hashar) [14:44:43] (03Merged) 10jenkins-bot: Add tox for integration/gear [integration/config] - 10https://gerrit.wikimedia.org/r/616819 (https://phabricator.wikimedia.org/T258630) (owner: 10Hashar) [14:45:59] (03PS1) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [integration/gear] - 10https://gerrit.wikimedia.org/r/616820 [14:46:51] (03CR) 10jerkins-bot: [V: 04-1] Jenkins job validation (DO NOT SUBMIT) [integration/gear] - 10https://gerrit.wikimedia.org/r/616820 (owner: 10Hashar) [14:56:53] (03PS2) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [integration/gear] - 10https://gerrit.wikimedia.org/r/616820 [14:56:55] (03PS1) 10Hashar: Remove python 3.4 test support [integration/gear] - 10https://gerrit.wikimedia.org/r/616823 [14:56:57] (03PS1) 10Hashar: wmf: adjust .gitreview for Wikimedia gerrit [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 [15:04:05] (03CR) 10Hashar: "That is merely a convenience for our forked repository :)" [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:04:10] (03Abandoned) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [integration/gear] - 10https://gerrit.wikimedia.org/r/616820 (owner: 10Hashar) [15:04:59] (03PS2) 10Hashar: wakeConnections: Randomize connections before scanning them [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) (owner: 10Ahmon Dancy) [15:06:02] (03CR) 10Hashar: "I have enabled CI on the repository and rebased your change on top of a change to fix the CI build. I guess next we want to tag a dummy re" [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) (owner: 10Ahmon Dancy) [15:07:48] (03CR) 10Ahmon Dancy: [C: 04-1] wmf: adjust .gitreview for Wikimedia gerrit (031 comment) [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:11:54] (03CR) 10Hashar: wmf: adjust .gitreview for Wikimedia gerrit (031 comment) [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:11:59] (03PS2) 10Hashar: wmf: adjust .gitreview for Wikimedia gerrit [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 [15:13:19] (03PS3) 10Hashar: wakeConnections: Randomize connections before scanning them [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) (owner: 10Ahmon Dancy) [15:20:02] Project beta-update-databases-eqiad build #43839: 04STILL FAILING in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43839/ [15:22:00] PROBLEM - Parsoid on deployment-parsoid11 is CRITICAL: connect to address 172.16.1.115 and port 8000: Connection refused [15:24:33] (03CR) 10Ahmon Dancy: [C: 03+1] "LGTM" [integration/gear] - 10https://gerrit.wikimedia.org/r/616823 (owner: 10Hashar) [15:25:20] (03CR) 10Ahmon Dancy: [C: 03+1] wmf: adjust .gitreview for Wikimedia gerrit [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:25:34] dancy: good morning :] I guess you can CR+2 those? [15:25:49] ok [15:25:54] (03CR) 10Hashar: [C: 03+2] wakeConnections: Randomize connections before scanning them [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) (owner: 10Ahmon Dancy) [15:25:58] (03CR) 10Ahmon Dancy: [C: 03+2] Remove python 3.4 test support [integration/gear] - 10https://gerrit.wikimedia.org/r/616823 (owner: 10Hashar) [15:26:03] and I +2ed your [15:26:19] so that once you +2 mines, they should all be merged eventually [15:26:56] (03Merged) 10jenkins-bot: Remove python 3.4 test support [integration/gear] - 10https://gerrit.wikimedia.org/r/616823 (owner: 10Hashar) [15:27:51] (03CR) 10Ahmon Dancy: [C: 03+2] wmf: adjust .gitreview for Wikimedia gerrit [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:28:42] (03Merged) 10jenkins-bot: wmf: adjust .gitreview for Wikimedia gerrit [integration/gear] - 10https://gerrit.wikimedia.org/r/616824 (owner: 10Hashar) [15:28:50] (03Merged) 10jenkins-bot: wakeConnections: Randomize connections before scanning them [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) (owner: 10Ahmon Dancy) [15:34:35] dancy: so the devil now is to upgrade the requirement for our Zuul deployment [15:34:51] which currently uses gear 0.7.0 when the head of master is 0.15 ish [15:36:29] I looked at the log of commits past 0.7.0 and they look ok. [15:37:55] The most interesting might be: b24b4d2 Add a send lock to the base Connection class [15:42:43] and some change to prefer ipv6 [15:42:50] but we explicitly use 127.0.0.1 ;D [15:45:20] btw I notice there isn't a version number in any of the source files. How does that work? [15:48:18] !log integration/gear : git tag 0.15.1+wmf1 5fd9c37 for T258630 [15:48:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:48:20] T258630: Improve scheduling of CI jobs invoked by zuul - https://phabricator.wikimedia.org/T258630 [15:48:38] dancy: the dependencies are definied in a requirements.txt file [15:49:20] I mean the gear code doesn't specify its version anywhere. Does it come from the git tag? [15:49:26] ah yeah [15:49:38] they use their own wrapper on top of setuptools: pbr [15:49:55] so the setup.py is very trivial [15:50:19] and pbr has a shit ton of conventions such as determining the version automatically when running from a git repo [15:50:29] rest can be configured via setup.cfg [15:50:40] https://pypi.org/project/pbr/ [15:50:42] ok. Makes sense. Thanks for the info. [15:50:57] openstack wrote that to avoid repeating themselves over and over on all the python projects they maintain [15:51:42] I'm noticing that a lot of openstack web links are failing. [15:52:00] (such as https://docs.openstack.org/pbr/latest/ ) [15:52:09] ah yeah [15:52:12] they rebranded as opendev [15:52:22] damn rebrands!! [15:52:28] to extend their scope [15:52:39] openstack now being one of the projects they support [15:54:25] (03PS1) 10Hashar: WMF: use gear from our forked version [integration/zuul] (patch-queue/debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/616858 (https://phabricator.wikimedia.org/T258630) [15:56:32] (03PS2) 10Umherirrender: Move encoding and extensions arg to ruleset.xml [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/614285 (https://phabricator.wikimedia.org/T200956) [15:59:24] damn [15:59:25] (03PS2) 10Hashar: WMF: use gear from our forked version [integration/zuul] (patch-queue/debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/616858 (https://phabricator.wikimedia.org/T258630) [15:59:33] I have lost my gpg key passphrase again :-\ [15:59:59] antoine! :) [16:02:21] hashar: vote mot de passe est "L'ÉtatContinuous Integration c'est moi" [16:02:33] ;D [16:02:43] *votre [16:03:58] if someone can help https://phabricator.wikimedia.org/T259023 be resolved in a near-future backport window, that would be wonderful [16:20:02] Project beta-update-databases-eqiad build #43840: 04STILL FAILING in 1.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43840/ [16:47:34] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10thcipriani) [16:49:39] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10observability, and 2 others: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10herron) I looked around briefly for a zuul promethe... [17:05:44] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Krinkle) >>! In T257218#6321698, @Tgr wrote: >>>! In T257218#6316493, @Krinkle wrote: >> I'd recommend filling in the internal blanks with white pixels to make it stand out more in typical browser tabs... [17:20:02] Project beta-update-databases-eqiad build #43841: 04STILL FAILING in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43841/ [17:42:10] 10Phabricator, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Operations, 10Traffic, and 2 others: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10mmodell) 05Stalled→03Open [17:42:16] 10Phabricator, 10RelEng-Archive-FY201718-Q2, 10Patch-For-Review: Enable notification server (real-time pop-up notifications) in Phabricator - https://phabricator.wikimedia.org/T765 (10mmodell) [18:01:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10observability, and 2 others: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10dancy) I noticed that there are several messages of... [18:10:05] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T257971 (10brennen) [18:10:07] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) [18:13:12] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) a:05LarsWirzenius→03brennen Keeping one eye on it, but now unblocked... [18:24:34] Project beta-update-databases-eqiad build #43842: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43842/ [18:57:08] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T257971 (10Jdforrester-WMF) [19:20:02] Project beta-update-databases-eqiad build #43843: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43843/ [19:26:06] (03CR) 10Hashar: "integration/zuul does not have any CI cause the testsuite is broken somehow and I never managed to fix it :)" [integration/zuul] (patch-queue/debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/616858 (https://phabricator.wikimedia.org/T258630) (owner: 10Hashar) [19:53:25] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Krinkle) >>! In T257218#6340562, @QChris wrote: > It seems we're left now with too many choices to move forward :-) > > * "lame minimum-effort version" from https://phabricator.wikimedia.org/T257218#6... [20:21:55] Yippee, build fixed! [20:21:55] Project beta-update-databases-eqiad build #43844: 09FIXED in 1 min 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43844/ [20:25:25] (03PS1) 10Ahmon Dancy: Fix handling of 'local' copies [blubber] - 10https://gerrit.wikimedia.org/r/616907 (https://phabricator.wikimedia.org/T259069) [20:29:16] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Legoktm) >>! In T257218#6340562, @QChris wrote: > ...I'll be bold and switch our Gerrit favicon to a picture of my grinning face :-) ... or something similar). This option is missing from the poll. [20:41:47] (03CR) 10Jeena Huneidi: [C: 03+1] "Tested against a blubberfile. LGTM" [blubber] - 10https://gerrit.wikimedia.org/r/616907 (https://phabricator.wikimedia.org/T259069) (owner: 10Ahmon Dancy) [20:42:08] (03CR) 10Dduvall: [C: 03+2] "Fixes the referenced issue and the fix implementation looks good to me." [blubber] - 10https://gerrit.wikimedia.org/r/616907 (https://phabricator.wikimedia.org/T259069) (owner: 10Ahmon Dancy) [20:45:09] (03Merged) 10jenkins-bot: Fix handling of 'local' copies [blubber] - 10https://gerrit.wikimedia.org/r/616907 (https://phabricator.wikimedia.org/T259069) (owner: 10Ahmon Dancy) [20:57:58] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [20:59:16] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [21:10:52] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [21:13:42] 10Phabricator, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Operations, 10Traffic, and 2 others: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10Dzahn) @20after4 Current stat... [21:18:46] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Tgr) > Why would use of white be disturbing in light mode? For the most part, I would expect it to be an unnoticable change since on such a bright background transparent is effectively identical to whi... [21:34:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10observability, and 2 others: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10dancy) Prototype mtail program for the error log fo... [21:58:14] (03PS1) 10MarcoAurelio: Edit Repo Config [debs/prometheus-es-exporter] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/616926 [21:58:59] (03Abandoned) 10MarcoAurelio: Edit Repo Config [debs/prometheus-es-exporter] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/616926 (owner: 10MarcoAurelio) [22:31:55] 10Beta-Cluster-Infrastructure: Clarify where to find error logs in beta cluster - https://phabricator.wikimedia.org/T259081 (10Urbanecm) [22:40:56] James_F: is there any rough date for when rc0 will be out for 1.35? or have I missed an email? [22:47:26] I've not seen one [22:47:54] (03PS1) 10Cooltey: Edit Repo Config [services/wikifeeds] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/616927 [22:51:23] Reedy: I must be been asked daily now when will be 1.35 be out? and all I can say is there's not even a release candidate [22:51:40] no one seems to understand the release timeline [22:52:09] AIUI there's still various things not finished [22:52:29] https://lists.wikimedia.org/pipermail/mediawiki-announce/2020-June/000250.html [22:52:35] >We anticipate that MediaWiki 1.35 will be released at the beginning of August. [22:52:40] I'd presume that's not gonna happen [22:53:17] Reedy: I feel like I've said that about 10 thousand times to Miraheze's users over the last 2 weeks [22:53:37] * Reedy shrugs [22:53:43] I'm trying to see if anyone knows more than it might not be August [22:53:55] I'd presume Miraheze wouldn't be upgrading everywhere immediately anyway [22:53:58] https://www.garron.me/en/blog/debian-it-ships-when-it-is-ready.html [22:55:15] Reedy: normally within a week or so of release [23:11:26] (03Abandoned) 10Cooltey: Edit Repo Config [services/wikifeeds] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/616927 (owner: 10Cooltey) [23:13:36] (03PS1) 10Krinkle: jjb: Add 'assert-no-mediawiki-errors' to quibble jobs (take 2) [integration/config] - 10https://gerrit.wikimedia.org/r/616923 (https://phabricator.wikimedia.org/T246358) [23:18:27] Project beta-scap-eqiad build #310622: 04FAILURE in 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/310622/ [23:23:48] I think we need more help addressing release blockers [23:24:31] Project beta-scap-eqiad build #310623: 04STILL FAILING in 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/310623/ [23:27:38] also https://webrtc-review.googlesource.com/c/src/+/176422 <-- the binary size changes pane is super cool [23:27:52] would be neat if we could do something similar for our test coverage and perf reports [23:28:01] (03PS2) 10Krinkle: jjb: Add 'assert-no-mediawiki-errors' to quibble jobs (take 2) [integration/config] - 10https://gerrit.wikimedia.org/r/616923 (https://phabricator.wikimedia.org/T246358) [23:29:07] legoktm: interesting, are those submitted in a comment and extracted? [23:29:32] I don't see any comment with that kind of info [23:30:01] last I checked we couldn't even vary the comment by more than true/false,never mind e.g. Jenkins abort/unstable states [23:30:22] supporting more than 2 states would be cool, suporting artibtrary text from Jenkiins out would be aweseome. [23:30:27] it's coming from their tryjobs builder I believe [23:30:28] https://ci.chromium.org/p/webrtc/builders/try/android_compile_arm64_rel/26039? [23:30:44] right, it's pushed to the status API [23:30:45] not pulled [23:30:50] makes sense [23:31:15] that would require support from Zuul to do that securely [23:31:27] basically the same problem [23:31:43] Yippee, build fixed! [23:31:44] Project beta-scap-eqiad build #310624: 09FIXED in 3 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/310624/ [23:54:20] PROBLEM - Host deployment-sentry01 is DOWN: CRITICAL - Host Unreachable (172.16.5.16)