[00:07:34] (03PS3) 10Ahmon Dancy: Use docker-compose exec instead of ssh [tools/train-dev] - 10https://gerrit.wikimedia.org/r/701993 [02:14:10] Hi releng, I made a smol mistake when rushing to try to get something merged in time for the branch cut. Here is the offending patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/702001 Thanks so much to DannyS712 for spotting this nearly instantaneously :) [02:14:35] The issue is that the commit message lists some patches as included in the merge that actually are not there [02:15:51] 10Phabricator, 10User-DannyS712: In train deployment tasks, link to create a subtask for a blocker should prefill unbreak now - https://phabricator.wikimedia.org/T285733 (10DannyS712) [02:16:39] Then a secondary issue is that other patches that fix CI for CentralNotice are among those not included. I don't believe this would hold up the train, though I guess can be put on a backport, because there is another actual functional patch that we should try to get out [02:18:20] Also my fault was for self-verifying, done since I was trying to get this in before the hour. Not sure if CI would have caught something if I hadn't done that, I imagine not [02:18:46] awaiting guidance on how to mop up, thanks in advance and many apologies [02:20:23] on second thought, it might be best to get this fixed before the train rolls though [02:23:34] brennen dduvall ^ thoughts? looks like you're on tomorrow's train? [02:28:23] 10Phabricator, 10User-DannyS712: In train deployment tasks, link to create a subtask for a blocker should prefill unbreak now - https://phabricator.wikimedia.org/T285733 (10DannyS712) [02:28:54] 10Phabricator, 10Patch-For-Review, 10User-DannyS712: In train deployment tasks, link to create a subtask for a blocker should prefill unbreak now - https://phabricator.wikimedia.org/T285733 (10DannyS712) p:05Triage→03Low [02:28:58] 10Phabricator, 10Patch-For-Review, 10User-DannyS712: In train deployment tasks, link to create a subtask for a blocker should prefill unbreak now - https://phabricator.wikimedia.org/T285733 (10DannyS712) [02:31:22] 10Phabricator, 10Patch-For-Review, 10User-DannyS712: In train deployment tasks, link to create a subtask for a blocker should prefill unbreak now - https://phabricator.wikimedia.org/T285733 (10DannyS712) (Notes for next time I need to create a patch in differential: repository is at https://phabricator.wikim... [02:34:56] ^ here is a fix: https://gerrit.wikimedia.org/r/702004 [03:39:17] AndyRussG: just saw this - thanks for flagging. i'll note on ticket that 702004 needs backported before tomorrow's deploy. [03:46:04] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T281153 (10brennen) Noting here that [[https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/702004/|Merge branch 'master' into wmf_deploy]] needs backport... [07:26:32] anyone got a TLDR of what mediawiki/tools/train-dev is? :D [08:19:17] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Jenkins agents start complains: Java is not in the PATH nor configured with the javaPath setting - https://phabricator.wikimedia.org/T285743 (10hashar) [08:25:25] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Jenkins agents start complains: Java is not in the PATH nor configured with the javaPath setting - https://phabricator.wikimedia.org/T285743 (10hashar) 05Open→03Resolved a:03hashar Explicitly set javaPath to `/usr/bin/java`... [08:29:18] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Doing): Upgrade Jenkins to 2.289.x - https://phabricator.wikimedia.org/T285531 (10hashar) 05Open→03Resolved I have removed the SSHD and Ant plugins from both Jenkins servers. Javadoc is still installed since that is an indir... [08:36:31] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.11 deployment blockers - https://phabricator.wikimedia.org/T281152 (10Joe) [08:37:12] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.11 deployment blockers - https://phabricator.wikimedia.org/T281152 (10Joe) 05Resolved→03Open Boldly reopening since we have a UBN! task that seems clearly related to this train release. [08:53:16] kostajh: so hmm can't do quibble weekly cause I gotta roll back the train [08:53:38] the mw servers local cache are overloaded since last week. Joyce [09:05:05] hashar: ok, good luck [09:29:46] 10Beta-Cluster-Infrastructure, 10GrowthExperiments, 10Growth-Team (Current Sprint): [betalabs-regression] Several controls display MediaWiki labels not text - https://phabricator.wikimedia.org/T285283 (10kostajh) I can't reproduce this and I assume it's some oddity with betalabs. @Tgr @mewoph I'm moving it... [09:30:50] 10Beta-Cluster-Infrastructure, 10GrowthExperiments, 10Growth-Team (Current Sprint): [beta cluster-regression] Several controls display MediaWiki labels not text - https://phabricator.wikimedia.org/T285283 (10Majavah) [09:31:43] 10Beta-Cluster-Infrastructure, 10GrowthExperiments, 10Growth-Team (Current Sprint): [beta cluster-regression] Several controls display MediaWiki labels not text - https://phabricator.wikimedia.org/T285283 (10Majavah) [09:31:52] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks localisation cache - https://phabricator.wikimedia.org/T285345 (10Majavah) [09:37:27] kostajh: no roll back needed :] [09:37:49] \o/ [10:38:04] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks localisation cache - https://phabricator.wikimedia.org/T285345 (10Jdforrester-WMF) Last merged patch in scap was {d6663929dbba2eced45b7e5ecf6fe00ec250de6e}, landing 12 days ago. Presumably that's not the direct... [10:46:56] (03PS1) 10Lucas Werkmeister (WMDE): Support linking to individual doc.wikimedia.org tiles [integration/docroot] - 10https://gerrit.wikimedia.org/r/702098 [10:48:16] (03CR) 10Lucas Werkmeister (WMDE): "This is somewhere between sketch, proof of concept, and fully featured patch. The id attribute could maybe be tweaked (escape some special" [integration/docroot] - 10https://gerrit.wikimedia.org/r/702098 (owner: 10Lucas Werkmeister (WMDE)) [10:49:54] (03CR) 10Lucas Werkmeister (WMDE): Support linking to individual doc.wikimedia.org tiles (031 comment) [integration/docroot] - 10https://gerrit.wikimedia.org/r/702098 (owner: 10Lucas Werkmeister (WMDE)) [11:00:12] 10Continuous-Integration-Config, 10Patch-For-Review: Introduce non-voting jobs with quibble+apache - https://phabricator.wikimedia.org/T276428 (10kostajh) >>! In T276428#6916188, @Addshore wrote: > So it looks like every non voting selenium job for Wikibase using apache as the webserver has failed since being... [11:13:57] 10Continuous-Integration-Config, 10Patch-For-Review: Introduce non-voting jobs with quibble+apache - https://phabricator.wikimedia.org/T276428 (10Addshore) I'd be interested to look at the VM running the tests and just see if it is even more stuff like {T281122} and VM memory compaction etc making things freez... [11:22:52] Project mediawiki-core-doxygen-docker build #26020: 04FAILURE in 18 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/26020/ [11:45:13] (03PS4) 10Kosta Harlan: Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 [12:07:55] (03CR) 10Michael Große: [C: 03+1] Support linking to individual doc.wikimedia.org tiles [integration/docroot] - 10https://gerrit.wikimedia.org/r/702098 (owner: 10Lucas Werkmeister (WMDE)) [12:19:10] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T281153 (10Ladsgroup) not a risky patch but if you see any errors with metadata and files, ping me. We had several changes going in (while being much smaller than previous... [13:14:25] E assert SystemExit(143) == SystemExit(143) [13:14:25] E +SystemExit(143) [13:14:25] E -SystemExit(143) [13:14:28] python I love you [13:16:22] huh [13:18:51] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.11 deployment blockers - https://phabricator.wikimedia.org/T281152 (10Joe) [13:18:57] (03CR) 10Hashar: Delete lock file on SIGTERM/SIGINT (033 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) (owner: 10Hashar) [13:22:21] (03PS2) 10Hashar: Delete lock file on SIGTERM/SIGINT [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) [13:25:16] (03CR) 10jerkins-bot: [V: 04-1] Delete lock file on SIGTERM/SIGINT [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) (owner: 10Hashar) [13:27:35] (03CR) 10Hashar: Delete lock file on SIGTERM/SIGINT (033 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) (owner: 10Hashar) [13:41:02] (03CR) 10Hashar: "The test fails cause assert_called_once() is python 3.6+." [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) (owner: 10Hashar) [13:41:14] (03PS3) 10Hashar: Delete lock file on SIGTERM/SIGINT [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) [13:45:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review, 10User-zeljkofilipin: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10hashar) [13:47:25] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.11 deployment blockers - https://phabricator.wikimedia.org/T281152 (10Jdforrester-WMF) 05Open→03Resolved Re-closing; blocker appears fixed. [13:51:04] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10hashar) Yes and it seems that has the l10n bot and/or LibraryUpdater bots r... [13:51:20] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10hashar) a:03hashar [13:51:37] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Doing), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10hashar) [13:56:18] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Doing), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10Jdforrester-WMF) >>! In T283883#7183766, @hashar wrote: > Yes and it seems... [13:57:59] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks localisation cache - https://phabricator.wikimedia.org/T285345 (10Jdforrester-WMF) OK, it looks like the issue might be that prod machines generally seem to have Python 3.7, but the Beta Cluster machines have Py... [14:01:09] Project beta-scap-sync-world build #10823: 04FAILURE in 8 min 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/10823/ [14:03:20] (03CR) 10Hashar: [C: 03+1] "> The id attribute could maybe be tweaked (escape some special characters, somehow? ensure uniqueness?)" [integration/docroot] - 10https://gerrit.wikimedia.org/r/702098 (owner: 10Lucas Werkmeister (WMDE)) [14:06:08] Yippee, build fixed! [14:06:09] Project beta-scap-sync-world build #10824: 09FIXED in 3 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/10824/ [14:18:13] Yippee, build fixed! [14:18:13] Project mediawiki-core-doxygen-docker build #26021: 09FIXED in 14 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/26021/ [14:33:05] Amir1: What's your wonderful bot to replace Herald? I've forgotten its name and so how to find and configure it. :-) [14:33:30] James_F: Hi, Maintenance bot [14:33:33] let me grab it [14:33:53] https://github.com/Ladsgroup/Phabricator-maintenance-bot/commit/7cc11de702b1a1b38dea743d5ec374f07f9a3118 [14:34:02] James_F: does this example make sense ^ [14:34:25] Sure, I guess? I'll write a patch and you can shout at me if it's wrong. :-) [14:35:28] <3 [14:37:28] Amir1: https://github.com/Ladsgroup/Phabricator-maintenance-bot/pull/35 [14:39:38] I think the _ should be replaced with " " but not super sure, technically it should work either way but phabricator is full of surprises [14:39:48] let's merge and test? [14:40:12] I can switch to spaces first if you want?\ [14:40:25] And how clever is it? [14:40:48] Because Phab search sometimes doesn't understand sub-project, but sometimes does, and I can never understand why. [14:40:50] it might have some trouble with the phase milestones on the team workboard [14:40:51] it's clever for the "in" part [14:41:01] But not for the not-in? [14:41:16] yeah [14:41:22] https://phabricator.wikimedia.org/maniphest/query/DfCGi_J.Sh7u/#R is filled with things that are not in the top-level project because they're in a milestone, which is not what we want. :-( [14:41:23] let's just merge [14:41:40] It might break things? [14:45:03] oh no [14:45:08] ? [14:45:16] worst case, it wouldn't work for you case [14:45:32] which we can test now [14:45:36] It won't edit war to move things from the milestone to the top-level? [14:45:40] Yeah, let's. [14:46:23] AFAIK no [14:46:39] One way to find out. :-) [14:47:22] James_F: make changes, and let me know to run the bot [14:47:33] so we don't have to wait half an hour [14:48:04] 10Project-Admins: Create project tag for WikiProject Music group - https://phabricator.wikimedia.org/T285767 (10Moebeus) [14:50:57] Amir1: OK, I've created T285768 and the bot should tag it with AW. I've also moved T285094 into Phase Zeta again, and hopefully the bot shouldn't move it up. :-) [14:50:58] T285768: Provide a means by which to display an automatically-populated "most used functions" list on the Main Page - https://phabricator.wikimedia.org/T285768 [14:50:59] T285094: Provide an API to search for implementation by Z14K1/Function - https://phabricator.wikimedia.org/T285094 [14:51:30] noted, let me start the bot [14:51:34] (03CR) 10Ahmon Dancy: [C: 03+1] "LGTM" [tools/scap] - 10https://gerrit.wikimedia.org/r/700520 (https://phabricator.wikimedia.org/T192703) (owner: 10Hashar) [14:54:26] "Maintenance_bot edited projects for T285367: UX for aliases, added: Abstract Wikipedia team; removed Abstract Wikipedia team (Phase ζ)." :-( [14:54:27] T285367: UX for aliases - https://phabricator.wikimedia.org/T285367 [14:54:56] On the plus side, "Maintenance_bot added a project to T285768: Provide a means by which to display an automatically-populated "most used functions" list on the Main Page: Abstract Wikipedia team." [14:56:52] Amir1: Should we revert? [14:57:18] I'm thinking a bit [14:57:26] let me see what can be done here [14:57:30] Sure. [14:59:34] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks localisation cache - https://phabricator.wikimedia.org/T285345 (10Majavah) >>! In T285345#7183782, @Jdforrester-WMF wrote: > OK, it looks like the issue might be that prod machines generally seem to have Python... [15:10:34] James_F: I think I found a somewhat easy solution, It'll be done soon [15:10:43] * James_F hugs Amir1 for being wonderful. [15:11:15] I wish I could do better, sorry for the mess [15:21:13] 10VPS-project-Codesearch, 10Performance-Team (Radar): Add performance/* repos to Codesearch - https://phabricator.wikimedia.org/T285771 (10Krinkle) [15:27:22] James_F: fixed and deployed, tested and it works fine, feel free to run further tests [15:34:31] Amir1: Thank you! [15:59:28] thcipriani: turns out I am fine doing the 1/1 if you wanna do it [15:59:39] hashar: sure [16:00:20] brennen: thanks so much!!! I'm here if needed for anything [16:02:33] brennen: there is actually one more thing that it would be nice to add to the pre-train backport, if it just happens that if would be no extra effort for you. It's not essential for the train, i.e. no ill effects if it doesn't ride the train, but it would be nice to get it out on the train instead of a separate backport deploy later, again, provided it doesn't make extra work for u :) thanks agian! [16:03:04] and apologies again for the mistake yesterday 8p [16:06:43] AndyRussG: i haven't checked the branch out locally yet, so backports are trivial at the moment [16:07:26] AndyRussG: soon would be good, however. :) [16:07:28] brennen: ah okok I'll merge the other thing into master and link it here, then, thanks!!!! [16:09:14] 10VPS-project-Codesearch, 10Performance-Team (Radar): Add performance/* repos to Codesearch - https://phabricator.wikimedia.org/T285771 (10Ladsgroup) The relationship between those marks and groups is a bit different. The mapping is basically this: `lang=python make_conf('core', args, core=True) make_c... [16:10:01] 10VPS-project-Codesearch, 10Performance-Team (Radar): Add performance/* repos to Codesearch - https://phabricator.wikimedia.org/T285771 (10Ladsgroup) We can also have a dedicated group called performance and make it show up in deployed, operations, or whatever you want. [16:10:48] brennen: it's this one: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/702151 [16:10:50] thanks again!!! [16:11:37] AndyRussG: k, i'll cherry pick and merge that once it passes CI [16:11:55] brennen: cool hugely appreciated, I'm about if anything needed! [16:36:53] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T281153 (10brennen) Also backporting https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/702167 per conversation with @AndyRussG, and waiting on https://... [16:38:05] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T281153 (10AndyRussG) @brennan thanks!!! [16:38:53] (^ oops corrected wrong spelling ahhh sorry) [16:41:19] no worries! :) [16:52:57] :) [17:11:12] 10Release-Engineering-Team (Radar), 10MW-on-K8s: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10thcipriani) p:05Triage→03High [17:13:20] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Regression: Unable to view mw-debug log in Jenkins - https://phabricator.wikimedia.org/T285335 (10thcipriani) p:05Triage→03Low [17:13:23] 10Release-Engineering-Team, 10user-sbassett: Permission to view mediawiki-config-pipeline-wmf-publish jobs - https://phabricator.wikimedia.org/T285782 (10sbassett) [17:14:36] 10Release-Engineering-Team, 10user-sbassett: Permission to view mediawiki-config-pipeline-wmf-publish jobs - https://phabricator.wikimedia.org/T285782 (10thcipriani) p:05Triage→03Medium a:03thcipriani Hey @sbassett I can get you all access to releases jenkins. [17:15:00] 10Release-Engineering-Team, 10user-sbassett: Permission to view mediawiki-config-pipeline-wmf-publish jobs - https://phabricator.wikimedia.org/T285782 (10sbassett) Thanks! [17:15:08] 10Release-Engineering-Team (Doing), 10user-sbassett: Permission to view mediawiki-config-pipeline-wmf-publish jobs - https://phabricator.wikimedia.org/T285782 (10thcipriani) [17:19:50] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10SRE, 10serviceops, 10Patch-For-Review: Check out www-portals repo in the mediawiki-webserver and in the mediawiki-multiversion images - https://phabricator.wikimedia.org/T285325 (10thcipriani) p:05Triage→03Medium [17:20:32] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks mediawiki - https://phabricator.wikimedia.org/T285345 (10thcipriani) [17:21:14] 10Release-Engineering-Team (Next), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks mediawiki - https://phabricator.wikimedia.org/T285345 (10thcipriani) [17:21:23] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Scap, 10User-Ladsgroup: image sizes not displayed on beta - https://phabricator.wikimedia.org/T285125 (10thcipriani) [17:21:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Radar): puppet CI task failing with: Cannot allocate memory - https://phabricator.wikimedia.org/T284998 (10thcipriani) [17:35:02] 10Release-Engineering-Team (Radar), 10Release Pipeline (Blubber): Can't use git master in blubber for dependencies without caching issues - https://phabricator.wikimedia.org/T283981 (10thcipriani) > Alternatively, is there a way I can look up the sha1 of HEAD or master of a Gerrit repo when the blubber file is... [17:35:45] 10VPS-project-Codesearch, 10Performance-Team (Radar): Add performance/* repos to Codesearch - https://phabricator.wikimedia.org/T285771 (10Krinkle) I see. OK. In that case let's do the first few under group `operations`, and WikimediaDebug under `wikimedia` since that one runs on MW web pages client-side. [17:37:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Developer Productivity, 10Regression: Unable to view mw-debug log in Jenkins - https://phabricator.wikimedia.org/T285335 (10Krinkle) [17:38:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Radar): puppet CI task failing with: Cannot allocate memory - https://phabricator.wikimedia.org/T284998 (10jbond) @hashar i have now merged #649684 and things feel smoother to me i think we can close this task now. however it would be usefu... [17:41:05] 10Release-Engineering-Team (Radar), 10Abstract Wikipedia team (Phase ζ), 10Release Pipeline (Blubber): More flexible treatment of requirements in `python`; control over `PYTHONPATH` - https://phabricator.wikimedia.org/T282795 (10thcipriani) >>! In T282795#7147340, @Jdforrester-WMF wrote: > @jeena suggests th... [17:42:45] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10phan: Phan job should use composer instead of vendor for release branches ( undeclared class \Wikimedia\Equivset\Equivset ) - https://phabricator.wikimedia.org/T231966 (10thcipriani) a:03hashar [17:45:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul, 10Documentation: Document Zuul problems caused by force merge - https://phabricator.wikimedia.org/T225955 (10thcipriani) 05Open→03Resolved a:03thcipriani Please improve https://www.mediawiki.org/wiki/Continuous_integration/Z... [17:46:03] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10Analytics-Radar, 10ChangeProp, and 3 others: Run EventBus tests in MediaWiki core CI - https://phabricator.wikimedia.org/T257583 (10thcipriani) Let us know if there's additional guidance needed here [17:48:52] 10Release-Engineering-Team (Radar), 10Prod-Kubernetes, 10serviceops, 10Kubernetes: CI pipeline/job to build and release helm chart artifacts - https://phabricator.wikimedia.org/T257333 (10thcipriani) Is this task superseded by the cronjob on the deployment machines that publishes to the chart museum? [17:51:15] 10Release-Engineering-Team (Radar), 10Quality-and-Test-Engineering-Team (QTE), 10Epic, 10Platform Team Initiatives (API Integration Tests), 10Testing-Roadblocks: Create and run a suite of end-to-end tests for the Wikimedia environment - https://phabricator.wikimedia.org/T248683 (10thcipriani) [17:53:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Librarization, 10Quibble: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T220723 (10thcipriani) [17:54:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Doing), 10ci-test-error: Various CI jobs failing after "mkdir: cannot create directory ‘log’: Permission denied" - https://phabricator.wikimedia.org/T282893 (10thcipriani) a:03hashar @hashar did you unassign because this is an upstream bu... [18:27:02] PROBLEM - SSH on contint2001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [18:38:52] 10Phabricator: Update Herald (H260) to include upcoming CommTech sprint milestones (5410, 5411, 5412, 5413) - https://phabricator.wikimedia.org/T285794 (10ldelench_wmf) [18:41:07] 10VPS-project-Codesearch, 10Patch-For-Review, 10Performance-Team (Radar), 10User-Ladsgroup: Add performance/* repos to Codesearch - https://phabricator.wikimedia.org/T285771 (10Ladsgroup) 05Open→03Resolved a:03Ladsgroup It will be live by tomorrow. [18:58:06] 10Release-Engineering-Team (Logspam), 10Growth-Team, 10MediaWiki-Watchlist, 10Patch-For-Review, and 2 others: WatchedItemStore should not write MapCacheLRU instances to a BagOStuff - https://phabricator.wikimedia.org/T282105 (10dmitriy.sky) @tstarling Question: in order to test what's in private property W... [19:17:28] 10Phabricator: Update Herald (H260) to include upcoming CommTech sprint milestones (5410, 5411, 5412, 5413) - https://phabricator.wikimedia.org/T285794 (10MBinder_WMF) 05Open→03Resolved Should be all set :) [19:18:22] 10Release-Engineering-Team (Doing), 10Scap, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks mediawiki - https://phabricator.wikimedia.org/T285345 (10dduvall) a:03dduvall [19:27:17] RECOVERY - SSH on contint2001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [19:39:19] 10Release-Engineering-Team (Next), 10Scap: Fix or drop disabled unit tests in Scap - https://phabricator.wikimedia.org/T283018 (10dancy) The problem is that assert_called() does not exist in Python 3.5. It showed up in 3.6. [19:39:59] (03PS1) 10Ahmon Dancy: Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) [19:40:54] (03CR) 10Ahmon Dancy: [C: 03+1] Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) (owner: 10Ahmon Dancy) [20:13:25] (03PS4) 10Ahmon Dancy: Use docker-compose exec instead of ssh [tools/train-dev] - 10https://gerrit.wikimedia.org/r/701993 [20:27:10] Project beta-scap-sync-world build #10867: 04FAILURE in 2 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/10867/ [20:37:27] Yippee, build fixed! [20:37:27] Project beta-scap-sync-world build #10868: 09FIXED in 2 min 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/10868/ [20:49:07] (03PS1) 10Ahmon Dancy: cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) [20:49:40] (03CR) 10jerkins-bot: [V: 04-1] cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) (owner: 10Ahmon Dancy) [20:51:09] (03PS2) 10Ahmon Dancy: cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) [20:52:01] (03PS3) 10Ahmon Dancy: cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) [21:17:37] 10Release-Engineering-Team (Doing), 10Scap, 10Patch-For-Review, 10WMDE-TechWish-Sprint-2021-06-23: Python3 scap breaks mediawiki - https://phabricator.wikimedia.org/T285345 (10dancy) Scap has been reverted on all beta notes and CDB files have been regenerated and redeployed. [21:27:08] (03PS4) 10Dduvall: cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) (owner: 10Ahmon Dancy) [21:27:43] (03CR) 10Ahmon Dancy: [C: 03+1] cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) (owner: 10Ahmon Dancy) [21:28:05] (03CR) 10Dduvall: [C: 03+2] cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) (owner: 10Ahmon Dancy) [21:28:44] (03Merged) 10jenkins-bot: cdblib.py: Python 3 fixes [tools/scap] - 10https://gerrit.wikimedia.org/r/702233 (https://phabricator.wikimedia.org/T285345) (owner: 10Ahmon Dancy) [21:45:33] !log urbanecm@deployment-deploy01:~$ mwscript extensions/GrowthExperiments/maintenance/updateMenteeData.php --wiki=cswiki # T285811 [21:45:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:45:37] T285811: Mentee overview module: Run updateMenteeData.php regularly - https://phabricator.wikimedia.org/T285811 [21:56:38] (03PS2) 10Brennen Bearnes: rename buster to fundraising-buster, create common [releng/dev-images] - 10https://gerrit.wikimedia.org/r/698878 (https://phabricator.wikimedia.org/T273100) [22:06:00] (03CR) 10Thcipriani: [C: 03+1] "One inline question about removing a "fixme_". Other than that: LGTM! (+2 in spirit)" (032 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) (owner: 10Ahmon Dancy) [22:06:11] (03PS2) 10Jeena Huneidi: WIP: add buster php images [releng/dev-images] - 10https://gerrit.wikimedia.org/r/698880 (https://phabricator.wikimedia.org/T273100) (owner: 10Brennen Bearnes) [22:06:49] 10Gerrit, 10Mobile: Can't type slash character ("/") in Gerrit web editor - https://phabricator.wikimedia.org/T282387 (10Paladox) This seems fixed at least on gerrit master... I'm not sure what change fixed it tho. [22:07:13] 10Gerrit, 10Upstream: Gerrit: issues with Responsive Web Design for mobile view - https://phabricator.wikimedia.org/T256547 (10Paladox) Fixed with https://gerrit-review.googlesource.com/c/gerrit/+/310663 [22:13:04] (03PS2) 10Ahmon Dancy: Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) [22:13:41] 10Release-Engineering-Team (Doing), 10user-sbassett: Permission to view mediawiki-config-pipeline-wmf-publish jobs - https://phabricator.wikimedia.org/T285782 (10thcipriani) 05Open→03Resolved You all should have permission to login and read now: https://releases-jenkins.wikimedia.org/ [22:15:38] (03PS3) 10Ahmon Dancy: Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) [22:16:47] (03CR) 10Thcipriani: [C: 03+2] Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) (owner: 10Ahmon Dancy) [22:16:56] (03CR) 10Ahmon Dancy: Fix disabled unit tests in Scap (032 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) (owner: 10Ahmon Dancy) [22:17:26] (03Merged) 10jenkins-bot: Fix disabled unit tests in Scap [tools/scap] - 10https://gerrit.wikimedia.org/r/702221 (https://phabricator.wikimedia.org/T283018) (owner: 10Ahmon Dancy) [22:18:03] urbanecm: hello, just asking. Usually I see sal log entries like your one above (https://sal.toolforge.org/log/9YC8WXoBa_6PSCT9C0-v) in the production log and not the releng one. Was this a mistake? [22:18:52] hi zabe, no, it was intention. If you look at the start of my SAL message, you'll see the command was ran at deployment-deploy01. In another words, it was executed at the beta cluster [22:19:48] ok, thanks for the explanation :) [22:20:06] in general, production log should have entries about production service. Beta runs in WMCS, not in production :-) [22:20:14] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review, 10User-brennen: Design m8s deployment workflows and tooling - https://phabricator.wikimedia.org/T279322 (10brennen) [22:20:15] Also see https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep, which says "This RelEng-specific SAL seems to be preferred over the generic WMCS one included above (if you know more, please edit this note!)" [22:22:34] zabe: ^ [22:22:44] (03PS3) 10Brennen Bearnes: WIP: add buster php images [releng/dev-images] - 10https://gerrit.wikimedia.org/r/698880 (https://phabricator.wikimedia.org/T273100) [22:23:29] yes, thanks, I'm gonna take a look at that [22:23:52] any time :) [22:29:25] 10Release-Engineering-Team (Next), 10Scap, 10Patch-For-Review: Fix or drop disabled unit tests in Scap - https://phabricator.wikimedia.org/T283018 (10dancy) 05Open→03Resolved [23:15:21] 10Gerrit, 10Mobile: Can't type slash character ("/") in Gerrit web editor - https://phabricator.wikimedia.org/T282387 (10Paladox) Ok, I know how it was fixed upstream though I'm not sure if that change can be back ported. [23:56:18] 10Release-Engineering-Team: TrainBranchBot merges code to mediawiki-config master branch, causing undeployed code problem - https://phabricator.wikimedia.org/T285819 (10Urbanecm) [23:56:45] 10Release-Engineering-Team: TrainBranchBot merges code to mediawiki-config master branch, causing undeployed code problem - https://phabricator.wikimedia.org/T285819 (10Urbanecm) p:05Triage→03Unbreak! Per description (blocks wmf.12 => UBN by definition). [23:57:14] 10Release-Engineering-Team: TrainBranchBot merges code to mediawiki-config master branch, causing undeployed code problem - https://phabricator.wikimedia.org/T285819 (10Urbanecm) [23:57:32] 10Release-Engineering-Team: TrainBranchBot merges code to mediawiki-config master branch, causing undeployed code problem - https://phabricator.wikimedia.org/T285819 (10Urbanecm)