[00:01:02] Hm.. in my mind this makes sense if wgAutoloadClasses was defined in LocalSettings, but it's not, it's defined in Setup.php, which we dont load in that way [00:01:09] Ah I see what you mean. [00:01:23] wgAutoloadClasses is defined Setup, which is loaded using requireOnceInGlobalScope which will "fix" the scope right after that file returns [00:01:34] but we call finalSetup() from _CALLBACK which Setup.php calls midway [00:02:52] so.. should adding ` global $wgAutoloadClasses;` to the wgScopeTest block fix it? [00:03:09] I have your patch checked out with the Setup.php part reverted and the above added instead [00:03:15] but it does not resolve the null + array fatal [00:05:16] Adding ` global $wgAutoloadClasses;` to TestSetup::requireOnceInGlobalScope before the `require_once` does work however. [00:09:02] I think it applies to wgConf as well [00:09:09] and basically any other variables that are newly created by Setup.php [00:11:38] Yes, your explanation is correct [00:12:08] And also yes, it does apply to other variables, too [00:12:29] All non-config variables, in fact [00:15:17] Daimona: Right, but those will be taken care of by the end of requireOnceInGlobalScope [00:15:46] so we really only need to hotpatch vars that we internally use within Setup.php, and afaik we have no calls to the outside from there except our own TestsAutoLoader [00:16:25] As for adding wgAutoloadClasses to the wgScopeTest hack: the hack runs after the variable's been declared ;-) [00:16:43] Yes, when Setup.php returns everything will be fine [00:16:58] Problems can only happen in things that are not Setup.php, but run before Setup.php returns [00:17:08] Setup callbacks are an example, extension functions are another [00:19:18] Your fix to requireOnceInGlobalScope will also work, I think [00:20:31] But I have to say, config being in global scope is a source of massive headaches, you can never really be sure what works and what doesn't... [00:26:28] 10Project-Admins: Create project tag for JWTAuth - https://phabricator.wikimedia.org/T341537 (10JeffreyWang) [06:31:36] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [07:10:26] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [07:14:32] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [07:47:14] 10Project-Admins: Create project tag for JWTAuth - https://phabricator.wikimedia.org/T341537 (10Aklapper) Hi, do you know any implementation details already? Will this be a MediaWiki extension, or part of MediaWiki core, or something else? Thanks! [07:55:06] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [07:58:07] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [07:58:41] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [08:01:27] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=fb9b83f1-475c-4737-a872-7868377e05ee) set by jelto@cumin1... [08:01:54] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=af763fea-db6d-494f-8c4c-8139c0ceab0c) set by jelto@cumin1... [08:03:29] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [08:10:17] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [08:15:33] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Traffic-Icebox: Investigate ATS/Varnish serving stall/cached Zuul status json - https://phabricator.wikimedia.org/T341548 (10hashar) [08:20:23] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [08:21:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Traffic-Icebox: Investigate ATS/Varnish serving stall/cached Zuul status json - https://phabricator.wikimedia.org/T341548 (10hashar) p:05Triage→03Low [08:26:42] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [08:34:12] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [08:38:55] PROBLEM - Check no envoy runtime configuration is left persistent on doc1003 is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string entries: {} not found on http://localhost:9631/runtime - 433 bytes in 0.003 second response time https://wikitech.wikimedia.org/wiki/Application_servers/Runbook%23Envoy [08:42:26] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [08:45:28] !log integration: removed security rules allowing contint2001 [208.80.153.15] on `integration`, `puppet-diffs` and `deployment-prep`: services got moved to contint2002 | T324659 [08:45:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:45:31] T324659: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 [08:56:38] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Jelto) [09:20:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Traffic-Icebox: Investigate ATS/Varnish serving stall/cached Zuul status json - https://phabricator.wikimedia.org/T341548 (10hashar) [09:27:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Traffic-Icebox: Investigate ATS/Varnish serving stall/cached Zuul status json - https://phabricator.wikimedia.org/T341548 (10hashar) [09:27:56] 10Phabricator (Upstream), 10I18n, 10Upstream: Says “1 Story Points”, needs proper singular - https://phabricator.wikimedia.org/T142285 (10Aklapper) We use a custom name "Story Points" instead of "Points" but the thingy at the top comes from `src/applications/maniphest/controller/ManiphestTaskDetailControlle... [09:28:26] 10Release-Engineering-Team (Deployment Training Requests): Deployment Training Request for jebe - https://phabricator.wikimedia.org/T341559 (10JEbe-WMF) [09:29:29] 10Phabricator: Enhance tokens by adding rainbow-related ones - https://phabricator.wikimedia.org/T337986 (10Aklapper) p:05Triage→03Low [09:30:46] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10User-brennen: Clean up Phabricator production error logs - https://phabricator.wikimedia.org/T337500 (10Aklapper) p:05Triage→03Low [09:32:20] 10Phabricator: Phabricator confirmation e-mail never arrives for @Raimund-Liebert-WMAT, leaving the account unable - https://phabricator.wikimedia.org/T163878 (10Aklapper) 05Open→03Resolved Assuming this is obsolete as the account shows as verified nowadays. Please reopen if this is still an issue. [09:33:30] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟): Clarify GPL 2.0 or later license on Phabricator - https://phabricator.wikimedia.org/T338762 (10Aklapper) p:05Triage→03Low [09:33:48] 10Phabricator, 10Dumps-Generation: Compress phabricator dump - https://phabricator.wikimedia.org/T262744 (10Aklapper) p:05Triage→03Low [09:34:21] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Traffic-Icebox: Investigate ATS/Varnish serving stall/cached Zuul status json - https://phabricator.wikimedia.org/T341548 (10hashar) The backend service providing the data for https://integration.wikimedia.org/zuul/status.json was stopped... [09:35:43] Project mediawiki-core-doxygen-docker build #44742: 04FAILURE in 31 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/44742/ [09:36:15] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Lucas_Werkmeister_WMDE) Just because I first saw that error after CI came back from maintenance: do you think there’s any... [09:37:50] 10Phabricator, 10DC-Ops: Update SRE Procurement Request template - https://phabricator.wikimedia.org/T305366 (10Aklapper) Is there consensus to make this change / could a discussion and decision take place, please? Otherwise this task should be declined. Current line is `**Networking Setup:** # of Connections... [09:38:12] 10Phabricator: Uninstall Differential (Phabricator application) - https://phabricator.wikimedia.org/T330797 (10Aklapper) p:05Triage→03Medium [09:38:33] 10Phabricator, 10DC-Ops: Update SRE Procurement Request template - https://phabricator.wikimedia.org/T305366 (10Aklapper) p:05Triage→03Low [09:38:46] 10Phabricator: Uninstall Owners (Phabricator application) - https://phabricator.wikimedia.org/T336907 (10Aklapper) p:05Triage→03Low [09:39:09] 10Phabricator: Convert (confusing) #Triagers to #acl*Batch-Editors or similar - https://phabricator.wikimedia.org/T126953 (10Aklapper) p:05Triage→03Low [09:39:36] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10collaboration-services, 10Patch-For-Review, 10User-brennen: Migrate phabricator.wikimedia.org to Phorge as upstream - https://phabricator.wikimedia.org/T333885 (10Aklapper) p:05Triage→03High [09:39:50] 10Phabricator, 10Release-Engineering-Team, 10User-brennen: Move phabricator/deployments repository from Differential to Gerrit - https://phabricator.wikimedia.org/T321380 (10Aklapper) p:05Triage→03High [09:40:25] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Document scap3 deployment of Phabricator - https://phabricator.wikimedia.org/T313956 (10Aklapper) p:05Triage→03Medium [09:41:10] 10Phabricator, 10SRE, 10Traffic: Accessing Phabricator from Tor (some ranges blocked but not others) - https://phabricator.wikimedia.org/T254568 (10Aklapper) 05Open→03Resolved Optimistically resolving as T253632 is resolved. Please reopen if this is still an issue - thanks! [09:42:00] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) >>! In T324659#9004463, @Lucas_Werkmeister_WMDE wrote: > Just because I first saw that error after CI came back fr... [09:42:08] 10Phabricator (Upstream): Allow partially locking Task comments - https://phabricator.wikimedia.org/T292505 (10Aklapper) [09:42:55] 10Phabricator: move_project script allows using --keep-members for milestones which cannot have members - https://phabricator.wikimedia.org/T224420 (10Aklapper) p:05Triage→03Low [09:43:34] 10Phabricator: Exclude specific tags globally when subtasking directly - https://phabricator.wikimedia.org/T220835 (10Aklapper) p:05Triage→03Low [09:44:20] 10Phabricator, 10phabricator maintenance bot, 10Documentation: Cover when to use Herald and when to prefer @Maintenance_bot - https://phabricator.wikimedia.org/T282367 (10Aklapper) p:05Triage→03Medium [09:44:24] 10Phabricator, 10User-DannyS712: Combine H32 and H194 - https://phabricator.wikimedia.org/T236647 (10Aklapper) p:05Triage→03Lowest [09:46:11] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) [09:58:54] 10Phabricator, 10Commons: Review Global Herald H109 - Add #Commons project to #MediaWiki-File-management and #MediaWiki-extensions-GWToolset - https://phabricator.wikimedia.org/T250026 (10Aklapper) p:05Triage→03Low [09:59:19] 10Phabricator, 10Commons: Review Global Herald H109 - Add #Commons project to #MediaWiki-File-management and #MediaWiki-extensions-GWToolset - https://phabricator.wikimedia.org/T250026 (10Aklapper) Hmm, how to make decisions and change... The #Commons project tag in Phab has 32 "members"; 15 have been active i... [09:59:25] 10Phabricator: Extend move_project.php to allow moving child projects - https://phabricator.wikimedia.org/T253214 (10Aklapper) p:05Triage→03Low [09:59:35] 10Phabricator: Autogeneration of Subtasks - https://phabricator.wikimedia.org/T244782 (10Aklapper) p:05Triage→03Lowest [10:00:59] 10Phabricator: Cannot create new Mock: RuntimeException: Illegal offset type in isset or empty - https://phabricator.wikimedia.org/T333089 (10Aklapper) 05Stalled→03Declined @Bennylin: If you can reproduce this, please reopen and provide a full list of steps to reproduce which leaves zero room for interpretat... [10:10:57] 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10API Platform, 10Anti-Harassment, and 18 others: Migrate PipelineLib repos to GitLab - https://phabricator.wikimedia.org/T332953 (10fnegri) [10:14:26] 10Phabricator, 10User-DannyS712: Combine H32 and H194 - https://phabricator.wikimedia.org/T236647 (10Aklapper) I think I'd prefer not to do this. Indeed, the outcome/actions are the same, but the condition feel like very unrelated to another :) [10:18:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10collaboration-services: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10hashar) [10:18:19] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) 05In progress→03Resolved [10:18:23] 10Continuous-Integration-Infrastructure, 10SRE, 10collaboration-services, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10hashar) a:03Jelto As far as I can tell, the services were successfully switched over from contint2001 to contint2002. I... [10:35:43] Project mediawiki-core-doxygen-docker build #44743: 04STILL FAILING in 31 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/44743/ [10:56:13] (03CR) 10Func: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/936740 (owner: 10Func) [10:59:28] 10Phabricator, 10cloud-services-team: Organize additional projects as #cloud-services subprojects - https://phabricator.wikimedia.org/T177787 (10fnegri) [11:07:59] (03PS1) 10Hashar: Switch Jenkins maintenance jobs to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937068 (https://phabricator.wikimedia.org/T324659) [11:09:31] (03PS1) 10Hashar: jjb: switch jobs from contint2001 to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937069 (https://phabricator.wikimedia.org/T324659) [11:09:57] (03CR) 10Hashar: [C: 03+2] Switch Jenkins maintenance jobs to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937068 (https://phabricator.wikimedia.org/T324659) (owner: 10Hashar) [11:11:10] (03Merged) 10jenkins-bot: Switch Jenkins maintenance jobs to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937068 (https://phabricator.wikimedia.org/T324659) (owner: 10Hashar) [11:12:03] Yippee, build fixed! [11:12:04] Project mediawiki-core-doxygen-docker build #44744: 09FIXED in 7 min 57 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/44744/ [11:12:21] !log integration: applied label `contint2001` to agent `contint2002` to let it processes pending jobs after the host switch over (T324659). Jobs updated by https://gerrit.wikimedia.org/r/937068 and https://gerrit.wikimedia.org/r/937069 [11:12:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:12:24] T324659: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 [11:12:45] (03CR) 10Hashar: [C: 03+2] jjb: switch jobs from contint2001 to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937069 (https://phabricator.wikimedia.org/T324659) (owner: 10Hashar) [11:13:56] (03Merged) 10jenkins-bot: jjb: switch jobs from contint2001 to contint2002 [integration/config] - 10https://gerrit.wikimedia.org/r/937069 (https://phabricator.wikimedia.org/T324659) (owner: 10Hashar) [12:57:38] Hi :) [12:57:43] Patches made to AQS 1.0 would trigger the node16-docker to run test which will obviously fail because the last I can remember, it runs on 10 - at most 12 version of Node [12:57:43] Integration config says AQS 1.0 should use node 16 - https://gerrit.wikimedia.org/g/integration/config/+/8aea91b31d80ba9309615d44453806fade670035/zuul/layout.yaml#8709 [12:57:43] This won’t work and all tests will fail CI. I want to change that to a lower version of node or is there a way to force the node version to bypass CI version of node? [12:57:52] The patch I’m referring to is: https://gerrit.wikimedia.org/r/c/analytics/aqs/+/937073 [12:58:30] Should I just change the node version in integration config for that repo or is there something I can do to use node 10 - 12? [13:05:47] 10GitLab, 10Patch-For-Review: Add tests+linting to docpub - https://phabricator.wikimedia.org/T341225 (10CodeReviewBot) jnuche opened https://gitlab.wikimedia.org/repos/releng/docpub/-/merge_requests/3 add linting and tests to the project [13:06:11] xSavitar: Argh, there's no support for node 10 or 12 in CI any more and hasn't been for years. [13:06:36] xSavitar: Theoretically you could move the repo to the pipeline and manage that risk yourself, but you really should fix that urgently. [13:07:18] xSavitar: Node 14 is on the way out (only around because of selenium tests needing to be upgraded). Soon we'll be moving repos to node 18. [13:08:12] people have been force-merging changes in that repo for almost two years? /o\ https://gerrit.wikimedia.org/r/c/analytics/aqs/+/713910 [13:08:20] Oh dear. [13:09:21] yeah Lucas_WMDE and that is what I want to avoid. [13:09:40] The best solution is to get RESTBase or AQS 1.0 work on Node 16 but meh [13:10:03] Isn't RESTBase dead now? [13:10:30] James_F, not yet! Some things still rely on it and we're trying to migrate everything away from it to API Gateway for example. [13:10:45] Ah. :-( [13:11:02] I'm loosing my mind here. So many bottle necks :D [13:11:48] The worse part is AQS 1.0 was a fork of RESTBase and there is really little I can do to take out the RB piece out of it until AQS 2.0 is shipped. [13:12:04] Ick. [13:26:19] xSavitar: there is no node10 support anymore :/ [13:29:20] theorically you could use the service pipeline to run the tests in a Docker image which manually installs node10 from eg https://nodejs.org/download/release/v10.24.1/node-v10.24.1.tar.gz [13:31:27] similar to what we are doing https://gerrit.wikimedia.org/g/integration/config/+/refs/heads/master/dockerfiles/node16/Dockerfile.template [13:31:47] but inside the analytics/aqs repo [13:52:31] xSavitar: also the build states `Cannot find module 'test/features/mediawiki-history-metrics/fake-druid.js'` so maybe it can't find it cause it is relative but the current working directory when running tests is different [13:52:48] yeah that is definitely path related [13:53:36] I got 281 tests passing using node14 by using absolute path in test/aqs_test_module.yaml for test/features/mediawiki-history-metrics/fake-druid.js [13:53:52] and using absolute paths for the the various sys/* paths in projects/aqs_default.yaml [13:55:23] so I guess the tests should run from a different directory or be given a base path when resolving relative paths :] [14:14:17] Thanks hashar, I'll poke around and see how to resolve this. <2 [14:14:25] *<3 [14:15:19] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Jenkins plugins security advisory - 2023-07-12 - https://phabricator.wikimedia.org/T341584 (10jnuche) [14:15:46] hashar, what about Node 16? If absolute paths are given and that is the only issue, I can fix that and make a patch to resolve the failure. [14:47:55] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10PWaigi-WMF) Hi, I'd like to request access to additional permissions because I will be creating new projects & boards for the new year. I'm the Product Manager for [[ https://phabri... [14:59:52] 10Release-Engineering-Team, 10API Platform, 10AQS2.0, 10Platform Engineering, and 4 others: Define a procedure/pattern to populate test environments - https://phabricator.wikimedia.org/T334851 (10Sfaci) [15:02:11] 10Phabricator, 10Browser-Support-Firefox: Firefox: Minor UI glitch in task Subscribers field: Unneeded space at beginning of a line if previous line included CJK characters - https://phabricator.wikimedia.org/T246434 (10Aklapper) 05Open→03Resolved I cannot reproduce anymore in recent Firefox and Chromium b... [15:09:26] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Ladsgroup) >>! In T706#9005642, @PWaigi-WMF wrote: > Hi, > I'd like to request access to additional permissions because I will be creating new projects & boards for the new year. I'... [15:15:38] xSavitar: I'd guess node 12 made some change to how `require` works [15:17:19] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T340245 (10matmarex) [15:22:12] 10Differential, 10Diffusion, 10Phabricator, 10Diffusion-Repository-Administrators: Rename repository names in Diffusion which have different names than in (canonical) Gerrit or Github (consistency) - https://phabricator.wikimedia.org/T135403 (10Aklapper) [15:24:37] hashar, what's the absolute path you're using? [15:32:53] local to my env so I have used eg `/aqs/sys/table.js` instead of the relative `sys/table.js` [15:33:39] fun thing, the repo depends on phantomjs which ceased to be a thing back in 2015 or so :-] [15:34:20] xSavitar: the build failure had a stacktrace showing the require failed to find `test/features/mediawiki-history-metrics/fake-druid.js` [15:34:53] then I have no idea from where in the code that is resolved but the stracktrace should have a clue [15:41:53] 10Release-Engineering-Team, 10API Platform, 10AQS2.0, 10Platform Engineering, and 4 others: Define a procedure/pattern to populate test environments - https://phabricator.wikimedia.org/T334851 (10Sfaci) Just to summarize and collect what we have discussed and learned so far: - We have discovered a new wa... [15:48:04] 10Release-Engineering-Team (They Live 🕶️🧟), 10Developer-Advocacy (Jul-Sep 2023): phabricator: quarterly_metrics.sh: Improve Bitergia instructions - https://phabricator.wikimedia.org/T341064 (10Aklapper) 05In progress→03Resolved Thanks! [15:53:25] 10Phabricator (Upstream), 10Upstream: Allow locking adding Task comments to members of certain projects - https://phabricator.wikimedia.org/T292505 (10Aklapper) [16:30:24] 10Release-Engineering-Team, 10API Platform, 10AQS2.0, 10Platform Engineering, and 4 others: Define a procedure/pattern to populate test environments - https://phabricator.wikimedia.org/T334851 (10Htriedman) I would be strongly in favor of using mock data over synthetic data, at least for the moment. We sho... [16:40:33] 10Phabricator, 10SRE, 10Traffic, 10SecTeam-Processed: Accessing Phabricator from Tor (some ranges blocked but not others) - https://phabricator.wikimedia.org/T254568 (10sbassett) [16:51:37] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: GitLab CI: "ENOSPC: no space left on device, mkdir" - https://phabricator.wikimedia.org/T340586 (10hashar) The Jenkins agents now have a 90G disk via the flavor `g3.cores8.ram24.disk20.ephemeral90.4xiops`. I have rebuild them all last week and they no mo... [16:52:49] 00:00:26.580 E RecursionError: maximum recursion depth exceeded [16:52:50] joy [16:52:51] of [16:52:53] python [16:53:03] (which of course I can not reproduce bah) [17:03:49] 10Gerrit, 10collaboration-services: Gerrit LFS objects lack an automatic sync to gerrit replicas - https://phabricator.wikimedia.org/T257741 (10hashar) >>! In T257741#8991066, @Dzahn wrote: > Isn't this a conflict with the statement that LFS data is not needed on replica hosts? Although there is nothing fetch... [17:07:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10collaboration-services: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10hashar) 05Open→03Resolved a:05Jelto→03Dzahn @Jelto and I did the switch over of contint2002 this morning So I believe that is f... [18:05:47] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: GitLab CI: "ENOSPC: no space left on device, mkdir" - https://phabricator.wikimedia.org/T340586 (10thcipriani) [18:05:58] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥): WMCS GitLab runners running frequently running out of disk space - https://phabricator.wikimedia.org/T340887 (10thcipriani) [18:06:53] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥): WMCS GitLab runners running frequently running out of disk space - https://phabricator.wikimedia.org/T340887 (10thcipriani) Closed T340586 as duplicate. Latest comment there probably relevant here, too: >>! In T340586#9006156, @hashar wrote: > The Jen... [18:10:54] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84 (10Aklapper) [18:15:18] For the records, I'm performing some project policy changes in Phabricator to harmonize settings (related to see less vandalism in Phab), so folks might get some uninteresting mail notifications - sorry [18:18:15] * TheresNoTime misread that as "interesting mail notifications" :( [18:20:19] TheresNoTime: depends on your interests I guess [18:20:29] :-P [18:21:07] https://phabricator.wikimedia.org/T341604 if you're curious - not a public task currently because WP:BEANS [19:29:45] 10Continuous-Integration-Config, 10Wikimedia-Site-requests, 10User-Urbanecm: CI should ensure that wmf-config/logos.php matches logos/config.yaml - https://phabricator.wikimedia.org/T329231 (10Stang) Hi @Urbanecm, could you explain why only python 3.9 is supported in tox.ini? I'm on a newly installed Debian... [20:10:52] 10Gerrit, 10collaboration-services: Gerrit LFS objects lack an automatic sync to gerrit replicas - https://phabricator.wikimedia.org/T257741 (10Dzahn) > systemd timers triggering rsync For that all we need is to use rsync::quickdatacopy like we use it all over the place and set "autosync = true". It already c... [20:24:07] 10Phabricator, 10wikimedia.biterg.io: Closed tickets in Bugzilla migrated without a closing date - https://phabricator.wikimedia.org/T107254 (10matmarex) It looks like you have no special handling for bugs that were closed and then reopened, so e.g. bug 1 (T2001) is listed as having been resolved in 2006. I do... [20:32:33] 10Phabricator, 10wikimedia.biterg.io: Closed tickets in Bugzilla migrated without a closing date - https://phabricator.wikimedia.org/T107254 (10Dzahn) >>! In T107254#8999820, @valerio.bozzolan wrote: > Fortunately the Glorious Bugzilla Archivers, also archived these pages: > > https://static-bugzilla.wikimedi... [21:30:41] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T340245 (10dduvall) [21:40:01] (03PS1) 10Superpes15: zuul: Add Anoop (~aanzx) to allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/937186 [21:41:38] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10collaboration-services, 10Patch-For-Review, 10User-brennen: Migrate phabricator.wikimedia.org to Phorge as upstream - https://phabricator.wikimedia.org/T333885 (10CodeReviewBot) brennen closed https://gitlab.wikimedia.org/repos/phabricator/depl... [21:42:25] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10collaboration-services, 10Patch-For-Review, 10User-brennen: Migrate phabricator.wikimedia.org to Phorge as upstream - https://phabricator.wikimedia.org/T333885 (10CodeReviewBot) brennen opened https://gitlab.wikimedia.org/repos/phabricator/depl... [21:46:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Machine-Learning-Team: Python torch fills disk of CI Jenkins instances - https://phabricator.wikimedia.org/T338317 (10hashar) As an update, I have rebuild all the Jenkins agent instances last week. The disk space allocated to Docker went... [22:33:48] (03CR) 10Hashar: [C: 03+2] zuul: Add Anoop (~aanzx) to allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/937186 (owner: 10Superpes15) [22:34:32] (03CR) 10Hashar: [C: 03+2] zuul: Add Dringsim (沈澄心) to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/936740 (owner: 10Func) [22:34:52] (03Merged) 10jenkins-bot: zuul: Add Anoop (~aanzx) to allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/937186 (owner: 10Superpes15) [22:35:39] (03Merged) 10jenkins-bot: zuul: Add Dringsim (沈澄心) to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/936740 (owner: 10Func) [22:40:43] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/937186 & https://gerrit.wikimedia.org/r/936740 [22:40:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:40:52] (03CR) 10Hashar: [C: 03+2] "Deployed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/937186 (owner: 10Superpes15) [22:41:03] (03CR) 10Hashar: [C: 03+2] "Deployed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/936740 (owner: 10Func) [23:03:42] 10GitLab: GitLab push rules for PhpStorm project security - https://phabricator.wikimedia.org/T341638 (10tstarling) [23:55:41] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥): WMCS GitLab runners running frequently running out of disk space - https://phabricator.wikimedia.org/T340887 (10dduvall) I've been unable to reproduce this issue locally, but I'm in conversation with upstream about the issue. I will submit the `/var/li...