[00:06:56] * DannyS712 is debugging on beta cluster [00:14:22] Project beta-scap-eqiad build #337016: 04FAILURE in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/337016/ [00:24:20] ^ is me [00:24:26] Project beta-scap-eqiad build #337017: 04STILL FAILING in 0.73 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/337017/ [00:28:58] * DannyS712 is done [00:53:53] Project beta-scap-eqiad build #337018: 04STILL FAILING in 19 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/337018/ [01:08:54] Yippee, build fixed! [01:08:54] Project beta-scap-eqiad build #337019: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/337019/ [08:08:38] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [08:11:12] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [08:41:54] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MediaWiki-Docker, 10dev-images, 10User-brennen: XDebug not present in docker-registry.wikimedia.org/dev/stretch-php72-fpm:2.0.0 - https://phabricator.wikimedia.org/T273682 (10kostajh) A workaround i... [08:43:36] 10Beta-Cluster-Infrastructure, 10SRE, 10SRE-swift-storage: Beta cluster Swift backend instances are missing profile::swift::storage::rsync_limit_memory_percent (puppet fails) - https://phabricator.wikimedia.org/T274092 (10hashar) [09:06:56] <_joe_> !log cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/657139 in deployment-prep [09:06:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:07:41] (03Abandoned) 10Giuseppe Lavagetto: Move blubber to use buster [blubber] - 10https://gerrit.wikimedia.org/r/656137 (https://phabricator.wikimedia.org/T271901) (owner: 10Giuseppe Lavagetto) [09:18:53] (03CR) 10Hashar: [C: 03+2] Archive the OpenStreetMapSlippyMap extension [integration/config] - 10https://gerrit.wikimedia.org/r/662016 (https://phabricator.wikimedia.org/T257873) (owner: 10Zoranzoki21) [09:20:48] (03Merged) 10jenkins-bot: Archive the OpenStreetMapSlippyMap extension [integration/config] - 10https://gerrit.wikimedia.org/r/662016 (https://phabricator.wikimedia.org/T257873) (owner: 10Zoranzoki21) [10:55:04] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [cloud/toolforge/jobs-framework-api] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/662651 [10:55:06] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [cloud/toolforge/jobs-framework-api] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/662651 (owner: 10QChris) [10:55:12] (03PS1) 10QChris: Import done. Revoke import grants [cloud/toolforge/jobs-framework-api] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/662652 [10:55:15] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [cloud/toolforge/jobs-framework-api] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/662652 (owner: 10QChris) [11:42:44] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services): releng/phpmetrics should use composer.json / composer.lock - https://phabricator.wikimedia.org/T269203 (10hashar) 05Open→03Declined Self declining too many ideas. [11:59:42] (03PS1) 10Hashar: dockerfiles: update quibble-buster-php72 packages [integration/config] - 10https://gerrit.wikimedia.org/r/662663 (https://phabricator.wikimedia.org/T250515) [11:59:44] (03PS1) 10Hashar: jjb: update jobs for quibble-buster-php72:0.0.45-s4 [integration/config] - 10https://gerrit.wikimedia.org/r/662664 (https://phabricator.wikimedia.org/T250515) [12:00:05] (03CR) 10Hashar: [C: 03+2] dockerfiles: update quibble-buster-php72 packages [integration/config] - 10https://gerrit.wikimedia.org/r/662663 (https://phabricator.wikimedia.org/T250515) (owner: 10Hashar) [12:01:25] (03Merged) 10jenkins-bot: dockerfiles: update quibble-buster-php72 packages [integration/config] - 10https://gerrit.wikimedia.org/r/662663 (https://phabricator.wikimedia.org/T250515) (owner: 10Hashar) [12:55:17] Hi o/ is there a "nice" way to rebuild a bunch of docker images from different projects to have https://gerrit.wikimedia.org/r/c/blubber/+/660771 included? [13:01:45] (03PS1) 10Hashar: Remove 3d2png Jessie based image [integration/config] - 10https://gerrit.wikimedia.org/r/662693 (https://phabricator.wikimedia.org/T225678) [13:02:12] (03PS2) 10Hashar: Remove 3d2png Jessie based image [integration/config] - 10https://gerrit.wikimedia.org/r/662693 (https://phabricator.wikimedia.org/T225678) [13:03:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10hash... [13:09:41] (03PS3) 10Hashar: Remove 3d2png Jessie based image [integration/config] - 10https://gerrit.wikimedia.org/r/662693 (https://phabricator.wikimedia.org/T225678) [13:09:43] (03PS1) 10Hashar: Remove tilerator / kartotherian NodeJS 6 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/662698 (https://phabricator.wikimedia.org/T211784) [13:11:13] 10Scap, 10good first task: Fix typo in Scap release checklist - https://phabricator.wikimedia.org/T255856 (10Iamzac) @LarsWirzenius is this issue already resolved? [13:14:34] (03PS2) 10Hashar: Remove tilerator / kartotherian NodeJS 6 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/662698 (https://phabricator.wikimedia.org/T211784) [13:16:29] (03PS1) 10Hashar: jjb: delete pywikibot-i18n-npm-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662702 (https://phabricator.wikimedia.org/T211784) [13:26:00] 10Beta-Cluster-Infrastructure, 10SRE, 10SRE-swift-storage, 10Patch-For-Review: Beta cluster Swift backend instances are missing profile::swift::storage::rsync_limit_memory_percent (puppet fails) - https://phabricator.wikimedia.org/T274092 (10fgiunchedi) My bad, I introduced the new parameter in the context... [13:29:14] (03PS1) 10Hashar: jjb: rm wikidata-query-gui-npm-browser-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662704 (https://phabricator.wikimedia.org/T254027) [13:29:16] (03PS1) 10Hashar: Remove NodeJS 6 jjb templates and images [integration/config] - 10https://gerrit.wikimedia.org/r/662705 [13:30:23] jayme: I guess all repositories using blubber would need a change to be send for them [13:30:44] jayme: the change then would pass CI, get a +2 which result in a new image being build/tested and published ;] [13:31:50] hashar: so it needs to be an actual change them...hmm. Okay. Thanks [13:32:39] (03CR) 10Hashar: [C: 03+2] "Job last builds:" [integration/config] - 10https://gerrit.wikimedia.org/r/662693 (https://phabricator.wikimedia.org/T225678) (owner: 10Hashar) [13:33:51] (03Merged) 10jenkins-bot: Remove 3d2png Jessie based image [integration/config] - 10https://gerrit.wikimedia.org/r/662693 (https://phabricator.wikimedia.org/T225678) (owner: 10Hashar) [13:35:14] jayme: well that is the sole way to get the image build indeed :] [13:36:06] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10hash... [13:36:41] (03CR) 10Hashar: [C: 03+2] Remove tilerator / kartotherian NodeJS 6 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/662698 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:38:29] (03Merged) 10jenkins-bot: Remove tilerator / kartotherian NodeJS 6 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/662698 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:39:04] (03CR) 10Hashar: [C: 03+2] jjb: delete pywikibot-i18n-npm-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662702 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:39:30] (03CR) 10Hashar: [C: 03+2] jjb: rm wikidata-query-gui-npm-browser-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662704 (https://phabricator.wikimedia.org/T254027) (owner: 10Hashar) [13:40:23] (03Merged) 10jenkins-bot: jjb: delete pywikibot-i18n-npm-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662702 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:41:21] (03Merged) 10jenkins-bot: jjb: rm wikidata-query-gui-npm-browser-node-6-docker [integration/config] - 10https://gerrit.wikimedia.org/r/662704 (https://phabricator.wikimedia.org/T254027) (owner: 10Hashar) [13:45:22] (03PS1) 10Hashar: Remove npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662706 (https://phabricator.wikimedia.org/T211784) [13:46:42] (03CR) 10Hashar: [C: 03+2] Remove npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662706 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:47:57] (03Merged) 10jenkins-bot: Remove npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662706 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:48:38] (03PS2) 10Hashar: Remove NodeJS 6 jjb templates and images [integration/config] - 10https://gerrit.wikimedia.org/r/662705 [13:49:16] (03CR) 10Hashar: [C: 03+2] Remove NodeJS 6 jjb templates and images (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/662705 (owner: 10Hashar) [13:49:43] (03PS3) 10Hashar: Remove NodeJS 6 jjb templates and images [integration/config] - 10https://gerrit.wikimedia.org/r/662705 (https://phabricator.wikimedia.org/T211784) [13:50:00] (03CR) 10Hashar: [C: 03+2] Remove NodeJS 6 jjb templates and images [integration/config] - 10https://gerrit.wikimedia.org/r/662705 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:53:04] (03Merged) 10jenkins-bot: Remove NodeJS 6 jjb templates and images [integration/config] - 10https://gerrit.wikimedia.org/r/662705 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [13:58:35] (03PS1) 10Hashar: dockerfiles: remove Jessie npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) [14:01:42] (03CR) 10Hashar: [C: 03+2] dockerfiles: remove Jessie npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [14:02:03] (03CR) 10Hashar: [C: 04-2] dockerfiles: remove Jessie npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [14:10:07] (03PS2) 10Hashar: dockerfiles: unused npm-test and npm-test-stretch [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) [14:11:01] (03CR) 10Hashar: [C: 03+2] "Turns out we don't need npm-test-stretch either :)" [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [14:11:57] (03Merged) 10jenkins-bot: dockerfiles: unused npm-test and npm-test-stretch [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [14:18:12] 10Scap, 10good first task: Fix typo in Scap release checklist - https://phabricator.wikimedia.org/T255856 (10thcipriani) 05Open→03Resolved a:03LarsWirzenius >>! In T255856#6810946, @Iamzac wrote: > @LarsWirzenius is this issue already resolved? Looks like it, my spell checker doesn't find anything on th... [14:25:19] hashar: o/ [14:25:20] yt? [14:25:33] trying to move along https://phabricator.wikimedia.org/T272863 but i can't follow what is happening [14:25:58] It looks like https://gerrit.wikimedia.org/r/c/integration/config/+/658946 needs merged, but it says not ready beacuse https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/658922/ needs to pass, which it has, and then has been abandoned [14:37:45] Lucas_WMDE: hiya sorry if you are not the right person to ping about ^, but you worked on https://phabricator.wikimedia.org/T272998 which I think is related, but I'm not totally sure how [14:38:00] * Lucas_WMDE looks [14:39:29] if that EventBus change passed then I’d guess that the integration/config change is okay? [14:39:34] but hashar would know better [14:40:26] it looks like it might have been blocked on that Wikibase issue at some point, which should now be resolved [14:43:55] ok, so likely if we get https://gerrit.wikimedia.org/r/c/integration/config/+/658946 merged, the eventbus patch will run ci proplery? [14:44:16] paladox: yt? you are added as a reviewer of https://gerrit.wikimedia.org/r/c/integration/config/+/658946, could you help? [14:45:10] hi, i don't have +2 :/ [14:45:10] ottomata: hopefully, yes [14:45:27] ottomata looks like hashar wrote the patch :) [14:45:27] I suppose you could always upload another blank MediaWiki change and run “check experimental” on it, to verify [14:48:27] well I guess https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/658922/was hashar doing just that? [14:48:36] but he abandoned it...meaning...all is well? [14:48:43] haha, i guess I just need to wait for hashar [14:48:52] yeah, that’s my interpretation too [14:49:35] Lucas_WMDE: ottomata hi [14:49:44] I am swamped in some migration of doom bah [14:49:51] 10Scap, 10good first task: Fix typo in Scap release checklist - https://phabricator.wikimedia.org/T255856 (10LarsWirzenius) I opened this task to have an easy onboarding task for a new co-worker. It's not really useful otherwise. @Iamzac, the git repository is at https://gerrit.wikimedia.org/r/admin/repos/med... [14:49:51] oh no :( [14:50:03] oof [14:50:04] hi [14:50:18] so v [14:50:21] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/658922/ [14:50:29] that is a dummy change I have send to test jobs pass on CI [14:50:52] I just routinely abandon them from time to time cause they are often just one off changes [14:51:43] and the wmf-quibble jobs passed there (via check experimental) [14:51:51] so I guess we can add EventBus yes [14:52:59] ok awesome so https://gerrit.wikimedia.org/r/c/integration/config/+/658946 is good to merge? [14:53:52] (03CR) 10Hashar: [C: 03+2] "The wmf-quibble jobs passed when triggered from the EventBus repository: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/" [integration/config] - 10https://gerrit.wikimedia.org/r/658946 (https://phabricator.wikimedia.org/T272863) (owner: 10Hashar) [14:53:55] ottomata: yeah [14:54:05] guess I have missed the notifications originating from the task [14:54:18] will deploy it once merged [14:54:27] thank you [14:54:28] then we can comment "recheck" on some EventBus change to validate [14:54:32] ok awesome [14:54:42] and from now on, the EventBus tests will be run for all repositories participating [14:55:01] ok [14:55:01] (03Merged) 10jenkins-bot: Add EventBus to the gate [integration/config] - 10https://gerrit.wikimedia.org/r/658946 (https://phabricator.wikimedia.org/T272863) (owner: 10Hashar) [14:55:02] great [14:55:03] which also explain why those jobs are soo slow: too many tests [14:55:54] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/658946 # T272863 [14:55:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:55:58] T272863: EventLogging PHP EventServiceClient should use EventBus->send(). - https://phabricator.wikimedia.org/T272863 [14:56:01] Lucas_WMDE: done! Danke :) [14:56:09] \o/ [14:56:10] ottomata: I have deployed the config change so you can try again ;] [14:57:48] (03PS1) 10Hashar: dockerfiles: remove unused seed_image from docker-pkg confg [integration/config] - 10https://gerrit.wikimedia.org/r/662721 [14:58:03] I have so wasted my afternoon [14:58:08] (03CR) 10Hashar: [C: 03+2] dockerfiles: remove unused seed_image from docker-pkg confg [integration/config] - 10https://gerrit.wikimedia.org/r/662721 (owner: 10Hashar) [14:59:15] ok hashar thank you [14:59:24] (03Merged) 10jenkins-bot: dockerfiles: remove unused seed_image from docker-pkg confg [integration/config] - 10https://gerrit.wikimedia.org/r/662721 (owner: 10Hashar) [15:04:39] (03CR) 10Lars Wirzenius: test: make test failures under Python3 fail tox test run (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/656951 (owner: 10Lars Wirzenius) [15:12:18] 10Release-Engineering-Team-TODO, 10MediaWiki-Docker, 10Developer Productivity, 10Patch-For-Review, 10User-Addshore: Port mediawiki-docker-dev "mwdd" v1 cli in to go - https://phabricator.wikimedia.org/T253313 (10Addshore) > What would this look like for non Wikimedia services like PHP-FPM, MariaDB, httpd... [15:17:59] (03PS1) 10Zoranzoki21: Add the Validator extension as dependency for the ParserFun extension [integration/config] - 10https://gerrit.wikimedia.org/r/662725 [15:58:59] (03PS1) 10Zoranzoki21: Archive the QuizTabulate extension [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) [16:23:35] Project mediawiki-core-doxygen-docker build #22496: 04FAILURE in 19 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/22496/ [16:27:45] (03CR) 10Jforrester: "Amazing. Thank you!" [integration/config] - 10https://gerrit.wikimedia.org/r/662707 (https://phabricator.wikimedia.org/T211784) (owner: 10Hashar) [16:31:47] James_F: yeah I went clearing out most of the jessie based stuff finally :] [16:31:55] cause docker-pkg failed to build on contint* machines [16:32:07] turns out it is unrelated but instead a docker-pkg config change in puppet is required [16:32:14] then I guess it is worth removing jessie stuff [16:32:30] will look at dropping the very last one which is to bring up a specific version of npm ;] [16:36:00] apergos: thanks for the reply on the train blocker. Cause I really do not know how to move forward :-\ [16:36:36] hey I am not the authority. but cunningham's law. so yeah. [16:36:57] every day that goes by and we still don't have info I get twitchier and twitchier [16:37:40] (03CR) 10Hashar: [C: 03+2] Archive the QuizTabulate extension [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) (owner: 10Zoranzoki21) [16:40:26] (03PS2) 10Jforrester: jjb: update jobs for quibble-buster-php72:0.0.45-s4 [integration/config] - 10https://gerrit.wikimedia.org/r/662664 (https://phabricator.wikimedia.org/T250515) (owner: 10Hashar) [16:42:46] (03CR) 10Zoranzoki21: "@Hashar Please +2 and this one." [integration/config] - 10https://gerrit.wikimedia.org/r/662725 (owner: 10Zoranzoki21) [16:43:12] !log Reloading zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/662729 [16:43:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:43:55] hashar: I see. :-) [16:44:17] hashar: There's a task tree I created for npm 7.x, BTW. ;-) [16:44:18] grr reloaded too fast [16:44:23] the change is not even merged bah [16:44:27] James_F: earghhh [16:44:29] ;D [16:44:31] * James_F grins. [16:44:42] there was a task for some service that wanted npm 6 on buster iirc [16:44:46] Unfortunately npm 7.x has a different format for package-lock.json. [16:44:59] so the package-lock.json changes? [16:45:03] So it's forward-compatible but not backwards-compatible. [16:45:24] I.e. we can move CI to npm 7.x right now, but once devs/repos move to npm 7.x it'll break with npm 6.x in CI. [16:45:27] elukey had an issue with a change having a lockVersion 1 > 2, I guess that would be the reason [16:45:27] Which sucks. [16:45:37] Yes, lockVersion 2 is the npm 7.x one. [16:45:39] then I guess stick to npm 6 for now? [16:45:53] No, I think we should probably bump CI to npm 7 ASAP. [16:46:09] That way teams can stick with npm 6, or move to npm 7, and it just works either way. [16:46:53] But there are also other things to work on. :-) [16:48:06] (03PS2) 10Hashar: Archive the QuizTabulate extension [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) (owner: 10Zoranzoki21) [16:48:19] (03CR) 10Hashar: [C: 03+2] Archive the QuizTabulate extension [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) (owner: 10Zoranzoki21) [16:49:35] James_F: ok got it ;) [16:49:55] (03Merged) 10jenkins-bot: Archive the QuizTabulate extension [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) (owner: 10Zoranzoki21) [16:51:05] !log Now really reloading zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/662729 [16:51:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:51:20] (03CR) 10Hashar: "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/662729 (https://phabricator.wikimedia.org/T274045) (owner: 10Zoranzoki21) [16:59:58] (03CR) 10Hashar: "I gave it a try on a dummy change https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ParserFun/+/436823 , CI indeed does not inject any" [integration/config] - 10https://gerrit.wikimedia.org/r/662725 (owner: 10Zoranzoki21) [17:03:17] requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.39/images/create?tag=latest&fromImage=wikimedia-stretch [17:03:19] pff [17:03:22] my day never ends [17:08:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10hash... [17:08:41] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Drop jessie testing support - https://phabricator.wikimedia.org/T224908 (10hashar) [17:09:19] hashar: Should we put some we-know-it-doesn't-work-but-oh-well node 10 CI for 3d2png etc.? [17:09:29] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Drop jessie testing support - https://phabricator.wikimedia.org/T224908 (10hashar) The last task to do is to migrate the npm container from Jessie straight to Buster. [17:10:08] hashar: Can you not drop the npm container? Everything should be using the node-10 ones now? [17:12:50] (03PS1) 10Jforrester: dockerfiles: Drop npm, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/662744 (https://phabricator.wikimedia.org/T224908) [17:12:52] (03PS1) 10Jforrester: dockerfiles: Drop ci-jessie, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/662745 (https://phabricator.wikimedia.org/T224908) [17:14:13] (03CR) 10jerkins-bot: [V: 04-1] dockerfiles: Drop ci-jessie, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/662745 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [17:15:04] James_F: have to dig. I think we still bootstrap npm at a specific version using something like COPY --from=npm-jessie [17:15:04] (03CR) 10jerkins-bot: [V: 04-1] dockerfiles: Drop npm, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/662744 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [17:15:09] or maybe that is no more needed [17:15:16] will look later this week ;] [17:15:19] I think I killed that. [17:15:23] * James_F leaves it to you. :-) [17:15:44] but yeah maybe the node10 package provides npm and we are using that [17:15:51] but I think we still relied on integration/npm.git [17:17:40] !log Building some docker images on contint.wikimedia.org [17:17:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:21:27] Yippee, build fixed! [17:21:27] Project mediawiki-core-doxygen-docker build #22497: 09FIXED in 16 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/22497/ [17:22:05] !log Built image docker-registry.discovery.wmnet/releng/quibble-buster-php72:0.0.45-s4 [17:22:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:35:35] (03CR) 10Zoranzoki21: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/662725 (owner: 10Zoranzoki21) [17:35:45] (03Abandoned) 10Zoranzoki21: Add the Validator extension as dependency for the ParserFun extension [integration/config] - 10https://gerrit.wikimedia.org/r/662725 (owner: 10Zoranzoki21) [18:00:37] "train log triage" meeting, on Wednesdays at 19:00 UTC, Is the a gcal event? :D [18:00:57] Or for the Thursdays at 10:00 UTC one ;) [18:01:30] (03CR) 10Hashar: "I have found the reason. ParserFun defines in composer.json:" [integration/config] - 10https://gerrit.wikimedia.org/r/662725 (owner: 10Zoranzoki21) [18:03:27] (03CR) 10Ahmon Dancy: [C: 03+2] Set setup.branch to local branch name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662000 (owner: 10Dduvall) [18:04:02] (03CR) 10Jeena Huneidi: [C: 03+2] Sanitize all image names/tags [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662003 (owner: 10Dduvall) [18:04:12] (03Merged) 10jenkins-bot: Set setup.branch to local branch name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662000 (owner: 10Dduvall) [18:13:19] 10Release-Engineering-Team-TODO, 10MW-on-K8s, 10Release Pipeline, 10serviceops: Create restricted docker-registry namespace for security patched images - https://phabricator.wikimedia.org/T273521 (10Joe) >>! In T273521#6807963, @Legoktm wrote: > Also, is there a chance developers will ask to be able to pul... [18:19:08] 10Release-Engineering-Team-TODO, 10MW-on-K8s, 10Release Pipeline, 10serviceops: Create restricted docker-registry namespace for security patched images - https://phabricator.wikimedia.org/T273521 (10JMeybohm) I think we could make kubernetes use docker credentials distributed to the nodes by default, see h... [18:52:19] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [18:56:13] hey releng, fr-tech is looking at simplifying our code structure and have a patch to match it in CI up here https://gerrit.wikimedia.org/r/c/integration/config/+/661829 [18:56:13] what is the process for getting something like that merged? [19:15:48] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release Pipeline (Blubber): Blubber forces installation of pip from PyPI, which no longer works with Python 2 - https://phabricator.wikimedia.org/T273793 (10dduvall) a:05dduvall→03LarsWirzenius Fix deployed. @LarsWirzenius, reassigning to y... [19:28:53] (03PS2) 10Dduvall: Sanitize all image names/tags [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662003 [19:30:26] (03CR) 10Dduvall: [C: 03+2] "Redoing CR+2 following rebase." [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662003 (owner: 10Dduvall) [19:30:54] (03Merged) 10jenkins-bot: Sanitize all image names/tags [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/662003 (owner: 10Dduvall) [19:39:15] 10Release-Engineering-Team (Logspam), 10Readers-Web-Backlog, 10JavaScript, 10MediaWiki-Interface (Tables), 10Wikimedia-production-error: Some tables cannot be sorted (TypeError: $nextRows[i] is undefined in table sorting) / Uncaught TypeError: Cannot read proper... - https://phabricator.wikimedia.org/T265503 [19:40:16] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1391.eqiad.wmnet'] ` an... [19:43:47] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review: Define a PipelineLib based MW image build job on releases-jenkins.wikimedia.org - https://phabricator.wikimedia.org/T271477 (10dduvall) Finally, suc... [19:43:53] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s, 10Release Pipeline: Determine whether PipelineLib can run on releases-jenkins.wikimedia.org - https://phabricator.wikimedia.org/T271472 (10dduvall) [19:43:55] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review: Define a PipelineLib based MW image build job on releases-jenkins.wikimedia.org - https://phabricator.wikimedia.org/T271477 (10dduvall) 05Open→03... [19:45:13] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s, 10Release Pipeline: Determine whether PipelineLib can run on releases-jenkins.wikimedia.org - https://phabricator.wikimedia.org/T271472 (10dduvall) 05Open→03Resolved p:05Triage→03Medi... [19:45:15] 10Release-Engineering-Team, 10Release Pipeline, 10serviceops-radar, 10Epic: Get production MW-land images built and published - https://phabricator.wikimedia.org/T238771 (10dduvall) [19:47:30] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s: Multi-version MediaWiki image is built and published - https://phabricator.wikimedia.org/T274182 (10dduvall) [19:47:57] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s: Multi-version MediaWiki image is built and published - https://phabricator.wikimedia.org/T274182 (10dduvall) p:05Triage→03Medium [19:55:29] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [19:56:18] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [19:59:47] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [20:00:07] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1389.eqiad.wmnet'] ` Of... [20:00:21] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [20:01:57] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [20:37:10] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1390.eqiad.wmnet'] ` an... [20:41:06] (03CR) 10Eileen: "Note with this patch CI passes with & without our structure change" [integration/config] - 10https://gerrit.wikimedia.org/r/661829 (https://phabricator.wikimedia.org/T273822) (owner: 10Eileen) [20:42:10] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1389.eqiad.wmnet'] ` an... [20:56:48] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by legoktm on cumin1001.eq... [21:00:08] Project mwcore-phpunit-coverage-master build #1205: 04FAILURE in 6 hr 0 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1205/ [21:09:45] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [21:10:40] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1305.eqiad.wmnet'] ` an... [21:15:37] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1304.eqiad.wmnet'] ` an... [21:17:03] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [21:21:47] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [21:26:53] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [21:40:04] !log upgraded gitlab instance on gitlab-test from 13.3.3-ce.0 to 13.8.3-ce.0 [21:40:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:40:40] !log enabled administrative approval for new account signups on gitlab-test [21:40:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:51:28] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1388.eqiad.wmnet'] ` an... [21:52:37] (03PS1) 10Thcipriani: deployment calendar: Fully automated deployment calendar [tools/release] - 10https://gerrit.wikimedia.org/r/662778 [21:54:01] (03CR) 10jerkins-bot: [V: 04-1] deployment calendar: Fully automated deployment calendar [tools/release] - 10https://gerrit.wikimedia.org/r/662778 (owner: 10Thcipriani) [21:58:48] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1397.eqiad.wmnet'] ` an... [21:59:10] (I'm going to be so happy when that ^ reimaging is done) [22:06:41] 53%! [22:18:14] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1271.eqiad.wmnet', 'mw12... [22:24:17] (03PS1) 10Krinkle: doc: Add wikimedia/minify [integration/docroot] - 10https://gerrit.wikimedia.org/r/662785 (https://phabricator.wikimedia.org/T273247) [22:24:37] (03CR) 10Krinkle: [C: 04-1] "Awaiting first doc run and composer release tag." [integration/docroot] - 10https://gerrit.wikimedia.org/r/662785 (https://phabricator.wikimedia.org/T273247) (owner: 10Krinkle) [22:30:47] Hi - I'm wondering if someone here is able to +2 this https://gerrit.wikimedia.org/r/c/integration/config/+/661829 [22:30:57] for our fundraising tech ci image [22:31:56] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: [Regression] some beta cluster wikis using official logos - https://phabricator.wikimedia.org/T274210 (10bd808) [22:32:22] (03CR) 10AndyRussG: "This change is ready for review." [releng/dev-images] - 10https://gerrit.wikimedia.org/r/632173 (https://phabricator.wikimedia.org/T262976) (owner: 10AndyRussG) [22:33:03] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Vector, 10Regression: [Regression] some beta cluster wikis using official logos - https://phabricator.wikimedia.org/T274210 (10Reedy) {T274196} and {T274199} [22:36:10] brennen: hi! so after some surprisingly emotional debate, we're ready for this to be reviewed and potentially +2'ed: https://gerrit.wikimedia.org/r/c/releng/dev-images/+/632173 [22:36:34] brennen: along with the follow-on one: https://gerrit.wikimedia.org/r/c/releng/dev-images/+/635361 [22:37:20] they're not perfect, especially the first one, but we've more or less decided on an approach to pursue and I think it'd be useful for the team to get this work so far into the main branch and also built images into the registry [22:38:42] AndyRussG: right on, thanks for the heads up! i'm neck deep in gitlab documentation this afternoon, but if i don't get to it before then, longma and i have a regular pairing session on tuesdays that we can probably use to give it a look. [22:39:47] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw1303.eqiad.wmnet'] ` an... [22:39:54] brennen: it's worth noting this is just 'iteration 1' - we are going to keep iterating but it's hard to express our proposed changes without a merged baseline [22:40:30] brennen: thanks! [22:41:37] eileen: duly noted. :) we'll try not to block you very long on getting that published. [22:41:45] thanks [22:43:11] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw2245.codfw.wmnet'] ` an... [22:44:19] brennen: I don't know if you are the person to ask but I also want to make some changes to our fundraising CI build. One that there is already a gerrit for - but the other to make some mysql settings changes to change the charset & row format - I guess I just figure it out & put up another gerrit on that? [22:48:58] (ref 661829 for the one there is already a gerrit for) [22:53:18] eileen: Sorry it wasn't clear to me - Which part do you need help with? [22:53:44] longma: thanks - so 2 things [22:53:50] 1) getting 661829 merged [22:54:12] 2) figuring out how to change the charset mysql is configured to on our ci [22:54:39] - I'm digging into 2 at the moment but I haven't figured out where the mysql config (or even the version ) is coming from so far [22:55:07] I'm not sure about that but I can take a look as well [22:55:26] I guess this must be the image docker-registry.wikimedia.org/releng/civicrm:0.2.0-s2 [22:55:51] (I haven't dug into these repos before ) [22:56:41] ah ok [22:58:25] that should be defined here I think https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/dockerfiles/civicrm/ [23:01:28] OK - I'm trying to fire it up locally - I guess confirming the mysql details it IS using is a good start since I *think* the right version of maria db would have the right defaults [23:01:51] (03PS3) 10Eileen: Switch fundraising tech setup to use recurse_submodules [integration/config] - 10https://gerrit.wikimedia.org/r/661829 (https://phabricator.wikimedia.org/T273822) [23:02:22] that's the change that I think we just need you to +2 [23:04:19] I can't quite figure out why there are 3 * .sh files in there [23:06:59] I'm not sure eithere [23:08:30] it takes a deploy as well for the jjb changes i think; not sure about the dockerfile change [23:08:33] for your existing patch I'd rather confirm with hashar since I'm not too familiar with it. I'll ask him about it tomorrow [23:08:50] (sorry, got lost in a maze of tabs and chat windows for a while there.) [23:08:57] yeah that's right. For the dockerfile we'd need to build that as well I believe [23:09:00] i am definitely shakier on this part of things than i should be. [23:10:25] :-) [23:11:11] thanks - I'm wondering if maybe only one of the 3 files is actually used & the other are just 'there' [23:11:56] eileen: so run-with-mysqld.sh uses run.sh [23:12:02] but I guess the example is just that? [23:12:19] yeah - example doesn't seem helpful to me [23:12:32] so probably ci calls run-with-mysqld.sh [23:12:40] yes, it's in the entrypoint [23:12:43] that's the entrypoint... yeah. [23:12:59] maybe example's just leftover from initial setup? [23:13:19] cool - I think it would be better to remove example.... [23:13:39] hasha.r made it so we can ask him :P [23:14:08] :-) [23:15:11] example run isn't included in the dockerfile [23:15:26] actually, it runs docker itself, so I wouldn't worry about editing it [23:15:49] unless you need to update something for running docker [23:17:00] yeah - we just need to update in general to use recurse-submodules & I need to figure out this mysql config thing & then lastly we want to switch away from the db pre-creates (but I'm not looking at that yet) so we can unfork buildkit [23:17:40] we don't normally run this docker image ourselves - just via jenkins [23:22:13] the image is based on a stretch image and installs https://packages.debian.org/stretch/default-mysql-server [23:27:18] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [23:28:36] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [23:28:48] eileen: it seems like you can pass the charsets option to mysqld when the server is started in run-with-mysqld, but I think the option name depends on the mysql version [23:29:20] thanks longma - looks like the image is mariadb 10.1 & live is 10.1.48 - which doesn't sound different [23:30:13] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [23:30:58] I'll look at trying to run it & pass params. I'm just checking how to get from a docker.template file to a docker file [23:32:23] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10SRE, 10serviceops, 10User-jijiki: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqia... [23:34:14] I think you need to use docker-pkg: https://www.mediawiki.org/wiki/Continuous_integration/Docker#Build_images_locally [23:34:30] brennen has used it a lot so he could probably help with that more [23:34:42] eileen: are you familiar with docker-pkg? [23:35:02] https://doc.wikimedia.org/docker-pkg/ [23:35:19] I'm digging in it now - I did get my head into it a bit last year but it's out of it now [23:36:16] So I use that to turn the Docker.template into a file I can use I think [23:36:38] It will build the images [23:37:39] OK - thanks - I was actually looking in the wrong place (a related repo) [23:37:56] eileen: one sec, i'll dig you up the command you need [23:38:06] (or try, at any rate. :) ) [23:39:44] I think I've found it in this readme https://gitlab.com/andyrussg/fundraising-dev [23:40:10] for testing a single thing from dev-images at a time, i use something like: [23:40:14] docker-pkg -c dockerfiles/config.yaml build --no-pull --select '*/stretch-php72-fpm-apache2-xdebug:*' dockerfiles [23:40:49] cool - that looks similar but more build out than what I found! [23:41:17] the `--select` param can be kind of finicky [23:42:03] but i think that general pattern ought to work [23:42:06] if following the fundraising dev-instructions, just remember you need to use the integration/config repo, not the dev-images one [23:42:49] longma that was exactly what I was trying to get my head around :-) [23:43:28] https://www.mediawiki.org/wiki/Continuous_integration/Docker#Build_images_locally should be what you need to do, unless it's out of date :) [23:44:57] i think it's likely right, --select just keeps you from having to build _everything_ [23:45:01] great that looks useful - thanks [23:45:14] yeah, --select would be helpful [23:45:41] and --no-pull keeps it from grabbing a copy of the image from the remote docker registry, which can be helpful to keep in mind if you haven't tweaked the version in the changelog file [23:46:18] docker-pkg -c dockerfiles/config.yaml build --no-pull --select '*/civicrm:*' dockerfiles [23:46:18] definitely seems to be doing something useful [23:46:42] eileen: one thing to keep in mind - if you're changing a dockerfile template for an image built with docker-pkg, you need to update the changelog to create the new version [23:47:09] if you look at recent commits to dev-images you can see some examples of that [23:47:34] ok - should I add that to the commit I put up for review? [23:47:38] (apologies if you already know this, but it's one thing i've seen people seem to miss.) [23:47:45] no I really don't [23:48:01] yeah, please do for those changes [23:48:01] I guess we leaned on Elliott to deal with it all under he had a baby [23:48:19] it's not exactly an obvious workflow [23:48:59] :-) [23:52:49] (03PS4) 10Eileen: Switch fundraising tech setup to use recurse_submodules [integration/config] - 10https://gerrit.wikimedia.org/r/661829 (https://phabricator.wikimedia.org/T273822) [23:53:54] they all seem to have high urgency....