[00:32:18] <wikibugs>	 (03CR) 10Dduvall: [C: 04-1] Enable deployment-charts update for published image (032 comments) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/622231 (https://phabricator.wikimedia.org/T255835) (owner: 10Jeena Huneidi)
[02:27:13] <wikibugs>	 10Gerrit, 10User-DannyS712: 500 Server Error for gerrit change 17620 - https://phabricator.wikimedia.org/T262632 (10DannyS712)
[02:27:28] <wikibugs>	 10Gerrit, 10User-DannyS712: 500 Server Error for gerrit change 17620 - https://phabricator.wikimedia.org/T262632 (10DannyS712)
[07:48:58] <wikibugs>	 10Continuous-Integration-Config: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only run PHP 7.2 - https://phabricator.wikimedia.org/T262640 (10SamanthaNguyen)
[09:27:11] <wikibugs>	 10Continuous-Integration-Config: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only run PHP 7.2 - https://phabricator.wikimedia.org/T262640 (10hashar) `extension-quibble` refers to a template of jobs and is indeed the default for most extensions. In short: * when a p...
[09:34:50] <wikibugs>	 (03PS1) 10Hashar: zuul: php7.4 only quibble template [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640)
[09:35:12] <hashar>	 James_F: if you are arond, shouldn't we run php74 tests on REL1_35 ? :]
[09:35:45] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] zuul: php7.4 only quibble template [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[09:35:49] <James_F>	 hashar: We do?
[09:35:58] <James_F>	 hashar: (On merge, not on test.)
[09:36:04] <hashar>	 apparently the php74 jobs are not in REL1_35 pipelines
[09:36:05] <hashar>	 ah
[09:36:26] <James_F>	 Test only runs one php flavour, gate-and-submit runs them all, like you said we should do.
[09:36:34] <James_F>	 Unless I screwed it up somewhere.
[09:36:42] <hashar>	     gate-and-submit-1_35:
[09:36:42] <hashar>	       - mwgate-node10-docker
[09:36:42] <hashar>	       - quibble-composer-mysql-php73-noselenium-docker
[09:36:42] <hashar>	       - quibble-composer-mysql-php73-selenium-docker
[09:36:46] <hashar>	 that is for extension-quibble
[09:36:55] <hashar>	 it lacks the php74 there
[09:37:03] <James_F>	 Oh! Yes, for things that's currently lacking php74 everywhere.
[09:37:14] <James_F>	 Because Wikibase had issues, and I never got around to enabling it everywhere.
[09:37:17] <James_F>	 There's a patch.
[09:37:19] <James_F>	 Let me look.
[09:37:24] <hashar>	 AHHH
[09:37:45] <James_F>	 Ah, no patch, just a task?
[09:37:49] <James_F>	 Let me write one quickly.
[09:37:54] <James_F>	 What could go wrong? ;-)
[09:38:17] <wikibugs>	 10Continuous-Integration-Config, 10PHP 7.4 support, 10Patch-For-Review: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only run PHP 7.2 - https://phabricator.wikimedia.org/T262640 (10hashar) Note that we do not have a php7.4 for #phan yet :-\
[09:39:13] <James_F>	 Extension-quibble looks fine?
[09:39:27] <James_F>	 https://www.irccloud.com/pastebin/YwZNScq5/
[09:39:39] <wikibugs>	 (03CR) 10Hashar: "Fails because:" [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[09:40:17] <hashar>	 James_F: yeah php74 is enforced for master indeed
[09:40:25] <hashar>	 but not for 1.35 :]
[09:40:56] <hashar>	 I guess we would need some matrix of which php versions we support for each releases
[09:43:05] <James_F>	 There.
[09:43:21] <wikibugs>	 (03PS1) 10Jforrester: layout: Add PHP74 jobs to REL1_35 pipelines where missing [integration/config] - 10https://gerrit.wikimedia.org/r/626628
[09:46:25] <wikibugs>	 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Wikimedia-General-or-Unknown, 10PHP 7.4 support: Make Wikimedia Production MediaWiki compatible with PHP 7.4 - https://phabricator.wikimedia.org/T247658 (10Jdforrester-WMF) 05Open→03Resolved To my knowledge, this is now fully done. If I'm...
[09:46:50] <James_F>	 hashar: Want me to make the php74 phan/etc. jobs? I just did it for php73 last month after all. :-)
[09:47:52] <wikibugs>	 (03CR) 10Jforrester: zuul: php7.4 only quibble template (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[09:51:54] <hashar>	 James_F: if you feel like it sure! :]
[09:56:13] <wikibugs>	 (03PS1) 10Jforrester: dockerfiles: Provide mediawiki-phan-php74 [integration/config] - 10https://gerrit.wikimedia.org/r/626632
[09:56:15] <James_F>	 hashar: Done.
[09:56:15] <wikibugs>	 (03PS1) 10Jforrester: jjb: Provide PHP 7.4 variants of phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/626633
[09:56:31] <hashar>	 you are fast :]
[09:56:32] <James_F>	 hashar: Can I leave it to you to merge and deploy? :-)
[09:56:36] <hashar>	 will review those after lunch I guess
[09:56:42] <hashar>	 thank you very much!
[09:56:44] <James_F>	 Excellent.
[09:57:15] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] dockerfiles: Provide mediawiki-phan-php74 [integration/config] - 10https://gerrit.wikimedia.org/r/626632 (owner: 10Jforrester)
[09:57:24] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] jjb: Provide PHP 7.4 variants of phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/626633 (owner: 10Jforrester)
[09:58:05] <wikibugs>	 (03PS2) 10Jforrester: dockerfiles: Provide mediawiki-phan-php74 [integration/config] - 10https://gerrit.wikimedia.org/r/626632
[09:58:07] <wikibugs>	 (03PS2) 10Jforrester: jjb: Provide PHP 7.4 variants of phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/626633
[10:03:59] <wikibugs>	 10Gerrit, 10User-DannyS712: 500 Server Error for gerrit change 17620 - https://phabricator.wikimedia.org/T262632 (10MarcoAurelio) Looks like Gerrit has troubles finding commit fc0d6b15d78a2e2e3a560ad961f58cbcb20ebc94:  ` 11/09/20 9:56:40 /changes/mediawiki%2Fextensions%2FAbuseFilter~17620/detail?O=916314 GET E...
[10:11:07] <hashar>	 f*** docker really
[10:11:08] <hashar>	 bah
[10:11:12] <hashar>	 so much copy pasting
[10:12:18] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Reedy, that enforces php7.4 compatibility for REL1_35 ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/626628 (owner: 10Jforrester)
[10:12:31] <wikibugs>	 (03PS2) 10Jforrester: zuul: php7.4 only quibble template [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[10:12:51] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] dockerfiles: Provide mediawiki-phan-php74 [integration/config] - 10https://gerrit.wikimedia.org/r/626632 (owner: 10Jforrester)
[10:13:08] <wikibugs>	 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Operations, 10Patch-For-Review, 10git-protocol-v2: Upgrade git fleet wide to git 2.20 - https://phabricator.wikimedia.org/T262244 (10MoritzMuehlenhoff) Deployment servers and some other cl...
[10:13:18] <wikibugs>	 (03Merged) 10jenkins-bot: layout: Add PHP74 jobs to REL1_35 pipelines where missing [integration/config] - 10https://gerrit.wikimedia.org/r/626628 (owner: 10Jforrester)
[10:14:02] <wikibugs>	 (03Merged) 10jenkins-bot: dockerfiles: Provide mediawiki-phan-php74 [integration/config] - 10https://gerrit.wikimedia.org/r/626632 (owner: 10Jforrester)
[10:14:37] <hashar>	 !log Reloading Zuul for "Add PHP74 jobs to REL1_35 pipelines where missing" https://gerrit.wikimedia.org/r/c/integration/config/+/626628/
[10:14:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:14:57] <hashar>	 !log Building docker image releng/mediawiki-phan74
[10:15:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:19:03] <wikibugs>	 (03CR) 10Hashar: [V: 03+2 C: 03+2] "Lets be bold and do it :]" [integration/reporting] - 10https://gerrit.wikimedia.org/r/589083 (owner: 10Dduvall)
[10:19:47] <hashar>	 !log Successfully tagged docker-registry.discovery.wmnet/releng/mediawiki-phan-php74:0.1.0 
[10:19:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:27:02] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Code-Health-Metrics: Move Dan's CI metrics cron script / HTML output from people.wikimedia.org to doc.wikimedia.org - https://phabricator.wikimedia.org/T255701 (10hashar) I someho...
[10:29:51] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Copy pasting should be banned! Then got done for the php73 as well and all those templates can be refactored later on." [integration/config] - 10https://gerrit.wikimedia.org/r/626633 (owner: 10Jforrester)
[10:30:57] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Provide PHP 7.4 variants of phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/626633 (owner: 10Jforrester)
[10:31:01] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Thanks for the update! :]" [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[10:32:07] <wikibugs>	 (03Merged) 10jenkins-bot: zuul: php7.4 only quibble template [integration/config] - 10https://gerrit.wikimedia.org/r/626605 (https://phabricator.wikimedia.org/T262640) (owner: 10Hashar)
[10:34:59] <hashar>	 !log Reloaded Zuul for https://gerrit.wikimedia.org/r/626605
[10:35:01] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:45:34] <wikibugs>	 10Continuous-Integration-Config, 10PHP 7.4 support, 10Patch-For-Review: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only run PHP 7.2 - https://phabricator.wikimedia.org/T262640 (10hashar) I did a `recheck` on https://gerrit.wikimedia.org/r/c/mediawiki/extension...
[10:46:01] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10PHP 7.4 support, 10Patch-For-Review: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI test... - https://phabricator.wikimedia.org/T262640
[10:51:58] <addshore>	 legoktm: https://libraryupgrader2.wmcloud.org/ os dead? D:
[10:52:08] <James_F>	 What, again?
[10:52:17] * James_F sighs.
[10:55:07] <James_F>	 hashar: BTW, FR-tech didn't respond to my query about if https://gerrit.wikimedia.org/r/c/integration/config/+/624755 can go ahead. :-(
[10:55:45] <hashar>	 James_F: yeah :-\
[10:55:55] <hashar>	 maintaining CI is a very lonesome job hehe
[10:56:00] <James_F>	 Maybe we should just do it and see if they notice?
[10:57:38] <addshore>	 James_F: could just be the webservice is stopped? (do you have powers?)
[10:58:01] <James_F>	 addshore: Could be, and no. Only legoktm and mutante, both of whom are probably still asleep.
[10:58:12] <addshore>	 ack
[11:01:51] <hashar>	 addshore: you can poke https://phabricator.wikimedia.org/T262228 :)
[11:01:54] <hashar>	 and there is no doc on https://www.mediawiki.org/wiki/Libraryupgrader
[11:01:58] <hashar>	 as to how the service runs
[11:02:12] <James_F>	 "Badly".
[11:02:35] <hashar>	 there are a bunch of those tools we should probably productionize
[11:02:48] <hashar>	 codesearch.wmflabs.org is another example ;]
[11:06:34] <wikibugs>	 (03PS3) 10Jforrester: layout: [FundraisingEmailUnsubscribe] Make special FR REL1_35-php73 job voting [integration/config] - 10https://gerrit.wikimedia.org/r/624755
[11:06:36] <wikibugs>	 (03PS2) 10Jforrester: layout: [FundraisingEmailUnsubscribe] Drop special FR REL1_31-php70 job [integration/config] - 10https://gerrit.wikimedia.org/r/624791
[11:06:38] <wikibugs>	 (03PS2) 10Jforrester: jjb: Drop special FR REL1_31 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/624793
[11:06:40] <wikibugs>	 (03PS1) 10Jforrester: layout: [FundraisingEmailUnsubscribe] Re-enable regular quibble job [integration/config] - 10https://gerrit.wikimedia.org/r/626642
[11:06:42] <wikibugs>	 (03PS1) 10Jforrester: layout: [DonationInterface] Add special FR PHP73/REL1_35 job [integration/config] - 10https://gerrit.wikimedia.org/r/626643
[11:06:44] <wikibugs>	 (03PS1) 10Jforrester: layout: Drop quibble-donationinterface-REL1_31-php70-docker job [integration/config] - 10https://gerrit.wikimedia.org/r/626644
[11:06:45] <James_F>	 hashar: Or at least make "official".
[11:06:46] <wikibugs>	 (03PS1) 10Jforrester: jjb: Drop quibble-donationinterface-REL1_31-php70-docker job [integration/config] - 10https://gerrit.wikimedia.org/r/626645
[11:06:59] <James_F>	 (More stack of FR cleanup stuff, probably not ready yet. :-()
[11:11:29] <hashar>	 :-\
[11:11:45] <James_F>	 Now working on the security-checker one.
[11:11:49] <hashar>	 but rejoice, french folks are now playing music with watermelons https://www.youtube.com/watch?v=RPf28jaiU90 
[11:11:57] <James_F>	 Rejoice?
[11:12:00] <James_F>	 Or lament?
[11:12:25] <hashar>	 ;D
[11:27:59] <wikibugs>	 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Operations, 10Patch-For-Review, 10git-protocol-v2: Upgrade git fleet wide to git 2.20 - https://phabricator.wikimedia.org/T262244 (10hashar) >>! In T262244#6453583, @MoritzMuehlenhoff wrot...
[11:40:05] <wikibugs>	 (03PS2) 10Jforrester: dockerfiles: [composer-php70] Make security-check ignore php version & cascade [integration/config] - 10https://gerrit.wikimedia.org/r/626499 (https://phabricator.wikimedia.org/T262465) (owner: 10SBassett)
[11:40:07] <wikibugs>	 (03PS1) 10Jforrester: jjb: [php-composer-security-docker] Switch to image that ignores PHP version [integration/config] - 10https://gerrit.wikimedia.org/r/626650 (https://phabricator.wikimedia.org/T262465)
[11:40:09] <wikibugs>	 (03PS1) 10Jforrester: jjb: Bump everything descended from composer-php70 to latest [integration/config] - 10https://gerrit.wikimedia.org/r/626651
[11:41:00] <James_F>	 OK, I'm going back to my actual job. :-)
[11:41:21] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] jjb: Bump everything descended from composer-php70 to latest [integration/config] - 10https://gerrit.wikimedia.org/r/626651 (owner: 10Jforrester)
[12:06:11] <jgleeson>	 Hey James_F, I'm one of the folks over at fr-tech. were you waiting on us for something?
[12:06:42] <jgleeson>	 anything I can help with
[12:08:20] <James_F>	 jgleeson: Oh, we were wondering when we can dismantle all the special fundraising/REL1_31 stuff now that there's fundraising/REL1_35 stuff.
[12:12:10] <jgleeson>	 thanks James_F. it looks like the updates needing to move us to REL1_35 are merged according to this task https://phabricator.wikimedia.org/T260621  but the task isn't closed so I'll check in with ejegg|away and cstone who've been looking at that once they log on for the day and see if we can close that and give you the green light 
[12:12:35] <James_F>	 jgleeson: Right. Thanks for checking!
[12:12:51] <James_F>	 I'm very familiar with how reality and Phabricator can lag behind each other. :-)
[12:13:14] <jgleeson>	 :))
[12:32:56] <wikibugs>	 (03CR) 10Lars Wirzenius: [V: 03+2 C: 03+2] Use single vdisk for train-dev [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626453 (owner: 10Ahmon Dancy)
[13:38:07] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10PHP 7.4 support: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only... - https://phabricator.wikimedia.org/T262640
[13:38:17] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10PHP 7.4 support: Attempting to run StructuredNavigation that requires PHP 7.4 fails CI tests, as tests setup only... - https://phabricator.wikimedia.org/T262640
[13:40:57] <hashar>	 my puppet is rusty :-\
[13:44:18] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] operations-dnslist: create version 0.0.8 [integration/config] - 10https://gerrit.wikimedia.org/r/626507 (owner: 10BBlack)
[13:45:50] <wikibugs>	 (03Merged) 10jenkins-bot: operations-dnslist: create version 0.0.8 [integration/config] - 10https://gerrit.wikimedia.org/r/626507 (owner: 10BBlack)
[13:57:51] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Passed on https://gerrit.wikimedia.org/r/c/operations/dns/+/626656 ! Excellent :]" [integration/config] - 10https://gerrit.wikimedia.org/r/626508 (owner: 10BBlack)
[13:58:52] <wikibugs>	 (03Merged) 10jenkins-bot: operations-misc: bump dnslint to 0.0.8 [integration/config] - 10https://gerrit.wikimedia.org/r/626508 (owner: 10BBlack)
[14:07:43] <wikibugs>	 10MediaWiki-Releasing, 10Documentation, 10MW-1.35-notes, 10MW-1.35-release, 10MW-1.36-notes (1.36.0-wmf.9; 2020-09-15): Finish up RELEASE-NOTES for 1.35.0 - https://phabricator.wikimedia.org/T259349 (10Reedy)
[14:13:17] <hashar>	 errands stuff and other duties
[14:13:24] <hashar>	 will be back later tonight if all goes well
[14:16:31] <hauskatze>	 hashar: est-ce que tu vas faire les courses? :P
[14:41:45] <ejegg>	 James_F: we're still working on getting the tests running on REL1_35
[14:41:51] <ejegg>	 various little code updates needed
[14:42:11] <ejegg>	 then we'll test it on our prod sites in the backup cluster
[14:42:22] <ejegg>	 and finally switch over and upgrade the main cluster
[14:42:34] <ejegg>	 hopefully within a week or two
[14:52:51] <James_F>	 Ack.
[15:55:32] <wikibugs>	 (03PS1) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[16:16:02] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 04-1] "Looks good.  Just some minor nits." (036 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[16:24:13] <wikibugs>	 (03CR) 10Lars Wirzenius: "Thanks, uploading new patch set momentarily." (036 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[16:37:53] <wikibugs>	 (03PS2) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[16:48:39] <wikibugs>	 (03PS3) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[16:50:51] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 04-1] refactor: combine assorted scripts into one (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[16:53:20] <wikibugs>	 (03PS4) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[16:56:44] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+1] "LGTM.  I'm going to test before merging." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[17:05:22] <wikibugs>	 (03PS5) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[17:08:42] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+1] "Much better.  Testing now..." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[17:13:46] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 04-1] refactor: combine assorted scripts into one (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[17:17:38] <wikibugs>	 (03PS6) 10Lars Wirzenius: refactor: combine assorted scripts into one [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709
[17:30:17] <legoktm>	 James_F: addshore: I know what's wrong with libup now, I'll fix it once I'm awake
[17:34:18] <wikibugs>	 10MediaWiki-Releasing, 10MW-1.35-notes, 10MW-1.35-release, 10Platform Team Workboards (Initiatives): Release MW 1.35.0 - https://phabricator.wikimedia.org/T256375 (10ssastry)
[17:36:52] <wikibugs>	 (03CR) 10Ahmon Dancy: [V: 03+2 C: 03+2] "Beautiful" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626709 (owner: 10Lars Wirzenius)
[17:38:04] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10Golbez81) Might be too late, but can I highly urge the project to step back the PHP requirem...
[17:43:28] <James_F>	 legoktm: Did it come to you in a dream? :-)
[17:48:05] <legoktm>	 James_F: I woke up, saw the ping, and then it just immediately hit me what was wrong
[17:48:21] <James_F>	 Aha. Nice.
[17:48:55] <legoktm>	 By default systemd will only restart a service so many times before giving up
[17:49:42] <legoktm>	 So the new daily restarts only work for a few days, whoops.
[17:59:15] <wikibugs>	 10LibUp, 10Patch-For-Review: 502 Bad Gateway on https://libraryupgrader2.wmcloud.org/ - https://phabricator.wikimedia.org/T262228 (10Legoktm) It went down again last night, ^ (which I already applied) should automatically restart it if it does fail in some way.
[18:00:18] <legoktm>	 Actually it seems to have been something else
[18:00:26] <legoktm>	 It was timing out somehow
[18:00:37] <legoktm>	 But my fix will hopefully fix it.
[18:09:22] <wikibugs>	 10Release-Engineering-Team, 10GitLab-Test, 10Cloud-VPS (Quota-requests), 10User-brennen: Request for floating IP / DNS for gitlab-test.wmcloud.org - https://phabricator.wikimedia.org/T261900 (10brennen)
[18:10:57] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10GitLab-Test, 10User-brennen: Unable to push repository to remote, hostname is inaccessible - https://phabricator.wikimedia.org/T262516 (10brennen) p:05Triage→03High
[18:31:45] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Wikimedia-Logstash: scap on beta fails canary check: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T260667 (10EBernhardson) Taking a quick loo...
[18:43:00] <wikibugs>	 (03PS1) 10Ahmon Dancy: Ensure train-dev.qcow2 vdisk is standalone [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626729
[18:44:03] <wikibugs>	 (03CR) 10Ahmon Dancy: [V: 03+2 C: 03+2] Ensure train-dev.qcow2 vdisk is standalone [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626729 (owner: 10Ahmon Dancy)
[18:44:20] <wikibugs>	 (03PS1) 10Ahmon Dancy: Run ansible playbooks in parallel [tools/train-dev] - 10https://gerrit.wikimedia.org/r/626730
[19:14:12] <wikibugs>	 (03CR) 10Jforrester: "Sorry, should re-do this now I've released composer-php70 0.2.3 with the underlying change." [integration/config] - 10https://gerrit.wikimedia.org/r/626480 (owner: 10Hashar)
[19:16:35] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10MarkAHershberger) RHEL 8 has PHP 7.3.20 available.  To use it: ` yum module install php:7.3/* `
[19:23:52] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10Reedy) >>! In T257879#6454753, @Golbez81 wrote: > Might be too late, but can I highly urge t...
[19:27:58] <brennen>	 !log gitlab-test may experience some downtime as a result of zone transfer for T262516, T261900
[19:28:00] <wikibugs>	 10MediaWiki-Codesniffer, 10MediaWiki-General, 10TechCom, 10at-ease, 10Proposal: Liberate the @ for AtEase - https://phabricator.wikimedia.org/T253461 (10Krinkle) 05Open→03Resolved a:03Krinkle Done ([diff](https://www.mediawiki.org/w/index.php?title=Manual:Coding_conventions/PHP&diff=4098856&oldid=4...
[19:28:04] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[19:28:04] <stashbot>	 T262516: Unable to push repository to remote, hostname is inaccessible - https://phabricator.wikimedia.org/T262516
[19:28:05] <stashbot>	 T261900: Request for floating IP / DNS for gitlab-test.wmcloud.org - https://phabricator.wikimedia.org/T261900
[19:29:08] <wikibugs>	 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T257976 (10jeena) 05Open→03Resolved
[19:38:38] <wikibugs>	 10LibUp: libup could remove eslint.options.extensions from Gruntfile.js for newer eslint-config-wikimedia - https://phabricator.wikimedia.org/T262221 (10Umherirrender) It could also remove the first version in the file list:  `     '*.{js,json}',     '**/*.{js,json}',  `  Unfortunally eslint does not check dot f...
[19:39:27] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10MarkAHershberger) Reedy, it is especially hard to find the official RH versions without a su...
[19:39:29] <wikibugs>	 10Project-Admins, 10XTools, 10User-Matthewrbowker: Proposal: Separate Phabricator project for semi-automated edit requests - https://phabricator.wikimedia.org/T262721 (10Pppery)
[19:42:32] <wikibugs>	 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10MediaWiki Train Development Environment: Create static host keys for train dev vm - https://phabricator.wikimedia.org/T259279 (10dancy) 05Open→03Declined I have been built and rebuilt the train-dev environment a zillion times without issues...
[19:45:10] <wikibugs>	 10Project-Admins, 10XTools, 10User-DannyS712, 10User-Matthewrbowker: Proposal: Separate Phabricator project for semi-automated edit requests - https://phabricator.wikimedia.org/T262721 (10DannyS712) a:03DannyS712 I'd like to see what @MusikAnimal thinks before proceeding, but sounds like a good idea to me
[19:46:09] <ebernhardson>	 !log deployment-prep delete elastic indices dated prior to 2020.06 from deployment-logstash03
[19:46:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[19:48:13] <wikibugs>	 10Project-Admins, 10XTools, 10User-DannyS712, 10User-Matthewrbowker: Proposal: Separate Phabricator project for semi-automated edit requests - https://phabricator.wikimedia.org/T262721 (10Matthewrbowker) That's why I didn't tag any project admins yet, I wanted to open it for wider discussion from the XTool...
[19:52:03] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10MoritzMuehlenhoff) >>! In T257879#6455213, @Reedy wrote: >>>! In T257879#6454753, @Golbez81...
[19:52:07] <wikibugs>	 (03PS1) 10Mholloway: Add dev image definition for eventgate-wikimedia [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739
[19:53:38] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10Reedy) Yay, paywalls! :)  So if 7.3.20 is officially supported by RH... The question lies wi...
[19:53:46] <wikibugs>	 10Project-Admins, 10XTools, 10User-DannyS712, 10User-Matthewrbowker: Proposal: Separate Phabricator project for semi-automated edit requests - https://phabricator.wikimedia.org/T262721 (10MusikAnimal) I don't think we need a entirely separate project, personally. Maybe instead a new column on the board and...
[19:56:18] <wikibugs>	 (03PS2) 10Mholloway: Add dev image definition for eventgate-wikimedia [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739
[19:57:34] <wikibugs>	 (03PS3) 10Mholloway: Add dev image definition for eventgate-wikimedia [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739
[20:20:24] <wikibugs>	 (03CR) 10Jeena Huneidi: "I only took a quick look, but this looks like what is described in the blubber config for eventgate. Is there a reason we couldn't use pip" [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739 (owner: 10Mholloway)
[20:31:01] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10MinervaNeue, 10Readers-Web-Backlog, 10ci-test-error: Can't call getText on element with selector ".section-heading" because element wasn't found - https://phabricator.wikimedia.org/T262617 (10Jdlrobson) I've seen this a few times before. It's a race con...
[20:39:11] <brennen>	 !log gitlab-test.wmcloud.org back up, now with more SSH (hopefully)
[20:39:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[20:43:26] <hauskatze>	 brennen: heh, I was trying to take a look and NX_DOMAIN wasn't particularly helpful :)
[20:43:29] <hauskatze>	 thanks
[20:44:46] <wikibugs>	 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), and 2 others: Stop using jsonlint and instead use eslint-plugin-json for the linting - https://phabricator.wikimedia.org/T220036 (10Umherirrender)
[20:44:52] <wikibugs>	 (03CR) 10Jeena Huneidi: [C: 04-1] "> Patch Set 3:" [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739 (owner: 10Mholloway)
[20:44:57] <brennen>	 i'm having a heckuva time connecting to gitlab itself over ssh, for whatever that's worth.  still trying to figure out why.
[20:45:42] <hauskatze>	 brennen: to commit?
[20:45:46] <hauskatze>	 ssh:// ?
[20:45:59] <hauskatze>	 still, I'm still kinda like gerrit :-)
[20:46:12] * hauskatze hides inside a nuclear shelter
[20:50:37] <longma>	 I don't think you are alone hauskatze :P
[20:51:05] <brennen>	 :P indeed.
[20:52:52] <hauskatze>	 longma: really? Well, thanks heh
[20:53:16] <brennen>	 if anyone feels like adding an ssh key to an account on gitlab-test and attempting to clone a repo there...
[20:53:35] <brennen>	 i'm not quite sure if i need to tweak something on the gitlab end or if my ssh config is getting in the way.
[20:54:16] <longma>	 Well, I think there are plenty of gerrit lovers and plenty of gerrit haters
[20:54:19] <hauskatze>	 brennen: known_hosts issues?
[20:54:45] <hauskatze>	 anyway I'm not in my 'dev' computer to attempt a fix :(
[20:54:51] <greg-g>	 opinions? people have them? strongly? no way!
[20:54:55] <brennen>	 doesn't seem to be - i just get the always helpful `Permission denied (publickey).`
[20:54:59] <mutante>	 is that the regular sshd that runs on any cloud instance or a separate sshd run by a gitlab application?
[20:55:16] <mutante>	 (thinking of gerrit-ssh server)
[20:55:48] <brennen>	 mutante: as i understand it, gitlab uses the regular sshd and runs a binary when the git user connects.
[20:57:16] <mutante>	 ah, ok. maybe then see if the /var/log/auth.log  on the instance also show the "failed public key" 
[20:57:32] <mutante>	 if there is nothing there maybe it already fails at the bastion it jumps through
[20:58:26] <mutante>	 and bastion host changed name
[21:00:18] <mutante>	 brennen: could be related to changes described in the list thread "[Cloud] [Cloud-announce] Phasing out the .wmflabs tld on September 8th" ?
[21:00:46] <mutante>	 i remembered it had that quote "Update your .ssh/config now so you aren't confused tomorrow!"
[21:00:55] <brennen>	 not going through a bastion host (using a floating IP instead), but auth.log is probably a fruitful line of inquiry
[21:00:56] <greg-g>	 yeah, same "permission denied (publickey)" error
[21:01:16] <greg-g>	 verified it's using the right key via .ssh/config (I just added the Host *.wmcloud.org stanza today!)
[21:02:34] <mutante>	 is this a new instance that has been created recently? has anyone connected to it before or first time?
[21:02:50] <mutante>	 sometimes it happens (for some time) after creating new instances
[21:02:53] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Wikimedia-Logstash: scap on beta fails canary check: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T260667 (10EBernhardson) * deleted logstash...
[21:03:38] <brennen>	 mutante: it's been around for a bit, and connecting via regular user works fine.  (i.e., i'm connected as brennen@gitlab-test.wmcloud.org as i debug it.)
[21:03:56] <brennen>	 so it seems likely specific to the gitlab setup.
[21:04:12] <mutante>	 oh, so this is not a regular user since it was created by gitlab 
[21:04:28] <mutante>	 then it sounds like that conflicts with the setup that ties ssh users to LDAP 
[21:04:46] <brennen>	 hmm, yeah, that would make sense now that i think about it.
[21:05:10] <mutante>	 sometimes we have this issue when puppet roles create users and you can't just apply that in cloud
[21:05:23] <brennen>	 i.e. it's essentially just not finding a `git` username in ldap and then it doesn't fall back to locally defined users?
[21:05:29] <brennen>	 something like that would make sense.
[21:05:29] <mutante>	 there have been odd workardounds like creating a wikitech user that is a system user...
[21:05:40] <mutante>	 yea
[21:05:43] <mutante>	 that.. i think
[21:07:20] <mutante>	 so there is "ldapvi" and cloud team uses it. but i don't know if that means they can just create a bunch of special users for this
[21:08:02] <mutante>	 maybe ask wmcs if that is an option
[21:08:42] <brennen>	 mutante:  yeah, will do.  thanks for the input.
[21:09:28] <mutante>	 yep, np
[21:20:51] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10Golbez81) I have module php:7.3 installed on CentOS Linux release 8.2.2004 and it's currentl...
[21:29:14] <wikibugs>	 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Operations, 10Patch-For-Review, 10git-protocol-v2: Upgrade git fleet wide to git 2.20 - https://phabricator.wikimedia.org/T262244 (10hashar) **status update**  @MoritzMuehlenhoff started u...
[21:31:35] <wikibugs>	 (03CR) 10Brennen Bearnes: "> Patch Set 3: Code-Review-1" [releng/dev-images] - 10https://gerrit.wikimedia.org/r/626739 (owner: 10Mholloway)
[21:35:04] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10Golbez81) Ok I guess CentOS 8 is just lagging a little behind as usual.  My RHEL8 Azure mach...
[21:38:47] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 5 others: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19 - https://phabricator.wikimedia.org/T257879 (10MarkAHershberger) FWIW, the maintainer of the RH package is the same person who maintains th...
[22:05:35] <wikibugs>	 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.5; 2019-05-14), and 2 others: Stop using jsonlint and instead use eslint-plugin-json for the linting - https://phabricator.wikimedia.org/T220036 (10Umherirrender)
[22:06:55] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Operations, 10observability: Beta puppet patch "prometheus: make ferm DNS record type configurable" - https://phabricator.wikimedia.org/T244624 (10dpifke) 05Open→03Resolved a:03dpifke Calling this resolved, as it's been a week and nobody has missed the associated patch.
[22:10:06] <dpifke>	 !log Testing https://gerrit.wikimedia.org/r/c/operations/puppet/+/626779 in beta, should only affect deployment-webperf12.
[22:10:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[22:33:09] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10GitLab-Test, 10User-brennen: Unable to push repository to remote, hostname is inaccessible - https://phabricator.wikimedia.org/T262516 (10brennen)
[22:33:11] <wikibugs>	 10Release-Engineering-Team, 10GitLab-Test, 10Cloud-VPS (Quota-requests), 10User-brennen: Request for floating IP / DNS for gitlab-test.wmcloud.org - https://phabricator.wikimedia.org/T261900 (10brennen) 05Open→03Resolved
[22:38:18] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10GitLab-Test, 10User-brennen: Unable to push repository to remote, hostname is inaccessible - https://phabricator.wikimedia.org/T262516 (10brennen) p:05High→03Medium While the gitlab bo...