[00:00:48] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) [00:07:19] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) >>! In T218729#5670450, @fgiunchedi wrote: > If deployment-logstash03 has the same classes applied than d... [00:15:48] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) 19:10 < mutante> !log phab2001 - restart ssh-phab service after repooli... [00:16:21] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) [00:19:49] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) [00:21:36] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) - phab1001 is on buster - phab2001 is now also on buster - next: declar... [00:24:12] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) Technically this ticket is resolved. The "switch prod from phab1003 to... [00:29:41] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) horizon-based hieradata changes: https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/5... [00:30:36] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) a:03Krenair [00:47:34] !log T238707 moved kibana4/logstash-beta proxies to deployment-logstash03, copied /etc/logstash/htpasswd file [00:47:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:47:37] T238707: Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 [00:51:19] !log T238707 created old-logstash-beta proxy to point at old instance, created default Index Pattern on new logstash-beta [00:51:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:53:47] !log T238707 changed dateFormat (under management -> advanced settings) from 'MMMM Do YYYY, HH:mm:ss.SSS' to 'YYYY-MM-DDTHH:mm:ss', and dateFormat:tz from Browser to UTC to match old instance [00:53:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:53:50] T238707: Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 [00:56:00] Why not go to buster? [01:01:24] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10Krenair) logstash-beta now points at deployment-logstash03, deployment-logstash2 is still acc... [01:02:12] paladox, are you volunteering? [01:02:40] nope, was just asking :) [01:03:30] (i was talking jessie -> buster) [01:03:33] https://phabricator.wikimedia.org/T218729#5143033 [01:04:27] ok [01:05:37] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation), 10Patch-For-Review: Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) Wondering what we need to do next. Do we need to copy dashboards over somehow? [03:29:39] PROBLEM - Puppet staleness on deployment-kafka-jumbo-2 is CRITICAL: (Service Check Timed Out) [06:02:32] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201911), 10serviceops, and 2 others: Make PHD run on the backup phabricator server (phab2001, currently) - https://phabricator.wikimedia.org/T232883 (10Marostegui) >>! In T232883#5676791, @Dzahn wrote: > > -... [06:23:51] 10Phabricator, 10LDAP: Specific user cannot log into Phabricator with LDAP account (Phab thinks it's registering a new account, hence email address is already in use) - https://phabricator.wikimedia.org/T168731 (10Reviyo1238) Specific user cannot log into phabricator with LDAP account. Account information is j... [06:46:39] new 1k batch of uploads/captions/depicts started, 8 seconds between edits since this worked fine yesterday [06:52:01] 10Phabricator, 10Operations, 10Traffic, 10serviceops: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10Joe) I don't think the solution is removing aphlict, but instead proxying to it directly from envoy or ATS, our choice. @ema @Dz... [07:09:19] 10Continuous-Integration-Config, 10BlueSpice: BlueSpice related tests fails on gate-and-submit-1.31 in quibble-composer tests - https://phabricator.wikimedia.org/T235807 (10Osnard) How about changing the pattern from `**` to `*/` (or even `*/`)? This would... [07:46:51] (03PS1) 10Pwirth: layout, parameter_function: Fix BlueSpiceUEModulePDFSubpages entries [integration/config] - 10https://gerrit.wikimedia.org/r/551962 [07:55:39] (03PS1) 10Thiemo Kreuz (WMDE): Minor fixups for the AssertEquals sniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/551965 [07:57:33] (03CR) 10Thiemo Kreuz (WMDE): "I just realized this was sitting here for more than 2 weeks already. So all good, I think. Thanks for the merge!" (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/547721 (owner: 10Thiemo Kreuz (WMDE)) [08:30:51] 10Release-Engineering-Team-TODO, 10Operations, 10serviceops, 10Patch-For-Review, 10Wikimedia-Incident: docker-registry: some layers has been corrupted due to deleting other swift containers - https://phabricator.wikimedia.org/T228196 (10akosiaris) 05Open→03Resolved a:03akosiaris I 'll boldly resolv... [09:46:46] uploads are done, captions startnig [09:52:37] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10User-Ladsgroup: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 (10Urbanecm) 05Open→03Resolved >>! In T237077#5637822, @Tgr wrote: > Seems like RESTBase needs to be configured: https://cs.wikipedia.beta.wmflabs.org/api/rest_v1/ is a 404. Wor... [09:54:30] 10Beta-Cluster-Infrastructure, 10Growth-Team: cswiki beta has incorrect wgGENewcomerTasksRemoteApiUrl - https://phabricator.wikimedia.org/T238321 (10Urbanecm) ` urbanecm@deployment-deploy01:~$ mwscript shell.php cswiki Psy Shell v0.9.9 (PHP 7.2.24-1+0~20191026.31+debian9~1.gbpbbacde+wmf1 — cli) by Justin Hilem... [09:58:15] (03PS1) 10Awight: Remove broken coverage report for codesniffer [integration/config] - 10https://gerrit.wikimedia.org/r/552025 [10:07:29] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10fgiunchedi) >>! In T238707#5677116, @Krenair wrote: > Wondering what we need to do next. Do we need to copy dashbo... [10:16:52] (03Abandoned) 10Awight: Remove broken coverage report for codesniffer [integration/config] - 10https://gerrit.wikimedia.org/r/552025 (owner: 10Awight) [10:25:40] (03PS1) 10Awight: Fix code coverage reporting for this repo [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 [10:25:51] (03CR) 10Daimona Eaytoy: "> Abandoned" [integration/config] - 10https://gerrit.wikimedia.org/r/552025 (owner: 10Awight) [10:58:41] (03CR) 10Daimona Eaytoy: "Seems fine. Could you please tag T237004?" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 (owner: 10Awight) [11:02:05] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Operations, 10Wikidata, 10Wikidata-Query-Service: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 (10Mathew.onipe) [11:06:42] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Operations, 10Wikidata, 10Wikidata-Query-Service: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 (10Gehel) @Mathew.onipe needs to be a memeber of... [11:07:28] is there anyone around with admin rights on gerrit to help with T238733 ? [11:07:28] T238733: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 [11:07:30] thanks! [11:07:35] onimisionipe: ^ [11:08:46] ack [12:22:13] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Gerrit-Privilege-Requests, 10Operations, 10Wikidata, 10Wikidata-Query-Service: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 (10MarcoAurelio) [12:25:38] (03PS2) 10Awight: Fix code coverage reporting for this repo [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 (https://phabricator.wikimedia.org/T237004) [12:25:48] (03CR) 10Awight: "> Seems fine. Could you please tag T237004?" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 (https://phabricator.wikimedia.org/T237004) (owner: 10Awight) [12:43:41] captions have finished up and depicts statements have started [13:21:53] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201911), 10serviceops, and 2 others: Make PHD run on the backup phabricator server (phab2001, currently) - https://phabricator.wikimedia.org/T232883 (10Dzahn) Yes, it was. But it was not expected that phd fai... [13:24:23] 10Phabricator, 10Operations, 10Traffic, 10serviceops: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10Dzahn) Either way when switching the Hiera key it should either enable or disable all the things and not just some of them. A... [13:28:24] (03CR) 10Daimona Eaytoy: [C: 03+2] Fix code coverage reporting for this repo [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 (https://phabricator.wikimedia.org/T237004) (owner: 10Awight) [13:28:51] (03Merged) 10jenkins-bot: Fix code coverage reporting for this repo [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/552032 (https://phabricator.wikimedia.org/T237004) (owner: 10Awight) [13:30:36] 10Gerrit, 10Phabricator: define requirements for new devtools cloud vps project to replace "git" and "phabricator" - https://phabricator.wikimedia.org/T236309 (10Dzahn) I commented both here in this ticket right above and on the wiki page you link to that this project can be removed. We are offering to have 3... [13:31:34] 10Gerrit, 10Phabricator: define requirements for new devtools cloud vps project to replace "git", "gerrit" and "phabricator" - https://phabricator.wikimedia.org/T236309 (10Dzahn) [13:34:38] 10MediaWiki-Codesniffer: mediawiki/tools/codesniffer phpunit coverage fatal error - https://phabricator.wikimedia.org/T237004 (10Daimona) 05Open→03Resolved a:03awight Aye https://doc.wikimedia.org/cover/mediawiki-tools-codesniffer/index.html Thanks! [13:47:00] 10Continuous-Integration-Infrastructure, 10MediaWiki-Codesniffer, 10Test-Coverage: Post-merge build failed for mediawiki/tools/codesniffer - https://phabricator.wikimedia.org/T191637 (10awight) [13:47:03] 10MediaWiki-Codesniffer: mediawiki/tools/codesniffer phpunit coverage fatal error - https://phabricator.wikimedia.org/T237004 (10awight) [14:36:25] Hey folks. gerrit seems super slow today. Is something up? [14:36:34] Still working. Just taking forever to clone some repos. [14:41:07] Looks fine to me. If your cloning the puppet repo or mediawiki/core then the clone may take a while due to sizew [14:41:17] *size [14:41:33] Hmm. I have a different issue when I try to clone using https [14:41:36] warning: remote HEAD refers to nonexistent ref, unable to checkout. [14:41:48] For git clone "https://halfak@gerrit.wikimedia.org/r/a/scoring/ores/deploy" [14:42:38] https://gerrit.wikimedia.org/r/#/admin/projects/scoring/ores/deploy,branches [14:42:43] dosen't have any master branch [14:42:53] huh. That's... wrong. [14:43:04] * halfak checks an old working version of the repo. [14:43:07] you need to create the branch locally [14:44:28] Oh FFS. It looks like that's not the right repo. I wonder why it even exist. The right repo is at mediawiki/services/ores/deploy [14:44:30] !!! [14:45:14] :P [14:45:28] I thought we moved all of the repos to a sane location [14:45:42] Apparently we were half-way done and then just gave up. [14:46:23] paladox, if I push the history of mediawiki/services/ores/deploy to scoring/ores/deploy and we just continue work from there, would that work? [14:46:47] Yes. You can git push --mirror [14:47:29] you may need to grant "Forge Author Identity" or "Forge Committer Identity" on that repo [14:47:45] Gotcha. [14:54:11] 10Release-Engineering-Team (Pipeline), 10Release Pipeline, 10Wikimedia-Portals: Migrate www.wikimedia.org (the portal) to be hosted as a service - https://phabricator.wikimedia.org/T238747 (10Jdforrester-WMF) [14:54:23] 10Release-Engineering-Team (Pipeline), 10Release Pipeline, 10Wikimedia-Portals: Migrate www.wikimedia.org (the portal) to be hosted as a service - https://phabricator.wikimedia.org/T238747 (10Jdforrester-WMF) Related to T213806. [15:02:01] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10TechCom: Expand Gerrit Manager permissions - https://phabricator.wikimedia.org/T234474 (10Dzahn) [15:04:57] James_F: re. portals; indeed, fun [15:05:08] too many Node/Npm deps [15:10:54] 10Phabricator, 10User-MModell: On Phabricator workboard, show status of associated Gerrit patches - https://phabricator.wikimedia.org/T215148 (10mmodell) [15:15:06] 10Gerrit, 10Phabricator: define requirements for new devtools cloud vps project to replace "git", "gerrit" and "phabricator" - https://phabricator.wikimedia.org/T236309 (10Andrew) For the new project, please file a ticket here: https://phabricator.wikimedia.org/project/profile/2875/ [15:39:33] iya [15:39:34] hiya [15:39:38] can someone fix some gerrit perms for me? [15:39:38] https://gerrit.wikimedia.org/r/admin/projects/eventgate-wikimedia,access [15:40:00] ottomata what would you want the group to be? [15:40:02] it only has gerrit managers able to do anything [15:40:10] i think it could inherit from mediawiki/services [15:40:12] and it would be good [15:40:47] ok, that needs a admin when re-inheriting [15:40:55] thcipriani twentyafterfour ^ [15:41:52] ottomata: I think we have a task for that? [15:42:16] oh? [15:42:27] oh, nope, sorry, it's for another group [15:42:30] a research repo [15:43:59] paladox's right - changing the inheritances requires an admin [15:44:14] surprise surprise, the Admin group is empty lol [15:44:24] that's because it uses the ldap group [15:44:42] and that let them use the UI as before paladox ? [15:44:52] or do they need to use CLI to do stuff? [15:45:13] it lets them use the UI as before. [15:45:25] :) [15:46:10] hauskater also fyi, i'm now on the Gerrit Frontend Team! [15:46:22] on what? [15:46:40] anyway, congrats paladox [15:46:41] on? [15:46:49] for PolyGerrit :) [15:46:51] (the ui) [15:46:56] I mean, it's WMF or Google team? [15:47:10] google though i doin't work for google :) [15:48:50] I'm glad for you nonetheless [15:48:51] ah, congrats, that's great news! [15:49:19] thanks! [15:59:17] about 70 depicts statements left and that will be it from me for the day [16:14:10] and it's done. more tomorrow [16:34:54] (03PS1) 10Jforrester: Add initial service pipeline config for production.git [integration/config] - 10https://gerrit.wikimedia.org/r/552084 [16:36:13] (03CR) 10Jforrester: [C: 03+2] Add initial service pipeline config for production.git [integration/config] - 10https://gerrit.wikimedia.org/r/552084 (owner: 10Jforrester) [16:37:00] (03Merged) 10jenkins-bot: Add initial service pipeline config for production.git [integration/config] - 10https://gerrit.wikimedia.org/r/552084 (owner: 10Jforrester) [16:37:18] !log Zuul: Add CI for mediawiki/services/eventstreams T238658 [16:37:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:37:21] T238658: Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 [16:37:53] !log Zuul: Add CI for wikimedia/production [16:37:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:38:00] another gerrit q! [16:38:10] is it possible to vary permissions based on directory paths in the repository itself? [16:38:26] e.g. only group A can merge changes that touch restricted/ directory? [16:38:38] On the branch yes, but not on the path, I think. For that we recommend a git submodule with different rights. [16:49:10] You can use prolog rules for path [16:49:19] though i think we recommend against that. [16:51:01] Yeah, please don't try. We don't want to try to support that. [16:57:54] prolog? [16:58:44] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Gerrit-Privilege-Requests, 10Operations, 10Wikidata, 10Wikidata-Query-Service: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 (10crusnov) p:05T... [17:00:06] Yeah. [17:01:24] I don't know what that is, sorry :) [17:01:51] Still trying to learn what git-fat/git-lfs is/does :P [17:02:48] ancient programming language [17:03:06] rather cool in some ways, but not really something you’d want to support in a modern stack [17:03:29] I think there’s some Java implementation that Gerrit presumably uses [17:03:34] oh [17:03:43] thanks Lucas_WMDE :) [17:15:08] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Gerrit-Privilege-Requests, 10Operations, 10Wikidata, 10Wikidata-Query-Service: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe - https://phabricator.wikimedia.org/T238733 (10hashar) 05Open... [17:16:04] 10Continuous-Integration-Infrastructure (Slipway), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Release Pipeline, 10Wikimedia-Portals: Migrate wikimedia-portals-build to Docker container - https://phabricator.wikimedia.org/T213806 (10hashar) [17:16:06] 10Release-Engineering-Team (Pipeline), 10Release Pipeline, 10Wikimedia-Portals: Migrate www.wikimedia.org (the portal) to be hosted as a service - https://phabricator.wikimedia.org/T238747 (10hashar) [17:17:51] 10Release-Engineering-Team (Pipeline), 10Release Pipeline, 10Wikimedia-Portals: Migrate www.wikimedia.org (the portal) to be hosted as a service - https://phabricator.wikimedia.org/T238747 (10hashar) I had a quick talk about this with @Jdrewniak . If we the blubber/pipeline thing setup, that will address T2... [17:20:49] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Analytics, and 4 others: Delete eventgate-ci repository from gerrit - https://phabricator.wikimedia.org/T229111 (10hashar) [17:25:50] Ciao Daimona [17:25:59] Hola :) [17:31:49] (03PS1) 10Jforrester: jjb, layout: Make wikimedia/production bespoke, publish-only [integration/config] - 10https://gerrit.wikimedia.org/r/552099 [17:32:44] (03CR) 10jerkins-bot: [V: 04-1] jjb, layout: Make wikimedia/production bespoke, publish-only [integration/config] - 10https://gerrit.wikimedia.org/r/552099 (owner: 10Jforrester) [17:35:05] !log JJB: Pushed wikimedia-production-pipeline-publish and trigger-wikimedia-production-pipeline-publish [17:35:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:35:28] (03CR) 10Umherirrender: "Would be nice to have this patch set together with Ic386bc26c29c862cc71c852ca7ebb7c63c1c7d9a in one release to avoid issues with the /****" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/551557 (https://phabricator.wikimedia.org/T238470) (owner: 10Umherirrender) [17:36:53] (03PS2) 10Jforrester: jjb, layout: Make wikimedia/production bespoke, publish-only [integration/config] - 10https://gerrit.wikimedia.org/r/552099 [17:38:16] (03CR) 10Jforrester: [C: 03+2] jjb, layout: Make wikimedia/production bespoke, publish-only [integration/config] - 10https://gerrit.wikimedia.org/r/552099 (owner: 10Jforrester) [17:39:03] (03Merged) 10jenkins-bot: jjb, layout: Make wikimedia/production bespoke, publish-only [integration/config] - 10https://gerrit.wikimedia.org/r/552099 (owner: 10Jforrester) [17:40:27] 10Release-Engineering-Team, 10MediaWiki-User-management, 10MediaWiki-extensions-FlaggedRevs, 10Regression, 10User-DannyS712: Pending changes: autoreview randomly fails - https://phabricator.wikimedia.org/T233561 (10Neolexx) The problem still persists at ru-wiki, MBH had to write a bot so once a day to pa... [17:58:09] (03CR) 10Jforrester: [C: 03+2] Remove experimental warning from README [blubber] - 10https://gerrit.wikimedia.org/r/549900 (owner: 10Dduvall) [17:58:29] (03CR) 10jerkins-bot: [V: 04-1] Remove experimental warning from README [blubber] - 10https://gerrit.wikimedia.org/r/549900 (owner: 10Dduvall) [18:00:55] (lol) [18:02:03] 10Release-Engineering-Team-TODO, 10Release Pipeline, 10serviceops-radar, 10Epic: Deploy MediaWiki to Wikimedia production in containers - https://phabricator.wikimedia.org/T238770 (10Jdforrester-WMF) [18:02:37] 10Release-Engineering-Team-TODO, 10Release Pipeline, 10serviceops-radar, 10Epic: Get production MW-land images built and published - https://phabricator.wikimedia.org/T238771 (10Jdforrester-WMF) [18:02:42] thcipriani \o/ caffeine was merged :) [18:03:32] 10Release-Engineering-Team-TODO (201911), 10Release Pipeline, 10serviceops-radar: Create initial production MW-land images with blubber - https://phabricator.wikimedia.org/T238773 (10Jdforrester-WMF) [18:03:43] 10Release-Engineering-Team-TODO (201911), 10Release Pipeline, 10serviceops-radar: Create initial production MW-land images with blubber - https://phabricator.wikimedia.org/T238773 (10Jdforrester-WMF) [18:04:27] 10Release-Engineering-Team, 10Release Pipeline, 10serviceops: Provide the official production base images for Wikimedia use - https://phabricator.wikimedia.org/T238774 (10Jdforrester-WMF) [18:06:09] 10Release-Engineering-Team-TODO, 10Release Pipeline, 10serviceops-radar: Add production config injection - https://phabricator.wikimedia.org/T238775 (10Jdforrester-WMF) [18:06:32] 10Release-Engineering-Team-TODO, 10Release Pipeline, 10serviceops-radar: Add testing entry point - https://phabricator.wikimedia.org/T238776 (10Jdforrester-WMF) [18:15:00] blubberoid is returning 503s: https://integration.wikimedia.org/ci/blue/organizations/jenkins/service-pipeline-test/detail/service-pipeline-test-and-publish/3762/pipeline/ [18:23:18] !log gerrit: Created `operations/software/netbox-extras.git`; request on mediawiki [18:23:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:26:54] Now the tedious creation of the mirrors [18:27:03] * hauskater head-desks [18:44:19] ok no path perms [18:44:25] good to know, i didn't think so [18:44:30] ty [18:46:35] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to webperf-002 - https://phabricator.wikimedia.org/T180761 (10Dzahn) [18:47:47] !log Manual replication cmd issued for new repo. [18:47:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:18:11] 10Phabricator, 10Operations, 10Traffic, 10serviceops: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10Dzahn) [19:43:25] 10Release-Engineering-Team (Pipeline), 10Analytics, 10Release Pipeline, 10Patch-For-Review, 10Services (watching): Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10Ottomata) [19:48:58] 10Release-Engineering-Team (Code Health), 10Code-Stewardship-Reviews, 10MediaWiki-extensions-FlaggedRevs: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10MarcoAurelio) At risk of saying or repeating the obvious, FlaggedRevs needs a maintainer. It is a Wikimedia-deployed ex... [20:22:11] mdholloway: delayed response, but blubberoid should now be good, there was a varnish issue before [20:22:28] greg-g: thanks! [20:31:53] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) [20:49:35] greg-g: still seeing the same response ("503: Backend fetch failed"): https://integration.wikimedia.org/ci/blue/organizations/jenkins/service-pipeline-test/detail/service-pipeline-test-and-publish/3763/pipeline/ [20:50:29] 10Release-Engineering-Team, 10Operations, 10serviceops, 10Performance-Team (Radar), and 2 others: PHP Fatal error: The UdpSocket to 127.0.0.1:10514 has been closed (from Monolog/SyslogUdp on mwdebug1002) - https://phabricator.wikimedia.org/T214734 (10Krinkle) Looks like it is still happening: https://log... [20:54:24] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to xhui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [20:55:25] 10Phabricator, 10Operations, 10Traffic, 10serviceops, 10Patch-For-Review: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10mmodell) >>! In T238593#5678148, @Dzahn wrote: > As Mukunda pointed out the aphlict service does not even... [20:56:25] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to xhui-001 - https://phabricator.wikimedia.org/T180761 (10Dzahn) [21:02:05] mdholloway: ugh, yes, debugging happening in -sre [21:06:52] (03CR) 10Jforrester: [C: 03+2] "…" [blubber] - 10https://gerrit.wikimedia.org/r/549900 (owner: 10Dduvall) [21:10:21] (03CR) 10jerkins-bot: [V: 04-1] Remove experimental warning from README [blubber] - 10https://gerrit.wikimedia.org/r/549900 (owner: 10Dduvall) [21:19:09] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) >>! In T238707#5677730, @fgiunchedi wrote: > If the new elasticsearch cluster has access to the old one... [21:25:49] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Move XHGui from tungsten to xghui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [21:25:57] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Move XHGui from tungsten to xhgui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [21:28:12] !log Create deployment-xhgui01 in Beta Cluster as m1.small/Debian 10 Buster. – T238098, T180761 [21:28:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:28:16] T180761: Move XHGui from tungsten to xhgui-001 - https://phabricator.wikimedia.org/T180761 [21:28:16] T238098: vm request for xhgui - https://phabricator.wikimedia.org/T238098 [21:28:31] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Move XHGui from tungsten to xhgui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [21:30:46] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Move XHGui from tungsten to xhgui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [21:30:54] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Move XHGui from tungsten to xhgui-001 - https://phabricator.wikimedia.org/T180761 (10Krinkle) [21:40:06] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate from deployment-logstash2 (jessie) to deployment-logstash03 (stretch) - https://phabricator.wikimedia.org/T238707 (10Krenair) looks like each of the logstash hosts runs its own elasticsearch cluster locally, would our source be som... [22:15:52] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Scoring-platform-team: gerrit: scoring/ores/editquality takes a long time to git gc - https://phabricator.wikimedia.org/T237807 (10Halfak) This could be an old repo that used before we had git LFS. We should be... [22:18:12] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10CPT Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10CCicalese_WMF) [22:18:23] 10Continuous-Integration-Config, 10Release-Engineering-Team: Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10CCicalese_WMF) [22:19:05] 10Gerrit, 10ORES, 10Scoring-platform-team (Current): Write a cookbook for the workaround for getting LFS to gerrit - https://phabricator.wikimedia.org/T226055 (10Halfak) [22:19:47] 10Gerrit, 10ORES, 10Scoring-platform-team (Current): Write a cookbook for the workaround for getting LFS to gerrit - https://phabricator.wikimedia.org/T226055 (10Halfak) I started some work here for describing how to build the wheels. This is part of the same deployment prep process so it should be included... [23:22:20] 10Gerrit: Rename Gerrit repository "LdapGroups" to "LDAPGroups" - https://phabricator.wikimedia.org/T200736 (10MarkAHershberger) Who manages github mirroring? It looks like there is no mirror of [the current LDAPGroups](https://www.mediawiki.org/w/index.php?title=Topic:Uz1b87najx66wm7f&topic_showPostId=vbfv7y0e... [23:24:27] 10Gerrit: Rename Gerrit repository "LdapGroups" to "LDAPGroups" - https://phabricator.wikimedia.org/T200736 (10MarcoAurelio) >>! In T200736#5680335, @MarkAHershberger wrote: > Who manages github mirroring? It looks like there is no mirror of [the current LDAPGroups](https://www.mediawiki.org/w/index.php?title=T... [23:30:19] paladox: ping [23:31:29] hi [23:32:11] paladox: could you do a replication for me? [23:32:16] yup [23:32:21] I don't have git/ssh in this laptop [23:32:24] thank you [23:32:53] replication start mediawiki/extensions/LDAPGroups --wait [23:33:12] I just created the github mirror for MaxSem [23:33:21] there are 2 extensions with the same name except capitalization [23:33:25] beware of conflicts [23:33:29] one is archived mutante [23:33:36] !log ssh -p 29418 paladox@gerrit.wikimedia.org replication start mediawiki/extensions/LDAPGroups --wait (per hauskater) [23:33:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:33:41] o_0 [23:33:42] Replicate mediawiki/extensions/LDAPGroups ref ..all.. to gerrit2001.wikimedia.org, Succeeded! (OK) [23:33:52] Replicate mediawiki/extensions/LDAPGroups ref ..all.. to github.com, Succeeded! (OK) [23:33:53] hauskater done! [23:33:55] and the github mirror of LdapGroups is deleted [23:34:12] Perfect, thanks paladox [23:34:18] your welcome :) [23:34:33] I was too busy to go downstairs, start the old computer that runs on charcoal, etc. [23:34:52] charcoal? lol [23:35:02] so to speak, it's quite old [23:35:08] heh [23:35:11] Intel Centrino [23:35:41] I recently wiped it with DBAN and installed Ubuntu on it [23:35:46] so far so good [23:35:59] lol [23:39:01] oh sorry MaxSem for the ping, it was for mark A. H. not you [23:39:02] apologies [23:39:29] 10Gerrit: Rename Gerrit repository "LdapGroups" to "LDAPGroups" - https://phabricator.wikimedia.org/T200736 (10MarcoAurelio) Done. Created at https://github.com/wikimedia/mediawiki-extensions-LDAPGroups. [23:41:11] night