[01:17:04] 10Gerrit, 10Phabricator: define requirements for new devtools cloud vps project to replace "git" and "phabricator" - https://phabricator.wikimedia.org/T236309 (10Dzahn) Another thing this would replace: https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2019_Purge#gerrit https://wikitech.wikimedia.org/wiki/N... [01:27:32] 10Gerrit, 10Phabricator: define requirements for new devtools cloud vps project to replace "git" and "phabricator" - https://phabricator.wikimedia.org/T236309 (10Dzahn) >>! In T236309#5609145, @Krenair wrote: > I don't know if we want to be introducing new things to deployment-prep > right now. I'm also not su... [06:05:08] Project mediawiki-core-doxygen-docker build #10982: 04FAILURE in 1 min 4 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10982/ [07:05:04] Project mediawiki-core-doxygen-docker build #10983: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10983/ [07:38:18] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 (10elukey) p:05Triage→03High [08:00:04] Project mwcore-phpunit-coverage-master build #267: 04STILL FAILING in 5 hr 0 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/267/ [08:05:03] Project mediawiki-core-doxygen-docker build #10984: 04STILL FAILING in 58 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10984/ [08:35:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Cloud-VPS, 10cloud-services-team: Investigate Stretch VM being slower than Jessie VM - https://phabricator.wikimedia.org/T236675 (10hashar) `sssd` does not make any not... [08:52:26] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 (10Krenair) You may remove this particular commit. Generally though, just fix the rebase conflict. [09:40:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Cloud-VPS, 10cloud-services-team: Investigate Stretch VM being slower than Jessie VM - https://phabricator.wikimedia.org/T236675 (10hashar) 05Open→03Stalled p:05Tr... [09:40:42] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Test-Coverage: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 (10hashar) [09:43:33] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 (10elukey) Sure, even if I might not resolve the conflict in the best way and break Beta :) Also we don't have any alarm about puppet half... [09:44:09] !log manual rebase fix on deployment-puppetmaster03 to unblock git-sync - T236876 [09:44:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:44:12] T236876: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 [09:45:12] !log mediawiki mcrouter's on deployment-prep are now using deployment-memc-08 (Buster node with memcached) - to report any issue T213089 [09:45:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:45:15] T213089: Upgrade memcached for Debian Stretch/Buster - https://phabricator.wikimedia.org/T213089 [09:46:47] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 (10elukey) 05Open→03Resolved [10:09:02] 10Continuous-Integration-Config, 10Core Platform Team, 10Code-Health: Enable API integration tests in CI for MediaWiki core - https://phabricator.wikimedia.org/T236680 (10daniel) @kostajh suggested to move the API integration test infrastructure into the MW core repo, along with the tests themselves. We alre... [10:46:55] PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [10:48:18] (03PS1) 10Pwirth: layout, parameter_functions: Add BlueSpiceNSFileRepoConnector [integration/config] - 10https://gerrit.wikimedia.org/r/547162 [10:53:39] (03CR) 10Hashar: [C: 03+2] layout, parameter_functions: Add BlueSpiceNSFileRepoConnector [integration/config] - 10https://gerrit.wikimedia.org/r/547162 (owner: 10Pwirth) [10:58:20] Project mediawiki-core-doxygen-docker build #10985: 04STILL FAILING in 1 min 6 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10985/ [10:59:43] RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [11:01:21] (03Merged) 10jenkins-bot: layout, parameter_functions: Add BlueSpiceNSFileRepoConnector [integration/config] - 10https://gerrit.wikimedia.org/r/547162 (owner: 10Pwirth) [11:01:51] (03CR) 10Hashar: [C: 03+2] "deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/547162 (owner: 10Pwirth) [11:06:33] Project mediawiki-core-doxygen-docker build #10986: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10986/ [11:17:47] !log Bring back online a couple instances I have put offline. That was preventing some jobs to run :/ [11:17:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:36:00] 10Phabricator, 10User-DannyS712: Combine H309 and H310 - https://phabricator.wikimedia.org/T236648 (10akosiaris) Easy enough. I just disabled H309, renamed H310 and added myself to it [11:36:17] 10Phabricator, 10User-DannyS712: Combine H309 and H310 - https://phabricator.wikimedia.org/T236648 (10akosiaris) 05Open→03Resolved a:03akosiaris Marking as resolved, feel free to reopen [12:05:14] Project mediawiki-core-doxygen-docker build #10987: 04STILL FAILING in 1 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10987/ [12:18:28] (03CR) 10Pwirth: "> deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/547162 (owner: 10Pwirth) [12:18:41] (03PS2) 10Zfilipin: WIP npm ERR! missing script: selenium-daily WIP [integration/config] - 10https://gerrit.wikimedia.org/r/546679 (https://phabricator.wikimedia.org/T199113) [12:21:49] 10Release-Engineering-Team, 10Operations, 10Prod-Kubernetes, 10Release Pipeline, 10serviceops: Allow testing of feature-flag-protected features in deployment-charts CI - https://phabricator.wikimedia.org/T236899 (10Joe) [12:22:00] 10Release-Engineering-Team, 10Operations, 10Prod-Kubernetes, 10Release Pipeline, 10serviceops: Allow testing of feature-flag-protected features in deployment-charts CI - https://phabricator.wikimedia.org/T236899 (10Joe) p:05Triage→03High a:03Joe [12:33:59] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10MW-1.35-notes (1.35.0-wmf.5; 2019-11-05), and 3 others: All repositories with Selenium tests should use wdio-mediawiki - https://phabricator.wikimedia.org/T199113 (10zeljkofilipin) [13:00:18] finally got the pycairo border stuff sorted out. here come another 1k of 32x32 png uploads [13:00:22] to commons beta. [13:05:17] Project mediawiki-core-doxygen-docker build #10988: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10988/ [13:40:28] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201910), 10Release, 10Train Deployments: 1.35.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T233851 (10LarsWirzenius) 05Open→03Resolved I declare this train to be finished. [13:59:20] 10Gerrit, 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scap: `scap clean` failure - https://phabricator.wikimedia.org/T218783 (10zeljkofilipin) [13:59:36] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scap, 10Performance-Team (Radar): Changing dblist files requires mtime touch of InitialiseSettings.php - https://phabricator.wikimedia.org/T217830 (10zeljkofilipin) [14:04:45] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing: ERROR webdriver: Request failed due to Error: session not created: Chrome version must be between 70 and 73 - https://phabricator.wikimedia.org/T234610 (10zeljkofilipin) @hashar I sti... [14:05:12] Project mediawiki-core-doxygen-docker build #10989: 04STILL FAILING in 1 min 7 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10989/ [14:06:10] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing: ERROR webdriver: Request failed due to Error: session not created: Chrome version must be between 70 and 73 - https://phabricator.wikimedia.org/T234610 (10zeljkofilipin) p:05High→0... [14:16:07] (03PS1) 10Tpt: Start branching Wikisource for wmf/ [tools/release] - 10https://gerrit.wikimedia.org/r/547216 (https://phabricator.wikimedia.org/T210174) [14:19:46] Hey folks. Does gerrit use the same username/keys as production shell access? [14:19:59] halfak: I don't think so [14:20:23] No [14:20:24] To upload changes there you need to use your Labs ssh keypair [14:20:28] It uses what is in ldap [14:20:32] so wikitech [14:20:34] Aha. Trying to work this out for my new engineer. It looks like I'm not using my labs keys. [14:20:36] no and it explicitly should not [14:20:41] ^ [14:20:52] Gotcha. So it should be using a labs key? [14:20:52] kalimera apergos :) [14:21:02] καλησπέρα! [14:21:02] halfak: aye [14:21:06] Thanks. [14:21:18] the one you have at toolsadmin.wikimedia.org halfak [14:24:15] 10Continuous-Integration-Infrastructure: mwgate-composer-php72-docker missing php-intl - https://phabricator.wikimedia.org/T236907 (10Reedy) [14:25:43] 10Continuous-Integration-Infrastructure: mwgate-composer-php72-docker missing php-intl - https://phabricator.wikimedia.org/T236907 (10Reedy) [14:25:45] 10Release-Engineering-Team (Deployment services), 10Security-Team, 10Wikimedia-Extension-setup, 10Wikimedia-Site-requests, 10Wikimedia-extension-review-queue: Deploy WebAuthn to Wikimedia Wikis - https://phabricator.wikimedia.org/T227242 (10Reedy) [14:29:12] (03CR) 10Jforrester: [C: 03+2] "Security review signed off." [tools/release] - 10https://gerrit.wikimedia.org/r/547216 (https://phabricator.wikimedia.org/T210174) (owner: 10Tpt) [14:29:49] (03Merged) 10jenkins-bot: Start branching Wikisource for wmf/ [tools/release] - 10https://gerrit.wikimedia.org/r/547216 (https://phabricator.wikimedia.org/T210174) (owner: 10Tpt) [14:30:02] (03PS1) 10Reedy: Add php-intl to php72 and php73 docker templates [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) [14:31:21] (03CR) 10Reedy: [C: 04-1] "Guess it needs changelogs doing too" [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) (owner: 10Reedy) [14:31:49] (03CR) 10Jforrester: "> Patch Set 1: Code-Review-1" [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) (owner: 10Reedy) [14:33:40] about 320 images in out of 1020 in my bulk upload [14:34:00] of *today's bulk upload, I should say [14:36:13] (03PS2) 10Reedy: Add php-intl to php72 and php73 docker templates [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) [14:38:05] (03CR) 10jerkins-bot: [V: 04-1] Add php-intl to php72 and php73 docker templates [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) (owner: 10Reedy) [14:39:18] (03PS3) 10Reedy: Add php-intl to php72 and php73 docker templates [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) [14:39:27] (03CR) 10Tpt: "Thank you!" [tools/release] - 10https://gerrit.wikimedia.org/r/547216 (https://phabricator.wikimedia.org/T210174) (owner: 10Tpt) [14:50:57] 10Scap, 10MediaWiki-Vendor: Scap sync fails because it tries to lint test files - https://phabricator.wikimedia.org/T202984 (10Reedy) [14:53:31] (03CR) 10Jforrester: "Should we do php71 (and php74 which isn't landed yet) too?" [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) (owner: 10Reedy) [14:54:52] (03CR) 10Reedy: "php74 (when it lands) for sure, we don't want it to regress" [integration/config] - 10https://gerrit.wikimedia.org/r/547220 (https://phabricator.wikimedia.org/T236907) (owner: 10Reedy) [14:56:34] 10Release-Engineering-Team-TODO (201910), 10Wikimedia-Extension-setup, 10Wikisource: Deploy Wikisource extension to beta cluster - https://phabricator.wikimedia.org/T236502 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF [14:57:25] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10MediaWiki-Vendor, 10Documentation: Document what requires mediawiki-vendor required libraries - https://phabricator.wikimedia.org/T220720 (10Reedy) [14:59:19] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10CPT Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10hashar) [15:01:35] (03CR) 10Reedy: [C: 04-1] dockerfiles: Provide initial PHP74 images (based on 7.4.0-rc.3 for now) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/545269 (https://phabricator.wikimedia.org/T236132) (owner: 10Jforrester) [15:05:10] Project mediawiki-core-doxygen-docker build #10990: 04STILL FAILING in 1 min 6 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10990/ [15:06:35] Reedy: ^^ has that r/p/ old links and do core.git as well [15:06:43] not sure which server is using it [15:07:14] integration-agent-docker-1008 ? [15:07:41] Should be easy enough to fix [15:07:57] and "$wgServer must be set in LocalSettings.php" [15:10:26] (03PS2) 10Jforrester: dockerfiles: Provide initial PHP74 images (based on 7.4.0-rc.3 for now) [integration/config] - 10https://gerrit.wikimedia.org/r/545269 (https://phabricator.wikimedia.org/T236132) [15:14:13] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10MW-1.35-notes (1.35.0-wmf.5; 2019-11-05), and 3 others: All repositories with Selenium tests should use wdio-mediawiki - https://phabricator.wikimedia.org/T199113 (10zeljkofilipin) [15:14:15] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10Patch-For-Review, 10User-zeljkofilipin: Upgrade webdriverio to version 5 for all repositories - https://phabricator.wikimedia.org/T234314 (10zeljkofilipin) [15:16:33] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Update files required by Selenium in core, extensions and skins - https://phabricator.wikimedia.org/T210726 (10zeljkofilipin) [15:20:40] I can look in a few hours. Travelling atm [15:20:53] Reedy: of course, safe travel [15:25:35] (03CR) 10Andrew-WMDE: [C: 03+1] "Looks good to me, would +2 but I can't" [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [15:30:10] (03PS3) 10Jforrester: dockerfiles: Provide initial PHP74 images (based on 7.4.0-rc.4 for now) [integration/config] - 10https://gerrit.wikimedia.org/r/545269 (https://phabricator.wikimedia.org/T236132) [15:30:12] (03PS1) 10Jforrester: dockerfiles: Fix unbuildable references to removed composer-package image [integration/config] - 10https://gerrit.wikimedia.org/r/547240 [15:31:28] (03CR) 10Jforrester: [C: 03+2] "Landing this now. The existing images in docker-registry work fine, but if they're lost this will let them re-build correctly. No need to " [integration/config] - 10https://gerrit.wikimedia.org/r/547240 (owner: 10Jforrester) [15:31:57] (03CR) 10Jforrester: "rc.5 got released yesterday but isn't in Debian yet. Will rev this until .0 is released." [integration/config] - 10https://gerrit.wikimedia.org/r/545269 (https://phabricator.wikimedia.org/T236132) (owner: 10Jforrester) [15:33:19] (03Merged) 10jenkins-bot: dockerfiles: Fix unbuildable references to removed composer-package image [integration/config] - 10https://gerrit.wikimedia.org/r/547240 (owner: 10Jforrester) [15:33:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (201910), 10PHP 7.4 support, 10Patch-For-Review: Provide CI testing for PHP 7.4 - https://phabricator.wikimedia.org/T236132 (10Jdforrester-WMF) 05Open→03Stalled Waiting on upstream release. [15:38:33] a little over halfway done with the uploads, then we move onto captions [15:45:06] (03PS1) 10Matthias Mullie: Add Echo dependency for MachineVision [integration/config] - 10https://gerrit.wikimedia.org/r/547253 [15:51:17] (03CR) 10Eric Gardner: [C: 03+1] Add Echo dependency for MachineVision [integration/config] - 10https://gerrit.wikimedia.org/r/547253 (owner: 10Matthias Mullie) [15:56:19] !log gerrit : created https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/XLS # T236914 [15:56:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:56:22] T236914: Requesting new Gerrit repository for XSL extension - https://phabricator.wikimedia.org/T236914 [15:56:55] 10Project-Admins: Request User Project for jbond - https://phabricator.wikimedia.org/T236922 (10jbond) [16:02:13] 10Phabricator, 10User-DannyS712: Disable H192 - https://phabricator.wikimedia.org/T236925 (10DannyS712) [16:02:46] 10Phabricator, 10User-DannyS712: Disable H192 - https://phabricator.wikimedia.org/T236925 (10DannyS712) See also {T108586} [16:05:14] Project mediawiki-core-doxygen-docker build #10991: 04STILL FAILING in 1 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10991/ [16:06:41] 10Phabricator, 10User-DannyS712: Disable H192 - https://phabricator.wikimedia.org/T236925 (10Aklapper) 05Open→03Resolved a:03Aklapper Good catch! Thanks, disabled. [16:24:55] 10Project-Admins, 10User-MarcoAurelio: Request User Project for jbond - https://phabricator.wikimedia.org/T236922 (10MarcoAurelio) a:03MarcoAurelio [16:26:48] 10Project-Admins, 10User-MarcoAurelio: Request User Project for jbond - https://phabricator.wikimedia.org/T236922 (10MarcoAurelio) 05Open→03Resolved p:05Triage→03Normal Done. Created as #user-jbond. Best regards. [16:28:26] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Update files required by Selenium in core, extensions and skins - https://phabricator.wikimedia.org/T210726 (10zeljkofilipin) [16:31:37] (03CR) 10Andrew-WMDE: Move argument parsing to module scope (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/546125 (owner: 10Awight) [16:31:52] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Release new version of wdio-mediawiki - https://phabricator.wikimedia.org/T236927 (10zeljkofilipin) [16:32:32] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Release new version of wdio-mediawiki - https://phabricator.wikimedia.org/T236927 (10zeljkofilipin) p:05Triage→03Normal [16:33:09] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Release new version of wdio-mediawiki - https://phabricator.wikimedia.org/T236927 (10zeljkofilipin) [16:33:11] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10Patch-For-Review, 10User-zeljkofilipin: Upgrade webdriverio to version 5 for all repositories - https://phabricator.wikimedia.org/T234314 (10zeljkofilipin) [16:36:25] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Release new version of wdio-mediawiki - https://phabricator.wikimedia.org/T236927 (10zeljkofilipin) [16:50:32] 10Continuous-Integration-Infrastructure, 10Operations, 10Packaging, 10User-jbond: PCC always has an ERROR when compiling for servers with profile::redis::slave - https://phabricator.wikimedia.org/T228266 (10jbond) [16:52:11] hm I have a message that an upload failed because 'the file repository "local" is in read-only mode.' [16:52:16] (to commons beta) [16:52:46] guess I'll try captions for today's uploads that succeeded [16:54:10] (03CR) 10Jforrester: [C: 03+2] Add Echo dependency for MachineVision [integration/config] - 10https://gerrit.wikimedia.org/r/547253 (owner: 10Matthias Mullie) [16:56:26] (03Merged) 10jenkins-bot: Add Echo dependency for MachineVision [integration/config] - 10https://gerrit.wikimedia.org/r/547253 (owner: 10Matthias Mullie) [16:58:27] !log Zuul: Add Echo as a phan and real dependency for MachineVision [16:58:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:05:18] Project mediawiki-core-doxygen-docker build #10992: 04STILL FAILING in 1 min 14 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10992/ [17:10:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Cloud-VPS, 10cloud-services-team: Investigate Stretch VM being slower than Jessie VM - https://phabricator.wikimedia.org/T236675 (10hashar) [17:20:43] (03CR) 10Jeena Huneidi: "I'm not sure when this happened but it looks like the password "adminpass" is invalid, so perhaps we should change that as well? I tried "" [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546988 (owner: 10Zfilipin) [17:23:23] "only" 700 more captions to go [17:24:29] 10Release-Engineering-Team, 10Operations, 10puppet-compiler, 10Puppet, 10User-jbond: add compiler1003 to jenkins - https://phabricator.wikimedia.org/T236468 (10jbond) [17:25:21] (03CR) 10Jforrester: [C: 03+1] "> Patch Set 1:" [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546988 (owner: 10Zfilipin) [17:26:54] (03PS1) 10MarcoAurelio: [XLS] Add new XLS extension on zuul [integration/config] - 10https://gerrit.wikimedia.org/r/547265 [17:31:57] 10Continuous-Integration-Config, 10Operations, 10User-jbond: operations/puppet CI errors not being displayed in console log - https://phabricator.wikimedia.org/T214726 (10jbond) [17:37:33] James_F: hi. Do you think we can use extension-quibble for XLS or we need something different? Repo is currently empty but the author wants to import from https://github.com/freephile/XSL apparently [17:38:07] oh crap [17:38:10] it's XSL [17:38:15] 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide, 10Patch-For-Review: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10Dzahn) I created a new keypair for deployment (ke... [17:38:16] Ha. [17:38:19] fark [17:38:33] And yes, extension-quibble should be fine for all extensions. [17:38:39] (03CR) 10Zfilipin: "> Patch Set 1:" [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546988 (owner: 10Zfilipin) [17:38:47] damn damn damn [17:38:54] hauskater lol [17:38:59] renaming projects in gerrit is not possible right? [17:39:05] delete the project [17:39:12] but it has a commit paladox [17:39:16] ohh [17:39:18] won't it cause logspam? [17:39:25] yeh [17:40:23] 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide, 10Patch-For-Review: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10mmodell) heh amazing timing! I just tried it and... [17:40:26] I'll just archive it [17:40:29] 10Project-Admins, 10User-DannyS712: Archive #iOS-app-feature-WP0 component - https://phabricator.wikimedia.org/T236932 (10DannyS712) [17:41:33] hauskater i guess we can delete it once upstream fixed that issue [17:41:38] there's a patch for this [17:41:42] (03CR) 10MarcoAurelio: [C: 04-1] "It's XSL, not XLS." [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [17:41:55] https://gerrit-review.googlesource.com/c/plugins/delete-project/+/242132 [17:41:57] yeah [17:42:13] for now, I'll tag it as archived and 'deleteme' for when that patch is merged [17:42:32] ok! [17:42:54] 10Phabricator, 10User-DannyS712: Disable H195 - https://phabricator.wikimedia.org/T236933 (10DannyS712) [17:43:09] 10Phabricator, 10User-DannyS712: Disable H195 - https://phabricator.wikimedia.org/T236933 (10DannyS712) See also {T108586} [17:43:56] 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide, 10Patch-For-Review: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10Dzahn) Once deployment has been confirmed keep in... [17:44:55] 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide, 10Patch-For-Review: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10Dzahn) >>! In T235677#5620454, @mmodell wrote: >... [17:45:13] (03PS2) 10MarcoAurelio: Add new XSL extension on zuul [integration/config] - 10https://gerrit.wikimedia.org/r/547265 [17:45:18] 10Phabricator, 10User-DannyS712: Disable H200 - https://phabricator.wikimedia.org/T236934 (10DannyS712) [17:45:27] 10Phabricator, 10User-DannyS712: Disable H200 - https://phabricator.wikimedia.org/T236934 (10DannyS712) See also {T108586} [17:45:39] (03PS3) 10MarcoAurelio: Add new XSL extension on zuul [integration/config] - 10https://gerrit.wikimedia.org/r/547265 [17:45:57] (03CR) 10MarcoAurelio: "Not yet created. I'll un-WIP this when done." [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [17:47:29] (03PS1) 10MarcoAurelio: Edit Project Config [extensions/XLS] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/547267 [17:47:39] (03Abandoned) 10MarcoAurelio: Edit Project Config [extensions/XLS] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/547267 (owner: 10MarcoAurelio) [17:47:52] PROBLEM - Parsoid on deployment-parsoid09 is CRITICAL: connect to address 172.16.5.63 and port 8000: Connection refused [17:47:52] PROBLEM - Parsoid on deployment-mediawiki-parsoid10 is CRITICAL: connect to address 172.16.0.141 and port 8000: Connection refused [17:47:55] 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide, 10Patch-For-Review: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10Dzahn) [17:51:22] 10Phabricator, 10User-DannyS712: Edit of global herald rule H322 - https://phabricator.wikimedia.org/T235145 (10DannyS712) [17:51:35] I could edit the group paladox though [17:52:47] yes [17:52:52] that'll rename it everywhere :) [17:53:22] recreating as XSSSSSSSSSSSSSSSL [17:53:59] (03CR) 10Jforrester: "Looks fine." [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [17:54:43] lololol [17:56:27] too similar to XSS, don’t scare us like that D: [17:56:42] :P [17:57:47] or http://bash.org/?400459 :D [18:01:46] thcipriani: hashar: I wonder how long we want to keep cobalt around. [18:02:04] i have the decom patches ready but so far i was waiting "just in case" [18:02:06] we got a backup [18:02:14] copy of /srv/git has been synced to gerrit1001 [18:02:14] yea, i just confirmed bacula has cobalt [18:02:19] that too [18:02:28] hopefully the backup works ;] [18:02:46] and beside the issues with references.. I haven't noticed anything [18:03:05] so that all sounds like cobalt can be recycled [18:03:18] where's jenkins-bot for integration/config? [18:03:24] No checks for my patches now? [18:03:32] hashar: paladox: i note that the backup pathes in Bacula for gerrit are only /srv/gerrit/ but nothing outside /srv/ [18:04:14] since we rsynced more on migration. does it mean we want to add a path .. looking [18:04:41] we are backing up /srv/gerrit/git [18:05:05] Project mediawiki-core-doxygen-docker build #10993: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10993/ [18:06:36] mutante well indexes can easily be rebuilt but expensive in terms of you have to rebuild it [18:09:36] thcipriani: would like your thoughts on whether it’s worth backin the indexes up ^ [18:11:28] 10Continuous-Integration-Config, 10CPT Initiatives (API Integration Tests), 10Code-Health: Enable API integration tests in CI for MediaWiki core - https://phabricator.wikimedia.org/T236680 (10WDoranWMF) [18:15:30] i am starting with _some_ decom steps. like first removing it from ssh/known_hosts on the prod gerrit server [18:15:43] (03CR) 10Jforrester: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [18:17:54] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Documentation: mediawiki-core-doxygen-docker broken / MediaWiki doc not generated - https://phabricator.wikimedia.org/T236939 (10hashar) [18:18:25] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201910), 10Documentation: mediawiki-core-doxygen-docker broken / MediaWiki doc not generated - https://phabricator.wikimedia.org/T236939 (10hashar) p:05Triage→03Unbreak! Easy fix: going... [18:21:56] (03PS1) 10Hashar: Set $wgServer for Doxygen jobs [integration/config] - 10https://gerrit.wikimedia.org/r/547276 (https://phabricator.wikimedia.org/T236939) [18:23:01] James_F: are you able to SWAT? [18:23:59] Project mediawiki-core-doxygen-docker build #10994: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10994/ [18:26:41] (03PS2) 10Hashar: Set $wgServer for Doxygen jobs [integration/config] - 10https://gerrit.wikimedia.org/r/547276 (https://phabricator.wikimedia.org/T236939) [18:27:20] integration-config-zuul-layout-diff-docker seems stuck somewhat [18:27:32] it takes 30 min and bails out [18:29:21] Yeah, looks like upstream issues with the job. hashar, ideas? [18:30:33] (03CR) 10Hashar: [C: 03+2] "Jobs updated :]" [integration/config] - 10https://gerrit.wikimedia.org/r/547276 (https://phabricator.wikimedia.org/T236939) (owner: 10Hashar) [18:30:46] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201910), 10Documentation, 10Patch-For-Review: mediawiki-core-doxygen-docker broken / MediaWiki doc not generated - https://phabricator.wikimedia.org/T236939 (10hashar) 05Open→03Resolve... [18:31:16] hauskater: James_F looking [18:31:26] thanks :) [18:31:52] no idea :- _ [18:31:59] OH [18:32:01] you mean it times out! [18:32:27] well that works for others [18:32:29] so hmm [18:32:41] something is wrong with https://gerrit.wikimedia.org/r/#/c/547265/ [18:32:44] Yippee, build fixed! [18:32:44] Project mediawiki-core-doxygen-docker build #10995: 09FIXED in 7 min 24 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10995/ [18:33:13] (03Merged) 10jenkins-bot: Set $wgServer for Doxygen jobs [integration/config] - 10https://gerrit.wikimedia.org/r/547276 (https://phabricator.wikimedia.org/T236939) (owner: 10Hashar) [18:33:29] or somewhere else hmm [18:34:30] I don't know really [18:34:34] we should lower that job time out anyway [18:34:38] it should be fast most of the time [18:34:44] and I noticed an issue in Jenkins which is that [18:34:49] when the console spams way too much data [18:34:54] the console is super slow to process [18:35:00] which can lead to the issue there [18:35:26] maybe related to the collapsible section plugin [18:35:31] it does bunch of regex [18:35:35] that might be the reason [18:35:40] I am in meeting now soryr :-\ [18:46:57] 10Project-Admins, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-WP0, 10User-DannyS712: Archive #iOS-app-feature-WP0 component - https://phabricator.wikimedia.org/T236932 (10Aklapper) Thanks. That requires input from folks using the #iOS-app-feature-WP0 component... https://phabricator.wikimedia.org/projec... [18:49:37] (03CR) 10Jforrester: [C: 03+2] "Let's merge this for now, and worry about CI being broken later." [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [18:50:09] 10Phabricator, 10User-DannyS712: Disable H195 - https://phabricator.wikimedia.org/T236933 (10Aklapper) 05Open→03Resolved a:03Aklapper Thanks for filing this. Rule disabled. [18:51:21] (03Merged) 10jenkins-bot: Add new XSL extension on zuul [integration/config] - 10https://gerrit.wikimedia.org/r/547265 (owner: 10MarcoAurelio) [18:51:41] !log Zuul: Add CI for new XSL extension [18:51:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:51:49] hauskater: Good luck. [18:51:56] yeah the failure is not related [18:52:04] James_F: for anything in special? :) [18:52:08] not sure why it suddenlty gets super slow though :-\ [18:52:43] fyi: i just removed cobalt from CI firewall [18:52:55] (in case it was still cobalt IP in some forgotten place or something) [18:53:42] i also saw "worry about CI being broken later" for unrelated things at the same time so i wasn't worried :) [18:58:29] 10Phabricator, 10User-DannyS712: Disable H200 - https://phabricator.wikimedia.org/T236934 (10Aklapper) 05Open→03Resolved a:03Aklapper Indeed, thanks a lot for finding this! Disabled. [18:58:56] (03CR) 10Krinkle: [C: 03+2] "It won't be optimal if a commit is tagged from another branch or from a detached tree, but it will work fine as it's only a cache." [integration/config] - 10https://gerrit.wikimedia.org/r/546340 (https://phabricator.wikimedia.org/T232055) (owner: 10Krinkle) [19:10:00] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations: Reimage cobalt and gerrit2001 as buster - https://phabricator.wikimedia.org/T176774 (10Dzahn) [19:11:48] !log Disabled explicit mirroring of rCICF to GitHub via Diffusion URI; let Gerrit replication take over [19:11:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:12:59] there are a few 30 minutes spikes from time to time https://integration.wikimedia.org/ci/job/integration-config-zuul-layout-diff-docker/buildTimeTrend [19:13:28] hauskater: nice [19:14:27] mutante: It just duplicates the work [19:14:53] diffusion replicates refs/changes too [19:14:57] I'll continue checking which repos are still not using replication [19:15:06] though i guess we could configure gerrit to do that now that the github bug is fixed [19:15:38] hauskater: i like it just because it removes diffusion related things. thanks [19:15:41] https://github.com/wikimedia/integration-config/commits/master <-- I just see refs/for/master [19:15:56] yeh [19:16:01] refs/changes are hidden in github [19:16:09] Diffusion is still there observing from Gerrit [19:16:09] but can be found if you know the change id [19:16:46] Flow is another one [19:18:56] for Flow, see #wikimedia-collaboration :p [19:19:53] no thanks :P [19:20:02] I Did The Neccessary [19:20:07] (TM) [19:20:27] (c) 2019, MarcoAurelio; All Rights Reserved [19:21:00] (03PS1) 10Hashar: Lower timeout for zuul-layout-diff [integration/config] - 10https://gerrit.wikimedia.org/r/547286 [19:21:25] (03PS2) 10Hashar: Lower timeout for zuul-layout-diff [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) [19:21:44] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Patch-For-Review: integration-config-zuul-layout-validate-docker takes too long in Jenkins due to huge output - https://phabricator.wikimedia.org/T232287 (10hashar) [19:21:50] hauskater: James_F the timeout bug is https://phabricator.wikimedia.org/T232287 [19:22:35] trying to compile something in puppet compiler and getting "DataBinding 'hiera': (): could not find expected ':' while scanning a simple key" which look unrelated to my actual change. more like it fails on anything that includes a Hiera lookup [19:22:46] thanks hashar [19:23:03] mutante: uh, Hiera [19:23:08] and puppet in general [19:23:11] "Ugh." [19:23:16] (03PS3) 10Hashar: Lower timeout for zuul outputting jobs [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) [19:23:18] that is all I have for tonight [19:23:20] bye bye ;) [19:23:24] (undeployed [19:23:29] adieu [19:23:59] not deployed I mean I gotta rush [19:24:11] * James_F deploys instead. [19:49:43] 250 captions to go... bored of watching it [19:52:05] (03CR) 10Jforrester: [C: 03+1] "LGTM but I don't know how to push new jjb updates; the README docs don't work for me. ;-)" [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) (owner: 10Hashar) [20:00:03] Project mwcore-phpunit-coverage-master build #268: 04STILL FAILING in 5 hr 0 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/268/ [20:29:21] * hashar needs a time machine [20:31:42] (03PS1) 10Hashar: README.md separate tox and command args with -- [integration/config] - 10https://gerrit.wikimedia.org/r/547299 [20:31:47] PROBLEM - Host deployment-eventgate-1 is DOWN: CRITICAL - Host Unreachable (172.16.7.89) [20:32:00] PROBLEM - Host deployment-eventgate-2 is DOWN: CRITICAL - Host Unreachable (172.16.0.38) [20:32:15] (03CR) 10Hashar: "> LGTM but I don't know how to push new jjb updates; the README docs don't work for me. ;-)" [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) (owner: 10Hashar) [20:32:26] James_F: ^^ ;] [20:32:37] tox -e jenkins-jobs --conf jenkins.ini etc [20:32:42] that does not work indeed :-\\\ [20:32:47] I just deleted those hosts ^^ [20:32:49] gotta use -- to stpo tox [20:44:18] !log integration-agent-docker-1005 downgrade docker to docker-ce 18.06.2~ce~3-0~debian , the one we had on Jessie # T236675 [20:44:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:44:22] T236675: Investigate Stretch VM being slower than Jessie VM - https://phabricator.wikimedia.org/T236675 [20:47:01] so [20:47:14] really I must have missed something after my lunch break today [20:47:23] cause I supposedly wanted to test Docker at the same version [20:47:24] and failed [20:47:28] thcipriani: tldr: it is docker ;] [20:54:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Cloud-VPS, 10cloud-services-team: Investigate Stretch VM being slower than Jessie VM - https://phabricator.wikimedia.org/T236675 (10hashar) 05Stalled→03Open On the... [20:55:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Test-Coverage: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 (10hashar) [20:55:06] gotta rollback all CI instances to the previous Docker :-\\ [20:55:09] back later [21:03:11] (03CR) 10Jforrester: [C: 03+2] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/547299 (owner: 10Hashar) [21:04:19] (03Abandoned) 10Umherirrender: [DonationInterface] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/547031 (https://phabricator.wikimedia.org/T224763) (owner: 10Umherirrender) [21:04:20] hashar that was quick :P [21:04:43] hmm [21:04:45] took me 10 hours or so ?:-!!-! [21:04:47] not counting the few attempts earlier [21:04:49] I am stupid that is all [21:05:03] or did that docker upgrade without monitoring speed bah [21:05:11] (03Merged) 10jenkins-bot: README.md separate tox and command args with -- [integration/config] - 10https://gerrit.wikimedia.org/r/547299 (owner: 10Hashar) [21:05:23] James_F: so that should fix it. Sorry for the screw up in README.md :\ [21:07:19] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10Wikimedia-General-or-Unknown, 10Epic, and 2 others: Enable mediawiki/mediawiki-phan-config on all Wikimedia-deployed repositories - https://phabricator.wikimedia.org/T224783 (10Umherirrender) [21:09:23] Someone should change the bots.wmflabs link in the topic to use HTTPS instead [21:09:33] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Operations, 10serviceops: Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10hashar) Eventually a few days after I noticed jobs were slower than expected and spend a couple days narrowing down. Tha... [21:11:26] (03CR) 10Jforrester: [C: 03+1] "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) (owner: 10Hashar) [21:11:34] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) (owner: 10Hashar) [21:14:03] (03Merged) 10jenkins-bot: Lower timeout for zuul outputting jobs [integration/config] - 10https://gerrit.wikimedia.org/r/547286 (https://phabricator.wikimedia.org/T232287) (owner: 10Hashar) [21:14:27] hi qchris [21:14:35] Hi hauskater [21:17:42] !log Rolling back docker on CI Stretch instances to 18.06.2~ce~3-0~debian (matching jessie) [21:17:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:17:45] qchris: !!!! :] [21:17:56] Heya hashar :-) [21:18:01] we miss you :-\ [21:18:16] Miss you too! [21:19:20] remembers me I never followed up on moving git repositories creation from mediawiki.org toward Phabricator :\ [21:19:28] then it is nicely handled so .. it is not a concern reall [21:19:29] y [21:19:38] Hahahaha :-D [21:20:01] nicely handled because it's qchris and myself (way more him than me) that do all the work hashar :P [21:20:11] apparently we have swap https://gerrit.wikimedia.org/r/monitoring?part=graph&graph=usedSwapSpaceSize :P [21:20:15] exactly my point :] [21:20:29] hauskater: that is an indirect way to congratulates you two! [21:20:30] Last week when I saw all that bunch of BlueSpice.* repo creations... ohmy [21:20:39] I nearly fell off my chair [21:20:49] * qchris bows to hashar and hauskater. Thanks :-) [21:20:57] :-P [21:21:06] hashar: well I'd prefer something more tangible... gold bars or something like that :P [21:21:08] hauskater was there a chusion behind? :P [21:21:28] paladox: no, cold, hard, stone floor [21:21:38] heh [21:21:53] well actually my house floor is of oak [21:21:58] still, it's hard [21:23:15] twentyafterfo-ur said he'll see if we can create a tool to make the process easier for us [21:23:35] Hardest part is to find a name that has PH on it :P [21:23:37] I think there was a task to make it self do [21:23:55] make a plugin in gerrit for it to setup the github mirror/repo [21:24:03] and Diffusion [21:24:16] that wasen't on it, but it can [21:25:00] he's created a task recently for this [21:25:08] yup [21:25:11] saw that too [21:25:22] But hardest part is to have the name contain 'ph' [21:25:30] like phatality :P [21:25:38] or maniphest [21:25:42] such phun [21:26:02] lol [21:27:37] James_F: so we're moving with WebAuthn? [21:27:44] curious to see how that works [21:28:01] hauskater: Apparently so. I just work here. ;-) [21:29:29] You're ubiquous [21:29:44] You work everywhere [21:31:09] Yeah, so everything's my fault. [21:31:14] *ubiquitous [21:31:17] Of course [21:31:48] But the successes are also yours! [21:37:09] marxarelli: I am downgrading docker on the CI hosts, that will show up in the jenkins metrics you are generating [21:38:16] apparently I upgraded them on Sep 23 [21:41:01] (03PS2) 10Jforrester: Add Echo as phan dependency for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/546955 (https://phabricator.wikimedia.org/T231395) (owner: 10Urbanecm) [21:46:41] hashar: kk [21:47:11] (03CR) 10Jforrester: [C: 03+2] Add Echo as phan dependency for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/546955 (https://phabricator.wikimedia.org/T231395) (owner: 10Urbanecm) [21:48:49] !log Triggering build of mwcore-phpunit-coverage-master , should use Docker 18.06.2 and be faster # T232706 [21:48:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:48:53] T232706: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 [21:49:05] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Test-Coverage: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 (10hashar) p:05Triage→03High [21:49:09] (03Merged) 10jenkins-bot: Add Echo as phan dependency for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/546955 (https://phabricator.wikimedia.org/T231395) (owner: 10Urbanecm) [21:51:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Test-Coverage: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 (10hashar) I have rolled back the whole fleet to Docker... [21:51:33] !log Zuul: Add Echo as phan dependency for CentralAuth [21:51:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:51:53] taking a break [21:52:35] !log Rolled back all CI WMCS Docker to 18.06.2 [21:52:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:52:45] then will have to bisect it :-\\ [22:12:52] (03PS5) 10Jforrester: layout, parameter_functions: Add ExportTables, UEModuleTable2Excel [integration/config] - 10https://gerrit.wikimedia.org/r/546923 (owner: 10Pwirth) [22:19:14] (03CR) 10Jforrester: [C: 03+2] layout, parameter_functions: Add ExportTables, UEModuleTable2Excel [integration/config] - 10https://gerrit.wikimedia.org/r/546923 (owner: 10Pwirth) [22:21:04] (03Merged) 10jenkins-bot: layout, parameter_functions: Add ExportTables, UEModuleTable2Excel [integration/config] - 10https://gerrit.wikimedia.org/r/546923 (owner: 10Pwirth) [22:22:33] !log added jdl to deployment-prep https://www.mediawiki.org/w/index.php?title=Wikimedia_Product/Analytics_Infrastructure&diff=3487111&oldid=3487110&diffmode=source [22:22:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:23:18] PROBLEM - Puppet errors on integration-agent-docker-1005 is CRITICAL: CRITICAL: 2.25% of data above the critical threshold [3.0] [22:31:53] (03CR) 10Jforrester: [C: 03+2] [WikimediaMaintenance] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/546736 (https://phabricator.wikimedia.org/T224770) (owner: 10Umherirrender) [22:33:19] (03Merged) 10jenkins-bot: [WikimediaMaintenance] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/546736 (https://phabricator.wikimedia.org/T224770) (owner: 10Umherirrender) [22:34:00] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: git-sync on deployment-puppetmaster03 fails to rebase since days ago - https://phabricator.wikimedia.org/T236876 (10Krenair) >>! In T236876#5618483, @elukey wrote: > Sure, even if I might not resolve the conflict in the best way and break Beta :) As l... [22:40:45] so CI looks all fine Docker wise. And hmm it is sleep time;) [22:41:20] * hashar waves [23:58:18] RECOVERY - Puppet errors on integration-agent-docker-1005 is OK: OK: Less than 1.00% above the threshold [2.0]