[00:06:15] uhhhh [00:07:22] oh I see [00:17:33] (03PS1) 10Legoktm: docker: Update bazel GPG key [integration/config] - 10https://gerrit.wikimedia.org/r/620807 (https://phabricator.wikimedia.org/T260608) [00:19:38] (03CR) 10Legoktm: [C: 03+2] docker: Update bazel GPG key [integration/config] - 10https://gerrit.wikimedia.org/r/620807 (https://phabricator.wikimedia.org/T260608) (owner: 10Legoktm) [00:20:39] (03Merged) 10jenkins-bot: docker: Update bazel GPG key [integration/config] - 10https://gerrit.wikimedia.org/r/620807 (https://phabricator.wikimedia.org/T260608) (owner: 10Legoktm) [00:21:19] !log rebuilding bazel & gerrit images (with https://gerrit.wikimedia.org/r/620807 fix) [00:21:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:37:04] (03PS1) 10Legoktm: docker: Fix installation path of bazel.gpg [integration/config] - 10https://gerrit.wikimedia.org/r/620809 (https://phabricator.wikimedia.org/T260608) [00:37:06] (03CR) 10Legoktm: [C: 03+2] docker: Fix installation path of bazel.gpg [integration/config] - 10https://gerrit.wikimedia.org/r/620809 (https://phabricator.wikimedia.org/T260608) (owner: 10Legoktm) [00:37:07] try #2 [00:38:06] (03Merged) 10jenkins-bot: docker: Fix installation path of bazel.gpg [integration/config] - 10https://gerrit.wikimedia.org/r/620809 (https://phabricator.wikimedia.org/T260608) (owner: 10Legoktm) [00:38:29] !log rebuilding bazel & gerrit images (try #2) [00:38:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:53:46] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure, 10DBA, 10Wikimedia-Rdbms, 10Epic: Enable MariaDB/MySQL's Strict Mode - https://phabricator.wikimedia.org/T108255 (10Reedy) [00:57:03] it's still building but I have to go afk [00:57:16] at least the fact that it's still building means it's succeeding? [01:05:52] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Try "deploy-promote testwikis" in train-dev, document what seems to be missing - https://phabricator.wikimedia.org/T260376 (10mm... [01:07:34] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: mirror-repos.sh snapshots created at different times will have different train releas... - https://phabricator.wikimedia.org/T260593 [04:55:43] 10Phabricator: Rename account Zoranzoki21 to Kizule on Phabricator - https://phabricator.wikimedia.org/T260645 (10Zoranzoki21) [04:56:34] 10Phabricator: Rename account Zoranzoki21 to Kizule on Phabricator - https://phabricator.wikimedia.org/T260645 (10Zoranzoki21) 05Open→03Stalled Until 18th September when account should be renamed on wikis. [04:56:43] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review: releng/bazel CI image fails to build - https://phabricator.wikimedia.org/T260608 (10Legoktm) 05Open→03Resolved a:03Legoktm... [04:57:43] 10Phabricator: Rename account Zoranzoki21 to Kizule on Phabricator - https://phabricator.wikimedia.org/T260645 (10Zoranzoki21) [05:02:19] 10Phabricator, 10DBA, 10Patch-For-Review: Upgrade m3 to Buster and MariaDB 10.4 - https://phabricator.wikimedia.org/T259589 (10Marostegui) Test read-only off [05:02:41] 10Phabricator, 10DBA: Upgrade m3 to Buster and MariaDB 10.4 - https://phabricator.wikimedia.org/T259589 (10Marostegui) [05:02:53] 10Phabricator, 10DBA, 10Operations: Upgrade m3 to Buster and MariaDB 10.4 - https://phabricator.wikimedia.org/T259589 (10Marostegui) [05:03:54] (03PS1) 10Legoktm: jjb: Add rust-coverage-publish job [integration/config] - 10https://gerrit.wikimedia.org/r/620817 (https://phabricator.wikimedia.org/T256827) [05:06:36] (03CR) 10Legoktm: [C: 03+2] jjb: Add rust-coverage-publish job [integration/config] - 10https://gerrit.wikimedia.org/r/620817 (https://phabricator.wikimedia.org/T256827) (owner: 10Legoktm) [05:06:45] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Drop MediaWiki testing in stretch and instead test only in buster - https://phabricator.wikimedia.org/T252432 (10greg) [05:07:36] (03Merged) 10jenkins-bot: jjb: Add rust-coverage-publish job [integration/config] - 10https://gerrit.wikimedia.org/r/620817 (https://phabricator.wikimedia.org/T256827) (owner: 10Legoktm) [05:08:03] !log reloading zuul for https://gerrit.wikimedia.org/r/620817 [05:08:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [05:09:38] 10Phabricator, 10DBA, 10Operations: Upgrade m3 to Buster and MariaDB 10.4 - https://phabricator.wikimedia.org/T259589 (10Marostegui) 05Open→03Resolved This was done successfully. m3 fully runs Buster and MariaDB 10.4 db1128 will be moved to m5, and that will be tracked at T260324 Thanks @mmodell for hel... [05:12:52] 10Phabricator: Custom task forms for #WMNO-General - https://phabricator.wikimedia.org/T258787 (10mmodell) Here is the onboarding form: https://phabricator.wikimedia.org/maniphest/task/edit/form/86/ [05:14:20] (03PS1) 10Legoktm: rust-coverage: Install cargo-tarpaulin as nobody rather than root [integration/config] - 10https://gerrit.wikimedia.org/r/620819 [05:14:22] (03CR) 10Legoktm: [C: 03+2] rust-coverage: Install cargo-tarpaulin as nobody rather than root [integration/config] - 10https://gerrit.wikimedia.org/r/620819 (owner: 10Legoktm) [05:15:37] 10Phabricator: Custom task forms for #WMNO-General - https://phabricator.wikimedia.org/T258787 (10mmodell) Please take a look and confirm that everything looks ok. [05:15:39] (03Merged) 10jenkins-bot: rust-coverage: Install cargo-tarpaulin as nobody rather than root [integration/config] - 10https://gerrit.wikimedia.org/r/620819 (owner: 10Legoktm) [05:15:59] !log rebuilding rust-coverage image https://gerrit.wikimedia.org/r/620819 [05:16:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [05:34:41] 10Gerrit, 10wikitech.wikimedia.org: Rename account Zoranzoki21 to Kizule on Gerrit - https://phabricator.wikimedia.org/T260647 (10Zoranzoki21) [05:35:24] 10Gerrit, 10wikitech.wikimedia.org: Rename account Zoranzoki21 to Kizule on Gerrit - https://phabricator.wikimedia.org/T260647 (10Zoranzoki21) [05:41:57] askdfhjdf [06:01:49] (03PS1) 10Legoktm: rust: Force all builds to be verbose [integration/config] - 10https://gerrit.wikimedia.org/r/620823 [06:01:51] (03CR) 10Legoktm: [C: 03+2] rust: Force all builds to be verbose [integration/config] - 10https://gerrit.wikimedia.org/r/620823 (owner: 10Legoktm) [06:01:53] (03PS1) 10Legoktm: rust-coverage: Use older cargo-tarpaulin to avoid bug with time [integration/config] - 10https://gerrit.wikimedia.org/r/620824 [06:01:55] (03CR) 10Legoktm: [C: 03+2] rust-coverage: Use older cargo-tarpaulin to avoid bug with time [integration/config] - 10https://gerrit.wikimedia.org/r/620824 (owner: 10Legoktm) [06:02:54] (03Merged) 10jenkins-bot: rust: Force all builds to be verbose [integration/config] - 10https://gerrit.wikimedia.org/r/620823 (owner: 10Legoktm) [06:02:58] (03Merged) 10jenkins-bot: rust-coverage: Use older cargo-tarpaulin to avoid bug with time [integration/config] - 10https://gerrit.wikimedia.org/r/620824 (owner: 10Legoktm) [06:03:39] !log rebuilding rust and rust-coverage images (https://gerrit.wikimedia.org/r/620823, https://gerrit.wikimedia.org/r/620824) [06:03:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:17:32] https://doc.wikimedia.org/cover/labs-tools-shorturls/ ahaha wonderful, (correctly) blocked by CSP [06:37:05] 10Phabricator: Custom task forms for #WMNO-General - https://phabricator.wikimedia.org/T258787 (10jhsoby-WMNO) 05Open→03Resolved Looks great, thank you very much @mmodell! [06:37:48] 10Continuous-Integration-Config, 10Patch-For-Review: Support Rust repositories in CI - https://phabricator.wikimedia.org/T256827 (10Legoktm) coverage report is now generated: https://doc.wikimedia.org/cover/labs-tools-shorturls/ ! except...the HTML view loads external JS from unpkg.com - I've filed https://git... [06:38:49] (03PS1) 10Legoktm: jjb: Fix rust-coverage-publish job, actually clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/620869 [06:38:51] (03CR) 10Legoktm: [C: 03+2] jjb: Fix rust-coverage-publish job, actually clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/620869 (owner: 10Legoktm) [06:38:53] (03PS1) 10Legoktm: jjb: Bump rust images [integration/config] - 10https://gerrit.wikimedia.org/r/620870 [06:38:55] (03CR) 10Legoktm: [C: 03+2] jjb: Bump rust images [integration/config] - 10https://gerrit.wikimedia.org/r/620870 (owner: 10Legoktm) [06:39:58] (03Merged) 10jenkins-bot: jjb: Fix rust-coverage-publish job, actually clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/620869 (owner: 10Legoktm) [06:40:11] (03Merged) 10jenkins-bot: jjb: Bump rust images [integration/config] - 10https://gerrit.wikimedia.org/r/620870 (owner: 10Legoktm) [07:01:52] 10Phabricator, 10DBA, 10Operations: replace phabricator db passwords with longer passwords - https://phabricator.wikimedia.org/T250361 (10jcrespo) [07:20:24] 10Phabricator, 10DBA, 10Operations: replace phabricator db passwords with longer passwords - https://phabricator.wikimedia.org/T250361 (10jcrespo) [07:30:44] 10Release-Engineering-Team-TODO, 10MediaWiki-Docker, 10Developer Productivity, 10User-Addshore: Create a core maintenance script that outputs if composer needs to be run, and always exits with code 0 - https://phabricator.wikimedia.org/T260656 (10Addshore) [07:31:44] 10Release-Engineering-Team-TODO, 10MediaWiki-Docker, 10Developer Productivity, 10Patch-For-Review, 10User-Addshore: Port mediawiki-docker-dev "mwdd" v1 cli in to go - https://phabricator.wikimedia.org/T253313 (10Addshore) [07:47:28] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10DBA, and 3 others: Improve privilege separation for phabricator's config files and mysql credentials - https://phabricator.wikimedia.org/T146055 (10jcrespo) 05Stalled→03Open [07:47:33] 10Phabricator, 10Security-Team, 10Striker, 10cloud-services-team (Kanban): Unable to mirror repository from git.legoktm.com into diffusion - https://phabricator.wikimedia.org/T143969 (10jcrespo) [07:56:05] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10DBA, and 4 others: Improve privilege separation for phabricator's config files and mysql credentials - https://phabricator.wikimedia.org/T146055 (10jcrespo) This is done: https://gerrit.wikimedia.org/r/c/op... [08:30:40] (03CR) 10Hashar: [C: 03+1] "PS4 makes it straightforward :]" (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/620771 (owner: 10Ahmon Dancy) [08:34:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review: releng/bazel CI image fails to build - https://phabricator.wikimedia.org/T260608 (10hashar) Thank you @legoktm ! [08:41:13] !log Restarted releases-jenkins for 'Email Extension' plugin update [08:41:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:47:21] 10Phabricator, 10DBA, 10Operations, 10Patch-For-Review: replace phabricator db passwords with longer passwords - https://phabricator.wikimedia.org/T250361 (10jcrespo) 05Open→03Resolved a:03jcrespo [09:05:33] !log Restarting CI Jenkins [09:05:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:12:42] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap: scap on beta fails canary check: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T260667 (10hashar) [09:26:39] !log gerrit: made operations-software-wmfmariadbpy self owned, added kormat to the group. # T260342 [09:26:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:26:41] T260342: Request for Gerrit Managers permissions - https://phabricator.wikimedia.org/T260342 [10:00:51] 10Release-Engineering-Team-TODO, 10MediaWiki-Docker, 10MediaWiki-Maintenance-system, 10Developer Productivity, 10User-Addshore: Create a core maintenance script that outputs if composer needs to be run, and always exits with code 0 - https://phabricator.wikimedia.org/T260656 (10Aklapper) [10:18:21] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Configure git user.email and user.name for the debian user on the deploy server in tr... - https://phabricator.wikimedia.org/T260671 [10:20:14] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add Gerrit to train-dev - https://phabricator.wikimedia.org/T260672 (10LarsWirzenius) [10:21:52] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Try "deploy-promote testwikis" in train-dev, document what seems to be missing - https://phabricator.wikimedia.org/T260376 (10La... [10:23:19] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add Gerrit to train-dev - https://phabricator.wikimedia.org/T260672 (10LarsWirzenius) [10:26:03] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add app servers to train-dev - https://phabricator.wikimedia.org/T260674 (10LarsWirzenius) [10:29:35] (03CR) 10Lars Wirzenius: [C: 03+2] "Well done." [tools/scap] - 10https://gerrit.wikimedia.org/r/620771 (owner: 10Ahmon Dancy) [10:32:12] (03Merged) 10jenkins-bot: Clearer error message if mwscript is not found [tools/scap] - 10https://gerrit.wikimedia.org/r/620771 (owner: 10Ahmon Dancy) [10:43:33] 10Phabricator: Rename account Zoranzoki21 to Kizule on Phabricator - https://phabricator.wikimedia.org/T260645 (10Zoranzoki21) [10:44:04] 10Gerrit, 10wikitech.wikimedia.org: Rename account Zoranzoki21 to Kizule on Gerrit - https://phabricator.wikimedia.org/T260647 (10Zoranzoki21) 05Open→03Stalled Stalled until 18th September when account should be renamed on wikis. [10:53:30] 10Phabricator: Custom task forms for #WMNO-General - https://phabricator.wikimedia.org/T258787 (10Aklapper) >>! In T258787#6356551, @mmodell wrote: > @aklapper: yes, duplicate form is the way to go. @mmodell: Thanks! [Documented](https://www.mediawiki.org/w/index.php?title=Phabricator/Help/Forms&type=revision&d... [11:28:21] (03PS1) 10Hashar: jjb: update jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620913 (https://phabricator.wikimedia.org/T256844) [11:31:32] (03PS1) 10Hashar: jjb: update quibble jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620914 (https://phabricator.wikimedia.org/T256844) [11:31:35] what a mess [11:38:59] (03CR) 10Hashar: [C: 03+2] jjb: update jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620913 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [11:39:03] (03CR) 10Hashar: [C: 03+2] jjb: update quibble jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620914 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [11:39:55] !log Updating all CI jobs based on ci-stretch for git protocol v2 https://gerrit.wikimedia.org/r/c/integration/config/+/620913/ | https://gerrit.wikimedia.org/r/c/integration/config/+/620914/ [11:39:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:40:04] (03Merged) 10jenkins-bot: jjb: update jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620913 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [11:40:07] (03Merged) 10jenkins-bot: jjb: update quibble jobs for git v2 / drop stretch-backports [integration/config] - 10https://gerrit.wikimedia.org/r/620914 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [11:42:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review, 10git-protocol-v2: Update Stretch CI containers to use a git that supports protocol v... - https://phabricator.wikimedia.org/T256844 [12:56:50] 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10MediaWiki Train Development Environment: Only builds a libvirt image, not a VirtualBox image - https://phabricator.wikimedia.org/T259612 (10LarsWirzenius) a:05LarsWirzenius→03None I can convert the train-dev image with qemu-img: `qemu-img... [13:02:49] (03PS1) 10Bearloga: Edit Repo Config [wikimedia/discovery/wmf] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/620685 [13:04:00] (03Abandoned) 10Bearloga: Edit Repo Config [wikimedia/discovery/wmf] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/620685 (owner: 10Bearloga) [13:29:38] Looks like the quibble CI jobs (and phan) have all begun insta-failing. Related to T256844, I suspect. [13:29:39] T256844: Update Stretch CI containers to use a git that supports protocol v2 - https://phabricator.wikimedia.org/T256844 [13:29:50] ` TypeError: PackingType of packed-Refs not understood: '# pack-refs with: peeled fully-peeled sorted'` [13:30:11] https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/30272/console [13:37:38] o/ here to also talk about broken CI for Wikibas e:P [13:58:50] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add app servers to train-dev - https://phabricator.wikimedia.org/T260674 (10LarsWirzenius) a:05LarsWirzenius→03None [13:59:07] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add Gerrit to train-dev - https://phabricator.wikimedia.org/T260672 (10LarsWirzenius) a:05LarsWirzenius→03None [13:59:56] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Configure git user.email and user.name for the debian user on the deploy server in tr... - https://phabricator.wikimedia.org/T260671 [14:00:13] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: mirror-repos.sh snapshots created at different times will have different train releas... - https://phabricator.wikimedia.org/T260593 [14:00:48] liw: you could do that when you create the sub task :) [14:02:10] yes, if I realised it at the time [14:12:26] hello. [14:12:26] any idea what might be happening with jenkins failing builds? [14:12:26] https://integration.wikimedia.org/ci/job/quibble-vendor-selenium-docker/28914/console [14:12:26] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/620919 [14:12:27] this is not my patch, I'm just trying to help a developer of my team [14:16:55] (03PS1) 10Jforrester: Revert "jjb: update quibble jobs for git v2 / drop stretch-backports" [integration/config] - 10https://gerrit.wikimedia.org/r/620966 [14:17:46] dvpita: ^ [14:18:04] awesome! [14:18:06] thanks! [14:18:31] Will leave to hashar . [14:18:34] yeah [14:19:49] (03CR) 10Hashar: [C: 03+2] Revert "jjb: update quibble jobs for git v2 / drop stretch-backports" [integration/config] - 10https://gerrit.wikimedia.org/r/620966 (owner: 10Jforrester) [14:20:17] (03CR) 10Hashar: [C: 03+2] "Rolled back jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/620966 (owner: 10Jforrester) [14:20:25] dvpita: that is rolling back [14:20:28] James_F: thanks for the link [14:20:35] Any time. [14:21:00] (03Merged) 10jenkins-bot: Revert "jjb: update quibble jobs for git v2 / drop stretch-backports" [integration/config] - 10https://gerrit.wikimedia.org/r/620966 (owner: 10Jforrester) [14:21:26] guess I should rollback the other as well [14:23:04] oh [14:23:13] that would be GitPython not supporting protocol v2 maybe [14:24:00] which might be https://github.com/gitpython-developers/GitPython/issues/687 [14:24:14] Oops. [14:25:49] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10git-protocol-v2: Update Stretch CI containers to use a git that supports protocol v2 - https://phabricator.wikimedia.org/T256844 (10hashar) 05Reso... [14:34:01] ;-\ [14:46:00] (03PS1) 10Hashar: dockerfile: stop using python3-git for Quibble Stretch [integration/config] - 10https://gerrit.wikimedia.org/r/620990 (https://phabricator.wikimedia.org/T256844) [14:46:29] dvpita: so that should work again now [14:47:23] !log Refreshing all JJB jobs to ensure rollback change https://gerrit.wikimedia.org/r/620966 is properly applied [14:47:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:47:46] looks like it https://usercontent.irccloud-cdn.com/file/C0UpyO7a/image.png [14:54:00] (03CR) 10Hashar: "I guess I should first build it locally and give it a try." [integration/config] - 10https://gerrit.wikimedia.org/r/620990 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [14:54:37] dvpita: so yeah in short, I have upgraded git in the CI image and Quibble the test runners used an obsolete git python module which does not work with the newer git command :\ [14:55:53] hashar: you will update the module or leave it? [14:56:06] update it [14:56:23] the one we used comes from Debian Stretch (python3-git) and is obsolete: version 2.1.1 [14:56:46] so I have send a patch to no more use that debian package and instead install from the python package management system (pypi) [14:58:45] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Try "deploy-promote testwikis" in train-dev, document what seems to be missing - https://phabricator.wikimedia.org/T260376 (10da... [15:43:04] 10Release-Engineering-Team, 10MediaWiki-Vagrant, 10Parsoid, 10RESTBase: Support RESTBase using Parsoid/PHP in MediaWiki-Vagrant - https://phabricator.wikimedia.org/T259989 (10Tgr) [16:35:00] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: For the Fresnel job, distinguish system failure from assert failure - https://phabricator.wikimedia.org/T216574 (10hashar) To rephrase, a Jenkins build status can take at least three different states. It is determined by the outcome of running b... [16:38:43] (03CR) 10Hashar: [C: 03+2] "Built locally and that gives us GitPython 2.1.15. I ran Quibble and it managed to clone the repositories." [integration/config] - 10https://gerrit.wikimedia.org/r/620990 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [16:39:49] (03Merged) 10jenkins-bot: dockerfile: stop using python3-git for Quibble Stretch [integration/config] - 10https://gerrit.wikimedia.org/r/620990 (https://phabricator.wikimedia.org/T256844) (owner: 10Hashar) [16:47:30] !log Rebuilding CI containers for Quibble to update GitPython T256844#6393265 | https://gerrit.wikimedia.org/r/c/integration/config/+/620990/ [16:47:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:47:33] T256844: Update Stretch CI containers to use a git that supports protocol v2 - https://phabricator.wikimedia.org/T256844 [16:47:49] BAH [16:47:55] and of course the docker registry now fails :\ [17:11:27] (03PS2) 10Hashar: fab: tail docker-pkg-build log [integration/config] - 10https://gerrit.wikimedia.org/r/620721 [17:12:38] (03CR) 10Hashar: fab: tail docker-pkg-build log (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/620721 (owner: 10Hashar) [18:20:58] 10Phabricator, 10WMF-NDA: Issue with WMF-NDA page - https://phabricator.wikimedia.org/T260715 (10JEumerus) [18:23:20] 10Phabricator, 10WMF-NDA: Issue with WMF-NDA page - https://phabricator.wikimedia.org/T260715 (10Dzahn) I made this small change: https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-PROJ-3vdjusjgztrpuo5/ Does that fix it? [18:32:13] 10Phabricator, 10WMF-NDA: Issue with WMF-NDA page - https://phabricator.wikimedia.org/T260715 (10JEumerus) Yes. [18:39:24] 10Phabricator, 10WMF-NDA: Issue with WMF-NDA page - https://phabricator.wikimedia.org/T260715 (10Dzahn) 05Open→03Resolved a:03Dzahn cool, well then i will claim it's resolved [18:40:11] 10Phabricator, 10WMF-NDA: #WMF-NDA project page has a broken sentence - https://phabricator.wikimedia.org/T260715 (10Aklapper) [18:48:27] 10Release-Engineering-Team, 10MediaWiki-Vagrant, 10Parsoid, 10WMDE-Templates-FocusArea, 10Patch-For-Review: Support Parsoid/PHP in MediaWiki-Vagrant - https://phabricator.wikimedia.org/T258940 (10Tgr) I'm not sure what if statement that would be. Parsoid/PHP gets automatically installed as part of MediaW... [18:54:24] 10Release-Engineering-Team, 10MediaWiki-Vagrant, 10Parsoid, 10WMDE-Templates-FocusArea, 10Patch-For-Review: Support Parsoid/PHP in MediaWiki-Vagrant - https://phabricator.wikimedia.org/T258940 (10Dzahn) Do you mean this in CommonSettings.php? ` // This is a temporary hack for hooking up Parsoid/PHP wit... [19:46:15] 10Continuous-Integration-Config, 10Release-Engineering-Team (Onboarding), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review: Clear out releases jenkins legacy jobs and move remaining ones to JJB - https://phabricator.wikimedia.org/T256164 (10hashar) [19:47:00] 10Continuous-Integration-Config, 10Release-Engineering-Team (Onboarding), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review: Clear out releases jenkins legacy jobs and move remaining ones to JJB - https://phabricator.wikimedia.org/T256164 (10hashar) I have removed the old... [19:56:13] dpifke (or anyone else): Do you mind if the deployment-mdb01 VM is down for a few minutes? I'd like to move it to different hardware. [19:56:45] No problem. [19:58:16] thanks — should only be 10-15 [20:03:09] done [20:03:20] looks like deployment-prep puppetdb is broken again :( [20:25:50] PROBLEM - jenkins_service_running on releases1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args .*/bin/java .*-jar /usr/share/jenkins/jenkins.war https://wikitech.wikimedia.org/wiki/Jenkins [20:27:15] ACKNOWLEDGEMENT - jenkins_service_running on releases1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args .*/bin/java .*-jar /usr/share/jenkins/jenkins.war daniel_zahn migration in progress https://wikitech.wikimedia.org/wiki/Jenkins [20:30:47] (03PS1) 10Southparkfan: Edit Repo Config [debs/wikistats] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/620974 [20:54:07] !log Restarted puppetdb on deployment-puppetdb03 [20:54:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:55:35] So there's a service file for puppetdb here, which specifies automatic restarts (`Restart=Always`): https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/puppetdb/templates/puppetdb.service.erb [20:55:46] But it's not being used on deployment-puppetdb03. [20:56:14] It doesn't seem referenced at all from https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/puppetdb/manifests/app.pp [21:16:39] 10Beta-Cluster-Infrastructure, 10Puppet: puppetdb on deployment-puppetdb03 keeps getting OOMKilled - https://phabricator.wikimedia.org/T248041 (10dpifke) Restarted again: https://sal.toolforge.org/log/aexZA3QBj_Bg1xd3wLvx (I'll reference this task next time, now that I know it exists.) Adding `Restart=always`... [21:29:56] (03PS1) 10Awight: Add Travis CI test configuration [integration/quibble] - 10https://gerrit.wikimedia.org/r/621071 [21:32:04] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [extensions/RelMicroformat] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/621072 [21:32:06] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [extensions/RelMicroformat] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/621072 (owner: 10QChris) [21:32:12] (03PS1) 10QChris: Import done. Revoke import grants [extensions/RelMicroformat] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/621073 [21:32:14] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [extensions/RelMicroformat] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/621073 (owner: 10QChris) [21:34:19] (03CR) 10Awight: "Runs successfully: https://travis-ci.org/github/adamwight/quibble/builds/719095889" [integration/quibble] - 10https://gerrit.wikimedia.org/r/621071 (owner: 10Awight) [21:35:25] (03PS1) 10Awight: Downgrade some error logging [integration/quibble] - 10https://gerrit.wikimedia.org/r/621074 [21:59:32] 10Release-Engineering-Team-TODO: Generate time.releng.team and people.json from one source of truth - https://phabricator.wikimedia.org/T250691 (10thcipriani) [22:19:24] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops: decom releases1001 and releases2001 - https://phabricator.wikimedia.org/T260742 (10Dzahn) [22:21:32] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: replace backends for releases.wikimedia.org with buster VMs - https://phabricator.wikimedia.org/T247652 (10Dzahn) both https://releases.wikimedia.org and https://releases-jenkins.wikimedia.org have been switched to the... [22:25:04] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: replace backends for releases.wikimedia.org with buster VMs - https://phabricator.wikimedia.org/T247652 (10Dzahn) 05Open→03Resolved [23:37:38] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)): Sketch MediaWiki production image composition and workflows - https://phabricator.wikimedia.org/T259817 (10dduvall) @jeena and I met today to review the following rough sketch. It's a work in progress, but t... [23:41:20] 10Continuous-Integration-Config, 10Research, 10Patch-For-Review: Enable CI on research/landing-page repo - https://phabricator.wikimedia.org/T257843 (10leila) @bmansurov can you look into this task and let me know what you recommend? (feel free to assign it to yourself.)