[01:24:50] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts: ` phab1001.eqiad.wmnet ` The log can be found in... [01:48:36] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) [01:56:50] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['phab1001.eqiad.wmnet'] ` Of which those **FAILED**: ` ['phab1001.eqiad.wmnet'] ` [03:15:02] 10Continuous-Integration-Config, 10Google-Code-in-2019: Add yourself to the Jenkins whitelist in Gerrit to trigger testing unit test failures or code style issues yourself - https://phabricator.wikimedia.org/T235286 (10Pppery) Do the `two merged patches` have to be from this year, or do merged patches from pas... [08:30:38] (03CR) 10Awight: "I like it!" (034 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/554571 (https://phabricator.wikimedia.org/T236680) (owner: 10Clarakosi) [09:13:47] (03CR) 10Daniel Kinzler: "> Patch Set 2:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/554571 (https://phabricator.wikimedia.org/T236680) (owner: 10Clarakosi) [10:08:29] (03CR) 10Hashar: [C: 03+2] "Jobs updated :)" [integration/config] - 10https://gerrit.wikimedia.org/r/554649 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [10:09:21] (03Merged) 10jenkins-bot: jjb: use a single label for debian packaging jobs [integration/config] - 10https://gerrit.wikimedia.org/r/554649 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [10:09:30] (03CR) 10Hashar: [C: 03+2] "That works for the few repositories / changes I have tested :]" [integration/config] - 10https://gerrit.wikimedia.org/r/554588 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [10:10:24] (03Merged) 10jenkins-bot: Stop using zuul-cloner in debian-glue jobs [integration/config] - 10https://gerrit.wikimedia.org/r/554588 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [10:10:34] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) p:05Triage→03Normal [10:10:47] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) [10:12:39] 10Beta-Cluster-Infrastructure, 10Wikidata, 10User-Addshore, 10Wikidata-Campsite (Wikidata-Campsite-Iteration-∞): beta wikidata trying to dispatch to wikis that don't exist - https://phabricator.wikimedia.org/T237551 (10Addshore) 05Open→03Resolved a:03Addshore Going to close this as it is resolved. [10:14:26] !log Deleted integration-slave-jessie-1001 , created integration-agent-pkgbuilder-1002 # TT224943 [10:14:26] 10Beta-Cluster-Infrastructure, 10Wikidata, 10User-Addshore, 10User-Ladsgroup, 10Wikidata-Campsite (Wikidata-Campsite-Iteration-∞): beta wikidata trying to dispatch to wikis that don't exist - https://phabricator.wikimedia.org/T237551 (10Ladsgroup) a:05Addshore→03Ladsgroup :D [10:14:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:14:30] !log Deleted integration-slave-jessie-1001 , created integration-agent-pkgbuilder-1002 # T224943 [10:14:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:14:33] T224943: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 [10:15:10] PROBLEM - Host integration-slave-jessie-1001 is DOWN: CRITICAL - Host Unreachable (172.16.0.86) [10:25:59] 10Beta-Cluster-Infrastructure, 10Wikidata, 10User-Addshore, 10User-Ladsgroup, 10Wikidata-Campsite (Wikidata-Campsite-Iteration-∞): beta wikidata trying to dispatch to wikis that don't exist - https://phabricator.wikimedia.org/T237551 (10Addshore) It looks like although this issue is solved, this has expo... [10:36:48] PROBLEM - Host integration-agent-pkgbuilder-1002 is DOWN: CRITICAL - Host Unreachable (172.16.0.121) [10:37:18] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Wikidata, and 3 others: Update Doxygen in CI to 1.8.15 or greater - https://phabricator.wikimedia.org/T239482 (10Addshore) a:03hashar [10:50:37] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) 05Open→03Resolved The jobs ar... [10:50:44] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Wikidata, and 3 others: Update Doxygen in CI to 1.8.15 or greater - https://phabricator.wikimedia.org/T239482 (10hashar) [10:51:49] RECOVERY - Host integration-agent-pkgbuilder-1002 is UP: PING OK - Packet loss = 0%, RTA = 0.74 ms [10:53:57] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) There are some oddities. For exam... [10:54:07] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) 05Resolved→03Open [10:54:10] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Wikidata, and 3 others: Update Doxygen in CI to 1.8.15 or greater - https://phabricator.wikimedia.org/T239482 (10hashar) [10:56:00] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review: Investigate gerrit session expiration - https://phabricator.wikimedia.org/T222472 (10LarsWirzenius) Occasional update: I last logged in on November 15, and am still logged in. [11:38:13] 10Release-Engineering-Team-TODO (201912): Write command line API client for Ick controller, in Go - https://phabricator.wikimedia.org/T239899 (10LarsWirzenius) [13:16:22] (03CR) 10Daimona Eaytoy: [C: 03+1] Factor out duplicated code from phan entrypoint scripts [integration/config] - 10https://gerrit.wikimedia.org/r/553440 (owner: 10Tim Starling) [13:20:35] (03PS1) 10Hashar: jjb: fix lintian report in debian-glue [integration/config] - 10https://gerrit.wikimedia.org/r/554850 (https://phabricator.wikimedia.org/T224943) [13:20:58] (03CR) 10Hashar: [C: 03+2] jjb: fix lintian report in debian-glue [integration/config] - 10https://gerrit.wikimedia.org/r/554850 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [13:22:13] (03Merged) 10jenkins-bot: jjb: fix lintian report in debian-glue [integration/config] - 10https://gerrit.wikimedia.org/r/554850 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [13:31:58] https://integration.wikimedia.org/ci/job/debian-glue-backports/9/console - "18:49:20 ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored." - any idea what's wrong with Jenkins? [13:39:54] (03PS1) 10Hashar: labs/toollabs drop Trusty debian packaging [integration/config] - 10https://gerrit.wikimedia.org/r/554857 [13:40:48] (03CR) 10jerkins-bot: [V: 04-1] labs/toollabs drop Trusty debian packaging [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (owner: 10Hashar) [13:44:50] (03PS1) 10Hashar: labs/toollabs also gate on jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 [13:45:15] (03CR) 10Hashar: [C: 03+2] labs/toollabs also gate on jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [13:47:25] (03PS1) 10Hashar: labs/toollabs: run Debian package for Buster [integration/config] - 10https://gerrit.wikimedia.org/r/554859 [14:07:38] 10Release-Engineering-Team, 10cloud-services-team (Kanban): labs/toollabs testsuite fails when building the Debian package - https://phabricator.wikimedia.org/T239910 (10hashar) [14:07:57] 10Release-Engineering-Team, 10cloud-services-team (Kanban): labs/toollabs testsuite fails when building the Debian package - https://phabricator.wikimedia.org/T239910 (10hashar) [14:07:59] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) [14:16:25] 10Release-Engineering-Team, 10cloud-services-team (Kanban): labs/toollabs testsuite fails when building the Debian package - https://phabricator.wikimedia.org/T239910 (10hashar) I have switched the CI hosts from Jessie to Buster (T224943) thus we have `pbuilder` updated from `0.215` to `0.230.4`. In the change... [14:21:27] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, 10Readers-Web-Backlog: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdlrobson) I dont mind copying local files or curl, however this is likely g... [14:38:14] ssh: connect to host git-ssh.wikimedia.org port 22: Connection refused [14:45:15] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] Enable code health reports for ext-Cite (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/554391 (owner: 10Awight) [14:57:20] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10herron) >>! In T233134#5713956, @Krinkle wrote: >>>! @colewhite wrote at 10Release-Engineering-Team-TODO (201912): Write howto for signing keys for keysigning party - https://phabricator.wikimedia.org/T239829 (10LarsWirzenius) I've updated the instructions at https://gerrit.wikimedia.org/r/plugins/gitiles/pgp-public-keys/+/refs/heads/master/process.mdwn and will be sending them out m... [15:00:52] 10Release-Engineering-Team-TODO (201912): Arrange intra-team PGP keysigning at Atlanta offsite - https://phabricator.wikimedia.org/T232990 (10LarsWirzenius) [15:00:54] 10Release-Engineering-Team-TODO (201912): Write howto for signing keys for keysigning party - https://phabricator.wikimedia.org/T239829 (10LarsWirzenius) 05Open→03Resolved [15:22:53] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) [15:22:55] 10Release-Engineering-Team, 10cloud-services-team (Kanban): labs/toollabs testsuite fails when building the Debian package - https://phabricator.wikimedia.org/T239910 (10hashar) [16:31:18] 10Release-Engineering-Team-TODO: jerkins-bot should not post on IRC for Gerrit changes marked 'WIP' - https://phabricator.wikimedia.org/T239928 (10CDanis) [16:32:34] 10Release-Engineering-Team-TODO: jerkins-bot should not post on IRC for Gerrit changes marked 'WIP' - https://phabricator.wikimedia.org/T239928 (10Reedy) jerkins is out to cause people misery [16:41:54] Reedy lol [16:48:20] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins: jerkins-bot should not post on IRC for Gerrit changes marked 'WIP' - https://phabricator.wikimedia.org/T239928 (10greg) p:05Triage→03Low [16:58:57] 10Release-Engineering-Team-TODO (201912), 10User-greg, 10Wikimedia-extension-review-queue: Investigate and make improvements to the extension review process - https://phabricator.wikimedia.org/T195244 (10greg) >>! In T195244#5713614, @Mholloway wrote: >>>! In T195244#4413456, @bd808 wrote: >> Flipping this r... [17:11:02] (03CR) 10Bstorm: [C: 03+1] labs/toollabs drop Trusty debian packaging [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (owner: 10Hashar) [17:11:27] (03CR) 10Bstorm: [C: 03+1] "We have no trusty anymore in cloud :)" [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (owner: 10Hashar) [17:13:35] (03CR) 10Bstorm: [C: 03+1] "Adding CC for some of my team members because we will probably need to mind this when updating the changelog in some debian packages we ma" [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (owner: 10Hashar) [17:14:17] (03CR) 10Bstorm: [C: 03+1] "We are in the process of deprecating jessie, but we still use it, so I think this would be good." [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [17:19:06] if nyone knows (and it's not clear to me from the deployment calendar) are we on deployment hiatus from next week through jan 3 or is it just that no one has updated the deployment calendar yet? [17:21:04] I thought we had a couple of weeks of deploys... [17:22:15] apergos: below... [17:22:15] Upcoming events [17:22:15] December 23-January 3rd - No deploys, Holiday break [17:22:24] Dec 23 [17:22:36] it doesn't say anything about next week but there's nothing in the calendar either, which seems odd [17:22:41] So I think it's all good until then, so someone just hasn't updated the calendar yet [17:22:53] ah so the freeze starts dec 23 then [17:22:58] thanks, that's what i wanted to know [17:25:52] (03CR) 10Jforrester: Enable code health reports for ext-Cite (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/554391 (owner: 10Awight) [17:27:00] RECOVERY - Host deployment-logstash2 is UP: PING OK - Packet loss = 0%, RTA = 0.50 ms [17:33:22] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10herron) Looking more closely the problem was due to a `Broker: Leader not available` issue in... [17:34:42] (03PS1) 10Majavah: Add Majavah to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/554918 [17:35:41] (03CR) 10Zoranzoki21: [C: 03+1] "Looks good, LGTM!" [integration/config] - 10https://gerrit.wikimedia.org/r/554918 (owner: 10Majavah) [17:45:25] apergos: what reedy said [17:45:38] good to know! thanks again [17:47:53] PROBLEM - Parsoid on deployment-parsoid09 is CRITICAL: connect to address 172.16.5.63 and port 8000: Connection refused [17:47:53] PROBLEM - Parsoid on deployment-mediawiki-parsoid10 is CRITICAL: connect to address 172.16.0.141 and port 8000: Connection refused [17:53:11] (03CR) 10Urbanecm: [C: 03+1] "LGTM" [integration/config] - 10https://gerrit.wikimedia.org/r/554918 (owner: 10Majavah) [18:02:13] (03CR) 10Jforrester: "> Patch Set 2:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/554571 (https://phabricator.wikimedia.org/T236680) (owner: 10Clarakosi) [18:03:05] * thcipriani takes queue to update calendar [18:23:05] (03CR) 10Jforrester: [C: 03+2] "Welcome!" [integration/config] - 10https://gerrit.wikimedia.org/r/554918 (owner: 10Majavah) [18:23:55] (03Merged) 10jenkins-bot: Add Majavah to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/554918 (owner: 10Majavah) [18:29:33] 10Gerrit, 10Operations, 10vm-requests: Gerrit VM to test data migration - https://phabricator.wikimedia.org/T239151 (10Dzahn) Just a thought.. If the 2 machines for contint are granted (T239880) first and are taken from available spare pool.. then possibly one of them could work as temp Gerrit test machine a... [18:31:45] 10Gerrit, 10Operations, 10vm-requests: Gerrit VM to test data migration - https://phabricator.wikimedia.org/T239151 (10Dzahn) a:05Dzahn→03None I'm unassigning from me for now because i'll be on a vacation and don't want this to be blocked on me for no reason. If anyone else can push this forward meanwhil... [18:36:16] !log Zuul: Add Majavah to CI whitelist [18:36:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:47:24] 10Gerrit, 10Operations, 10vm-requests: Gerrit VM to test data migration - https://phabricator.wikimedia.org/T239151 (10thcipriani) >>! In T239151#5716392, @Dzahn wrote: > Just a thought.. If the 2 machines for contint are granted (T239880) first and are taken from available spare pool.. then possibly one of... [18:54:32] (03PS2) 10Jforrester: jjb/layout: [labs/toollabs] Drop Ubuntu Trusty debian packaging jobs [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [18:59:08] (03CR) 10Jforrester: [C: 03+2] jjb/layout: [labs/toollabs] Drop Ubuntu Trusty debian packaging jobs [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [18:59:59] (03Merged) 10jenkins-bot: jjb/layout: [labs/toollabs] Drop Ubuntu Trusty debian packaging jobs [integration/config] - 10https://gerrit.wikimedia.org/r/554857 (https://phabricator.wikimedia.org/T224943) (owner: 10Hashar) [19:00:46] (03PS2) 10Jforrester: labs/toollabs also gate on jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [19:01:06] (03PS3) 10Jforrester: layout: [labs/toollabs] Gate, don't just test, on Ubuntu Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [19:01:13] (03CR) 10Jforrester: [C: 03+2] layout: [labs/toollabs] Gate, don't just test, on Ubuntu Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [19:02:04] (03Merged) 10jenkins-bot: layout: [labs/toollabs] Gate, don't just test, on Ubuntu Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/554858 (owner: 10Hashar) [19:15:20] !log Zuul: [labs/toollabs] Drop trusty, add jessie to gate as well as test T224943 [19:15:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:15:23] T224943: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 [19:22:02] !log killed stuck beta CI jobs, stuff is flowing again [19:22:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:39:24] 10Phabricator, 10Developer-Advocacy (Oct-Dec 2019): Decrease issues with assignee field set for years without progress (aka cookie licking) - https://phabricator.wikimedia.org/T228575 (10Dzahn) >>! In T228575#5714168, @Aklapper wrote: >>>! In T228575#5714093, @Dzahn wrote: >> Don't they have to be nagged to ..... [20:04:24] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10Krenair) What exactly is needed? Do we need to make a second logstash stretch host? Do we nee... [20:34:07] (03PS1) 10Jforrester: jjb: Provide operations-mw-config-php72-composer-diffConfig-docker [integration/config] - 10https://gerrit.wikimedia.org/r/554941 (https://phabricator.wikimedia.org/T223602) [20:44:52] (03PS2) 10Hashar: labs/toollabs: run Debian package for Buster [integration/config] - 10https://gerrit.wikimedia.org/r/554859 [20:46:33] (03PS1) 10Hashar: Make Debian glue voting for operations/debs/doxygen [integration/config] - 10https://gerrit.wikimedia.org/r/554944 (https://phabricator.wikimedia.org/T239482) [20:48:25] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Patch-For-Review: Migrate debian-glue jobs to Buster instances - https://phabricator.wikimedia.org/T224943 (10hashar) labs/toollabs.git is broken but t... [20:48:55] (03CR) 10Hashar: [C: 03+2] Make Debian glue voting for operations/debs/doxygen [integration/config] - 10https://gerrit.wikimedia.org/r/554944 (https://phabricator.wikimedia.org/T239482) (owner: 10Hashar) [20:49:43] (03Merged) 10jenkins-bot: Make Debian glue voting for operations/debs/doxygen [integration/config] - 10https://gerrit.wikimedia.org/r/554944 (https://phabricator.wikimedia.org/T239482) (owner: 10Hashar) [20:52:11] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/#/c/554944/ [20:52:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:54:56] (03CR) 10Reedy: [V: 03+2 C: 03+2] Setup default permissions [software/netbox-extras] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/554598 (owner: 10Volans) [21:27:50] (03PS1) 10Jforrester: layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) [21:28:46] (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [21:28:52] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Wikidata, and 3 others: Update Doxygen in CI to 1.8.15 or greater - https://phabricator.wikimedia.org/T239482 (10hashar) I have a working backport for Doxygen 1.8.16 but that is... [21:36:32] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Wikidata, and 3 others: Update Doxygen in CI to 1.8.15 or greater - https://phabricator.wikimedia.org/T239482 (10Jdforrester-WMF) >>! In T239482#5716831, @hashar wrote: > I have... [21:36:40] (03PS2) 10Jforrester: layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) [21:36:44] (03CR) 10Jforrester: [C: 03+2] jjb: Provide operations-mw-config-php72-composer-diffConfig-docker [integration/config] - 10https://gerrit.wikimedia.org/r/554941 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [21:37:39] (03Merged) 10jenkins-bot: jjb: Provide operations-mw-config-php72-composer-diffConfig-docker [integration/config] - 10https://gerrit.wikimedia.org/r/554941 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [21:37:41] (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [21:39:32] (03PS3) 10Jforrester: layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) [21:41:33] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [21:42:22] (03Merged) 10jenkins-bot: layout: [mediawiki-operations] Provide a non-voting config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554951 (https://phabricator.wikimedia.org/T223602) (owner: 10Jforrester) [22:00:45] (03PS1) 10Jforrester: layout: [mediawiki-operations] Provide custom responses for the config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554959 [22:02:54] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki-operations] Provide custom responses for the config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554959 (owner: 10Jforrester) [22:03:52] (03Merged) 10jenkins-bot: layout: [mediawiki-operations] Provide custom responses for the config diff job [integration/config] - 10https://gerrit.wikimedia.org/r/554959 (owner: 10Jforrester) [22:09:25] 10Diffusion, 10Phabricator: Viewing MediaWiki repository in diffusion results in an Unhandled Exception ("CommandException") - https://phabricator.wikimedia.org/T239786 (10Dzahn) This has been fixed. Sorry for the inconvenience. https://phabricator.wikimedia.org/source/mediawiki/ is working for me. [22:09:37] 10Diffusion, 10Phabricator: Viewing MediaWiki repository in diffusion results in an Unhandled Exception ("CommandException") - https://phabricator.wikimedia.org/T239786 (10Dzahn) 05Open→03Resolved a:03Dzahn [22:12:28] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) >>! In T238956#5712739, @Mainframe98 wrote: > I suspect that this move inadvertently caused {T239786} as I only started seeing it today. Yes,... [22:13:21] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) [22:15:48] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) 05Open→03Resolved Summary: We switched from phab1003 to phab1001. Then we realized phab1001 had wrong BIOS settings (disks in legacy IDE... [22:15:59] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10mmodell) >>! In T238956#5712104, @Lucas_Werkmeister_WMDE wrote: > The recently added Gerrit integration directly beneath a tasks description seems to... [22:17:19] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10SRE-swift-storage, 10User-MModell: Connect Phabricator to swift for storage of git-lfs and file uploads. - https://phabricator.wikimedia.org/T182085 (10mmodell) 05Open→03Stalled [22:17:22] 10Release-Engineering-Team (Kanban), 10Scap, 10Patch-For-Review: Support git-lfs in scap - https://phabricator.wikimedia.org/T180627 (10mmodell) [22:21:44] 10Release-Engineering-Team, 10serviceops: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) [22:26:07] 10Release-Engineering-Team, 10serviceops: switch prod Phabricator from phab1003 to phab1001 - https://phabricator.wikimedia.org/T238956 (10Dzahn) [23:05:13] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (201912), 10cloud-services-team (Kanban): labs/toollabs testsuite fails when building the Debian package - https://phabricator.wikimedia.org/T239910 (10Jdforrester-WMF) [23:22:26] 10Release-Engineering-Team-TODO, 10Release Pipeline: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10Jdforrester-WMF) >>! In T239137#5696351, @Addshore wrote: > Not sure why that would be incompatible with the wikibase directory struct... [23:22:37] 10Release-Engineering-Team-TODO, 10Release Pipeline: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10Jdforrester-WMF) [23:30:18] brennen: assuming the train gets unblocked soon, when is the next window when I should worry about the wikitech upgrade? [23:36:47] andrewbogott: i think we're just past my personal cutoff for willingess to deploy today, at 16:36 local. monday at the earliest. [23:36:53] * brennen looks at deployment calendar [23:37:11] oh yeah, I figured that today was out of the question :) [23:38:23] i'm guessing 18:00 UTC monday at earliest. [23:38:57] sorry, make that 17:00 UTC. [23:40:07] ok — so when the train is delayed it just catches up whenever, it doesn't wait for the next official window? [23:40:36] It kinda depends what's happened [23:40:44] Usually mondays get used to catch up [23:40:46] andrewbogott: i'm still new at this so i could be overruled here, but my understanding is that if we're delayed over the weekend we _try_ to catch up monday [23:41:02] but like Reedy said, situation-dependent. [23:41:25] what they said ^ [23:41:38] ok! I made a note on my calendar to be alert for this on Monday — I may or may not be awake. [23:41:39] Thanks! [23:42:12] https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#What_happens_next? [23:42:14] you betcha. i still have a note to check logins on wikitech. [23:42:51] ah, good to see that 3pm pacific is called out as a specific cutoff there.