[00:02:13] I think noc.wm.o in general is expected to be "fairly" up to date. Assuming we have E-Tags/Last-Modified enabled there, and no vary headers and request coalescing is working as it should, could also set max-age for the domain to something as low as 1minute. [00:02:25] Then revalidation will keep it up to date without too much overhead. [00:02:40] Or even just 1minute without all that. Why not? [00:03:05] does noc.wm.o actually get used for stuff? [00:03:52] gets used by X-Wikimedia-Debug extension and /versions/ tool. And humans view it too, with a reasonable expectation that it isn't 16 hours out of date. [00:04:09] currently it seems to get cached indefinitely. [00:04:17] well, 4-7 days probably per varnish default [00:15:23] 10Continuous-Integration-Config, 10PhabTaskGraph, 10Patch-For-Review: PhabTaskGraph extension cannot find phutil_library - https://phabricator.wikimedia.org/T202515 (10CCicalese_WMF) The git submodule was a great suggestion. Implemented in patch. [00:32:59] PROBLEM - Puppet errors on deployment-certcentral-testclient02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [01:00:19] 10Continuous-Integration-Config, 10PhabTaskGraph, 10Patch-For-Review: PhabTaskGraph extension cannot find phutil_library - https://phabricator.wikimedia.org/T202515 (10CCicalese_WMF) Hmm, that's still not quite right. Although it runs fine with the submodule loaded, there is a new test failure (see below). A... [01:11:11] PROBLEM - Puppet errors on deployment-webperf11 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [01:12:23] PROBLEM - Puppet errors on deployment-memc06 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [01:13:53] PROBLEM - Puppet errors on deployment-maps04 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [01:30:21] PROBLEM - Puppet errors on deployment-poolcounter04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [01:33:46] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [01:33:54] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [01:34:22] PROBLEM - Puppet errors on deployment-deploy01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [01:35:00] PROBLEM - Puppet errors on deployment-deploy02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [01:46:10] RECOVERY - Puppet errors on deployment-webperf11 is OK: OK: Less than 1.00% above the threshold [0.0] [01:47:22] RECOVERY - Puppet errors on deployment-memc06 is OK: OK: Less than 1.00% above the threshold [0.0] [01:48:11] 10Release-Engineering-Team (Next), 10MediaWiki-General-or-Unknown, 10MW-1.29-release: Formalise and Announce REL1_29 EOL - https://phabricator.wikimedia.org/T197669 (10Legoktm) I think everyone is in agreement that there will be one more 1.29 release, which will contain some security fixes and a pile of bug... [01:53:54] RECOVERY - Puppet errors on deployment-maps04 is OK: OK: Less than 1.00% above the threshold [0.0] [02:05:22] RECOVERY - Puppet errors on deployment-poolcounter04 is OK: OK: Less than 1.00% above the threshold [0.0] [02:08:45] RECOVERY - Puppet errors on deployment-db03 is OK: OK: Less than 1.00% above the threshold [0.0] [02:09:57] RECOVERY - Puppet errors on deployment-deploy02 is OK: OK: Less than 1.00% above the threshold [0.0] [02:13:56] RECOVERY - Puppet errors on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [02:14:20] RECOVERY - Puppet errors on deployment-deploy01 is OK: OK: Less than 1.00% above the threshold [0.0] [02:35:07] 10Continuous-Integration-Config, 10MediaWiki-Releasing: php55lint is not running for REL1_30 mediawiki/core patches - https://phabricator.wikimedia.org/T203054 (10Legoktm) p:05Triage>03High [03:29:19] 10Release-Engineering-Team, 10MediaWiki-extensions-WikimediaIncubator, 10Epic, 10I18n: Make creating a new Language project easier - https://phabricator.wikimedia.org/T165585 (10C933103) 1. If a new site is to be created for each incubator site, how will WMF turn them into a full site once they become elig... [04:48:00] RECOVERY - Puppet errors on deployment-certcentral-testclient02 is OK: OK: Less than 1.00% above the threshold [0.0] [05:34:34] (03PS1) 10Legoktm: Add tox-poolcounter image [integration/config] - 10https://gerrit.wikimedia.org/r/456073 [05:35:18] (03CR) 10Legoktm: "Hashar, do you have any suggestions how I can run `make` before tox without needing to fork/copy the entire run.sh script for this image?" [integration/config] - 10https://gerrit.wikimedia.org/r/456073 (owner: 10Legoktm) [06:08:31] https://tools.wmflabs.org/versions/ seems broken, group 0 is on -wmf.19 [06:09:12] Nikerabbit: it was discussed in here earlier, https://phabricator.wikimedia.org/T202734 [06:09:26] legoktm: oh I see [06:12:30] 10Release-Engineering-Team, 10Patch-For-Review: tools.wmflabs.org/versions caching - https://phabricator.wikimedia.org/T202734 (10Legoktm) I don't have arcanist set up, so here's a patch: ``` From 694d638621fbe02b0c1146e5e0418d737736ccd7 Mon Sep 17 00:00:00 2001 From: Kunal Mehta Date... [06:32:01] (03CR) 10Gehel: "@Hashar: any progress on this! Ping me if you need support!" [integration/config] - 10https://gerrit.wikimedia.org/r/415001 (owner: 10Hashar) [06:45:07] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [07:05:22] PROBLEM - Puppet errors on deployment-deploy01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:07:40] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10Legoktm) Given that Wikibase is now running the quibble jobs, I think we should just drop the old mwext-repo/client jobs. If sqlite... [07:10:13] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10Addshore) >>! In T188717#4541170, @Legoktm wrote: > Given that Wikibase is now running the quibble jobs, I think we should just dro... [07:12:41] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10WMDE-leszek) Isn't it that quibble job runs repo and client enabled variant only? We still should have a job that has only client e... [07:13:25] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10Legoktm) Well the goal of this ticket is to get rid of all the special casing for Wikibase :) [07:16:47] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10Addshore) >>! In T188717#4541180, @Legoktm wrote: > Well the goal of this ticket is to get rid of all the special casing for Wikiba... [07:19:28] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10WMDE-leszek) I guess it is kind of going to be solved once we've managed to split WikibaseClient off WikibaseRepo, and making it fu... [07:20:06] RECOVERY - Puppet errors on deployment-imagescaler01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:21:21] 10Continuous-Integration-Config, 10Wikidata, 10User-Addshore: Move Wikibase to using the normal mediawiki extension (quibble) jobs - https://phabricator.wikimedia.org/T188717 (10Addshore) >>! In T188717#4541195, @WMDE-leszek wrote: > I guess it is kind of going to be solved once we've managed to split Wikiba... [07:32:03] 10Release-Engineering-Team (Watching / External), 10Operations, 10SRE-Access-Requests: Add contint-roots to releases{1,2}001 - https://phabricator.wikimedia.org/T201470 (10ArielGlenn) I've talked about this a little with moritzm and we've decided to go back to the SRE meeting with it, since the solution prop... [07:40:21] RECOVERY - Puppet errors on deployment-deploy01 is OK: OK: Less than 1.00% above the threshold [0.0] [08:00:43] (03CR) 10Thiemo Kreuz (WMDE): [C: 031] "Ah, I see. So this does not trigger any additional warning on core? Well, ok then." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/455626 (owner: 10Umherirrender) [08:01:15] 10Release-Engineering-Team, 10MediaWiki-extensions-WikimediaIncubator, 10Epic, 10I18n: Make creating a new Language project easier - https://phabricator.wikimedia.org/T165585 (10Amire80) >>! In T165585#4540966, @C933103 wrote: > 1. If a new site is to be created for each incubator site, how will WMF turn t... [08:18:36] RECOVERY - Free space - all mounts on integration-slave-docker-1026 is OK: OK: All targets OK [08:51:44] !log cleaned up disk on integration-slave-docker-1001 and integration-slave-docker-1002 [08:51:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:24:48] (03PS3) 10Hashar: Move phpcs dryrun to the generic composer image [integration/config] - 10https://gerrit.wikimedia.org/r/455835 (https://phabricator.wikimedia.org/T200722) [09:31:09] (03PS4) 10Hashar: Move phpcs dryrun to the generic composer image [integration/config] - 10https://gerrit.wikimedia.org/r/455835 (https://phabricator.wikimedia.org/T200722) [09:31:30] (03PS5) 10Hashar: Move phpcs dryrun to the generic composer image [integration/config] - 10https://gerrit.wikimedia.org/r/455835 (https://phabricator.wikimedia.org/T200722) [09:31:40] (03CR) 10Hashar: [C: 032] "Adjusted to move codesniffer under mediawiki/core vendor directory. Else the codesniffer php files were being processed since they were u" [integration/config] - 10https://gerrit.wikimedia.org/r/455835 (https://phabricator.wikimedia.org/T200722) (owner: 10Hashar) [09:32:42] (03Restored) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/183013 (owner: 10Hashar) [09:32:46] (03PS3) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/183013 [09:33:33] (03Merged) 10jenkins-bot: Move phpcs dryrun to the generic composer image [integration/config] - 10https://gerrit.wikimedia.org/r/455835 (https://phabricator.wikimedia.org/T200722) (owner: 10Hashar) [09:40:38] (03Abandoned) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/183013 (owner: 10Hashar) [09:47:17] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Quibble, 10Patch-For-Review, 10User-zeljkofilipin: Quibble does not install ffmpeg - https://phabricator.wikimedia.org/T193157 (10zeljkofilipin) Thanks! ๐ŸŽ‰ [10:02:24] addshore: 00:01:25.964 The "http://packagist.org/p/provider-2018-07%248df114abe395b56fd2b053bcdcaf1c53db731223961315ac0bbfbcca96cfea8f.json" file could not be downloaded (HTTP/1.1 404 Not Found) [10:02:36] hmmmmmm? [10:02:41] addshore: hello. I guess the above issue is what you talked about yesterday. Wikibase fails randomly due to packagist / composer install [10:02:48] spotted that https://integration.wikimedia.org/ci/job/mwext-Wikibase-client-tests-mysql-hhvm-jessie/7798/console [10:02:53] yes, as an example [10:02:54] and i noticed it yesterday [10:03:05] it also happens in core and others tests sometimes [10:03:14] that is a huge break really [10:03:50] oh, its still 404ing, no thats not what I was talking about then, i am just talking about random failures and 503s etc [10:03:59] ahh [10:05:12] addshore: I guess we want to fill bug whenever something happens [10:05:28] track down the root cause and solve/work around it [10:21:08] 10Release-Engineering-Team, 10Operations: Keyholder phab repo duplicate work - https://phabricator.wikimedia.org/T203003 (10ArielGlenn) p:05Triage>03Normal [10:26:03] 10Continuous-Integration-Infrastructure (shipyard), 10Operations, 10Kubernetes: Evaluate VMWare's Harbour as a docker registry - https://phabricator.wikimedia.org/T202504 (10ArielGlenn) p:05Triage>03Normal [10:26:21] 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-TrustedXFF, 10Patch-For-Review: [TrustedXFF] on fresh install trusted-xff.cdb is missing and User::getDefaultOptions() cause a fatal - https://phabricator.wikimedia.org/T198120 (10hashar) Looping Core Platform Team members. TrustedXFF has a list of... [10:26:38] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-TrustedXFF, 10Patch-For-Review: [TrustedXFF] on fresh install trusted-xff.cdb is missing and User::getDefaultOptions() cause a fatal - https://phabricator.wikimedia.org/T198120 (10hashar) p:05Tr... [10:27:34] 10Continuous-Integration-Infrastructure (shipyard), 10ArticlePlaceholder, 10Wikidata, 10Patch-For-Review, 10Technical-Debt: ArticlePlaceholder should use MediaWiki qunit runner - https://phabricator.wikimedia.org/T180171 (10hashar) [10:27:56] 10Continuous-Integration-Infrastructure (shipyard), 10MediaWiki-extensions-WikibaseMediaInfo, 10Wikidata: [WikibaseMediaInfo] test fails with PHPUnit 6 - https://phabricator.wikimedia.org/T198192 (10hashar) [10:38:52] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [10:40:40] 10Phabricator: Reorganize columns on Phabricator workboard - https://phabricator.wikimedia.org/T203073 (10Aklapper) p:05Triage>03Normal [10:44:08] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [10:45:04] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [10:59:36] (03PS28) 10Zfilipin: Selenium daily tests for beta using Docker/wdio [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [10:59:52] zeljkof: I had it rebased yesterday :] [11:00:00] zeljkof: probably we can start deleting some of the old jobs? [11:00:31] (03CR) 10Zfilipin: "PS28 updates commit message with deleted jobs and adds a couple of missing Wikibase jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:00:44] hashar: working on it [11:00:52] will continue after SWAT [11:04:21] I've already deleted old jobs [11:17:29] 10Gerrit, 10Release-Engineering-Team (Kanban): Gerrit has created duplicate accounts for some users - https://phabricator.wikimedia.org/T197083 (10Aklapper) >>! In T197083#4504096, @mmodell wrote: > This should **hopefully** be resolved, at least for @kaldari, on Thursday. Do we know of more affected users? [11:20:50] (03PS29) 10Zfilipin: Selenium daily tests for beta using Docker/wdio [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:21:26] (03CR) 10Zfilipin: "PS29 adds a couple of missing Wikibase jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:32:59] (03PS30) 10Zfilipin: Selenium daily tests for beta using Docker/wdio [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:33:11] (03CR) 10Zfilipin: "PS30 is a rebase" [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:41:09] (03PS31) 10Zfilipin: Selenium daily tests for beta using Docker/wdio [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:41:19] Project beta-scap-eqiad build #220551: 04FAILURE in 14 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/220551/ [11:44:26] (03CR) 10Zfilipin: "PS31 fixes how Wikibase jobs are created." [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [11:56:39] Project beta-scap-eqiad build #220552: 04STILL FAILING in 14 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/220552/ [12:06:14] (03PS32) 10Zfilipin: Selenium daily tests for beta using Docker/wdio [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [12:06:54] (03CR) 10Zfilipin: "PS32 removes jobs for repositories that do not have Selenium tests in Node.js." [integration/config] - 10https://gerrit.wikimedia.org/r/443931 (https://phabricator.wikimedia.org/T188742) (owner: 10Hashar) [12:11:44] Project beta-scap-eqiad build #220553: 04STILL FAILING in 14 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/220553/ [12:13:42] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [12:16:31] 10Release-Engineering-Team (Watching / External), 10Core-Platform-Team, 10Wikidata: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663 (10Addshore) This sort of happened a week or so when mediawi... [12:17:01] 10Release-Engineering-Team (Watching / External), 10Core-Platform-Team, 10Wikidata: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663 (10Addshore) a:05Addshore>03None Unassigning myself as I... [12:20:52] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [12:22:25] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [12:26:57] Project beta-scap-eqiad build #220554: 04STILL FAILING in 14 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/220554/ [12:35:34] !log fixed keyholder on deployment-deploy01 [12:35:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:42:51] Yippee, build fixed! [12:42:51] Project beta-scap-eqiad build #220555: 09FIXED in 15 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/220555/ [12:48:33] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:04:09] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:04:26] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MediaWiki-User-preferences, 10MW-1.32-release-notes (WMF-deploy-2018-07-17 (1.32.0-wmf.13)), and 3 others: Selenium "User should be able to change preferences" test flaky - https://phabricator.wikimedia.org/T198137 (10Pablo-WMDE) [13:10:41] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:12:25] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:12:59] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:15:11] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10MW-1.32-release-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)), 10Patch-For-Review, 10User-zeljkofilipin: Run tests daily targeting beta cluster for all repositories with Selenium tests - https://phabricator.wikimedia.org/T188742 (10zeljkofilip... [13:26:58] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) p:05Triage>03Normal [13:27:21] (03PS1) 10Hashar: docker: Quibble container with PHP 5.6 [integration/config] - 10https://gerrit.wikimedia.org/r/456141 (https://phabricator.wikimedia.org/T203084) [13:37:09] (03CR) 10Hashar: [C: 032] docker: Quibble container with PHP 5.6 [integration/config] - 10https://gerrit.wikimedia.org/r/456141 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [13:38:39] (03Merged) 10jenkins-bot: docker: Quibble container with PHP 5.6 [integration/config] - 10https://gerrit.wikimedia.org/r/456141 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [13:40:28] 10Continuous-Integration-Config, 10RelEng-Archive-FY201718-Q2, 10Composer, 10Upstream, 10Wikimedia-log-errors (Shared Build Failure): Error "TransportException 404 Not Found" in Jenkins jobs using composer - https://phabricator.wikimedia.org/T182266 (10Tgr) I have not seen this error for months and now s... [13:42:37] (03PS1) 10Hashar: docker: fix quibble-jessie-php56 changelog [integration/config] - 10https://gerrit.wikimedia.org/r/456144 [14:07:01] 10Continuous-Integration-Config, 10Math, 10Quibble, 10Patch-For-Review, 10User-Addshore: MathHooks: table creation yields warnings on quibble - https://phabricator.wikimedia.org/T202266 (10Pablo-WMDE) Hi @Physikerwelt, > how a change in Wikibase can affect the Math extension configuration As I understa... [14:13:27] (03PS1) 10Hashar: Docker jobs for DonationInterface [integration/config] - 10https://gerrit.wikimedia.org/r/456148 (https://phabricator.wikimedia.org/T203084) [14:20:49] PROBLEM - SSH on integration-slave-docker-1004 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:24:45] (03PS2) 10Hashar: Docker jobs for DonationInterface [integration/config] - 10https://gerrit.wikimedia.org/r/456148 (https://phabricator.wikimedia.org/T203084) [14:24:47] (03PS1) 10Hashar: docker: quibble-jessie-php56 fix package name [integration/config] - 10https://gerrit.wikimedia.org/r/456152 (https://phabricator.wikimedia.org/T203084) [14:25:16] (03CR) 10Hashar: [C: 032] docker: quibble-jessie-php56 fix package name [integration/config] - 10https://gerrit.wikimedia.org/r/456152 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [14:28:02] (03Merged) 10jenkins-bot: docker: quibble-jessie-php56 fix package name [integration/config] - 10https://gerrit.wikimedia.org/r/456152 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [14:41:03] (03PS3) 10Hashar: Docker jobs for DonationInterface [integration/config] - 10https://gerrit.wikimedia.org/r/456148 (https://phabricator.wikimedia.org/T203084) [14:41:16] (03CR) 10Hashar: [C: 032] "Good enough for now" [integration/config] - 10https://gerrit.wikimedia.org/r/456148 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [14:43:09] (03Merged) 10jenkins-bot: Docker jobs for DonationInterface [integration/config] - 10https://gerrit.wikimedia.org/r/456148 (https://phabricator.wikimedia.org/T203084) (owner: 10Hashar) [14:45:40] RECOVERY - SSH on integration-slave-docker-1004 is OK: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u5 (protocol 2.0) [14:48:01] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) [14:53:45] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) I have deployed the new container that ships Zend php 5.6.... [15:13:58] 10Release-Engineering-Team (Kanban), 10User-greg, 10User-zeljkofilipin: Wikimedia Portals Update and European Mid-day SWAT windows at the same time on Mondays - https://phabricator.wikimedia.org/T201932 (10greg) Ack. Will do. [15:13:58] PROBLEM - Puppet errors on deployment-certcentral-testclient02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [15:39:08] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) [15:40:06] 10Release-Engineering-Team, 10Operations, 10Epic, 10Services (watching): FY2017/18 Program 6 - Outcome 2 - Objective 2: Set up a continuous integration and deployment pipeline - https://phabricator.wikimedia.org/T170481 (10thcipriani) [15:40:51] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) [15:42:02] 10Release-Engineering-Team (Kanban), 10Operations, 10Release Pipeline, 10Services (watching): Move Graphoid to Kubernetes via the deployment pipeline - https://phabricator.wikimedia.org/T203091 (10thcipriani) p:05Triage>03Normal [15:44:54] 10Release-Engineering-Team (Kanban), 10Operations, 10Release Pipeline, 10Services (watching): Create Graphoid .pipeline files - https://phabricator.wikimedia.org/T203092 (10thcipriani) p:05Triage>03Normal [15:45:02] 10Gerrit, 10Operations, 10Traffic, 10Patch-For-Review: Enable avatars in gerrit - https://phabricator.wikimedia.org/T191183 (10Paladox) @thcipriani or @mmodell wondering if you be able to comment here that releng supports this avatar change and maintaining it please. (Ops need releng to comment) [15:46:21] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Add service-pipeline to zuul for Graphoid - https://phabricator.wikimedia.org/T203093 (10thcipriani) p:05Triage>03Normal [15:48:27] 10Release-Engineering-Team, 10Analytics, 10MediaWiki-extensions-JsonConfig, 10MW-1.32-release-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), and 2 others: Config:Dashiki:* on Meta can't be opened - https://phabricator.wikimedia.org/T203029 (10Nuria) Some sample urls that are still broken: https://meta.wik... [15:50:17] 10Release-Engineering-Team, 10Analytics, 10MediaWiki-extensions-JsonConfig, 10MW-1.32-release-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), and 2 others: Config:Dashiki:* on Meta can't be opened - https://phabricator.wikimedia.org/T203029 (10Nuria) Note that dashiki just consumes these json files on meta... [15:52:14] (03PS1) 10Thcipriani: Migrate graphoid to service pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/456168 (https://phabricator.wikimedia.org/T203093) [15:55:20] 10Release-Engineering-Team (Someday), 10Scap, 10MediaWiki-General-or-Unknown, 10Wikimedia-Incident: Detect missing extension dependencies before production - https://phabricator.wikimedia.org/T200944 (10kostajh) > CI for wmf-config repository: It might be possible to detect this in CI for wmf-config. Speci... [15:56:13] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) The original job relies on mediawiki/vendor and then run `... [15:58:26] ejegg: hello. If you are around I could use some light about DonationInterface dependencies on https://phabricator.wikimedia.org/T203084 :D [15:58:50] hi hashar! [15:58:50] ejegg: seems we have to run "composer install" inside the extension directory since mediawiki/vendor fundraising rbranch does not provide all depenedencies [15:59:11] and seemingly, running composer update from mediawiki/core brings bad dependencies which cause tests to fail / autoloader to explode bah :/ [15:59:21] ooh, that's nasty [15:59:28] ejegg: I am trying to migrate the mwext-donationinterfacecore-REL1_27-zend56-jessie job to the new test runner Quibble (and in a docker container) [15:59:36] yeah, on local dev we generally do 'composer install' in the extension dir [15:59:54] that is what I eventually noticed by reading the jenkins job eek [15:59:57] for the move to 1.31, we we also planning to move to a unified vendor dir [16:00:00] under mediawiki [16:00:20] we were hoping to upgrade payments a couple weeks ago, but we needed to do more testing on stretch & php7 [16:00:29] using composer update and the merge plugin from mediawikicore? [16:00:35] yes, exactly [16:00:39] ahhhh [16:00:49] so I have added an experimental job that does clone mediawiki/vendor [16:00:55] hoping that will make things easier from CI point of view [16:01:03] then craft a composer.local.json and runs composer update from the root of mediawiki [16:01:18] good news, there is only 8 tests failling, all due to a missing class Class PayWithAmazon\Mocks\MockPaymentsClient :] [16:01:19] hashar ahhhh, that could work [16:01:49] d'oh, of course, that's the one repo we haven't been able to upstream any fixes in [16:02:00] but that uses "composer update" which probably does not handle the DonationInterface/composer.lock that is uspposed to be used with "composer install" [16:02:02] so we're still using a repositories block in the composer.json [16:02:08] OH MY GOD [16:02:11] I should just hard-fork it [16:02:12] that so explains it :]]]]]]]]]]]]]]]]]]]] [16:02:42] yeah, Amazon is super-resistant to merge any PRs in their payments SDK [16:02:49] even my freaking whitespace fix :P [16:02:55] I can imagine yeah [16:03:23] then you probably need the test to run with that composer.lock [16:03:42] I am not sure how acceptable it is to use composer update (which afaik would grab whatever is the latest version from packagist) [16:03:47] right, we're checking in composer.lock in the extension currently [16:03:47] but maybe for CI it is a non issue [16:04:05] and the idea for after the MW1.31 upgrade would be to check in composer.lock in the mw root dir [16:07:41] ejegg: if there is a way to use the composer update / merge plugin for REL1_27 that would be awesome [16:07:54] else I guess I can come up with a nasty hack in the test runner to accomodate for that usecase [16:10:34] hashar: we could use the merge plugin in 1.27 [16:11:01] i was just planning to do both at once cause it felt like less work [16:15:30] ejegg: could. If you want to reproduce using the merge plugin , on a DonationInterface change you can comment "check experimental" [16:15:33] and that will trigger the new job [16:18:04] 10Gerrit, 10Release-Engineering-Team (Kanban): Gerrit has created duplicate accounts for some users - https://phabricator.wikimedia.org/T197083 (10mmodell) 05Open>03Resolved a:03mmodell I manually fixed all of the others that I'm aware of. [16:18:09] cool, thanks hashar! [16:18:10] 10Gerrit, 10Release-Engineering-Team (Watching / External), 10Upstream: Unable to edit external-ids ref in notedb due to validation - https://phabricator.wikimedia.org/T197192 (10thcipriani) p:05High>03Normal @mmodell found a way for gerrit admins to be able to edit, although not ideal it does lower the... [16:18:16] 10Gerrit, 10Release-Engineering-Team (Kanban): Gerrit has created duplicate accounts for some users - https://phabricator.wikimedia.org/T197083 (10thcipriani) [16:18:22] ejegg: and I guess tomorrow I will add a job for REL1_31 ;] [16:18:48] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512 (10hashar) [16:19:31] Am I right in vaguely remembering that a RelEnger needs to do a command line Phab edit to make "old" projects able to have sub-projects? [16:21:30] Never mind, URL hacked my way to the panel anyway. [16:27:24] heh [16:30:10] (03CR) 10Hashar: [C: 031] Migrate graphoid to service pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/456168 (https://phabricator.wikimedia.org/T203093) (owner: 10Thcipriani) [17:07:21] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) Based on a discussion with @Ejegg , the issues would be be... [17:15:44] (03PS1) 10Reedy: Add EUCopyrightCampaign and EUCopyrightCampaignSkin [integration/config] - 10https://gerrit.wikimedia.org/r/456180 [17:19:07] (03PS2) 10Reedy: Add EUCopyrightCampaign and EUCopyrightCampaignSkin [integration/config] - 10https://gerrit.wikimedia.org/r/456180 [17:21:06] (03PS3) 10Reedy: Add EUCopyrightCampaign and EUCopyrightCampaignSkin [integration/config] - 10https://gerrit.wikimedia.org/r/456180 [17:21:25] (03CR) 10Reedy: [C: 032] Add EUCopyrightCampaign and EUCopyrightCampaignSkin [integration/config] - 10https://gerrit.wikimedia.org/r/456180 (owner: 10Reedy) [17:22:49] (03Merged) 10jenkins-bot: Add EUCopyrightCampaign and EUCopyrightCampaignSkin [integration/config] - 10https://gerrit.wikimedia.org/r/456180 (owner: 10Reedy) [17:23:42] !log reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/456180/ [17:23:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:57:28] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10thcipriani) p:05Triage>03Normal [18:07:02] 10Release-Engineering-Team: Add branch committers to train task - https://phabricator.wikimedia.org/T203111 (10thcipriani) [18:08:14] (03PS1) 10Krinkle: Enable CI for performance/perflogbot (standard npm job) [integration/config] - 10https://gerrit.wikimedia.org/r/456189 (https://phabricator.wikimedia.org/T203110) [18:08:24] (03PS2) 10Krinkle: Enable CI for performance/perflogbot (standard Node.js jobs) [integration/config] - 10https://gerrit.wikimedia.org/r/456189 (https://phabricator.wikimedia.org/T203110) [18:08:32] (03CR) 10Krinkle: [C: 032] Enable CI for performance/perflogbot (standard Node.js jobs) [integration/config] - 10https://gerrit.wikimedia.org/r/456189 (https://phabricator.wikimedia.org/T203110) (owner: 10Krinkle) [18:10:05] (03Merged) 10jenkins-bot: Enable CI for performance/perflogbot (standard Node.js jobs) [integration/config] - 10https://gerrit.wikimedia.org/r/456189 (https://phabricator.wikimedia.org/T203110) (owner: 10Krinkle) [18:10:29] 10Phabricator, 10Project-Admins, 10Security-Team: Create Tag for WMDE Fundraising Security issues - https://phabricator.wikimedia.org/T194286 (10Aklapper) 05Open>03declined >>! In T194286#4325103, @Aklapper wrote: > If there are further questions, please ask. > Also please tell how to proceed - as I sai... [18:10:41] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/456189 (T203110) [18:10:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:10:45] T203110: Move tools-perflogbot repository to Gerrit - https://phabricator.wikimedia.org/T203110 [18:18:47] 10Phabricator: Phabricator should recognize "Bug: Tnnn" in commit messages same as "refs Tnnn" and connect the commit to the task - https://phabricator.wikimedia.org/T117434 (10Aklapper) p:05Low>03Lowest Proposing to decline as "Related Objects" exists. [18:21:35] 10Phabricator, 10Wikimedia-Logstash: Improve error reporting / integration between Kibana and Phabricator - https://phabricator.wikimedia.org/T185155 (10Aklapper) p:05Low>03Lowest [18:23:35] 10Release-Engineering-Team: Add branch committers to train task - https://phabricator.wikimedia.org/T203111 (10thcipriani) @mmodell mentioned that there may be facility in phab to do this. [18:23:39] 10Phabricator, 10Cloud-Services, 10VPS-project-Phabricator: Applying role role::phabricator::main causes errors on instances - https://phabricator.wikimedia.org/T138881 (10Aklapper) Anyone knows what's left to do here and could elaborate? @Paladox maybe? Thanks! :) [18:23:48] Presumably 1.33.0-wmf.1 is going to be vaguely at the end of October? (1.32.0-wmf.1 was 24 April, so +6 months would give us 30 October as the six-months-plus-round-forward date.) That'd make wmf.27 the last branch of 1.32. [18:24:46] 10Phabricator, 10Cloud-Services, 10VPS-project-Phabricator: Applying role role::phabricator::main causes errors on instances - https://phabricator.wikimedia.org/T138881 (10Paladox) 05Open>03Resolved a:03Paladox I believe we resolved this a long time ago as itโ€™s been running successfully on phab.wmflabs... [18:28:34] 10Phabricator: Phabricator edit registered multiple times - https://phabricator.wikimedia.org/T185201 (10Aklapper) p:05Triage>03Lowest [18:28:50] For practical considerations, that gives us a week's breathing room after the data centre switch-over-back (wmf.26 will be 2018-10-16; based on precedent, there'll be at least a half-dozen UBNs from the fortnight of no deploys). [18:28:55] 10Scap, 10Operations, 10Patch-For-Review: Intermittent git-fat failure during deploy - https://phabricator.wikimedia.org/T202100 (10thcipriani) >>! In T202100#4515951, @Ottomata wrote: > we just updated it for wqds* hosts. If that worked fine for @Gehel and Erik, we'll update the rest of the flee (all nodes... [18:29:06] James_F: the last of 1.32 is https://phabricator.wikimedia.org/T191072 (wmf.26, week of Oct 15th), so 1.33.0-wmf.1 is Oct 22nd (well 23rd), TechConf week, fittingly :) [18:29:21] Hmm, OK. [18:31:52] 10Release-Engineering-Team, 10Scap, 10Operations: find a way to systematically update the deployment server name across all repos - https://phabricator.wikimedia.org/T197470 (10thcipriani) [18:31:53] each release gets 26 weeks (half of the year) [18:31:56] 10Scap, 10Patch-For-Review, 10User-fgiunchedi: Ensure deployment_server is global - https://phabricator.wikimedia.org/T162814 (10thcipriani) 05Open>03Resolved a:03thcipriani All repos that are deployed have merged patches unsetting `git_server` so they should now obey the globally set `git_server` in `... [18:33:41] OK, I've set up ReleaseTaggerBot through 'til 1.33 wmf.1 https://phabricator.wikimedia.org/project/view/3556/ [18:33:53] Now I don't need to think about RTB for another two months. [18:34:07] :) [18:36:44] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512 (10Legoktm) [18:36:47] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-TrustedXFF, 10Patch-For-Review: [TrustedXFF] on fresh install trusted-xff.cdb is missing and User::getDefaultOptions() cause a fatal - https://phabricator.wikimedia.org/T198120 (10Legoktm) 05Ope... [18:38:27] 10Scap, 10Operations, 10Patch-For-Review: Intermittent git-fat failure during deploy - https://phabricator.wikimedia.org/T202100 (10Ottomata) Done, worked everywhere except: ``` The following hosts were unreachable: cloudservices1004.wikimedia.org ``` [18:39:07] 10Phabricator, 10GlobalBlocking, 10User-Zppix: Have global blocks from any wikimedia project affect Phabricator as well - https://phabricator.wikimedia.org/T173463 (10Aklapper) [18:39:13] 10Phabricator, 10Release-Engineering-Team (Someday), 10Cloud-Services, 10LDAP, 10Wikimedia-Incident: Blocking an account on wikitech should disable LDAP logins - https://phabricator.wikimedia.org/T168692 (10Aklapper) [18:39:38] (03CR) 10Thcipriani: [C: 032] Migrate graphoid to service pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/456168 (https://phabricator.wikimedia.org/T203093) (owner: 10Thcipriani) [18:39:56] 10Scap, 10Operations, 10Patch-For-Review: Intermittent git-fat failure during deploy - https://phabricator.wikimedia.org/T202100 (10Dzahn) >>! In T202100#4543309, @Ottomata wrote: > The following hosts were unreachable: > cloudservices1004.wikimedia.org should be because of T201341#4543268 [18:41:08] (03Merged) 10jenkins-bot: Migrate graphoid to service pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/456168 (https://phabricator.wikimedia.org/T203093) (owner: 10Thcipriani) [18:42:27] 10Phabricator, 10GlobalBlocking, 10User-Zppix: Have global blocks from any wikimedia project affect Phabricator as well - https://phabricator.wikimedia.org/T173463 (10Aklapper) [18:43:32] !log reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/456168/ (T203093) [18:43:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:43:36] T203093: Add service-pipeline to zuul for Graphoid - https://phabricator.wikimedia.org/T203093 [18:51:54] 10Release-Engineering-Team (Kanban), 10Operations, 10Release Pipeline, 10Services (watching): Move Graphoid to Kubernetes via the deployment pipeline - https://phabricator.wikimedia.org/T203091 (10thcipriani) [18:51:58] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Add service-pipeline to zuul for Graphoid - https://phabricator.wikimedia.org/T203093 (10thcipriani) 05Open>03Resolved a:03thcipriani Patches for graphoid are now going through `service-pipeline` [18:52:55] 10Phabricator, 10Release-Engineering-Team (Someday), 10wikitech.wikimedia.org, 10LDAP, 10Wikimedia-Incident: Blocking an account on wikitech should disable LDAP logins - https://phabricator.wikimedia.org/T168692 (10Aklapper) [18:57:59] 10Phabricator, 10Wikibugs: Convert Wikibugs to use phabricator webhooks (instead of polling the feed) - https://phabricator.wikimedia.org/T197790 (10Legoktm) From https://secure.phabricator.com/book/phabricator/article/webhooks/: > Callbacks may be delayed for an arbitrarily long amount of time, up to the garb... [19:00:03] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10Aklapper) [19:00:09] 10Gerrit, 10Phabricator, 10Release-Engineering-Team (Someday): Stop using Differential for code review - https://phabricator.wikimedia.org/T191182 (10Aklapper) [19:02:13] 10Release-Engineering-Team, 10Patch-For-Review: tools.wmflabs.org/versions caching - https://phabricator.wikimedia.org/T202734 (10thcipriani) FWIW, you can create a new differential revision by pasting a patch in https://phabricator.wikimedia.org/differential/diff/create/ I had hoped that by creating the revi... [19:17:40] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10Jenkins, 10Patch-For-Review: Upgrade deployment-prep deployment servers to stretch - https://phabricator.wikimedia.org/T192561 (10thcipriani) >>! In T192561#4540050, @Dzahn wrote: > Thanks, i see: > > ``` > # Parsoid Jav... [19:31:11] 10Release-Engineering-Team, 10Analytics, 10MediaWiki-extensions-JsonConfig, 10MW-1.32-release-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), and 2 others: Config:Dashiki:* on Meta can't be opened - https://phabricator.wikimedia.org/T203029 (10Mholloway) This seems to have been fixed by today's deployment o... [19:48:06] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.32.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T191065 (10dduvall) Branch 1.32.0-wmf.19 has been deployed to group1. There are loads of log spam due to T202058 and T201082 but all else looks ok. [19:49:13] PROBLEM - Host deployment-certcentral02 is DOWN: CRITICAL - Host Unreachable (10.68.23.13) [19:51:34] PROBLEM - Host deployment-certcentral-testclient02 is DOWN: CRITICAL - Host Unreachable (10.68.16.95) [20:05:32] 10Release-Engineering-Team, 10Patch-For-Review: tools.wmflabs.org/versions caching - https://phabricator.wikimedia.org/T202734 (10bd808) I've applied @Legoktm's patch. When @Krinkle's apache config lands we can revert it to avoid useless cache pollution. [20:12:22] 10Phabricator (Upstream), 10Upstream: "Unknown Object" in feed should say "Restricted Task" instead - https://phabricator.wikimedia.org/T105332 (10Aklapper) [20:16:12] 10Phabricator, 10Puppet: Local config file contains escape characters - https://phabricator.wikimedia.org/T103924 (10Aklapper) [20:16:36] 10Phabricator, 10Puppet: Local config file contains escape characters - https://phabricator.wikimedia.org/T103924 (10Aklapper) Where to see that "they get processed as regular string"? [20:17:34] 10Phabricator: Tasks moving back to default column on their own - https://phabricator.wikimedia.org/T107103 (10Aklapper) Has anyone seen this again lately? [20:18:29] 10Release-Engineering-Team, 10Analytics, 10MediaWiki-extensions-JsonConfig, 10MW-1.32-release-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), and 2 others: Config:Dashiki:* on Meta can't be opened - https://phabricator.wikimedia.org/T203029 (10Nuria) yay! Resolving. [20:18:36] 10Release-Engineering-Team, 10Analytics, 10MediaWiki-extensions-JsonConfig, 10MW-1.32-release-notes (WMF-deploy-2018-08-28 (1.32.0-wmf.19)), and 2 others: Config:Dashiki:* on Meta can't be opened - https://phabricator.wikimedia.org/T203029 (10Nuria) 05Open>03Resolved [20:27:41] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-TrustedXFF, 10MW-1.32-release-notes (WMF-deploy-2018-09-04 (1.32.0-wmf.20)), 10Patch-For-Review: [TrustedXFF] on fresh install trusted-xff.cdb is missi... - https://phabricator.wikimedia.org/T198120 [20:28:08] legoktm: I fixed up the phpcs job on mediawiki/tools/codesniffer . It is a bit slow and hacky but it works ! [20:29:01] (03PS1) 10Hashar: Migrate TrustedXFF to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/456276 (https://phabricator.wikimedia.org/T183512) [20:29:19] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512 (10hashar) [20:29:22] (03CR) 10Hashar: [C: 032] Migrate TrustedXFF to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/456276 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [20:30:11] PROBLEM - Free space - all mounts on integration-slave-docker-1026 is CRITICAL: CRITICAL: integration.integration-slave-docker-1026.diskspace.root.byte_percentfree (<11.11%) [20:30:49] (03Merged) 10jenkins-bot: Migrate TrustedXFF to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/456276 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar) [20:32:17] hasharAway: thank you :) [20:32:56] legoktm: and if you could possibly get a look at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/ReadingLists/+/447086/ [20:33:11] extension registry does not load the extension configuration setting. But probably want tgr to +2 that one [20:37:26] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512 (10hashar) [20:39:57] (03PS1) 10Hashar: Remove deleted repositories [integration/config] - 10https://gerrit.wikimedia.org/r/456278 [20:40:11] RECOVERY - Free space - all mounts on integration-slave-docker-1026 is OK: OK: All targets OK [20:40:34] (03PS2) 10Hashar: Remove deleted repositories [integration/config] - 10https://gerrit.wikimedia.org/r/456278 [20:40:47] (03CR) 10Hashar: [C: 032] Remove deleted repositories [integration/config] - 10https://gerrit.wikimedia.org/r/456278 (owner: 10Hashar) [20:42:22] (03Merged) 10jenkins-bot: Remove deleted repositories [integration/config] - 10https://gerrit.wikimedia.org/r/456278 (owner: 10Hashar) [20:47:17] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10thcipriani) @faidon @Volans any preference on the hierarchy under which we create this repo? `operations/debs/keyholder` vs `operations/software/keyholder` vs `operations/keyholder`? [20:52:08] 10Phabricator, 10Release-Engineering-Team (Kanban): Phabricator DB query request: list repositories and assigned callsigns - https://phabricator.wikimedia.org/T202222 (10MarcoAurelio) Thank you :) [20:55:27] https://charity.wtf/2018/08/19/shipping-software-should-not-be-scary/ is good reading and it's nice to see how much of the advice we follow. [21:03:33] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10Paladox) i think "operations/debs/" is used for if your planning on using the repo as a debian package repo? [21:07:29] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10thcipriani) >>! In T203108#4543657, @Paladox wrote: > i think "operations/debs/" is used for if your planning on using the repo as a debian package repo? Yes, sorry, forgot the mention, that's the pl... [21:08:22] 10Release-Engineering-Team (Kanban): Create keyholder gerrit repo - https://phabricator.wikimedia.org/T203108 (10Paladox) I guess we should go with "operations/debs/" then :) i think we use "operations/software/" if you planning on deploying it a different way like scp. [21:16:23] the git commands to clone / fetch look soooo much better in the update in pg [21:16:59] https://phabricator.wikimedia.org/F25503488 [21:21:17] and now the delete project plugin supports polygerrit https://gerrit-review.googlesource.com/c/plugins/delete-project/+/140591 [21:22:38] https://gerrit-review.googlesource.com/c/gerrit/+/194453 [21:27:46] 10Release-Engineering-Team (Watching / External), 10Operations, 10Release Pipeline: Update Debian package of Blubber (0.5.0-1) - https://phabricator.wikimedia.org/T203121 (10thcipriani) [22:16:27] 10Release-Engineering-Team, 10GitHub-Mirrors, 10User-greg: Requesting access to the 'wikimedia' organization at GitHub for MarcoAurelio - https://phabricator.wikimedia.org/T201259 (10greg) 05Open>03Resolved a:03greg I am supportive of this. Added. Thanks for your on-going work here. [22:19:34] 10Release-Engineering-Team, 10Structured-Data-Commons, 10Wikidata, 10User-Urbanecm, 10Wiki-Setup (Create): Create a production test wiki in group0 to parallel Wikimedia Commons - https://phabricator.wikimedia.org/T197616 (10greg) 05Open>03declined After speaking with @MarkTraceur we agreed to setup B... [22:41:02] 10Release-Engineering-Team, 10Structured-Data-Commons, 10Wikidata, 10User-Urbanecm, 10Wiki-Setup (Create): Create a production test wiki in group0 to parallel Wikimedia Commons - https://phabricator.wikimedia.org/T197616 (10Jdforrester-WMF) >>! In T197616#4543890, @greg wrote: > After speaking with @Mark... [22:43:50] (03PS4) 10Dduvall: Statsd publisher that sends job/node metrics to statsd.eqiad.wmnet [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) [22:45:07] (03CR) 10jerkins-bot: [V: 04-1] Statsd publisher that sends job/node metrics to statsd.eqiad.wmnet [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) (owner: 10Dduvall) [22:51:21] (03PS5) 10Dduvall: Statsd publisher that sends job/node metrics to statsd.eqiad.wmnet [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) [22:58:09] (03PS6) 10Dduvall: Statsd publisher that sends job/node metrics to statsd.eqiad.wmnet [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) [23:03:31] (03PS7) 10Dduvall: Statsd publisher that sends job/node metrics to statsd.eqiad.wmnet [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) [23:03:59] (03CR) 10Dduvall: "Considering Krinkle's feedback and our IRC conversation in #wikimedia-releng on Friday, I've paired this patch down quite a bit to just wh" [integration/config] - 10https://gerrit.wikimedia.org/r/455269 (https://phabricator.wikimedia.org/T201972) (owner: 10Dduvall) [23:05:14] (03CR) 10Legoktm: "I guess I could just have tox run make??" [integration/config] - 10https://gerrit.wikimedia.org/r/456073 (owner: 10Legoktm) [23:53:22] 10Phabricator, 10Release-Engineering-Team (Kanban), 10User-Ryasmeen: 404 on workboard for an existing project (due to custom filter applied which did not exist in database) - https://phabricator.wikimedia.org/T199207 (10Deskana) After a few weeks with no issues, this has happened three times in the past 48 h...