[01:38:33] Project browsertests-Wikidata-SmokeTests-linux-firefox-sauce build #476: 04FAILURE in 21 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce/476/ [01:40:42] PROBLEM - Free space - all mounts on deployment-fluorine is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine.diskspace._srv.byte_percentfree (<33.33%) [02:18:43] RECOVERY - Puppet failure on deployment-eventlogging03 is OK: OK: Less than 1.00% above the threshold [0.0] [02:47:07] Project beta-scap-eqiad build #82947: 04FAILURE in 1 min 21 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/82947/ [02:57:50] Yippee, build fixed! [02:57:50] Project beta-scap-eqiad build #82948: 09FIXED in 8 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/82948/ [07:00:47] RECOVERY - Free space - all mounts on deployment-fluorine is OK: OK: All targets OK [09:15:45] 10MediaWiki-Codesniffer: phplint doesn't run on .inc files - https://phabricator.wikimedia.org/T116524#1893330 (10TasneemLo) [09:16:41] 10MediaWiki-Codesniffer, 5Patch-For-Review: Add sniff to avoid if/else/while/foreach without curly braces - https://phabricator.wikimedia.org/T113863#1893339 (10TasneemLo) [09:17:17] 10MediaWiki-Codesniffer, 5Patch-For-Review: Improve test suite for mediawiki phpcs standard - https://phabricator.wikimedia.org/T92751#1893345 (10TasneemLo) a:5TasneemLo>3None [09:38:24] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #710: 04FAILURE in 1 min 23 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/710/ [11:17:13] PROBLEM - Host deployment-cache-parsoid04 is DOWN: CRITICAL - Host Unreachable (10.68.19.197) [12:58:24] PROBLEM - Puppet failure on wmfbranch is CRITICAL: CRITICAL: 75.00% of data above the critical threshold [0.0] [12:58:31] Yippee, build fixed! [12:58:31] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #884: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/884/ [13:33:28] RECOVERY - Puppet failure on wmfbranch is OK: OK: Less than 1.00% above the threshold [0.0] [13:54:23] PROBLEM - Puppet failure on wmfbranch is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0] [14:34:24] RECOVERY - Puppet failure on wmfbranch is OK: OK: Less than 1.00% above the threshold [0.0] [15:47:12] 10Continuous-Integration-Infrastructure, 10MediaWiki-API, 10MediaWiki-Unit-tests, 10MediaWiki-extensions-CentralAuth: Fix ApiDocumentationTest on CentralAuth - https://phabricator.wikimedia.org/T122007#1893653 (10Umherirrender) 3NEW [16:04:36] 10Deployment-Systems, 6Release-Engineering-Team: scap-purge-l10n-cache - https://phabricator.wikimedia.org/T122008#1893667 (10Reedy) 3NEW [16:04:44] 10Deployment-Systems, 6Release-Engineering-Team: scap-purge-l10n-cache hanging - https://phabricator.wikimedia.org/T122008#1893676 (10Reedy) [16:05:17] 10Continuous-Integration-Infrastructure, 10MediaWiki-API, 10MediaWiki-Unit-tests, 10MediaWiki-extensions-CentralAuth: Fix ApiDocumentationTest on CentralAuth - https://phabricator.wikimedia.org/T122007#1893678 (10Reedy) [16:06:36] 10Deployment-Systems, 6Release-Engineering-Team: scap-purge-l10n-cache hanging - https://phabricator.wikimedia.org/T122008#1893667 (10Reedy) --verbose doesn't show very much [16:16:38] RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:44:16] (03CR) 10Hoo man: [C: 032] Remove unused variables from two utilities [tools/code-utils] - 10https://gerrit.wikimedia.org/r/258966 (owner: 10Thiemo Mättig (WMDE)) [16:44:34] (03Merged) 10jenkins-bot: Remove unused variables from two utilities [tools/code-utils] - 10https://gerrit.wikimedia.org/r/258966 (owner: 10Thiemo Mättig (WMDE)) [16:45:30] (03CR) 10Hoo man: [C: 032] Fix broken/incomplete PHPDoc tags in two utilities [tools/code-utils] - 10https://gerrit.wikimedia.org/r/258965 (owner: 10Thiemo Mättig (WMDE)) [16:45:36] (03PS2) 10Hoo man: Fix broken/incomplete PHPDoc tags in two utilities [tools/code-utils] - 10https://gerrit.wikimedia.org/r/258965 (owner: 10Thiemo Mättig (WMDE)) [17:25:56] (03PS1) 10Paladox: [ShoutWikiAPI] Add Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/260248 [17:26:16] (03PS2) 10Paladox: [ShoutWikiAPI] Add Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/260248 [17:48:32] 10Deployment-Systems, 6Release-Engineering-Team: scap-purge-l10n-cache hanging - https://phabricator.wikimedia.org/T122008#1893747 (10Reedy) Yup, so after it was depooled, scap-purge-l10n-cache runs fine, so I presume there's some excessive timeout being set to the dsh call by this module? [18:39:49] It seems that https://integration.wikimedia.org/zuul/ is stuck all tests aren't loading. The queue getting bigger but nothing is running. [18:47:10] (03CR) 10Paladox: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/260248 (owner: 10Paladox) [20:42:26] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 6Discovery, 10Wikimedia-Site-Requests, 7Wikimedia-log-errors: Warning when loading config - https://phabricator.wikimedia.org/T122017#1893898 (10Krenair) [20:42:35] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 6Discovery, 10Wikimedia-Site-Requests, 7Wikimedia-log-errors: Warning when loading config - https://phabricator.wikimedia.org/T122017#1893899 (10Krenair) a:3Krenair [20:46:12] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 6Discovery, 10Wikimedia-Site-Requests, 7Wikimedia-log-errors: Warning when loading config - https://phabricator.wikimedia.org/T122017#1893904 (10Krenair) 5Open>3Resolved From https://gerrit.wikimedia.org/r/#/c/255135/ ```krenair@deployment-bastion:/srv/... [20:47:29] 10Beta-Cluster-Infrastructure, 6Discovery, 10Wikimedia-Portals: git submodules under /srv/mediawiki-staging in beta - https://phabricator.wikimedia.org/T122018#1893907 (10Krenair) 3NEW [21:02:47] PROBLEM - Free space - all mounts on deployment-fluorine is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine.diskspace._srv.byte_percentfree (<37.50%) [21:03:17] There are a lot testpages of selenium user at test2, this account it inactive since february. Are the pages still needed, or can I delete them? [21:07:04] 5Gerrit-Migration, 10Analytics-Tech-community-metrics: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#1893921 (10Aklapper) [21:30:30] 5Gerrit-Migration, 10Analytics-Tech-community-metrics: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#1893963 (10jgbarah) We would like to follow your plans to migrate to Differential as much as possible. During th... [21:34:28] greg-g: Hey. Just a Heads-up: Since the last MW tarball release we've received several reports about Special pages being broken: https://phabricator.wikimedia.org/T121892 - Chad is already CC'ed on the task. [21:38:55] * Reedy looks [21:39:32] andre__: Easy fix [21:40:00] * Reedy does so [21:41:07] Reedy, oh thanks. [21:41:09] Still, wondering whether to push followup tarballs just to fix that issue. As it looks popular. [21:41:29] plus aftermath ("could we have avoided that?") [21:42:53] I think we should [21:43:07] The could we avoid that... Better static analysis [21:43:16] And/or better unit testing, so said could would've been exercised [21:46:14] Looks like it's an issue in 1.23 through 1.25 [21:46:20] I think 1.26 may be ok, but needs checking [21:46:38] Nope, 1.26 too [21:55:45] andre__: backports/cherry picks made [21:57:14] Reedy: I like you! [21:57:16] Thanks. [21:57:25] How many years has it taken? [21:57:26] :P [21:57:43] Many. Many, baby. Don't make me count. [21:57:46] :P [21:57:59] I wonder if I should've done the bumps and everything all in the same commit [21:58:21] * Reedy sends greg-g an invoice [22:14:33] Releases prepped [22:15:42] Reedy: there's a comment on the task that it's not a complete patch? :-/ [22:15:45] dunno [22:16:06] andre__: different errors [22:16:11] I can't fix them if I don't know they're there ;) [22:16:14] But we should fix those too [22:16:20] ah [22:18:09] 6Release-Engineering-Team, 5MW-1.23-release, 5MW-1.25-release, 5MW-1.26-release: Release 1.23.13, 1.24.6, 1.25.5, 1.26.2 - https://phabricator.wikimedia.org/T122023#1894018 (10Reedy) 3NEW [22:18:26] 6Release-Engineering-Team, 5MW-1.23-release, 5MW-1.25-release, 5MW-1.26-release: Release 1.23.13, 1.24.6, 1.25.5, 1.26.2 - https://phabricator.wikimedia.org/T122023#1894026 (10Reedy) [22:18:39] Those could just be 1.23 etc [22:18:59] * andre__ tries to slowly move afk [22:19:03] g'night. and thanks again. [23:10:43] 6Release-Engineering-Team, 5MW-1.23-release, 5MW-1.25-release, 5MW-1.26-release: Release 1.23.13, 1.24.6, 1.25.5, 1.26.2 - https://phabricator.wikimedia.org/T122023#1894053 (10Reedy) Bugfixes for T121892 are made for 1.23-1.26 https://gerrit.wikimedia.org/r/260306 https://gerrit.wikimedia.org/r/260307 htt... [23:17:02] 10Continuous-Integration-Config, 6Release-Engineering-Team, 6Collaboration-Team-Backlog, 10Flow, 5MW-1.25-release: Flow (or tests) broken in REL1_25 - https://phabricator.wikimedia.org/T122025#1894060 (10Reedy)