[02:42:51] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Release Pipeline, 10User-dancy: Failed docker build leaves dangling container - https://phabricator.wikimedia.org/T235680 (10dancy) 05Open→03Resolved [02:43:08] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Release Pipeline: Failed docker build leaves dangling container - https://phabricator.wikimedia.org/T235680 (10dancy) [06:20:31] maintenance-disconnect-full-disks build 196323 integration-agent-docker-1011 (/: 18%, /srv: 99%, /var/lib/docker: 70%): OFFLINE due to disk space [06:22:26] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<33.33%) [06:25:29] maintenance-disconnect-full-disks build 196324 integration-agent-docker-1011 (/: 17%, /srv: 73%, /var/lib/docker: 55%): RECOVERY disk space OK [06:32:28] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [06:35:30] maintenance-disconnect-full-disks build 196326 integration-agent-docker-1011 (/: 17%, /srv: 97%, /var/lib/docker: 55%): OFFLINE due to disk space [06:38:27] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<22.22%) [06:40:30] maintenance-disconnect-full-disks build 196327 integration-agent-docker-1011 (/: 17%, /srv: 73%, /var/lib/docker: 55%): RECOVERY disk space OK [06:53:18] 10Phabricator: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10Aklapper) > Dunno if this is actually a Phabricator (Upstream) bug, or a WMF issue If you want to find out, feel free to go to something like https://secure.phabricator.com/T12345 in upstr... [07:34:01] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: CRITICAL: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found)deployment-prep.deployment-snapshot01.diskspace.root.byte_percentfree (<10.00%) [07:44:00] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [08:07:26] Reedy: you did https://phabricator.wikimedia.org/T258541 last time, mind a look? [08:28:26] PROBLEM - Host deployment-changeprop is DOWN: CRITICAL - Host Unreachable (172.16.5.21) [08:28:26] PROBLEM - Host deployment-cpjobqueue is DOWN: CRITICAL - Host Unreachable (172.16.4.124) [08:28:26] PROBLEM - Host deployment-chromium02 is DOWN: CRITICAL - Host Unreachable (172.16.4.14) [08:37:03] 10Beta-Cluster-Infrastructure, 10Operations: deployment-puppetmaster04: git-sync-upstream is failing with a merge conflict since 2020-07-17T08:50:01Z - https://phabricator.wikimedia.org/T258451 (10jbond) @Krinkle I have manuly resolved the conflicts and things look good to me, is someone able to confirm that t... [10:24:52] James_F: are you able to look at Gerrit logs or is it a pain? [10:25:29] It's a pain. Also, not my team any more. [10:25:33] Sorry. :-( [10:25:40] ah, ok [10:26:10] Tyler or Dan or Antoine are the experts in that stuff, I believe. [10:59:58] 10Phabricator, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Developer Productivity: Enhance phabricator project metrics and reporting - https://phabricator.wikimedia.org/T255703 (10Aklapper) I'd guess the most related upstream ticket would be https://secu... [11:00:36] PROBLEM - Host deployment-mcs01 is DOWN: CRITICAL - Host Unreachable (172.16.5.64) [11:01:54] (03PS1) 10Addshore: Edit Repo Config [wikidata/query/gui-deploy] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/615428 [11:10:28] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<11.11%) [11:25:25] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [11:55:29] maintenance-disconnect-full-disks build 196390 integration-agent-docker-1011 (/: 17%, /srv: 99%, /var/lib/docker: 75%): OFFLINE due to disk space [12:00:28] maintenance-disconnect-full-disks build 196391 integration-agent-docker-1011 (/: 17%, /srv: 37%, /var/lib/docker: 48%): RECOVERY disk space OK [12:18:54] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10Tgr) >>! In T257218#6317485, @thcipriani wrote: > Yes, this logo is a mash up of the public domain community logo + the git-scm logo. IANAL, but this link seems relevant: https://creativecommons.org/fa... [13:01:47] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "I have never seen this sniff before, but I like it very much. The additions done in this patch look fine, as far as I can tell." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/611222 (https://phabricator.wikimedia.org/T257710) (owner: 10DannyS712) [13:10:31] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "I would love to see this simplification in all codebases. Even if this means we are not checking .inc files any more. What's the worst thi" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/614285 (https://phabricator.wikimedia.org/T200956) (owner: 10Umherirrender) [13:10:58] 10Diffusion, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Developer Productivity: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 (10fgiunchedi) Ping/bump ? Looking forward to stop gerritbo... [13:13:55] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+2] "All this does is, as far as I'm concerned, making the test files in this codebase a little more sane and easier to understand. This does n" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/614092 (owner: 10Esanders) [13:15:06] (03Merged) 10jenkins-bot: Set tab width correctly in test runner [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/614092 (owner: 10Esanders) [13:17:52] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "Script looks fine. How couldn't it. It's so tiny. ;-)" (032 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/424970 (owner: 10Legoktm) [13:28:35] 10MediaWiki-Codesniffer, 10Patch-For-Review, 10User-DannyS712: Treat assertEmpty() as problematic in PHPUnitAssertEquals - https://phabricator.wikimedia.org/T246674 (10thiemowmde) I'm still not really sure if it's a good idea to //block// people from using assertEmpty. I mean, it's still doing an assertion,... [14:00:28] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<33.33%) [14:00:30] maintenance-disconnect-full-disks build 196415 integration-agent-docker-1011 (/: 18%, /srv: 100%, /var/lib/docker: 62%): OFFLINE due to disk space [14:04:00] 10Phabricator: Set visibility policy of the Phabricator's Countdown application to "public" - https://phabricator.wikimedia.org/T258106 (10valerio.bozzolan) >>! In T258106#6315042, @Aklapper wrote: > Ah, looks like the application's "Can Use Application" policy set to "Trusted-Contributors (Project)" overwrites... [14:05:28] maintenance-disconnect-full-disks build 196416 integration-agent-docker-1011 (/: 18%, /srv: 28%, /var/lib/docker: 40%): RECOVERY disk space OK [14:10:27] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [14:27:43] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10ema) p:05Triage→03High [14:32:36] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10ema) Due to this issue I have so far been pointing people to the github mirrors of our repositories so that they can clone things and look around. Clearly not optimal. [14:36:27] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<44.44%) [14:40:16] 10Beta-Cluster-Infrastructure, 10Operations: deployment-puppetmaster04: git-sync-upstream is failing with a merge conflict since 2020-07-17T08:50:01Z - https://phabricator.wikimedia.org/T258451 (10Mholloway) 05Open→03Resolved a:03Mholloway Yes, this is working well again, thank you @jbond. [14:40:20] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10Mholloway) [14:40:30] 10Beta-Cluster-Infrastructure, 10Operations: deployment-puppetmaster04: git-sync-upstream is failing with a merge conflict since 2020-07-17T08:50:01Z - https://phabricator.wikimedia.org/T258451 (10Mholloway) a:05Mholloway→03jbond [14:40:56] (03PS1) 10Edward Tadros: Selenium: Remove Jon from MinervaNeue notifications [integration/config] - 10https://gerrit.wikimedia.org/r/615503 (https://phabricator.wikimedia.org/T258528) [14:46:26] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [14:46:49] 10Phabricator: Add a Phabricator policy capability "Can create Countdowns" - https://phabricator.wikimedia.org/T258599 (10valerio.bozzolan) [14:48:43] 10Beta-Cluster-Infrastructure, 10Operations: deployment-puppetmaster04: git-sync-upstream is failing with a merge conflict since 2020-07-17T08:50:01Z - https://phabricator.wikimedia.org/T258451 (10jbond) 05Resolved→03Open Great thanks, resolving [14:48:46] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10jbond) [14:48:51] 10Beta-Cluster-Infrastructure, 10Operations: deployment-puppetmaster04: git-sync-upstream is failing with a merge conflict since 2020-07-17T08:50:01Z - https://phabricator.wikimedia.org/T258451 (10jbond) 05Open→03Resolved [14:48:53] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10jbond) [14:49:53] 10Phabricator: Set visibility policy of the Phabricator's Countdown application to "public" - https://phabricator.wikimedia.org/T258106 (10valerio.bozzolan) See {T258599}. [14:54:10] 10Phabricator: Add a Phabricator policy capability "Can create Countdowns" - https://phabricator.wikimedia.org/T258599 (10valerio.bozzolan) [15:04:48] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10Reedy) >>! In T258529#6325050, @Aklapper wrote: >> Dunno if this is actually a Phabricator (Upstream) bug, or a WMF issue > If you want to find out, fe... [15:05:48] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10RhinosF1) This was declinded upstream was it not [15:06:46] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10RhinosF1) https://discourse.phabricator-community.org/t/phabricator-does-not-respect-have-a-request-desktop-site-function/3053 [15:06:49] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10Majavah) Its broken for me too with Android 10 and Firefox 68 [15:08:53] Reedy, Majavah: might be worth trying to convince upstream about it [15:09:14] either that or ask them to add all missing features to mobile [15:09:35] heh [15:09:41] file seperate bugs for each of them [15:09:43] it'll get annoying [15:10:17] I don't have an account on upstream [15:10:25] Majavah: yeah we could just file lots of bugs until they hate us on discourse [15:10:45] I seriously find advanced things much harder to find on mobile [15:11:18] I use desktop mode for like everything [15:21:57] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10Reedy) For me, the inconsistency is annoying. If "request desktop website" didn't work on any device, that's one thing (and they can say they didn't w... [15:22:00] PROBLEM - Parsoid on deployment-parsoid11 is CRITICAL: connect to address 172.16.1.115 and port 8000: Connection refused [15:42:31] 10Release-Engineering-Team, 10Analytics, 10Analytics-EventLogging, 10dev-images, 10Patch-For-Review: EventLogging dev image should have verbose output enabled - https://phabricator.wikimedia.org/T257378 (10Milimetric) 05Declined→03Open p:05Triage→03High my bad, we were speed-grooming. This is ac... [15:42:38] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10Aklapper) p:05Triage→03Low [16:51:05] (03PS1) 10Esanders: Enable Generic.ControlStructures.DisallowYodaConditions rule [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/615523 [17:06:28] (03CR) 10Jforrester: [C: 03+1] Enable Generic.ControlStructures.DisallowYodaConditions rule [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/615523 (owner: 10Esanders) [17:25:31] maintenance-disconnect-full-disks build 196456 integration-agent-docker-1009 (/: 17%, /srv: 100%, /var/lib/docker: 51%): OFFLINE due to disk space [17:29:22] PROBLEM - Free space - all mounts on integration-agent-docker-1009 is CRITICAL: CRITICAL: integration.integration-agent-docker-1009.diskspace._srv.byte_percentfree (<11.11%) [17:30:32] maintenance-disconnect-full-disks build 196457 integration-agent-docker-1009 (/: 17%, /srv: 78%, /var/lib/docker: 60%): RECOVERY disk space OK [17:32:11] 10Phabricator (Upstream), 10Mobile, 10Upstream: Phab doesn't respect "Request Desktop Website" browser setting - https://phabricator.wikimedia.org/T258529 (10mmodell) It essentially bases the mobile vs desktop choice on the width of the window. If you resize a desktop browser down small enough it will also t... [17:33:59] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Scap, 10Developer Productivity, and 2 others: Create a development & testing environment for scap + train deployment... - https://phabricator.wikimedia.org/T255695 [17:39:25] RECOVERY - Free space - all mounts on integration-agent-docker-1009 is OK: OK: All targets OK [19:11:36] 10Phabricator, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Developer Productivity: Enhance phabricator project metrics and reporting - https://phabricator.wikimedia.org/T255703 (10mmodell) [20:32:18] 10Gerrit, 10Patch-For-Review: Unbreak gerrit-replica web UI on gerrit2001 - https://phabricator.wikimedia.org/T256567 (10QChris) 05Open→03Declined Making the web UI work, would need more than just a few adaptions. Hence, declining. If we want to make it work at some point, it might be easier to run with t... [20:38:23] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: (Service Check Timed Out) [20:38:28] PROBLEM - Free space - all mounts on integration-agent-docker-1008 is CRITICAL: (Service Check Timed Out) [20:43:14] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [20:43:21] RECOVERY - Free space - all mounts on integration-agent-docker-1008 is OK: OK: All targets OK [20:48:17] !log restarted php7.2-fpm on deployment-mediawiki-{07,09} for T258628 [20:48:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:48:19] T258628: Call to a member function getFullURL() on null - https://phabricator.wikimedia.org/T258628 [21:04:25] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<11.11%) [21:19:26] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [21:52:33] 10Gerrit: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10QChris) Yup, that's not good. However, the URL can currently not be configured. It's hardwired to SSH, if SSH is available. Not sure if anyone is working on it or planning to work on it. Bu... [21:54:52] I want to have scap run a command that will import variables from the repository into the running application on promote, this is all doable from existing checks but this has to run as a specific user. Should i look into changing the deploy user on this repo, or using a restricted sudo rule from deploy-service user? [22:55:29] maintenance-disconnect-full-disks build 196522 integration-agent-docker-1011 (/: 18%, /srv: 99%, /var/lib/docker: 41%): OFFLINE due to disk space [23:00:28] maintenance-disconnect-full-disks build 196523 integration-agent-docker-1011 (/: 18%, /srv: 36%, /var/lib/docker: 40%): RECOVERY disk space OK [23:00:28] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<11.11%) [23:05:27] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [23:54:19] PROBLEM - Host deployment-sentry01 is DOWN: CRITICAL - Host Unreachable (172.16.5.16)