[00:03:12] (03PS3) 10Jforrester: [DNM] layout: Drop HHVM testing from all quibble jobs [integration/config] - 10https://gerrit.wikimedia.org/r/534521 (https://phabricator.wikimedia.org/T192166) [00:06:45] (03PS3) 10Jforrester: layout: Collapse -nohhvm jobs into their base as we've dropped that [integration/config] - 10https://gerrit.wikimedia.org/r/534522 [00:10:33] (03PS2) 10Jforrester: layout: Drop all HHVM testing except for Fundraising [integration/config] - 10https://gerrit.wikimedia.org/r/534523 [01:20:22] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [01:25:22] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [02:04:50] 10Phabricator (Upstream), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10epriestley) Since the downsides are fairly minor, I've landed the workaround above into the Phabricator upstream in PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [02:57:22] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [03:05:28] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn) [03:58:23] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [04:28:23] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [04:38:23] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [04:49:23] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [05:29:23] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [06:09:39] PROBLEM - Puppet errors on deployment-cache-text05 is CRITICAL: CRITICAL: 5.56% of data above the critical threshold [3.0] [08:08:45] 10Gerrit, 10Release-Engineering-Team: Update JavaMelody to 1.79.0 - https://phabricator.wikimedia.org/T232678 (10hashar) The javamelody plugin is 1.75.0: ` name=plugins/javamelody/external_plugin_deps.bzl load("//tools/bzl:maven_jar.bzl", "maven_jar") def external_plugin_deps(): maven_jar( name =... [08:11:19] 10Gerrit, 10Release-Engineering-Team (Development services): Update JavaMelody on Gerrit to 1.79.0 - https://phabricator.wikimedia.org/T232678 (10hashar) p:05Triage→03Normal [08:13:14] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10serviceops-radar, 10Upstream: Gerrit account cache has a faulty reentrant lock causing http/sendemail threads to stall completely - https://phabricator.wikimedia.org/T224448 (10hashar) [08:30:18] hashar: good day, any chance https://gerrit.wikimedia.org/r/c/integration/config/+/535668 could be deployed today? [08:39:26] 10Continuous-Integration-Config, 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, and 5 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10kostajh) Is it possible to have the defau... [08:41:39] kostajh: yeah [08:41:48] kostajh: I guess I can just blindly trust what you wrote? :] [08:43:40] (03CR) 10Hashar: "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/535668 (owner: 10Kosta Harlan) [08:43:46] (03CR) 10Hashar: [C: 03+2] Revert "jjb: [mwext-codehealth*] Only clone ZUUL_PROJECT" [integration/config] - 10https://gerrit.wikimedia.org/r/535668 (owner: 10Kosta Harlan) [08:43:53] kostajh: done [08:44:01] Thx very much [08:45:51] (03Merged) 10jenkins-bot: Revert "jjb: [mwext-codehealth*] Only clone ZUUL_PROJECT" [integration/config] - 10https://gerrit.wikimedia.org/r/535668 (owner: 10Kosta Harlan) [08:50:00] hashar: as long as you're arbitrarily trusting my patches, https://gerrit.wikimedia.org/r/c/integration/config/+/534423 is ready to go too [08:50:30] I did test that new image locally at least :) [08:54:45] 10Continuous-Integration-Config, 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, and 5 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10hashar) >>! In T225730#5490128, @kostajh... [08:54:51] kostajh: ah yeah [08:54:53] doing so :] [08:55:24] kostajh: at this point, we should probably just give you proper right to the CI system :] [08:56:25] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Number of jobs generated: 4" [integration/config] - 10https://gerrit.wikimedia.org/r/534423 (owner: 10Kosta Harlan) [08:56:32] kostajh: I dont scale :D [08:56:39] anyway jobs have been switched to the new container [08:57:09] ha [08:57:17] Thank you! [08:58:33] (03Merged) 10jenkins-bot: jjb: [*-codehealth]: Use java8-sonar-scanner 0.4.12 [integration/config] - 10https://gerrit.wikimedia.org/r/534423 (owner: 10Kosta Harlan) [09:18:41] 10Phabricator, 10Release-Engineering-Team-TODO, 10serviceops, 10User-MModell: Mukunda to set up a meeting with service ops to discuss operational best practices for phabricator - https://phabricator.wikimedia.org/T232058 (10mmodell) 05Open→03Resolved [09:34:11] hashar: FYI I think Ericsson are going to look at redoing caffeine for gerrit 2.15+ [09:34:12] Since they forked it and applied caffeine to there fork. [09:34:13] But that may be a while [09:34:32] paladox: cool :] [09:34:44] paladox: as for the mysterious lock that disappear/does not show up, that is an entire mystery to me :-\ [09:35:37] Yup [09:36:25] hashar: it’s very hard to debug [09:36:36] As I carnt seem to reproduce it locally [09:40:46] hashar: I’m hopping the new hardware will help with this :) [09:45:18] paladox: who knows really but that really sounds like a code issue [09:45:30] what puzzles me is that the threaddump / jstack output does list threads has waiting for a lock [09:45:47] but there is no thread holding it (at lesat in the output) [09:50:12] Hmm [12:37:43] !sal [12:37:43] https://tools.wmflabs.org/sal/releng [12:39:51] !log integration-slave-docker-1043 : killed stall container # T232706 [12:39:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:39:54] T232706: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 [12:41:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Test-Coverage: mwcore-phpunit-coverage-master times out after 5 hours - https://phabricator.wikimedia.org/T232706 (10hashar) 05Open→03Resolved a:03hashar There was... [12:53:47] (03PS6) 10Hashar: api-testing: Add job template and non voting jobs [integration/config] - 10https://gerrit.wikimedia.org/r/535747 (https://phabricator.wikimedia.org/T230340) (owner: 10Holger Knust) [12:56:57] (03CR) 10Hashar: [C: 03+2] "Looks like that work properly. I have rebased the change and made the jobs non voting so they start triggering now ;]" (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/535747 (https://phabricator.wikimedia.org/T230340) (owner: 10Holger Knust) [12:59:39] (03Merged) 10jenkins-bot: api-testing: Add job template and non voting jobs [integration/config] - 10https://gerrit.wikimedia.org/r/535747 (https://phabricator.wikimedia.org/T230340) (owner: 10Holger Knust) [13:06:22] 10Phabricator, 10Release-Engineering-Team-TODO (201909), 10Wikimedia-Logstash, 10observability, and 2 others: Deploy phatality into kibana - https://phabricator.wikimedia.org/T230752 (10fgiunchedi) >>! In T230752#5487593, @mmodell wrote: > The way I deployed it in beta is fairly straightforward - I copied... [13:34:45] Reedy: Are you able to give me sysop on test2wiki? Or is it better if I file a Phab task? [13:34:53] I can jfdi [13:34:54] what username? [13:34:56] :) [13:34:58] Adamw [13:35:25] done [13:35:50] Reedy: Thanks! [13:36:09] * awight immediately reverts vandalism rollbacks ;-) [13:36:28] uh that didn't even make sense. Anyway, [j/k intended] [14:41:01] fyi it appears the PolyGerrit team are working on customisable dashboards. [14:44:04] (03PS1) 10Alexandros Kosiaris: Edit Project Config [deployment-charts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536607 [14:44:12] for both Project and User dashboards! [14:44:21] (03Abandoned) 10Alexandros Kosiaris: Edit Project Config [deployment-charts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536607 (owner: 10Alexandros Kosiaris) [14:49:08] hashar: Now that I look at https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/536580/2/includes/libs/rdbms/lbfactory/LBFactoryMulti.php again, the first wfDebugLog will be called regardless of whether the key exist. Is that intended? [14:51:38] Daimona: yeah I though about having a comparison point [14:51:44] but that might end up being too spammy :-\ [14:52:02] Isn't it too spammy at its current version? [14:52:04] I wanted to get a reference point to compare aain [14:52:13] I wanted to get a reference point to compare against [14:52:36] Because right now we may get a log entry from that point without then triggering the coredump [14:52:42] but all that logging might well be useless. Tim went with gdb and confirmed that the constant and the empty string array key are indeed the same value [14:53:25] Daimona: yes definitely [14:53:55] or I could have checked whether the $groupLoads temporary variable actually has an empty string as a key [14:54:33] It's probably useless, but... [14:55:00] I was thinking about another array_key_exists right inside the if (line 203) [16:02:27] 10Diffusion, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 (10mmodell) @greg: not really but currently refs/changes are not, by default, "publishe... [16:06:28] PROBLEM - Host integration-agent-docker-1001 is DOWN: CRITICAL - Host Unreachable (172.16.3.36) [16:08:46] 10MediaWiki-Releasing, 10Epic: Expand the set of bundled skins - https://phabricator.wikimedia.org/T194266 (10CCicalese_WMF) Untagging from a particular release and making into an #epic. Requests to bundle skins into particular releases are subtasks. [16:12:23] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) p:05Triage→03Normal [16:14:37] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic: Expand the set of bundled extensions and skins to achieve a default MediaWiki experience that's comparable to Wikimedia sites - https://phabricator.wikimedia.org/T178349 (10CCicalese_WMF) [16:15:12] 10MediaWiki-Releasing, 10Epic: Expand the set of bundled skins - https://phabricator.wikimedia.org/T194266 (10CCicalese_WMF) [16:15:14] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic: Expand the set of bundled extensions and skins to achieve a default MediaWiki experience that's comparable to Wikimedia sites - https://phabricator.wikimedia.org/T178349 (10CCicalese_WMF) [16:18:27] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:18:56] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:19:24] !log Zuul: Add initial jobs for mediawiki/tools/api-testing T230340 [16:19:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:19:27] T230340: Set up CI for mediawiki/tools/api-testing - https://phabricator.wikimedia.org/T230340 [16:21:22] 10MediaWiki-Releasing, 10MediaWiki-extensions-Scribunto, 10MW-1.34-release: Bundle Scribunto extension with MediaWiki - https://phabricator.wikimedia.org/T191737 (10CCicalese_WMF) Any update on the license question? [16:22:25] 10MediaWiki-Releasing, 10Growth-Team, 10Notifications, 10MW-1.34-release: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10CCicalese_WMF) What is the situation with respect to database support? [16:23:26] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.35-release: Enhance installer to support dependencies between skins and extensions - https://phabricator.wikimedia.org/T196670 (10CCicalese_WMF) Yes, this and the tasks that are dependent upon it going to have to wait until 1.35. [16:24:14] 10MediaWiki-Releasing, 10MinervaNeue, 10MW-1.35-release, 10Readers-Web-Backlog (Tracking): Bundle Minerva Neue skin with MediaWiki - https://phabricator.wikimedia.org/T191743 (10CCicalese_WMF) [16:25:13] 10MediaWiki-Releasing, 10MinervaNeue, 10MW-1.35-release, 10Readers-Web-Backlog (Tracking): Bundle Minerva Neue skin with MediaWiki - https://phabricator.wikimedia.org/T191743 (10CCicalese_WMF) [16:25:15] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:26:49] 10MediaWiki-Releasing, 10MobileFrontend, 10MW-1.35-release, 10Readers-Web-Backlog (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734 (10CCicalese_WMF) [16:27:23] 10MediaWiki-Releasing, 10MobileFrontend, 10MW-1.35-release, 10Readers-Web-Backlog (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734 (10CCicalese_WMF) [16:27:25] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:27:27] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:29:27] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:29:48] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:31:13] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10CPT Initiatives (API Integration Tests), and 2 others: Set up CI for mediawiki/tools/api-testing - https://phabricator.wikimedia.org/T230340 (10Jdforrester-WMF) OK, the tests are... [16:31:29] mholloway@mholloway:~/code/wikimedia/kartotherian$ curl https://blubberoid.wikimedia.org/v1/test [16:31:30] Failed to read 'application/json' config from request body. Error: unexpected end of JSON input [16:31:45] ^ seems broken [16:34:49] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.34-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) >>! In T191740#4682331, @Daimona wrote: > Works with web installer. The DB part is dependent on several ta... [16:35:33] 10MediaWiki-Releasing, 10MediaWiki-extensions-LoginNotify, 10MW-1.34-release: Bundle LoginNotify extension with MediaWiki - https://phabricator.wikimedia.org/T191746 (10CCicalese_WMF) [16:36:25] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) >>! In T232538#5490768, @gerritbot wrote: > Change 536598 had... [16:37:51] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:37:53] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:38:38] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:39:30] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:39:34] 10MediaWiki-Releasing, 10Growth-Team, 10MediaWiki-Installer, 10Epic, 10MW-1.32-release: Expand the set of bundled extensions and skins in MediaWiki 1.32 - https://phabricator.wikimedia.org/T196650 (10Jdforrester-WMF) [16:39:46] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10Jdforrester-WMF) [16:39:49] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10Jdforrester-WMF) [16:41:32] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:41:40] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:44:22] 10MediaWiki-Releasing, 10MediaWiki-extensions-Scribunto, 10MW-1.34-release: Bundle Scribunto extension with MediaWiki - https://phabricator.wikimedia.org/T191737 (10Krinkle) [16:47:02] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle PageImages with MediaWiki - https://phabricator.wikimedia.org/T229128 (10CCicalese_WMF) [16:48:22] 10MediaWiki-Releasing, 10MediaWiki-extensions-LoginNotify, 10MW-1.34-release: Bundle LoginNotify extension with MediaWiki - https://phabricator.wikimedia.org/T191746 (10CCicalese_WMF) p:05Triage→03Normal [16:48:28] 10MediaWiki-Releasing, 10Growth-Team, 10Thanks, 10MW-1.34-release: Bundle Thanks extension with MediaWiki - https://phabricator.wikimedia.org/T191739 (10CCicalese_WMF) p:05Triage→03Normal [16:48:34] 10MediaWiki-Releasing, 10AntiSpoof, 10MW-1.34-release: Bundle AntiSpoof extension with MediaWiki - https://phabricator.wikimedia.org/T191736 (10CCicalese_WMF) p:05Triage→03Normal [16:48:44] 10MediaWiki-Releasing, 10MediaWiki-extensions-Scribunto, 10MW-1.34-release: Bundle Scribunto extension with MediaWiki - https://phabricator.wikimedia.org/T191737 (10CCicalese_WMF) p:05Triage→03Normal [16:48:49] 10MediaWiki-Releasing, 10Growth-Team, 10Notifications, 10MW-1.34-release: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10CCicalese_WMF) p:05Triage→03Normal [16:48:59] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.34-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) p:05Triage→03Normal [16:49:03] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle TextExtracts with MediaWiki - https://phabricator.wikimedia.org/T229127 (10CCicalese_WMF) p:05Triage→03Normal [16:49:08] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle PageImages with MediaWiki - https://phabricator.wikimedia.org/T229128 (10CCicalese_WMF) p:05Triage→03Normal [16:49:33] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.34-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10Daimona) @CCicalese_WMF It depends... Right now we only have two known incompatibilities with Postgres. For the first on... [16:49:51] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.34-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10DannyS712) >>! In T191740#5491206, @CCicalese_WMF wrote: >>>! In T191740#4682331, @Daimona wrote: >> Works with web insta... [16:52:07] 10MediaWiki-Releasing, 10MediaWiki-extensions-Scribunto, 10MW-1.34-release: Bundle Scribunto extension with MediaWiki - https://phabricator.wikimedia.org/T191737 (10Anomie) Ping #wmf-legal? I don't see there being an answer to that question until an actual tech lawyer steps up. And it wouldn't surprise me mu... [16:53:19] 10Phabricator: Not able to edit Phabricator task T218184: ERR_SPDY_PROTOCOL_ERROR - https://phabricator.wikimedia.org/T232478 (10Aklapper) 05Stalled→03Declined Thanks for the feedback! Declining as this cannot be reproduced anymore. [16:53:36] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) Thank you, @Daimona and @DannyS712! I will bump to 1.35 and have requested CPT review for the patches rela... [16:53:50] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:53:52] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:53:55] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) [16:54:08] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10CCicalese_WMF) [16:54:17] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) [16:58:54] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) [16:59:49] (03CR) 10Brennen Bearnes: "This change is ready for review." [releng/dev-images] - 10https://gerrit.wikimedia.org/r/525842 (https://phabricator.wikimedia.org/T222494) (owner: 10Brennen Bearnes) [17:03:43] 10MediaWiki-Releasing, 10Growth-Team, 10Notifications, 10MW-1.34-release: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10CCicalese_WMF) @Anomie, do you have any insight into the database support in Echo? [17:05:30] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle PageImages with MediaWiki - https://phabricator.wikimedia.org/T229128 (10CCicalese_WMF) @Jdforrester-WMF, could you please help assess this extension for bundling? [17:05:40] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle TextExtracts with MediaWiki - https://phabricator.wikimedia.org/T229127 (10CCicalese_WMF) @Jdforrester-WMF, could you please help assess this extension for bundling? [17:05:52] 10MediaWiki-Releasing, 10Growth-Team, 10Notifications, 10MW-1.34-release: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10Reedy) >>! In T191738#5491381, @CCicalese_WMF wrote: > @Anomie, do you have any insight into the database support in Echo? I think sqlite and mysql... [17:41:07] hauskatze https://gerrit-review.googlesource.com/c/plugins/rename-project/+/236553 :) [17:43:43] paladox: oh, awesome :) [17:43:57] yup! [17:44:18] Once we upgrade our Gerrit Install too [17:46:52] hauskatze well that works on gerrit 2.15 :P [17:51:20] gerrit 2 is being EOL'd so we'll update anyways right? [17:53:46] gerrit 2.15 is EOL in novemeber [17:53:52] 2.16 will still be supported [17:55:23] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-jhuneidi is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 hphp_invoke - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 353 bytes in 0.046 second response time [18:04:13] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10Daimona) @CCicalese_WMF Thanks! I'd also appreciate a lot some review for T220791, which is a bit more delicate... As fo... [18:14:14] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10DannyS712) @Daimona would you be willing to add me to T152394 and/or T187731? [18:15:54] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10DBA, and 2 others: Make PHD run on the backup phabricator server (phab2001, currently) - https://phabricator.wikimedia.org/T232883 (10mmodell) [18:26:58] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle TextExtracts with MediaWiki - https://phabricator.wikimedia.org/T229127 (10Jdforrester-WMF) >>! In T229127#5491384, @CCicalese_WMF wrote: > @Jdforrester-WMF, could you please help assess this extension for bundling? It seems like a good... [18:34:09] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MW-1.34-release: Bundle PageImages with MediaWiki - https://phabricator.wikimedia.org/T229128 (10Jdforrester-WMF) [18:41:21] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10thcipriani) >>! In T232538#5491215, @ssastry wrote: >>>! In T232538#549... [18:47:27] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10Daimona) >>! In T191740#5491715, @DannyS712 wrote: > @Daimona would you be willing to add me to T152394 and/or T187731?... [18:54:44] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Reimage both phab1001 and phab2001 to stretch - https://phabricator.wikimedia.org/T190568 (10Dzahn) 05Open→03Stalled [18:54:49] 10Phabricator, 10PHP 7.2 support, 10Patch-For-Review: Move Phabricator from PHP 7.0 to PHP 7.2 - https://phabricator.wikimedia.org/T151070 (10Dzahn) [18:54:52] 10Phabricator, 10Performance: /maniphest/report/project/ : Maximum execution time of 10 seconds exceeded - https://phabricator.wikimedia.org/T125357 (10Dzahn) [18:54:55] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations, 10serviceops, and 3 others: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832 (10Dzahn) [18:56:09] 10Phabricator, 10Release-Engineering-Team (Development services), 10Operations, 10hardware-requests, 10serviceops: The server, WMF7426, was given to us temporarily, we would like to make it permanent - https://phabricator.wikimedia.org/T232887 (10mmodell) [18:56:31] 10Phabricator, 10Release-Engineering-Team (Development services), 10Operations, 10hardware-requests, 10serviceops: The phabricator server, WMF7426, was given to us temporarily, we would like to make it permanent - https://phabricator.wikimedia.org/T232887 (10mmodell) [18:57:00] 10MediaWiki-Releasing, 10Growth-Team, 10Notifications, 10MW-1.34-release: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10Anomie) >>! In T191738#5491386, @Reedy wrote: > Postgres if it works, is more by accident (ie the sql works there too) than any sort of planning It... [19:01:53] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) [19:11:11] 10Gerrit, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Zuul: Allow hiding of non-discussion comments in Gerrit - https://phabricator.wikimedia.org/T48148 (10hashar) Gave it a try on https://gerrit.wikimedia.org/r/#/c/test/gerrit-ping/+/226272/ By using: ` gerrit r... [19:11:16] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10DBA: Switch phabricator production to codfw - https://phabricator.wikimedia.org/T164810 (10Dzahn) 05Open→03Stalled [19:14:43] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201909), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10Dzahn) [19:14:47] 10Phabricator, 10Operations, 10Traffic: Make phame cacheable - https://phabricator.wikimedia.org/T219978 (10Dzahn) [19:15:08] 10Phabricator, 10Operations, 10Traffic: Make phame cacheable - https://phabricator.wikimedia.org/T219978 (10Dzahn) merging into T226044 [19:18:27] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201909), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10mmodell) == From the merged task: Blog posts on phame can... [19:19:17] I'm seeing lots of patches get merged for BlueSpice* - the jenkins jobs seem to be holding up other things and seem redundant since the patches are getting merged without Jenkins [19:19:23] Is there any way to speed this up? I [19:19:42] I have potential train blockers that are taking 3 hrs to get merged by Jenkins [19:20:04] or maybe I'm misreading what's causing the delays here? [19:21:13] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201909), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10mmodell) a:05mmodell→03None This is unblocked on my en... [19:22:24] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Availability, and 2 others: Deploy phabricator to phab2001.codfw.wmnet - https://phabricator.wikimedia.org/T137928 (10Dzahn) 05Stalled→03Open [19:22:26] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10DBA: Switch phabricator production to codfw - https://phabricator.wikimedia.org/T164810 (10Dzahn) [19:22:29] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) [19:22:34] 10Phabricator, 10RelEng-Archive-FY201718-Q1, 10Operations: reinstall iridium (phabricator) as phab1001 with jessie - https://phabricator.wikimedia.org/T152129 (10Dzahn) [19:22:40] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201909), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10mmodell) Also important, @epriestley's comment at T219978#... [19:31:51] thcipriani hashar so are you effectively recommending renaming then? Which means spinning up a new instance and removing the old one and updating puppet references and any other configs and wikipages that might directly refer to that instance? i am trying to evaluate what is simpler and also better long-term maintenance wise. [19:32:18] subbu: maybe #wmcs people can just rename it? [19:32:36] else yeah, would be easier longterm to have it prefixed 'deployment-mediawiki-' :-\ [19:34:20] @hashar any idea what's slowing down CI today? [19:35:42] hashar, I am sitting here with andrewbogott and he doesn't recommend trying to renaming on the backend. ( andrewbogott ref: T232538 ) [19:35:43] T232538: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 [19:41:09] (03PS3) 10Jforrester: build: Upgrade mediawiki-codesniffer from 0.8.0 to 19.1.0 [tools/release] - 10https://gerrit.wikimedia.org/r/535595 (https://phabricator.wikimedia.org/T232473) (owner: 10Hashar) [19:41:11] (03PS1) 10Jforrester: build: Upgrade php-parallel-lint from 0.9.* to 1.0.0 [tools/release] - 10https://gerrit.wikimedia.org/r/536666 [19:41:13] (03PS1) 10Jforrester: build: Upgrade mediawiki-codesniffer from 19.1.0 to 26.0.0 and make pass [tools/release] - 10https://gerrit.wikimedia.org/r/536667 [19:41:15] (03PS1) 10Jforrester: composer: Explicitly state that we're trying to support PHP 5.6.99 (i.e. HHVM) for now [tools/release] - 10https://gerrit.wikimedia.org/r/536668 [19:42:05] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201909), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10Krinkle) [19:43:53] (03CR) 10Jforrester: [C: 03+2] build: Upgrade mediawiki-codesniffer from 0.8.0 to 19.1.0 (033 comments) [tools/release] - 10https://gerrit.wikimedia.org/r/535595 (https://phabricator.wikimedia.org/T232473) (owner: 10Hashar) [19:44:01] (03CR) 10Jforrester: [C: 03+2] build: Upgrade php-parallel-lint from 0.9.* to 1.0.0 [tools/release] - 10https://gerrit.wikimedia.org/r/536666 (owner: 10Jforrester) [19:44:19] subbu: :-\ [19:44:57] subbu: well I would recommend using deployment-mediawiki as a prefix so :^/ Unless one introduces a new puppet classes that makes it absolutely trivial to define an instance has been a beta cluster mediawiki target [19:45:11] (03Merged) 10jenkins-bot: build: Upgrade mediawiki-codesniffer from 0.8.0 to 19.1.0 [tools/release] - 10https://gerrit.wikimedia.org/r/535595 (https://phabricator.wikimedia.org/T232473) (owner: 10Hashar) [19:45:13] (03Merged) 10jenkins-bot: build: Upgrade php-parallel-lint from 0.9.* to 1.0.0 [tools/release] - 10https://gerrit.wikimedia.org/r/536666 (owner: 10Jforrester) [19:45:15] which really could jut be a role::beta::mediawiki::appserver that would include all the classes currently applied [19:45:30] (03CR) 10Jforrester: [C: 03+2] build: Upgrade mediawiki-codesniffer from 19.1.0 to 26.0.0 and make pass [tools/release] - 10https://gerrit.wikimedia.org/r/536667 (owner: 10Jforrester) [19:45:34] (03CR) 10Jforrester: [C: 03+2] composer: Explicitly state that we're trying to support PHP 5.6.99 (i.e. HHVM) for now [tools/release] - 10https://gerrit.wikimedia.org/r/536668 (owner: 10Jforrester) [19:45:40] Jdlrobson: it is crippled by multiples ongoing issues yes :-\ [19:46:09] (03Merged) 10jenkins-bot: build: Upgrade mediawiki-codesniffer from 19.1.0 to 26.0.0 and make pass [tools/release] - 10https://gerrit.wikimedia.org/r/536667 (owner: 10Jforrester) [19:46:34] (03Merged) 10jenkins-bot: composer: Explicitly state that we're trying to support PHP 5.6.99 (i.e. HHVM) for now [tools/release] - 10https://gerrit.wikimedia.org/r/536668 (owner: 10Jforrester) [19:47:14] (03CR) 10Hashar: "Thank you James and Daimona!" [tools/release] - 10https://gerrit.wikimedia.org/r/535595 (https://phabricator.wikimedia.org/T232473) (owner: 10Hashar) [19:49:17] 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Release-Tools, 10PHP 7.3 support, 10Patch-For-Review: PHP CodeSniffer on mediawiki/tools/release warns when running with php 7.3 - https://phabricator.wikimedia.org/T232473 (10hashar) 05Open→03Resolved ` $ php --version PHP 7.3.4-2 (cli)... ` ` $ c... [19:49:42] ah yeah hmm [19:49:44] zuul queue is crazy [19:51:45] (03PS8) 10Jforrester: [WIP] build: Clean up some PHPCS exclusions [tools/release] - 10https://gerrit.wikimedia.org/r/535588 (https://phabricator.wikimedia.org/T232473) (owner: 10Daimona Eaytoy) [19:52:53] (03CR) 10Jforrester: "Hey, I've hi-jacked this to be just the PHPCS improvements and done the others differently; hope that's OK." [tools/release] - 10https://gerrit.wikimedia.org/r/535588 (https://phabricator.wikimedia.org/T232473) (owner: 10Daimona Eaytoy) [19:56:42] :( [19:56:50] (03PS1) 10Jforrester: [DNM] Move to PHP7.2+ support [tools/release] - 10https://gerrit.wikimedia.org/r/536670 [19:57:16] (03PS2) 10Jforrester: Run CI for make-tarball-release again, once HHVM is no longer supported [tools/release] - 10https://gerrit.wikimedia.org/r/528191 [19:57:18] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Move to PHP7.2+ support [tools/release] - 10https://gerrit.wikimedia.org/r/536670 (owner: 10Jforrester) [19:57:35] (03PS3) 10Jforrester: Run CI for make-tarball-release again, once HHVM is no longer supported [tools/release] - 10https://gerrit.wikimedia.org/r/528191 [19:58:02] (03CR) 10jerkins-bot: [V: 04-1] Run CI for make-tarball-release again, once HHVM is no longer supported [tools/release] - 10https://gerrit.wikimedia.org/r/528191 (owner: 10Jforrester) [19:58:33] (03CR) 10Daimona Eaytoy: "> Hey, I've hi-jacked this to be just the PHPCS improvements and done" [tools/release] - 10https://gerrit.wikimedia.org/r/535588 (https://phabricator.wikimedia.org/T232473) (owner: 10Daimona Eaytoy) [20:01:57] paladox: Do you know if it's easy/possible to add extra syntax highlight themes in Gerrit? [20:02:25] Krinkle GWTUI uses codemirror. PolyGerrit uses highlight.js. Which ui do you use? [20:02:37] Old UI still, waiting for 2.16 and dashboards [20:02:38] gerrit 2.16 has added more syntax highlighting themes to PolyGerrit's ui! [20:02:59] Krinkle which file extension are you looking to add? [20:03:43] https://usercontent.irccloud-cdn.com/file/ax6JEyCI/current.png https://usercontent.irccloud-cdn.com/file/J3ceucjc/expected.png [20:03:47] 10Phabricator (Upstream), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10mmodell) 05Open→03Resolved a:03mmodell Thank you, @epriestley, for patching over chrome's lack of proper QA. Really heroic... [20:03:53] I mean the diff handling, not the syntax, sorry. [20:04:10] 10Phabricator (Upstream), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10mmodell) a:05mmodell→03epriestley [20:04:31] 10Phabricator (Upstream), 10Release-Engineering-Team-TODO (201909), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10Jdforrester-WMF) [20:04:42] Oh [20:04:43] Right now the Red and Green are relatively dark and hard to differentiate for some people. The red should be lighter and more towards yellow, and the green lighter and more towards blue to resolve this (we did this for Wikipedia as well, much stronger even) [20:05:11] I think you can change this if you know the css class it's applying [20:05:34] yeah, but is it something the syntax themes can do so that I can make it a preference? [20:05:38] 10MediaWiki-Releasing, 10MediaWiki-extensions-Scribunto, 10MW-1.34-release: Bundle Scribunto extension with MediaWiki - https://phabricator.wikimedia.org/T191737 (10Jdforrester-WMF) I think it's reasonable to proceed and let Debian do what they may. [20:05:41] (without changing Gerrit default/core hack) [20:05:48] we can see if overriding the styles in https://github.com/wikimedia/puppet/blob/production/modules/gerrit/files/homedir/review_site/etc/GerritSite.css will work [20:06:07] oh [20:06:18] some of the color themes already change the diff styles as well [20:06:26] https://usercontent.irccloud-cdn.com/file/dDqZVYA7/paradiso.png [20:06:36] so it seems to be part of the syntax highlight plugins [20:06:38] don't know for sure [20:06:59] Krinkle it's using codemirror (so uses the version from gerrit core) [20:07:42] the codemirror-editor plugin is for PolyGerrit only [20:09:31] Krinkle i doin't think it is possible to make it per user (styling). [20:10:05] paladox: Is it possible to add an extra theme? [20:10:10] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Release-Engineering-Team (Kanban), 10Zuul, 10Patch-For-Review: Zuul cancels all changes when a change is manually merged - https://phabricator.wikimedia.org/T203846 (10hashar) 05Resolved→03Open The patch I wrote is wrong. In `zuu``l/connection/gerr... [20:10:33] https://usercontent.irccloud-cdn.com/file/J1rm1iTn/diff-pref.png [20:11:10] Krinkle good question. [20:12:00] Nope :(https://github.com/GerritCodeReview/gerrit/blob/stable-2.15/lib/codemirror/cm.bzl#L41 [20:12:03] * https://github.com/GerritCodeReview/gerrit/blob/stable-2.15/lib/codemirror/cm.bzl#L41 [20:14:48] paladox: OK. but this has helped. [20:14:56] oh [20:15:00] I see that each active theme uses a document class to trigger the theme [20:15:04] Krinkle it'll be easier under PolyGerrit. [20:15:13] e.g. ".cm-s-eclipse span.cm-comment" [20:15:17] for theme eclipse [20:15:19] yeh. [20:15:31] which means from our own GerritSite.css I could change one of the themes nobody cares about to be my custom theme [20:15:35] without disrupting the others [20:15:49] yup [20:23:20] 10Gerrit: Gerrit red-green diff make the code hard to read (impossible for color-blind users) - https://phabricator.wikimedia.org/T232893 (10Krinkle) [20:24:37] RECOVERY - Puppet errors on deployment-cache-text05 is OK: OK: Less than 1.00% above the threshold [2.0] [20:25:17] ohh, i think that was fixed under PolyGerrit. [20:30:24] 10Phabricator (Upstream), 10Release-Engineering-Team-TODO (201909), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10RHo) \o/ {F30320108} [20:31:21] Krinkle paladox how can i make the submit button show on Minerva? [20:31:34] i think we configured it off, but during these times of zuul slowdown I think we need it [20:31:35] Jdlrobson you need to add the submit right to that repo [20:31:41] is that a config thing? [20:32:25] Jdlrobson https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/MinervaNeue/+/536683/ [20:32:46] (03PS1) 10Jdlrobson: Edit Project Config [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536682 [20:32:47] <3 wow that easy huh? Thank you paladox for being so helpful as always :) [20:32:48] (03PS1) 10Paladox: Add submit button for mediawiki-skins-MinervaNeue [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536683 [20:32:56] Jdlrobson your welcome :) [20:33:04] (03PS2) 10Jdlrobson: Add submit button for mediawiki-skins-MinervaNeue [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536683 (owner: 10Paladox) [20:33:07] (03CR) 10Jdlrobson: [C: 03+2] Add submit button for mediawiki-skins-MinervaNeue [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536683 (owner: 10Paladox) [20:33:22] Jdlrobson needs to be v+2 and manually submitted. [20:33:25] (03Abandoned) 10Jdlrobson: Edit Project Config [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536682 (owner: 10Jdlrobson) [20:33:33] (03CR) 10Jdlrobson: [V: 03+2 C: 03+2] Add submit button for mediawiki-skins-MinervaNeue [skins/MinervaNeue] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536683 (owner: 10Paladox) [20:35:20] thanks paladox [20:35:23] :) [20:35:40] 10Phabricator (Upstream), 10Release-Engineering-Team-TODO (201909), 10Browser-Support-Google-Chrome, 10Upstream: Some Phabricator boards do not load cards anymore in Chrome 77 - https://phabricator.wikimedia.org/T232600 (10epriestley) Nice! Thanks for everyone's help hunting this one down, this was definit... [20:35:48] Jdlrobson: It was disabled because our patchset 'test' jobs set V+2 (not V+1). This means that, enabling Submit, means you now by default see a "Submit" button on any patch set with passing tests. Pressing this means you instantly merge a change, bypassing the gate, and breaking Zuul's internal model for the next 20-30min, possibly causing other repos and your own to be in a failing state on master due ot tests that we don't run on [20:35:49] 'test'. [20:35:57] So be very careful not to press it by accident :) [20:36:27] we should probably make it so that only gate jobs set V+2 and use V+1 for the patchset test. Then you can only press "Submit" if you manually set V+2 [20:39:09] For my own repos, I usually keep it disabled for that reason, and for the 1 or 2 times a year I do a breaking change that I need to merge with failing tests (e.g. to do something tricky that depends on changes in multiple repos), I do a config change in refs/config for a few minutes, and then disable it again. [20:39:13] Jdlrobson: this can be done from the GUI at https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/skins/MinervaNeue,access [20:50:37] and https://phabricator.wikimedia.org/T203846#5492011 [20:50:45] force merge are still disturbing zuul :-\ [20:50:57] but force merge is not the only reason it is being slow [20:52:28] 10Gerrit: Gerrit red-green diff make the code hard to read (impossible for color-blind users) - https://phabricator.wikimedia.org/T232893 (10Krinkle) p:05Triage→03Normal a:03Krinkle [21:03:52] 10Gerrit, 10Patch-For-Review: Gerrit red-green diff make the code hard to read (impossible for color-blind users) - https://phabricator.wikimedia.org/T232893 (10Krinkle) |default theme (unchanged) |-- |{F30320424} |eclipse theme (before) | eclipse theme (after) |--|-- |{F30320426}|{F30320425} [21:07:51] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.35-release: Expand the set of bundled extensions and skins in MediaWiki 1.35 - https://phabricator.wikimedia.org/T232854 (10CCicalese_WMF) p:05Triage→03Normal [21:08:34] 10MediaWiki-Releasing, 10AbuseFilter, 10MW-1.35-release, 10Patch-For-Review, 10User-Daimona: Bundle AbuseFilter extension with MediaWiki - https://phabricator.wikimedia.org/T191740 (10CCicalese_WMF) [21:12:25] (03PS1) 10Hashar: Fix 0018-wmf-ignore-submit-error-on-merged-change [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/536689 (https://phabricator.wikimedia.org/T203846) [21:15:26] (03PS1) 10Jforrester: make-release: Include Scribunto in 1.34+ [tools/release] - 10https://gerrit.wikimedia.org/r/536690 (https://phabricator.wikimedia.org/T191737) [21:15:39] (03CR) 10Hashar: [C: 03+2] Fix flaw in wmf-ignore-submit-error-on-merged-change (031 comment) [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/502984 (https://phabricator.wikimedia.org/T203846) (owner: 10Hashar) [21:15:50] I am stupid [21:15:54] :-\ [21:16:26] (03CR) 10Hashar: [C: 03+2] Fix 0018-wmf-ignore-submit-error-on-merged-change [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/536689 (https://phabricator.wikimedia.org/T203846) (owner: 10Hashar) [21:19:26] (03Merged) 10jenkins-bot: Fix 0018-wmf-ignore-submit-error-on-merged-change [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/536689 (https://phabricator.wikimedia.org/T203846) (owner: 10Hashar) [21:20:04] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Release-Engineering-Team (Kanban), 10Zuul, 10Patch-For-Review: Zuul cancels all changes when a change is manually merged - https://phabricator.wikimedia.org/T203846 (10hashar) Would then need to rebuild the Debian package get it on apt.wikimedia.org an... [21:21:58] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) An update. So, as per @thcipriani and @hashar's recommendatio... [21:23:24] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Release-Engineering-Team-TODO (201909), 10Zuul, 10Patch-For-Review: Zuul cancels all changes when a change is manually merged - https://phabricator.wikimedia.org/T203846 (10Jdforrester-WMF) [21:38:23] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909), 10CirrusSearch: Unit tests in mediawiki-config repo don't pass under PHP72, only HHVM - https://phabricator.wikimedia.org/T232691 (10Jdforrester-WMF) Tagging back in Cirrus for the remaining issues. [21:39:44] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) Okay, a further update. @Krenair had to remove some of the ro... [21:49:43] 10Gerrit, 10Patch-For-Review: Gerrit red-green diff make the code hard to read (impossible for color-blind users) - https://phabricator.wikimedia.org/T232893 (10hashar) I liked the colors Trevor proposed back in 2012: {F4276} And they are apparently still used. Should we align with MediaWiki colors? And mayb... [21:54:28] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10hashar) That puppet prefix is there to not have to remember to apply th... [21:58:45] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) >>! In T232538#5492275, @hashar wrote: > That puppet prefix is... [22:04:01] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10Krenair) >>! In T232538#5492295, @ssastry wrote: >>>! In T232538#549227... [22:08:01] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) Next update. I applied all those roles and ran `sudo puppet a... [22:08:22] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [22:23:56] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [extensions/GenerateMochaConfig] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536695 [22:23:58] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [extensions/GenerateMochaConfig] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536695 (owner: 10QChris) [22:24:03] (03PS1) 10QChris: Import done. Revoke import grants [extensions/GenerateMochaConfig] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536696 [22:24:05] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [extensions/GenerateMochaConfig] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/536696 (owner: 10QChris) [22:28:21] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [22:32:00] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) >>! In T232538#5492320, @ssastry wrote: > Next update. > > I... [22:39:23] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [22:40:51] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10Krenair) >>! In T232538#5492336, @ssastry wrote: > Looks like @Krenair... [22:45:37] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) 05Open→03Resolved a:03ssastry Ok, I verified that the in... [22:47:40] 10Phabricator, 10PHP 7.2 support: Move Phabricator from PHP 7.0 to PHP 7.2 - https://phabricator.wikimedia.org/T151070 (10Dzahn) [22:47:42] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10Dzahn) 05Stalled→03Open [22:47:47] 10Phabricator, 10Performance: /maniphest/report/project/ : Maximum execution time of 10 seconds exceeded - https://phabricator.wikimedia.org/T125357 (10Dzahn) [22:47:50] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Operations, 10serviceops, and 3 others: Apache on phab1001 is gradually leaking worker processes which are stuck in "Gracefully finishing" state - https://phabricator.wikimedia.org/T182832 (10Dzahn) [22:49:24] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [22:51:11] 10Continuous-Integration-Config, 10TechCom: On CI, stop testing MediaWiki with php7.0 ahead of dropping support - https://phabricator.wikimedia.org/T225628 (10Jdforrester-WMF) [22:51:16] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10MediaWiki-General, 10PHP 7.0 support, 10Patch-For-Review: Drop PHP 7.0 support from MediaWiki - https://phabricator.wikimedia.org/T216165 (10Jdforrester-WMF) [22:51:36] 10Continuous-Integration-Config, 10TechCom: On CI, stop testing MediaWiki with php7.0 ahead of dropping support - https://phabricator.wikimedia.org/T225628 (10Jdforrester-WMF) Not a blocker to T216165. However, not Declining out-right, as keeping php70 going is not free. [23:00:24] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%) [23:01:44] 10Beta-Cluster-Infrastructure: Investigate cloud VM creation failures for instances with deployment-mediawiki prefix - https://phabricator.wikimedia.org/T232901 (10ssastry) [23:02:29] 10Beta-Cluster-Infrastructure, 10Parsoid-PHP, 10RESTBase, 10Core Platform Team Workboards (Purple), 10Patch-For-Review: Make the parsoid server on the beta cluster a mediawiki app server - https://phabricator.wikimedia.org/T232538 (10ssastry) >>! In T232538#5492336, @ssastry wrote: > I think that is wort... [23:06:55] PROBLEM - Host deployment-alex-test is DOWN: CRITICAL - Host Unreachable (172.16.6.31) [23:12:54] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin10... [23:40:03] !log deployment-deploy01: cleaning disk: rm /var/cache/hhvm/cli.hhbc.sq3 [23:40:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:59:45] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Reimage both phab1001 and phab2001 to stretch / buster - https://phabricator.wikimedia.org/T190568 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['phab1001.eqiad.wmne...