[01:52:37] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Chinese-Sites, 10Patch-For-Review, 10User-Urbanecm: Increase wgAbuseFilterEmergencyDisable on zh Beta Cluster - https://phabricator.wikimedia.org/T230305 (10Shizhao) [02:18:14] Project beta-scap-eqiad build #310331: 04FAILURE in 3 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/310331/ [02:28:15] Yippee, build fixed! [02:28:15] Project beta-scap-eqiad build #310332: 09FIXED in 3 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/310332/ [09:01:26] 10Phabricator: Limit mass-adding subscribers on Phabricator - https://phabricator.wikimedia.org/T254409 (10Aklapper) 05Stalled→03Declined Unfortunately closing this Phabricator task as no further information has been provided. @Yair_rand: After you have provided the information asked for, please set the sta... [09:10:49] 10Phabricator (Upstream), 10Upstream: Time zones on Phabricator events are confusing - https://phabricator.wikimedia.org/T158910 (10Aklapper) [09:41:19] 10Project-Admins, 10User-DannyS712: Create component for `Assert` library - https://phabricator.wikimedia.org/T251076 (10Aklapper) Makes sense to me; however wondering who are stewards of that extension (if existing) and what they think (as it touches their workflow) [09:42:32] 10Project-Admins, 10User-DannyS712: Create #mediawiki-namespaces - https://phabricator.wikimedia.org/T249998 (10Aklapper) Does #Platform_Engineering agree (as I guess they are code stewards) as it touches their workflows? If yes then let's do this please [09:43:11] 10Phabricator: Phabricator: "500 Internal Server Error" when using VPN (due to specific IP range) - https://phabricator.wikimedia.org/T258059 (10Aklapper) [09:51:12] 10MediaWiki-Codesniffer: Do not warn about missing function doc comment when parameter and return types are fully specified by type hints - https://phabricator.wikimedia.org/T258925 (10Tgr) [09:51:48] 10MediaWiki-Codesniffer: Do not warn about missing function doc comment when parameter and return types are fully specified by type hints - https://phabricator.wikimedia.org/T258925 (10Tgr) Somewhat related: {T178136} [09:53:52] (03PS1) 10JMeybohm: helm does not need backports as well [integration/config] - 10https://gerrit.wikimedia.org/r/616454 (https://phabricator.wikimedia.org/T258773) [09:54:51] (03PS2) 10JMeybohm: helm does now need backports as well [integration/config] - 10https://gerrit.wikimedia.org/r/616454 (https://phabricator.wikimedia.org/T258773) [09:55:05] what a bad typo :) [10:05:39] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki/services/mobileapps] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516517 (https://phabricator.wikimedia.org/T247012) (owner: 10Mholloway) [10:06:43] (03Merged) 10jenkins-bot: layout: [mediawiki/services/mobileapps] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516517 (https://phabricator.wikimedia.org/T247012) (owner: 10Mholloway) [10:08:15] !log Zuul: [mediawiki/services/mobileapps] Drop npm-docker job T247012 T218733 [10:08:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:08:49] T247012: Archive mediawiki/services/mobileapps/deploy once it's no longer used - https://phabricator.wikimedia.org/T247012 [10:08:50] T218733: Migrate mobileapps to k8s and node 10 - https://phabricator.wikimedia.org/T218733 [10:09:06] (03PS7) 10Jforrester: layout: [mediawiki/services/mobileapps/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516651 (https://phabricator.wikimedia.org/T247012) [10:09:14] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki/services/mobileapps/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516651 (https://phabricator.wikimedia.org/T247012) (owner: 10Jforrester) [10:10:18] (03Merged) 10jenkins-bot: layout: [mediawiki/services/mobileapps/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516651 (https://phabricator.wikimedia.org/T247012) (owner: 10Jforrester) [10:11:14] !log Zuul: [mediawiki/services/mobileapps/deploy] Archive T247012 [10:11:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:13:04] (03PS1) 10Jforrester: layout: Drop `npm-docker` template, no longer used [integration/config] - 10https://gerrit.wikimedia.org/r/616457 (https://phabricator.wikimedia.org/T211784) [10:17:25] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/n... - https://phabricator.wikimedia.org/T211784 [10:23:44] (03PS5) 10Jforrester: jjb: Drop mobileapps-deploy-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516721 (https://phabricator.wikimedia.org/T247012) [10:24:34] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/516721 (https://phabricator.wikimedia.org/T247012) (owner: 10Jforrester) [10:25:08] (03CR) 10Jforrester: [C: 03+2] "Hurrah." [integration/config] - 10https://gerrit.wikimedia.org/r/616457 (https://phabricator.wikimedia.org/T211784) (owner: 10Jforrester) [10:25:39] (03Merged) 10jenkins-bot: jjb: Drop mobileapps-deploy-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516721 (https://phabricator.wikimedia.org/T247012) (owner: 10Jforrester) [10:26:06] (03Merged) 10jenkins-bot: layout: Drop `npm-docker` template, no longer used [integration/config] - 10https://gerrit.wikimedia.org/r/616457 (https://phabricator.wikimedia.org/T211784) (owner: 10Jforrester) [10:26:18] (03CR) 10Jforrester: [C: 04-2] layout: [mediawiki/services/kartotherian] Run only the pipeline jobs [integration/config] - 10https://gerrit.wikimedia.org/r/534179 (https://phabricator.wikimedia.org/T216826) (owner: 10Jforrester) [10:26:26] (03CR) 10Jforrester: [C: 04-2] [DNM] layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [10:29:58] (03PS4) 10Jforrester: layout: [mediawiki/services/kartotherian] Drop the kartotherian-node10-browser-docker jobs [integration/config] - 10https://gerrit.wikimedia.org/r/534179 (https://phabricator.wikimedia.org/T216826) [10:30:00] (03PS4) 10Jforrester: layout: Archive the kartotherian and tilerator deploy repos now they use the pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/534180 (https://phabricator.wikimedia.org/T216826) [10:30:02] (03PS3) 10Jforrester: jjb: Drop old kartotherian jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/534181 (https://phabricator.wikimedia.org/T216826) [10:30:04] (03PS3) 10Jforrester: dockerfiles: Drop node10-kartotherian and npm-test-maps-service [integration/config] - 10https://gerrit.wikimedia.org/r/534182 (https://phabricator.wikimedia.org/T216826) [10:31:02] (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/services/kartotherian] Drop the kartotherian-node10-browser-docker jobs [integration/config] - 10https://gerrit.wikimedia.org/r/534179 (https://phabricator.wikimedia.org/T216826) (owner: 10Jforrester) [10:31:12] (03CR) 10jerkins-bot: [V: 04-1] layout: Archive the kartotherian and tilerator deploy repos now they use the pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/534180 (https://phabricator.wikimedia.org/T216826) (owner: 10Jforrester) [10:31:36] (03PS6) 10Jforrester: [DNM] layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [10:31:38] (03PS4) 10Jforrester: layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) [10:31:40] (03PS6) 10Jforrester: jjb: Drop recommendation API node 6 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/516719 (https://phabricator.wikimedia.org/T248355) [10:34:35] (03PS2) 10Jforrester: Revert "jjb: Codehealth pipeline NodeJS configuration" [integration/config] - 10https://gerrit.wikimedia.org/r/615193 (owner: 10MSantos) [10:35:33] (03CR) 10jerkins-bot: [V: 04-1] Revert "jjb: Codehealth pipeline NodeJS configuration" [integration/config] - 10https://gerrit.wikimedia.org/r/615193 (owner: 10MSantos) [10:36:24] (03PS3) 10Jforrester: Revert "jjb: Codehealth pipeline NodeJS configuration" [integration/config] - 10https://gerrit.wikimedia.org/r/615193 (owner: 10MSantos) [10:37:15] (03CR) 10Jforrester: [C: 03+2] Revert "jjb: Codehealth pipeline NodeJS configuration" [integration/config] - 10https://gerrit.wikimedia.org/r/615193 (owner: 10MSantos) [10:38:15] (03Merged) 10jenkins-bot: Revert "jjb: Codehealth pipeline NodeJS configuration" [integration/config] - 10https://gerrit.wikimedia.org/r/615193 (owner: 10MSantos) [12:15:12] 10Continuous-Integration-Infrastructure, 10Sonarqubebot, 10User-DannyS712: Sonarqubebot ignores qunit tests when measuring coverage - https://phabricator.wikimedia.org/T258941 (10DannyS712) [12:33:21] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Scap, 10Operations, 10Patch-For-Review: scap configuration in puppet defaults to forge the git repo name with 'med... - https://phabricator.wikimedia.org/T257413 [13:30:07] 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Patch-For-Review: Jenkins comments no longer formatted on Gerrit 3 - https://phabricator.wikimedia.org/T256575 (10hashar) p:05Triage→03Medium [13:32:03] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): CI fetching tags should also create the local tag references - https://phabricator.wikimedia.org/T186496 (10hashar) p:05Triage... [13:32:30] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Technical-Debt: Abstract out git commands for CI Docker containers - https://phabricator.wikimedia.org/T256462 (10hashar) p... [13:35:25] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Quibble, 10Regression: Regression: lost the ability to --dump-db-postrun - https://phabricator.wikimedia.org/T239396 (10hashar) Pending next release of Quibble [13:46:42] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Chinese-Sites, 10User-Urbanecm: Increase wgAbuseFilterEmergencyDisable on zh Beta Cluster - https://phabricator.wikimedia.org/T230305 (10Urbanecm) 05Open→03Resolved Should be all done :). [13:59:51] 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Patch-For-Review: Jenkins comments no longer formatted on Gerrit 3 - https://phabricator.wikimedia.org/T256575 (10hashar) [14:50:29] maintenance-disconnect-full-disks build 197865 integration-agent-docker-1011 (/: 17%, /srv: 95%, /var/lib/docker: 65%): OFFLINE due to disk space [14:55:37] maintenance-disconnect-full-disks build 197866 integration-agent-docker-1011 (/: 17%, /srv: 46%, /var/lib/docker: 51%): RECOVERY disk space OK [15:10:09] nice that that job is bringing nodes back online as they recover now [15:12:30] Yeah, that has saved a lot of manual fiddling. I also have collected data on why it happens: https://graphite.wikimedia.org/S/p [15:13:59] wow, 6GB [15:14:36] quibble selenium job sure is space hungry [15:16:27] If you extend the range of the graph (e.g., by adding `&from=-4days`) you'll see that wikidata-query-rdf-maven-java8-docker-site-publish grows past 12GB the few times that it runs. [15:19:52] those jobs used to be isolated on their own nodes, but now that they have "docker" in the name, I'm no longer sure if that's the case [15:22:28] PROBLEM - Free space - all mounts on integration-agent-docker-1011 is CRITICAL: CRITICAL: integration.integration-agent-docker-1011.diskspace._srv.byte_percentfree (<11.11%) [15:23:45] I know that data only goes back 4 days, but why does it appear that the php72 job is slowly growing? [15:24:03] (iow: this might be a fluke/data is unreliable so far) [15:28:57] I noticed that too. We'll see how it looks after a few more days. [15:32:59] 21WAABV1Z: your nick changed somehow, but /whois confirms it's you. a quick "/nick jrbranna" should fix it ;) [15:36:49] (03CR) 10Thcipriani: [C: 03+1] "LGTM. Seems to depend on Id63e271407dc17fe56c7009f16092ea27a18409f" [integration/config] - 10https://gerrit.wikimedia.org/r/616177 (https://phabricator.wikimedia.org/T224041) (owner: 10Jeena Huneidi) [15:37:29] RECOVERY - Free space - all mounts on integration-agent-docker-1011 is OK: OK: All targets OK [15:41:45] 10Release-Engineering-Team, 10Analytics-Radar, 10Product-Analytics, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10fdans) [16:24:25] 10Beta-Cluster-Infrastructure, 10CheckUser, 10Trust-and-Safety, 10WMF-Legal, and 2 others: Enable CheckUser for beta cluster - https://phabricator.wikimedia.org/T214820 (10Rxy) 05Open→03Declined >>! In T214820#6308084, @Aklapper wrote: > @Rxy: Could you answer the last question please? Thanks! Yes, We... [16:36:39] PROBLEM - Free space - all mounts on integration-castor03 is CRITICAL: CRITICAL: integration.integration-castor03.diskspace._srv.byte_percentfree (<11.11%) [16:42:15] Majavah: can you kill the spam bot that just hit my beta talk [16:42:18] https://en.wikipedia.beta.wmflabs.org/w/index.php?title=User_talk:RhinosF1&diff=441195&oldid=425447 [16:42:43] RhinosF1: locked [16:43:22] Majavah: ty, exact same post as 2 months ago. I wonder if an abuse filter or protection could shut them up [16:44:04] hmm [16:44:05] https://meta.m.wikimedia.beta.wmflabs.org/wiki/Special:CentralAuth?target=DanieleGarber81 was the last one [16:44:15] Majavah: maybe lock ^ [16:44:26] mobile link :/ [16:44:26] Both have only hit my talk [16:44:32] https://meta.wikimedia.beta.wmflabs.org/wiki/Special:CentralAuth?target=DanieleGarber81 [16:44:55] also gone [16:45:42] Ty Majavah [16:45:43] hopefully no new beta cluster user has a legitimate purpose to add "seo services" before they are autoconfirmed [16:45:51] Ye [16:47:10] Majavah: you done an abuse filter or? [16:47:12] Link? [16:47:23] doing, not finished yet [16:48:56] Ack [16:50:09] I made https://meta.wikimedia.beta.wmflabs.org/wiki/Special:AbuseFilter/62, log-only until I get couple of test hits [16:50:41] You may not view details of this filter, because it is hidden from public view. [16:52:13] now you can [16:54:35] Ty [16:57:34] 10Continuous-Integration-Config, 10Code-Health-Metrics, 10Patch-For-Review: Report results from SonarCloud to Gerrit - https://phabricator.wikimedia.org/T217008 (10thcipriani) All the checkboxes are checked—is there anything left to do here? [17:05:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10User-brennen, 10User-dancy: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10dancy) [17:14:49] 10Gerrit, 10Release-Engineering-Team, 10Jenkins: Jenkins is sometimes not picking up +2 votes, some patches don't get merged - https://phabricator.wikimedia.org/T257556 (10hashar) 05Open→03Resolved a:03hashar I looked at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/608927 , patchset 5 got a CR+2 r... [17:41:38] PROBLEM - Free space - all mounts on integration-castor03 is CRITICAL: CRITICAL: integration.integration-castor03.diskspace._srv.byte_percentfree (<22.22%) [17:43:47] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: HTTP CRITICAL: HTTP/1.1 302 Found - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 716 bytes in 0.317 second response time [17:46:46] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) [17:48:10] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T257971 (10brennen) [18:08:58] (03PS2) 10Krinkle: Support DOCUMENT_ROOT being a symbolic link [integration/docroot] - 10https://gerrit.wikimedia.org/r/611377 (https://phabricator.wikimedia.org/T149924) (owner: 10Hashar) [18:10:58] (03PS3) 10Krinkle: Support DOCUMENT_ROOT being a symbolic link [integration/docroot] - 10https://gerrit.wikimedia.org/r/611377 (https://phabricator.wikimedia.org/T149924) (owner: 10Hashar) [18:11:04] (03CR) 10Krinkle: [C: 03+1] Support DOCUMENT_ROOT being a symbolic link [integration/docroot] - 10https://gerrit.wikimedia.org/r/611377 (https://phabricator.wikimedia.org/T149924) (owner: 10Hashar) [18:21:36] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10observability, and 2 others: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10hashar) [18:26:30] (03CR) 10Krinkle: [C: 03+2] Support DOCUMENT_ROOT being a symbolic link [integration/docroot] - 10https://gerrit.wikimedia.org/r/611377 (https://phabricator.wikimedia.org/T149924) (owner: 10Hashar) [18:27:50] (03CR) 10Zfilipin: "> Patch Set 2:" [ruby/api] - 10https://gerrit.wikimedia.org/r/613666 (owner: 10Alex Dean) [18:30:01] (03Merged) 10jenkins-bot: Support DOCUMENT_ROOT being a symbolic link [integration/docroot] - 10https://gerrit.wikimedia.org/r/611377 (https://phabricator.wikimedia.org/T149924) (owner: 10Hashar) [19:12:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Investigate again a central cache for package managers - https://phabricator.wikimedia.org/T147635 (10dancy) 05Stalled→03Open a:03dancy [19:12:44] 10Continuous-Integration-Infrastructure, 10Zuul, 10User-brennen, 10User-dancy: Improve scheduling of CI jobs invoked by zuul - https://phabricator.wikimedia.org/T258630 (10hashar) [19:12:47] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10User-dancy: Investigate again a central cache for package managers - https://phabricator.wikimedia.org/T147635 (10dancy) [19:13:14] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T257969 (10jeena) 05Open→03Resolved [19:24:42] 10Release-Engineering-Team, 10Analytics-Radar, 10Product-Analytics, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10mpopov) [19:29:14] 10Continuous-Integration-Infrastructure, 10Zuul, 10User-brennen, 10User-dancy: Improve scheduling of CI jobs invoked by zuul - https://phabricator.wikimedia.org/T258630 (10hashar) For the patch, we can send it upstream https://opendev.org/opendev/gear.git . You would need an Ubuntu / launchpad account to p... [19:36:26] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Selenium quibble jobs have a huge cache overloading the CI agents small disk ... - https://phabricator.wikimedia.org/T258972 [19:37:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Collect samples of workspace sizes on integration-agent-docke... - https://phabricator.wikimedia.org/T258626 [19:37:35] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Selenium quibble jobs have a huge cache overloading the CI agents small disk ... - https://phabricator.wikimedia.org/T258972 [19:37:45] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Selenium quibble jobs have a huge cache overloading the CI agents small disk ... - https://phabricator.wikimedia.org/T258972 [19:53:12] (03PS1) 10DannyS712: Add `extension-javascript-documentation` for GlobalWatchlist [integration/config] - 10https://gerrit.wikimedia.org/r/616503 [19:54:16] 10Release-Engineering-Team (Onboarding), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10Release Pipeline (Blubber), 10User-dancy: Blubber: Allow more than one layer of copying - https://phabricator.wikimedia.org/T254629 (10dancy) @je... [19:54:17] (03PS2) 10DannyS712: Add `extension-javascript-documentation` for GlobalWatchlist [integration/config] - 10https://gerrit.wikimedia.org/r/616503 [19:54:25] (03CR) 10DannyS712: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/616503 (owner: 10DannyS712) [19:54:25] 10Release-Engineering-Team (Onboarding), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10Release Pipeline (Blubber), 10User-dancy: Blubber: Allow more than one layer of copying - https://phabricator.wikimedia.org/T254629 (10dancy) 05... [19:55:08] is there a way to schedule zuul patches for the backport windows? https://gerrit.wikimedia.org/r/c/integration/config/+/616503 [19:58:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))): Selenium quibble jobs have a huge cache overloading the CI agents small disk ... - https://phabricator.wikimedia.org/T258972 [20:00:12] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Zuul, 10observability, and 2 others: Monitor /var/log/zuul/error.log - https://phabricator.wikimedia.org/T258821 (10hashar) I have added #observability , they have all... [20:18:27] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Operations: Puppet failure on deployment-cache-text06 - https://phabricator.wikimedia.org/T256064 (10herron) p:05Triage→03Medium [20:34:39] 10Gerrit, 10DBA, 10Patch-For-Review: Make sure both `reviewdb-test` (used forgerrit upgrade testing) and `reviewdb` (formerly production) databases get torn down - https://phabricator.wikimedia.org/T255715 (10Dzahn) If it hasn't already happened you can remove the grant for the "gerritro" user on m2 as well.... [21:00:11] (03PS1) 10Ahmon Dancy: Squash castor-save into castor-save-workspace-cache [integration/config] - 10https://gerrit.wikimedia.org/r/616594 [21:10:38] (03PS1) 10Ahmon Dancy: Move castor-save-filter.bash check into castor-save-workspacecache.bash [integration/config] - 10https://gerrit.wikimedia.org/r/616595 [21:21:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Selenium quibble jobs have a huge cache overloading the CI ag... - https://phabricator.wikimedia.org/T258972 [21:47:41] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release Pipeline (Blubber): Document blubber helm deployment - https://phabricator.wikimedia.org/T252669 (10jeena) 05Open→03Resolved Documented here: https://wikitech.... [21:50:21] (03PS1) 10Ahmon Dancy: wakeConnections: Randomize connections before scanning them [integration/gear] - 10https://gerrit.wikimedia.org/r/616599 (https://phabricator.wikimedia.org/T258630) [22:06:09] (03PS1) 10Thcipriani: deploy-promote: usage and explicit error [tools/release] - 10https://gerrit.wikimedia.org/r/616600 [22:06:10] (03PS1) 10Thcipriani: deploy-promote: assert repo is up-to-date [tools/release] - 10https://gerrit.wikimedia.org/r/616601 [22:27:51] 10Beta-Cluster-Infrastructure, 10Wikidata, 10Beta-Cluster-reproducible: "Source map error: request failed with status 404" on beta - https://phabricator.wikimedia.org/T256360 (10Jdlrobson) [23:08:11] 10Continuous-Integration-Config, 10Code-Health, 10Readers-Web-Backlog (Kanbanana-2020-21-Q1), 10Vue.js (Vue.js-Search): Configure ESLint and stylelint for Vue.js search development ahead of it being done for all repos - https://phabricator.wikimedia.org/T249304 (10Jdrewniak) 05Open→03Resolved [23:15:31] PROBLEM - SSH on integration-cumin-01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds