[00:46:35] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10Aklapper) Are these supposed to be [subprojects](https://www.mediawiki.org/wiki/Phabricator/Project_management#Parent_Projects,_Subprojects_and_Milestones) of #Research? If not, [how is each of those tags desc... [01:00:33] 10Project-Admins, 10Core Platform Team, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Krinkle) [01:02:13] 10Gerrit, 10Upstream: Support posting screenshots in Gerrit - https://phabricator.wikimedia.org/T228084 (10Aklapper) [OT] @kostajh: Thanks for catching that! Fixed. [01:03:35] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10leila) >>! In T228350#5343995, @Aklapper wrote: > Are these supposed to be [subprojects](https://www.mediawiki.org/wiki/Phabricator/Project_management#Parent_Projects,_Subprojects_and_Milestones) of #Research?... [01:15:09] 10Gerrit, 10Upstream: Support posting screenshots in Gerrit - https://phabricator.wikimedia.org/T228084 (10kostajh) > That extension does not appear to support PolyGerrit. That's unfortunate. @Paladox are you aware of any alternatives? [01:30:36] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10Aklapper) > not well-versed I'm going to confuse you more now, I'm afraid. :-/ The #Research project tag has no [subprojects or milestones](https://www.mediawiki.org/wiki/Phabricator/Project_management#Paren... [01:45:55] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10leila) @Aklapper Your latest question made me think that it's perhaps best for me to step back and look at all our team's workflow on Phabricator and then come with my request to you. I /think/ there is some o... [01:53:30] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10Aklapper) @leila: Heh, alright. :) https://www.mediawiki.org/wiki/Phabricator/Project_management might provide some bits and pieces, but since Team Practices Group does not exist anymore I'm afraid that we're... [02:02:53] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10leila) Thanks for the link and the offer. :) [02:35:18] (03CR) 10Krinkle: [C: 03+1] Allow @slowThreshold annotation in tests [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/520691 (owner: 10MaxSem) [02:36:08] (03CR) 10Legoktm: [C: 03+2] Allow @slowThreshold annotation in tests [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/520691 (owner: 10MaxSem) [02:37:03] (03Merged) 10jenkins-bot: Allow @slowThreshold annotation in tests [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/520691 (owner: 10MaxSem) [02:37:33] (03CR) 10jenkins-bot: Allow @slowThreshold annotation in tests [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/520691 (owner: 10MaxSem) [05:42:02] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-SecureLinkFixer: Deploy SecureLinkFixer to beta cluster - https://phabricator.wikimedia.org/T228374 (10Legoktm) [05:59:38] (03PS1) 10Legoktm: Start branching SecureLinkFixer [tools/release] - 10https://gerrit.wikimedia.org/r/524089 (https://phabricator.wikimedia.org/T200751) [06:00:06] (03CR) 10jerkins-bot: [V: 04-1] Start branching SecureLinkFixer [tools/release] - 10https://gerrit.wikimedia.org/r/524089 (https://phabricator.wikimedia.org/T200751) (owner: 10Legoktm) [06:00:43] 01:59:51 docker: filesystem layer verification failed for digest sha256:e9dd2befc159629b6a09232c6478fa48bedfc34117be1a04c1e337ebd0a46d27. [06:15:44] 10Release-Engineering-Team-TODO (201907), 10Operations, 10serviceops, 10Patch-For-Review, 10Wikimedia-Incident: docker-registry: some layers has been corrupted due to deleting other swift containers - https://phabricator.wikimedia.org/T228196 (10Nikerabbit) Per https://integration.wikimedia.org/ci/job/tr... [06:19:44] 10Continuous-Integration-Config: Switch translatewiki repo from composer-test-hhvm to composer-test - https://phabricator.wikimedia.org/T228376 (10Nikerabbit) [06:28:04] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-SecureLinkFixer, 10Patch-For-Review: Deploy SecureLinkFixer to beta cluster - https://phabricator.wikimedia.org/T228374 (10greg) (Just a drive by reminder to add to Dev/Maintainers when you're ready) [06:35:08] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-SecureLinkFixer, 10Patch-For-Review: Deploy SecureLinkFixer to beta cluster - https://phabricator.wikimedia.org/T228374 (10Legoktm) >>! In T228374#5344550, @greg wrote: > (Just a drive by reminder to add to Dev/Maintainers when you're ready) Done: https:... [06:49:03] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<11.11%) [06:52:12] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Scap, 10serviceops, 10Patch-For-Review: 'scap pull' stopped working on appservers ? - https://phabricator.wikimedia.org/T228328 (10greg) [06:59:02] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:34:29] (03CR) 10Hashar: "And commits which are pushed directly can trigger a job via the 'post' pipeline. Though there is only one such example right now :)" [integration/config] - 10https://gerrit.wikimedia.org/r/523804 (https://phabricator.wikimedia.org/T226668) (owner: 10Ottomata) [07:51:04] ACKNOWLEDGEMENT - DPKG on contint1001 is CRITICAL: DPKG CRITICAL dpkg reports broken packages amusso Zuul / python oddity https://wikitech.wikimedia.org/wiki/Monitoring/Missing_notes_link [07:54:47] 10Continuous-Integration-Config: Switch translatewiki repo from composer-test-hhvm to composer-test - https://phabricator.wikimedia.org/T228376 (10hashar) integration/config.git has: ` name=zuul/layout.yaml,lang=yaml - name: translatewiki test: - translatewiki-rake-docker - translatewiki-compos... [07:55:04] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907): Switch translatewiki repo from composer-test-hhvm to composer-test - https://phabricator.wikimedia.org/T228376 (10hashar) [07:58:07] 10Continuous-Integration-Config, 10MediaWiki-extensions-Page_Forms, 10Patch-For-Review: Add phan to PageForms - https://phabricator.wikimedia.org/T228155 (10hashar) Given PageForms is supposedly usable without SMW, would it makes sense to have run TWO phan analysis? One with SMW and another one without it?... [08:00:13] (03CR) 10Awight: [C: 04-1] Run node browser tests in each repo (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) (owner: 10Awight) [08:03:53] 10Release-Engineering-Team-TODO (201907), 10Operations, 10serviceops, 10Patch-For-Review, 10Wikimedia-Incident: docker-registry: some layers has been corrupted due to deleting other swift containers - https://phabricator.wikimedia.org/T228196 (10akosiaris) p:05Normal→03High >>! In T228196#5342893, @t... [08:41:53] RECOVERY - DPKG on contint1001 is OK: All packages OK https://wikitech.wikimedia.org/wiki/Monitoring/Missing_notes_link [08:44:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Zuul: zuul systemd service: [/lib/systemd/system/zuul.service:15] Failed to parse usec_t value, ignoring: infinity - https://phabricator.wikimedia.org/T228381 (10hashar) [08:44:07] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Zuul: zuul systemd service: [/lib/systemd/system/zuul.service:15] Failed to parse usec_t value, ignoring: infinity - https://phabricator.wikimedia.org/T228381 (10hashar)... [08:54:03] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Zuul: zuul systemd service: [/lib/systemd/system/zuul.service:15] Failed to parse usec_t value, ignoring: infinity - https://phabricator.wikimedia.org/T228381 (10hashar)... [08:57:41] RECOVERY - puppet last run on contint1001 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures https://wikitech.wikimedia.org/wiki/Monitoring/Missing_notes_link [09:01:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907): contint1001 spurious disk space alarms - https://phabricator.wikimedia.org/T227605 (10hashar) 05Open→03Resolved Yes that looks good now. Thank you! [09:06:01] 10Release-Engineering-Team-TODO (201907), 10Operations, 10serviceops, 10Patch-For-Review, 10Wikimedia-Incident: docker-registry: some layers has been corrupted due to deleting other swift containers - https://phabricator.wikimedia.org/T228196 (10fsero) i've uploaded the missing layers from a backup, it w... [09:07:37] 10Release-Engineering-Team-TODO (201907), 10Operations, 10serviceops, 10Patch-For-Review, 10Wikimedia-Incident: docker-registry: some layers has been corrupted due to deleting other swift containers - https://phabricator.wikimedia.org/T228196 (10fsero) fixes also docker-registry.wikimedia.org/releng/comp... [10:57:49] (03PS16) 10Awight: Run node browser tests in each repo [integration/quibble] - 10https://gerrit.wikimedia.org/r/510709 (https://phabricator.wikimedia.org/T199116) [10:57:51] (03PS1) 10Awight: Clean-up: split out run_selenium [integration/quibble] - 10https://gerrit.wikimedia.org/r/524189 [11:09:42] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10Patch-For-Review: Add prometheus exporter to Gerrit - https://phabricator.wikimedia.org/T184086 (10fgiunchedi) >>! In T184086#5340821, @hashar wrote: > **`metrics-reporter-jmx`** > > The code is... [11:14:12] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10fgiunchedi) [11:23:15] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10awight) [11:27:31] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201907), 10Code-Stewardship-Reviews, 10Graphoid, and 3 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10dr0ptp4kt) Hi team, just to follow up on what I've let some of you know by email, I'm g... [12:03:15] 10Project-Admins, 10Collection, 10OCG-General: Decide what to do with open OCG tasks in Phabricator - https://phabricator.wikimedia.org/T161312 (10Aklapper) One year later, to allow average humans to understand the situation around PDFs, I'm simply going to drop my notes here (Partially quoting Tgr from `T17... [12:04:43] (03PS13) 10Effie Mouzeli: Add --canary-wait-time flag [tools/scap] - 10https://gerrit.wikimedia.org/r/495398 (https://phabricator.wikimedia.org/T217924) [12:04:54] (03CR) 10jerkins-bot: [V: 04-1] Add --canary-wait-time flag [tools/scap] - 10https://gerrit.wikimedia.org/r/495398 (https://phabricator.wikimedia.org/T217924) (owner: 10Effie Mouzeli) [12:06:03] (03CR) 10Effie Mouzeli: Add --canary-wait-time flag (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/495398 (https://phabricator.wikimedia.org/T217924) (owner: 10Effie Mouzeli) [12:07:03] (03CR) 10Effie Mouzeli: "Needs rebasing :(" [tools/scap] - 10https://gerrit.wikimedia.org/r/495398 (https://phabricator.wikimedia.org/T217924) (owner: 10Effie Mouzeli) [12:15:31] 10Project-Admins, 10Research: create tags for Research - https://phabricator.wikimedia.org/T228350 (10Aklapper) 05Open→03Stalled Setting task status to `stalled`; please set the status back to "Open" via the {nav name=Add Action... > Change Status} dropdown once more actionable. Thanks! [12:49:29] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Zuul, 10Patch-For-Review: zuul systemd service: [/lib/systemd/system/zuul.service:15] Failed to parse usec_t value, ignoring: ... - https://phabricator.wikimedia.org/T228381 [12:54:02] 10Project-Admins, 10Collection, 10OCG-General: Decide what to do with open OCG tasks in Phabricator - https://phabricator.wikimedia.org/T161312 (10Tgr) >>! In T161312#5345340, @Aklapper wrote: > * Currently, Tasks about the PDF rendering functionality go to #PDF-Rendering. Note that, with PDFs being generat... [13:02:40] (03CR) 10Hashar: [V: 03+2 C: 03+2] Allow registered users to delete refs in sandbox (031 comment) [test/gerrit-ping] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/522462 (https://phabricator.wikimedia.org/T227159) (owner: 10Hashar) [13:03:49] (03CR) 10Hashar: [V: 03+2 C: 03+2] Allow registered users to delete refs in sandbox (031 comment) [test/gerrit-ping] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/522462 (https://phabricator.wikimedia.org/T227159) (owner: 10Hashar) [13:04:56] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10LarsWirzenius) [13:04:59] (03PS1) 10Hashar: Allow registered users to delete refs in sandbox [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/524206 (https://phabricator.wikimedia.org/T227159) [13:05:48] (03PS1) 10Hashar: Revert "Allow registered users to delete refs in sandbox" [test/gerrit-ping] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/524207 [13:06:08] (03PS2) 10Hashar: Revert "Allow registered users to delete refs in sandbox" [test/gerrit-ping] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/524207 (https://phabricator.wikimedia.org/T227159) [13:06:20] (03CR) 10Hashar: [V: 03+2 C: 03+2] Revert "Allow registered users to delete refs in sandbox" [test/gerrit-ping] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/524207 (https://phabricator.wikimedia.org/T227159) (owner: 10Hashar) [13:06:34] (03CR) 10Hashar: [V: 03+2 C: 03+2] Allow registered users to delete refs in sandbox [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/524206 (https://phabricator.wikimedia.org/T227159) (owner: 10Hashar) [13:07:24] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review: Enable sandbox branches in gerrit - https://phabricator.wikimedia.org/T227159 (10hashar) 05Open→03Resolved Done on `All-Projects.git` via https://gerrit.wikimedia.org/r/#/c/All-Pro... [13:07:29] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO: Add Code Stewardship review to Review Queue process - https://phabricator.wikimedia.org/T203698 (10Aklapper) Is this supposed to be mandatory? If yes, would this be an item under "Preparing for deployment" under #3 on https://www.mediaw... [13:08:42] 10Release-Engineering-Team-TODO (201907), 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.13; 2019-07-09): ReleaseNotesTest:testReleaseNotesFilesExistAndAreNotMalformed takes ~ 4 seconds - https://phabricator.wikimedia.org/T227067 (10hashar) 05Open→03Resolved Too many changes to backport for REL1_3... [13:16:14] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Operations, 10serviceops: Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10hashar) 05Resolved→03Open Thanks, I can confirm the component is around and it addresses the concern of mixing up up... [13:16:14] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907): Rebuild integration-slave-docker-* instances to use less RAM, new name and Stretch - https://phabricator.wikimedia.org/T226233 (10hashar) [13:16:20] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Operations: Migrate contint* hosts to Stretch/Buster - https://phabricator.wikimedia.org/T224591 (10hashar) [13:20:38] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10LarsWirzenius) [13:37:53] !log integration: rm /etc/apt/sources.list.d/sury-php.list and remove php packages | https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/523148/ | T225735 [13:37:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:37:56] T225735: Cleanup CI puppet manifests - https://phabricator.wikimedia.org/T225735 [13:48:47] !log integration: cherry picked https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/523150/ for apt unattended_upgrades | T225735 [13:48:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:48:52] T225735: Cleanup CI puppet manifests - https://phabricator.wikimedia.org/T225735 [13:51:14] !log integration: in hiera project config, remove classes: [ contint::packages::apt ] | T225735 [13:51:16] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Anomie) I'm not sure about the #Schema-change part. That tag claims to be about actual schema changes. Glancing through the tasks currently in the s... [13:51:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:13:40] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Marostegui) So, from my side, this is the way I read those tags: #schema-change is where a discussion about an upcoming schema change might happen... [14:14:28] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: Code Stewardship Review: SpamBlacklist - https://phabricator.wikimedia.org/T224921 (10marcella) >>! In T224921#5292216, @Aklapper wrote: > @marcella: Could you please elaborate if the WMF Editing Team is re... [14:30:16] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Operations, 10serviceops: Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10akosiaris) >>! In T226236#5345617, @hashar wrote: > Thanks, I can confirm the component is around and it addresses the c... [14:31:19] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10LarsWirzenius) [14:32:11] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Other / Uncategorized), 10Release-Engineering-Team-TODO (201907), 10User-greg: Request access to deployment-prep - https://phabricator.wikimedia.org/T228021 (10aezell) Just in case manager approval is needed. I approve this access for Dom. [14:36:47] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Krinkle) >>! In T228360#5345746, @Anomie wrote: > I'm not sure about the #Schema-change part. Thanks, I'm not actually very familiar with the Phabr... [14:36:55] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Krinkle) p:05Triage→03Normal [14:43:35] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10jcrespo) I personally don't have any opinion about #MediaWiki-Database I don't interact with it or need it or I am not subscribed to it. It is nice... [15:09:04] (03CR) 10Jforrester: [C: 03+2] Add TheWikipediaLibrary extension [tools/release] - 10https://gerrit.wikimedia.org/r/524052 (https://phabricator.wikimedia.org/T132084) (owner: 10Catrope) [15:10:09] (03Merged) 10jenkins-bot: Add TheWikipediaLibrary extension [tools/release] - 10https://gerrit.wikimedia.org/r/524052 (https://phabricator.wikimedia.org/T132084) (owner: 10Catrope) [15:12:40] 10Phabricator, 10Operations, 10Traffic: Make phame cacheable - https://phabricator.wikimedia.org/T219978 (10epriestley) > I'm not sure what the purpose of the cookie is... This cookie mostly supports CSRF protection for login attempts (`phsid` is "**PH**abricator **S**ession **ID**"), and prevents an attack... [15:13:00] (03PS2) 10Jforrester: Start branching SecureLinkFixer [tools/release] - 10https://gerrit.wikimedia.org/r/524089 (https://phabricator.wikimedia.org/T200751) (owner: 10Legoktm) [15:13:09] (03CR) 10Jforrester: [C: 03+2] Start branching SecureLinkFixer [tools/release] - 10https://gerrit.wikimedia.org/r/524089 (https://phabricator.wikimedia.org/T200751) (owner: 10Legoktm) [15:13:39] (03Merged) 10jenkins-bot: Start branching SecureLinkFixer [tools/release] - 10https://gerrit.wikimedia.org/r/524089 (https://phabricator.wikimedia.org/T200751) (owner: 10Legoktm) [15:15:50] (03CR) 10Krinkle: "This might mean that RL and i18n -related patches can no longer be tested on mwdebug. Need to think about this.." [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [15:24:27] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10hashar) [15:31:30] hashar: rollback? [15:31:53] * Krinkle continues in -ops [15:48:43] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog: Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) >>! In T228170#5342317, @Jdforrester-WMF wrote: >>>! In T228170#5341437, @M... [15:49:38] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) [15:51:36] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10Jdforrester-WMF) a:05MSantos→03Jdforrester-WMF Oh, meh. Custom images s... [15:54:54] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10hashar) I have rollbacked due to T228436 I did it on behalf of @LarsWirzenius s... [15:56:14] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) >>! In T228170#5346418, @Jdforrester-WMF wrote: > Oh, meh. Custom... [16:05:17] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Anomie) >>! In T228360#5345910, @Krinkle wrote: > Would using #mediawiki-maintenance-scripts and/or #mediawiki-installer be a more suitable place f... [16:13:03] 10Continuous-Integration-Config, 10MediaWiki-extensions-Page_Forms, 10Patch-For-Review: Add phan to PageForms - https://phabricator.wikimedia.org/T228155 (10Daimona) @hashar Being a static analysis tool, it wouldn't make any difference. We only need SMW to provide phan with all of the classes/methods definit... [16:13:28] 10Phabricator, 10Developer-Advocacy, 10Developer-Wishlist (2017), 10Goal: Consolidate the many tech events calendars in Phabricator's calendar - https://phabricator.wikimedia.org/T1035 (10Aklapper) 05Open→03Declined Boldly declining this in favor of https://discuss-space.wmflabs.org/c/events/l/calendar [16:14:18] 10Phabricator: Subscriber changes not reported by email if Herald also changes subscribers in the same action - https://phabricator.wikimedia.org/T133073 (10epriestley) This is expected behavior, but not exactly desirable. See upstream for some context. [16:24:51] 10Release-Engineering-Team, 10Analytics, 10Analytics-Kanban: issues with artifact cache in an-coord1001 - https://phabricator.wikimedia.org/T227132 (10mmodell) >>! In T227132#5301532, @Nuria wrote: > Can the release engineering chime as to whether scap config settings should also delete artifacts from the ta... [16:30:04] 10Release-Engineering-Team, 10Analytics, 10Analytics-Kanban: issues with artifact cache in an-coord1001 - https://phabricator.wikimedia.org/T227132 (10mmodell) > Does this scap config take effect in the source as well as the target of the deploy? The cached revs are only used on the target, not on the source. [16:34:59] 10Release-Engineering-Team (Deployment services), 10Analytics, 10Analytics-Kanban: issues with artifact cache in an-coord1001 - https://phabricator.wikimedia.org/T227132 (10greg) [16:38:55] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10Jdforrester-WMF) Gods, this repo desperately needs to be in the pipeline. T... [16:42:19] 10MediaWiki-Releasing, 10Release-Engineering-Team (Other / Uncategorized), 10Release-Engineering-Team-TODO: Removal of EOL release branches broke SemanticMediaWiki CI - https://phabricator.wikimedia.org/T179754 (10greg) [16:43:05] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10Wikimedia-production-error: Fatalmonitor on logstash still includes deprecated channel:wfLogDBError - https://phabricator.wikimedia.org/T165675 (10greg) [16:43:20] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Operations, 10puppet-compiler, 10Puppet: Integrate the puppet compiler in the puppet CI pipeline - https://phabricator.wikimedia.org/T166066 (10greg) [16:43:35] 10Continuous-Integration-Config, 10Analytics, 10JavaScript: Fix the analytics/mediawiki-storage repo to work on node10 - https://phabricator.wikimedia.org/T228451 (10Jdforrester-WMF) [16:43:38] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scoring-platform-team, 10Performance: Investigate PEP 503 repo for production deployment of python wheels - https://phabricator.wikimedia.org/T192478 (10greg) [16:44:13] 10MediaWiki-Releasing, 10Release-Engineering-Team (Other / Uncategorized), 10Release-Engineering-Team-TODO, 10Security: Create fab task to deploy jenkins job to either ci-jenkins or releases-jenkins - https://phabricator.wikimedia.org/T208528 (10greg) [16:45:03] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10JavaScript: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [16:45:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) 05Open→03Stalled Blocked on... [16:46:24] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10MediaWiki-Vagrant: Create a test suite that compiles mediawiki-vagrant puppet manifests - https://phabricator.wikimedia.org/T183570 (10greg) [16:46:49] 10Continuous-Integration-Config, 10Analytics, 10JavaScript: Fix the analytics/mediawiki-storage repo to work on node10 - https://phabricator.wikimedia.org/T228451 (10Milimetric) p:05Triage→03High [16:47:01] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10MediaWiki-Vagrant: Create a test suite that compiles mediawiki-vagrant puppet manifests - https://phabricator.wikimedia.org/T183570 (10greg) 05Open→03Declined Declining for now. [16:47:03] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10User-zeljkofilipin: Continuous integration for mediawiki-vagrant - https://phabricator.wikimedia.org/T183456 (10greg) [16:47:15] (03PS1) 10Jforrester: dockerfiles: Provide node10-kartotherian until it moves to the services pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/524271 (https://phabricator.wikimedia.org/T228170) [16:47:16] (03PS1) 10Jforrester: jjb: Provide kartotherian-node10-browser-docker based on custom image [integration/config] - 10https://gerrit.wikimedia.org/r/524272 (https://phabricator.wikimedia.org/T228170) [16:47:18] (03PS1) 10Jforrester: layout: [wikimedia/services/kartotherian] Switch to custom job [integration/config] - 10https://gerrit.wikimedia.org/r/524273 (https://phabricator.wikimedia.org/T228170) [16:51:03] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Composer can't regenerate class map at operations/mediawiki-config - https://phabricator.wikimedia.org/T155151 (10greg) Sorry for not really seeing this until now, is this still an issue for you @Dereckson ? [16:52:42] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10CirrusSearch, 10Discovery-Search: Build failures for Elastica and CirrusSearch on REL1_32 - https://phabricator.wikimedia.org/T216612 (10greg) Sorry for the late response, is this still an issue for you? [16:57:48] 10Release-Engineering-Team (Deployment services), 10Analytics, 10Analytics-Kanban: issues with artifact cache in an-coord1001 - https://phabricator.wikimedia.org/T227132 (10Ottomata) We're having a problem where the cached revs are not removed in the case of a scap failure. Occasionally we run out of disk s... [16:58:22] (03PS3) 10Jforrester: Add branching for Extension:WikipediaExtracts [tools/release] - 10https://gerrit.wikimedia.org/r/485373 (https://phabricator.wikimedia.org/T149765) (owner: 10Sophivorus) [16:58:35] (03CR) 10Jforrester: [C: 04-2] "Blocked by T149766." [tools/release] - 10https://gerrit.wikimedia.org/r/485373 (https://phabricator.wikimedia.org/T149765) (owner: 10Sophivorus) [16:58:39] (03PS4) 10Jforrester: Add branching for Extension:WikipediaExtracts [tools/release] - 10https://gerrit.wikimedia.org/r/485373 (https://phabricator.wikimedia.org/T149765) (owner: 10Sophivorus) [16:59:58] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10Krinkle) [17:06:22] 10Release-Engineering-Team (Deployment services), 10Analytics, 10Analytics-Kanban: issues with artifact cache in an-coord1001 - https://phabricator.wikimedia.org/T227132 (10Nuria) @mmmodell: this an example of such a problem: https://phabricator.wikimedia.org/T228347 [17:06:30] (03CR) 10MSantos: dockerfiles: Provide node10-kartotherian until it moves to the services pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/524271 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [17:21:26] (03PS2) 10Umherirrender: build: Updating mediawiki/mediawiki-codesniffer to 19.2.0 [integration/docroot] - 10https://gerrit.wikimedia.org/r/523639 (owner: 10Libraryupgrader) [17:27:53] (03CR) 10Umherirrender: [C: 03+1] build: Updating mediawiki/mediawiki-codesniffer to 19.2.0 [integration/docroot] - 10https://gerrit.wikimedia.org/r/523639 (owner: 10Libraryupgrader) [17:34:02] (03CR) 10Jforrester: [C: 03+2] build: Updating mediawiki/mediawiki-codesniffer to 19.2.0 [integration/docroot] - 10https://gerrit.wikimedia.org/r/523639 (owner: 10Libraryupgrader) [17:34:47] (03CR) 10jenkins-bot: build: Updating mediawiki/mediawiki-codesniffer to 19.2.0 [integration/docroot] - 10https://gerrit.wikimedia.org/r/523639 (owner: 10Libraryupgrader) [17:37:25] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10LibUp: LibraryUpgrader CI normalisation tasks, June/July 2019 - https://phabricator.wikimedia.org/T225325 (10Umherirrender) pdate mediawiki-codesniffer -> 26.0.0 is marked as ready, but I... [17:40:42] 10Release-Engineering-Team (Deployment services), 10Release, 10Train Deployments: 1.34.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T220741 (10greg) a:03brennen [17:40:55] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Release, 10Train Deployments: 1.34.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T220741 (10greg) [17:43:08] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) >>! In T228170#5346681, @Jdforrester-WMF wro... [17:47:24] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10Jdforrester-WMF) Pushing it as a pre-built binary int... [17:47:32] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Provide node10-kartotherian until it moves to the services pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/524271 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [17:47:42] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: Add liveness/readiness checks to services in local-charts - https://phabricator.wikimedia.org/T224933 (10jeena) [17:47:45] 10Release-Engineering-Team-TODO (201907), 10Developer Productivity, 10local-charts, 10Patch-For-Review: Move local-charts helm charts to a chart repository - https://phabricator.wikimedia.org/T224935 (10jeena) [17:47:48] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: Add liveness/readiness checks to services in local-charts - https://phabricator.wikimedia.org/T224933 (10jeena) This is being done as part of migrating our charts to the deployment-charts repo [17:48:05] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/524272 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [17:48:14] 10Continuous-Integration-Config, 10Analytics, 10Analytics-Kanban, 10JavaScript: Fix the analytics/mediawiki-storage repo to work on node10 - https://phabricator.wikimedia.org/T228451 (10Milimetric) [17:49:04] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10Krinkle) [17:49:06] (03Merged) 10jenkins-bot: dockerfiles: Provide node10-kartotherian until it moves to the services pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/524271 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [17:51:14] (03Merged) 10jenkins-bot: jjb: Provide kartotherian-node10-browser-docker based on custom image [integration/config] - 10https://gerrit.wikimedia.org/r/524272 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [18:01:12] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Scap, 10serviceops, 10Patch-For-Review: 'scap pull' stopped working on appservers ? - https://phabricator.wikimedia.org/T228328 (10Dzahn) A side effect of this is also that when reimaging hosts fails now. wmf-au... [18:01:51] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Scap, 10serviceops, 10Patch-For-Review: 'scap pull' stopped working on appservers ? - https://phabricator.wikimedia.org/T228328 (10Dzahn) p:05Triage→03High [18:06:14] !log Docker: Creating docker-registry.discovery.wmnet/releng/node10-kartotherian:0.1.0 [18:06:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:06:29] (03CR) 10Jforrester: [C: 03+2] layout: [wikimedia/services/kartotherian] Switch to custom job [integration/config] - 10https://gerrit.wikimedia.org/r/524273 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [18:09:54] (03Merged) 10jenkins-bot: layout: [wikimedia/services/kartotherian] Switch to custom job [integration/config] - 10https://gerrit.wikimedia.org/r/524273 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [18:10:08] zuul service fails to reload on contint2001 - known? [18:10:51] !log Zuul: [wikimedia/services/kartotherian] Switch to custom job T228170 [18:10:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:10:53] T228170: Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 [18:12:39] (03CR) 10Jforrester: dockerfiles: Provide node10-kartotherian until it moves to the services pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/524271 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [18:13:13] 08:57 hashar: contint1001: stopped zuul, ran apt install to get the new python2.7 copied to Zuul virtualenv, restarted zuul/zuul-merger. That clears a couple Icinga alarms from yesterday [18:13:23] ^ this broke something on 2001 though it looks [18:13:37] Ah, did we need to do the same on 2001? [18:13:50] hashar's not around right now. [18:14:38] i am not sure but comparing the versions now [18:15:11] question would be why a manual install [18:15:22] peaks into bash_history [18:16:49] manually installing conflicts with puppet [18:17:00] /usr/bin/dpkg returned an error code (1) [18:17:08] Dependency Package[zuul] has failures: true [18:17:29] cp: cannot create regular file '/usr/share/python/zuul/bin/python2.7': Text file busy [18:19:20] running 'apt install zuul' does not upgrade zuul. it says it is already the newest version, which i can confirm. but why it still runs is "1 not fully installed or removed" [18:20:08] letting it do that it tries to setup zuul and that is the part that fails [18:20:56] so yea, gotta stop zuul and then do it again [18:22:00] !log contint2001 systemctl stop zuul ; apt install zuul ... still fails [18:22:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:22:27] !log contint2001 /usr/sbin/dpkg-reconfigure: zuul is broken or not fully installed [18:22:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:25:04] !log contint2001 /etc/init.d/zuul-merger stop; apt install zuul (this time it worked to configure zuul after also stopping zuul-merge) ; apt-get autoremove (removed unused python packages); puppet agent -tv [18:25:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:25:56] RECOVERY - DPKG on contint2001 is OK: All packages OK https://wikitech.wikimedia.org/wiki/Monitoring/Missing_notes_link [18:25:57] !log contint2001 - puppet run reinstalled python-dateutil and ruby-safe-yeaml, stopped zuul service but no more failures now [18:25:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:26:29] fixed. puppet and dpkg are happy again. zuul is stopped (per design) [18:27:54] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 1 second ago with 0 failures https://wikitech.wikimedia.org/wiki/Monitoring/Missing_notes_link [18:30:56] James_F: short answer: yes [18:31:31] Fun. [18:31:56] for a moment i thought i am logging to the same SAL in both channels [18:32:07] Nope. :-) [18:32:11] maybe a unified SAL could actually be nice [18:32:17] Why have one SAL when we can have confusion?! [18:33:00] heh, indeed. i would not have found h.ashar's action if he had used this channel :p [18:33:11] but now i put mine into the other one [18:42:34] 10Continuous-Integration-Config, 10Analytics, 10Analytics-Kanban, 10JavaScript, 10Patch-For-Review: Fix the analytics/mediawiki-storage repo to work on node10 - https://phabricator.wikimedia.org/T228451 (10Jdforrester-WMF) 05Open→03Resolved a:03Milimetric Thank you! [18:42:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [18:44:43] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) a:05Jdforrester-WMF→03MSantos [18:51:16] (03PS1) 10Jforrester: layout: Generalise generic-node10-browser-docker on test/gate as node10-browser-docker template [integration/config] - 10https://gerrit.wikimedia.org/r/524301 [18:51:16] (03PS1) 10Jforrester: layout: [analytics-mediawiki-storage] Drop node 6, switch to node10-browser-docker [integration/config] - 10https://gerrit.wikimedia.org/r/524302 [18:51:18] (03PS1) 10Jforrester: jjb: Drop analytics-mediawiki-storage-npm-browser-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/524303 (https://phabricator.wikimedia.org/T211784) [18:51:20] (03PS1) 10Jforrester: dockerfiles: [node10-kartotherian] Use SRE-backported custom components where needed [integration/config] - 10https://gerrit.wikimedia.org/r/524304 (https://phabricator.wikimedia.org/T228170) [18:59:39] (03CR) 10Jforrester: [C: 03+2] layout: Generalise generic-node10-browser-docker on test/gate as node10-browser-docker template [integration/config] - 10https://gerrit.wikimedia.org/r/524301 (owner: 10Jforrester) [18:59:44] (03CR) 10Jforrester: [C: 03+2] layout: [analytics-mediawiki-storage] Drop node 6, switch to node10-browser-docker [integration/config] - 10https://gerrit.wikimedia.org/r/524302 (owner: 10Jforrester) [19:02:04] (03Merged) 10jenkins-bot: layout: Generalise generic-node10-browser-docker on test/gate as node10-browser-docker template [integration/config] - 10https://gerrit.wikimedia.org/r/524301 (owner: 10Jforrester) [19:02:07] (03Merged) 10jenkins-bot: layout: [analytics-mediawiki-storage] Drop node 6, switch to node10-browser-docker [integration/config] - 10https://gerrit.wikimedia.org/r/524302 (owner: 10Jforrester) [19:03:54] !log Zuul: [analytics-mediawiki-storage] Drop node 6, switch to node10-browser-docker [19:03:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:04:24] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/524303 (https://phabricator.wikimedia.org/T211784) (owner: 10Jforrester) [19:06:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [19:06:47] (03Merged) 10jenkins-bot: jjb: Drop analytics-mediawiki-storage-npm-browser-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/524303 (https://phabricator.wikimedia.org/T211784) (owner: 10Jforrester) [19:11:59] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10LibUp: LibraryUpgrader CI normalisation tasks, June/July 2019 - https://phabricator.wikimedia.org/T225325 (10Jdforrester-WMF) 25.0.0 is [[https://www.mediawiki.org/w/index.php?title=Libra... [19:16:08] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [node10-kartotherian] Use SRE-backported custom components where needed [integration/config] - 10https://gerrit.wikimedia.org/r/524304 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [19:17:33] (03PS1) 10Jforrester: jjb: [kartotherian-node10-browser-docker] Bump to latest custom image [integration/config] - 10https://gerrit.wikimedia.org/r/524308 (https://phabricator.wikimedia.org/T228170) [19:17:41] (03Merged) 10jenkins-bot: dockerfiles: [node10-kartotherian] Use SRE-backported custom components where needed [integration/config] - 10https://gerrit.wikimedia.org/r/524304 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [19:18:26] !log Docker: Creating docker-registry.discovery.wmnet/releng/node10-kartotherian:0.1.1 [19:18:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:18:29] (03CR) 10Thcipriani: "> This might mean that RL and i18n -related patches can no longer be" [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [19:23:33] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/524308 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [19:26:42] (03Merged) 10jenkins-bot: jjb: [kartotherian-node10-browser-docker] Bump to latest custom image [integration/config] - 10https://gerrit.wikimedia.org/r/524308 (https://phabricator.wikimedia.org/T228170) (owner: 10Jforrester) [19:34:08] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-SecureLinkFixer: Deploy SecureLinkFixer to beta cluster - https://phabricator.wikimedia.org/T228374 (10Jdforrester-WMF) 05Open→03Resolved Deployed. [20:30:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [20:30:52] 10Continuous-Integration-Infrastructure (Slipway), 10Wikidata, 10Wikidata Query UI: Migrate wikidata-query-gui-build to Docker containers - https://phabricator.wikimedia.org/T210286 (10Jdforrester-WMF) [20:31:28] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10Jdforrester-WMF) 05Open→03Resolved https://gerrit.wikimedia.org/r/c/med... [20:31:34] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10Jdforrester-WMF) [20:36:12] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10aaron) [20:37:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10aaron) [20:39:08] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10aaron) [20:54:58] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10Reading-Infrastructure-Team-Backlog (Kanban): Implement CI rules for new kartotherian repo - https://phabricator.wikimedia.org/T228170 (10MSantos) @Jdforrester-WMF thank you so much! [21:14:59] James_F: does stylelint have an autofix? [21:16:53] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10MSantos) FWIW: - Now that... [21:16:53] gerrit is rebooting [21:17:05] !log rebooting gerrit server [21:17:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:17:21] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10local-charts: Ensure that local-charts installs and runs on Debian Buster - https://phabricator.wikimedia.org/T228477 (10brennen) [21:18:38] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10local-charts: Ensure that local-charts installs and runs on Debian Buster - https://phabricator.wikimedia.org/T228477 (10brennen) p:05Triage→03Normal [21:31:47] legoktm: Yes, --fix. [21:32:20] legoktm: But it's "experimental" and is likely to be a bit suspect. [21:32:39] E.g. it ignores stylelint-disable comments. [21:33:12] heh [21:33:15] that seems rather important [21:33:31] (As in, it'll try to fix it anyway.) [21:34:02] IME we generally don't do stylelint-disable for auto-fix-able issues, more like "too much depth" or "id reference". [21:36:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T220739 (10Krinkle) [21:42:30] (03CR) 10Catrope: [C: 03+1] refreshMessageBlobs: don't run on scap pull [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [21:44:17] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Scap, 10serviceops, 10Patch-For-Review: 'scap pull' stopped working on appservers ? - https://phabricator.wikimedia.org/T228328 (10greg) a:03thcipriani [21:45:07] (03CR) 10Krinkle: [C: 03+1] "Ha, interesting. I hadn't thought of that. Thanks. LGTM." [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [21:46:19] (03CR) 10Thcipriani: [C: 03+2] refreshMessageBlobs: don't run on scap pull [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [21:46:31] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10local-charts, 10Kubernetes: local-charts: Document Windows recommendations and notes - https://phabricator.wikimedia.org/T223659 (10jeena) Hi Phaebz, thanks for testing out the windows install! How is the documentation going? [21:47:36] (03Merged) 10jenkins-bot: refreshMessageBlobs: don't run on scap pull [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [21:48:55] (03CR) 10jenkins-bot: refreshMessageBlobs: don't run on scap pull [tools/scap] - 10https://gerrit.wikimedia.org/r/523999 (https://phabricator.wikimedia.org/T228328) (owner: 10Thcipriani) [21:59:52] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10LibUp: LibraryUpgrader CI normalisation tasks, June/July 2019 - https://phabricator.wikimedia.org/T225325 (10Legoktm) >>! In T225325#5347302, @Jdforrester-WMF wrote: > 25.0.0 is [[https:/... [22:02:40] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [22:05:01] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [22:05:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) >>! In T2... [22:13:15] (03PS1) 10Jforrester: layout: Archive maps/kartotherian and maps/tilerator (but not /deploy) [integration/config] - 10https://gerrit.wikimedia.org/r/524365 [22:13:17] (03PS1) 10Jforrester: jjb: Delete kartotherian-npm-node-6-docker and tilerator-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/524366 [22:15:44] (03CR) 10Jforrester: [C: 03+2] layout: Archive maps/kartotherian and maps/tilerator (but not /deploy) [integration/config] - 10https://gerrit.wikimedia.org/r/524365 (owner: 10Jforrester) [22:15:49] (03CR) 10Jforrester: [C: 03+2] jjb: Delete kartotherian-npm-node-6-docker and tilerator-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/524366 (owner: 10Jforrester) [22:17:19] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) [22:17:40] (03Merged) 10jenkins-bot: layout: Archive maps/kartotherian and maps/tilerator (but not /deploy) [integration/config] - 10https://gerrit.wikimedia.org/r/524365 (owner: 10Jforrester) [22:17:58] (03Merged) 10jenkins-bot: jjb: Delete kartotherian-npm-node-6-docker and tilerator-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/524366 (owner: 10Jforrester) [22:18:10] !log Zuul: Archive maps/kartotherian and maps/tilerator [22:18:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:20:05] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<55.56%) [22:24:42] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201907), 10Scap, 10serviceops: Deploy scap 3.11.1-1 - https://phabricator.wikimedia.org/T228482 (10thcipriani) [22:40:28] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdrewniak) > wikimedia/por... [22:46:15] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: Ensure that local-charts installs and runs on Debian Buster - https://phabricator.wikimedia.org/T228477 (10brennen) [22:51:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201907), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdforrester-WMF) >>! In T2... [23:04:48] thcipriani: i see your change is merged so i tried scap pull again on mw2250. i get a fresh error now that is unrelated. heh [23:05:01] so you fixed it but we have more issues [23:05:05] mutante: scap hates you apparently :) [23:05:08] ImportError: No module named concurrent.futures [23:05:23] File "/usr/lib/python2.7/dist-packages/scap/opcache_manager.py", line 3, in [23:05:26] from concurrent.futures import ThreadPoolExecutor [23:05:43] hauskatze: yep :) [23:05:55] probably Scap should be updated to use py3 just saying [23:06:05] I think py2 is going EOL soon [23:06:12] mutante: that should be a dependency in the debian package [23:06:21] that is, it should have gotten installed with the deb [23:06:51] https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/tools/scap/+/release/debian/control#22 [23:07:06] ^ python-concurrent.futures [23:07:54] i see.. yea. but that package is not installed yet [23:08:13] ah, yeah, I see you're still on scap 3.11.0-1 [23:08:19] ack [23:08:37] it is in the changelog for the bugfix I *just* posted: https://phabricator.wikimedia.org/T228482 [23:08:53] scap is already the newest version (3.11.0-1). [23:09:11] aha! alright [23:10:04] heh, right, I misunderstood: thought you already had it built and published :) [23:16:17] i tried to follow the docs.. building on boron worked.. copying over worked.. publishing did not work [23:17:32] ah, maybe it didn't.. on it [23:17:53] scap | 3.11.1-1 | stretch-wikimedia | amd64, i386, source [23:19:12] missing a key when copying it to jessie [23:20:55] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity: Improve documentation for running local-charts and building docker images using the temporary Dockerfiles - https://phabricator.wikimedia.org/T220771 (10brennen) 05Open→03Resolved a:03brennen Since we now... [23:20:57] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Epic: FY201819 TEC12 Program – Developer Productivity - https://phabricator.wikimedia.org/T212449 (10brennen) [23:25:19] 10Project-Admins, 10Core Platform Team, 10DBA, 10Performance-Team: Narrow scope of MediaWiki-Database workboard - https://phabricator.wikimedia.org/T228360 (10Legoktm) From a bug filer perspective, `Wikimedia-Rdbms` is really not a great project name. We usually don't prefix library phabricator projects wi... [23:26:25] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Release Pipeline, 10local-charts: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10brennen) [23:28:12] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Release Pipeline, 10local-charts: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10brennen) As written originally: > Define a base docker-pkg template and .pipeline/blubber.yaml for mediawiki/core... [23:31:13] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Release Pipeline, 10local-charts: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10brennen) [23:31:15] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts, 10Epic: Create official docker images for Mediawiki and services used in the local development environment - https://phabricator.wikimedia.org/T217872 (10brennen) [23:31:49] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Epic: FY201819 TEC12 Program – Developer Productivity - https://phabricator.wikimedia.org/T212449 (10greg) 05Open→03Resolved FY1819 is over, closing this task. Still relevant sub-tasks will be tracked separately. [23:45:41] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: Add tests to local-charts / configure local-charts for CI - https://phabricator.wikimedia.org/T217868 (10brennen) See also: T216049: add CI job into operations/deployments-charts repo that helm lint packages and perform the helm index...