[00:00:51] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:03:31] any keyholder experts around? [00:07:53] Krenair: I can maybe help, trying not to be around too much longer though :) [00:09:08] never mind [00:09:13] volans had the answer [00:09:36] see -traffic logs if interested, tl;dr agent refused to sign stuff until I restarted keyholder-proxy [00:10:01] ah [00:10:01] apparently it does this after first install by puppet [00:10:10] 00:09:09 sync-dir failed: Command 'find -O2 '/srv/mediawiki-staging/php-1.33.0-wmf.21/' -not -type d -name '*.php' -not -name 'autoload_static.php' -or -name '*.inc' | xargs -n1 -P30 -exec php -l >/dev/null 2>&1' returned non-zero exit status 125 [00:10:12] :( [00:11:18] linting fails? [00:11:46] Not sure [00:12:19] Or tooo many files [00:15:56] 10Release-Engineering-Team (Kanban), 10Code-Stewardship-Reviews, 10MediaWiki-extensions-UserMerge, 10Stewards-and-global-tools: UserMerge: Code Stewardship Review - https://phabricator.wikimedia.org/T204747 (10Zer00CooL) OK thank you for your answer. In this regard, I completed the thread, since the link m... [00:23:27] (03PS2) 10Paladox: Fix blocking users [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497429 [00:37:02] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: Work out how we are going to cut deploy branches without gerrit HTTP Token - https://phabricator.wikimedia.org/T218597 (10thcipriani) I've edited the setup section of the train docs to use ssh rather than http token... [00:38:46] 10Gerrit, 10Release-Engineering-Team (Next), 10DBA, 10Operations: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532 (10Paladox) p:05Normal→03High Per T200739#5034407 [01:05:53] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: Work out how we are going to cut deploy branches without gerrit HTTP Token - https://phabricator.wikimedia.org/T218597 (10Legoktm) Just to be clear, is the removal of the Gerrit HTTP token feature going to be a perma... [01:25:39] (03CR) 10Ottomata: "I think this is breaking tag push builds for eventgate-ci:" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/492225 (owner: 10Thcipriani) [01:32:40] 10Beta-Cluster-Infrastructure, 10Acme-chief: Write designate integration script for certcentral DNS challenges - https://phabricator.wikimedia.org/T206922 (10Krenair) I finally got around to writing the basic script today (copy at deployment-acme-chief03:/usr/local/bin/acme-chief-designate-sync.py) and it pret... [01:47:07] 10Release-Engineering-Team (Kanban), 10Code-Stewardship-Reviews, 10MediaWiki-extensions-UserMerge, 10Stewards-and-global-tools: UserMerge: Code Stewardship Review - https://phabricator.wikimedia.org/T204747 (10Tgr) >>! In T204747#5034543, @Zer00CooL wrote: > If UserMerge is no longer supported, how do you... [02:04:23] (03PS1) 10Krinkle: Disable 'release-quibble-vendor-mysql-php72-docker' job on wmf/ [integration/config] - 10https://gerrit.wikimedia.org/r/497437 [02:09:48] !log sudo aptly repo create -component="main" -distribution="buster-deployment-prep" buster-deployment-prep [02:09:48] Krenair: Failed to log message to wiki. Somebody should check the error logs. [02:21:02] !log sudo aptly publish --architectures="all,amd64" --skip-signing repo buster-deployment-prep [02:21:03] Krenair: Failed to log message to wiki. Somebody should check the error logs. [02:26:46] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Krinkle) [03:44:22] 10Release-Engineering-Team (Kanban): Find out about OS X / iOS building/testing from friend - https://phabricator.wikimedia.org/T217675 (10brennen) 05Open→03Resolved Had some discussion here with friends who've worked in the iOS ecosystem. Consensus was pretty much that virtualization is difficult and they... [03:44:24] 10Release-Engineering-Team (Kanban): Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10brennen) [05:25:59] (03PS2) 10Mill: vaaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/497437 (owner: 10Krinkle) [05:35:33] (03PS4) 10Mill: 7gaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/496894 (owner: 10Jforrester) [05:35:36] (03PS3) 10Mill: fhaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/497079 (owner: 10Jforrester) [05:35:38] brennen: greg-g: jrbranaa: liw: longma: mark: thcipriani: twentyafterfour: zeljkof: who can block a gerrit user? flooding with vandalism galore ^ [05:35:55] AndyRussG|ish: on it [05:36:08] twentyafterfour: fantastic, thanks [05:36:21] so far only looks like user Mill, who knows tho [05:36:40] hmm gerrit loading very slowly [05:36:47] thanks twentyafterfour - was about to try and figure out how to go about. [05:37:02] uploading patchsets like crazy, I hope that users has no +2 rights anywhere [05:42:29] (03PS2) 10Mill: ioaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/495997 (owner: 10Daimona Eaytoy) [05:42:42] (03PS3) 10Mill: toaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/495939 (owner: 10MacFan4000) [05:57:05] (03PS3) 10Mill: yqaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/491793 (owner: 10Volans) [05:57:14] (03PS3) 10Mill: 7qaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/491831 (owner: 10Mahveotm) [05:57:38] (03PS6) 10Mill: jraaaaaaaaaaaa [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/486702 (owner: 10Paladox) [05:59:34] (03PS27) 10Mill: praaaaaaaaaaaa [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/488579 (owner: 10Paladox) [06:00:17] (03PS4) 10Mill: 3taaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/487786 (owner: 10Gehel) [06:03:44] (03PS3) 10Mill: f7aaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/460552 (owner: 10Zfilipin) [06:04:41] (03PS5) 10Mill: (9aaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/441178 (owner: 10WMDE-leszek) [06:05:07] (03PS4) 10Mill: $abaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/400002 (owner: 10Florianschmidtwelzow) [06:05:45] (03PS3) 10Mill: adbaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/469069 (owner: 10Awight) [06:09:58] (03PS1) 10Smalyshev: Add WikibaseLexemeCirrusSearch extension [tools/release] - 10https://gerrit.wikimedia.org/r/497457 (https://phabricator.wikimedia.org/T216206) [06:11:05] (03CR) 10jerkins-bot: [V: 04-1] 7gaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/496894 (owner: 10Jforrester) [06:11:11] (03CR) 10jerkins-bot: [V: 04-1] fhaaaaaaaaaaaa [integration/config] - 10https://gerrit.wikimedia.org/r/497079 (owner: 10Jforrester) [06:49:27] PROBLEM - zuul_gearman_service on contint1001 is CRITICAL: connect to address 127.0.0.1 and port 4730: Connection refused [06:50:03] PROBLEM - zuul_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-server [06:50:11] PROBLEM - zuul_merger_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger [06:52:48] ACKNOWLEDGEMENT - zuul_gearman_service on contint1001 is CRITICAL: connect to address 127.0.0.1 and port 4730: Connection refused Vgutierrez maintenance [06:52:48] ACKNOWLEDGEMENT - zuul_merger_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger Vgutierrez maintenance [06:52:48] ACKNOWLEDGEMENT - zuul_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-server Vgutierrez maintenance [06:54:22] Project beta-code-update-eqiad build #239346: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239346/ [07:06:13] Project mediawiki-core-doxygen-docker build #5525: 04FAILURE in 2 min 9 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5525/ [07:06:34] Project beta-code-update-eqiad build #239347: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239347/ [07:18:44] Project beta-code-update-eqiad build #239348: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239348/ [07:25:08] perhaps not worth dealing with just this moment, probably fallout from, but for posterity uploading a new patchset to gerrit with Bug: Txxx created a phab post by gerritbot announcing a related patch set was uploaded, but gerritbot had already announced that patch a few days ago [07:25:38] https://phabricator.wikimedia.org/T212888#5018297 and https://phabricator.wikimedia.org/T212888#5034956 [07:30:58] Project beta-code-update-eqiad build #239349: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239349/ [07:38:22] 10Release-Engineering-Team (Kanban): Find out about OS X / iOS building/testing from friend - https://phabricator.wikimedia.org/T217675 (10hashar) For the IOS application, the development moved to Github to be able to use a third party that provides MacOS / Iphone etc as a service. That has been deemed easier an... [07:42:02] twentyafterfour: did you see my proposal re locking Gerrit down a bit? With some sort of trusted group? [07:42:25] 10Continuous-Integration-Infrastructure, 10Quibble, 10Wikimedia-production-error (Shared Build Failure): When git-submodule clone fails in Quibble, the error should not be ignored - https://phabricator.wikimedia.org/T215689 (10hashar) [07:42:45] addshore: that is a good suggestion, not sure how to implement it though [07:42:46] 10Continuous-Integration-Config, 10Growth-Team, 10StructuredDiscussions, 10Patch-For-Review, 10User-kostajh: Fix Flow random test failures - https://phabricator.wikimedia.org/T208988 (10hashar) Indeed, it did try to process the submodules but since Gerrit was not responsive/down the clones failed: ` INFO... [07:43:09] Project beta-code-update-eqiad build #239350: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239350/ [07:43:48] twentyafterfour: I'm sure we could figure it out if we could see the UI [07:44:04] :) [07:44:59] It depends exactly what groups we have on our Gerrit, and where the underlying access for "registered users" o [07:44:59] Comes from [07:47:36] I guess we can't get to Gerrit from within / proxying through the cluster to try and figure it out? [07:55:22] Project beta-code-update-eqiad build #239351: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239351/ [08:06:13] Project mediawiki-core-doxygen-docker build #5526: 04STILL FAILING in 2 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5526/ [08:07:30] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-production-error (Shared Build Failure): Merge blocker: The table 'l10n_cache' is full in quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T217654 (10Petar.petkovic) [08:07:33] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_jenkins CI slave scripts] [08:07:33] Project beta-code-update-eqiad build #239352: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239352/ [08:07:48] 10Continuous-Integration-Infrastructure, 10Upstream: chromium 72 crash when used with --remote-debugging-port - https://phabricator.wikimedia.org/T216702 (10Petar.petkovic) [08:08:22] 10Continuous-Integration-Infrastructure, 10Cleanup, 10Repository-Admins, 10Wikibase-Quality, and 3 others: Archive the WikibaseQualityExternalValidation extension - https://phabricator.wikimedia.org/T204490 (10Petar.petkovic) [08:10:01] 10RelEng-Archive-FY201718-Q1, 10ContentTranslation, 10Browser-Tests, 10User-zeljkofilipin: ContentTranslation: Migrate browser tests from Ruby to node.js - https://phabricator.wikimedia.org/T174595 (10Petar.petkovic) [08:10:57] 10Release-Engineering-Team (Kanban): Find out about OS X / iOS building/testing from friend - https://phabricator.wikimedia.org/T217675 (10TheDJ) My 2 cents... From having had to maintain Xcode on a Mac as CI for a number of years in house myself, I wholeheartedly agree that doing it externally is cheaper and ea... [08:19:46] Project beta-code-update-eqiad build #239353: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239353/ [08:21:19] Uh [08:21:33] I got pinged and looks like someone spammed [08:29:18] addshore: it’s easy to make gerrit read only for anon users [08:30:46] You just need to edit All-Projects and revoke the read access to ref/* for anons [08:31:58] Project beta-code-update-eqiad build #239354: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239354/ [08:44:11] Project beta-code-update-eqiad build #239355: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239355/ [08:56:22] Project beta-code-update-eqiad build #239356: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239356/ [09:06:13] Project mediawiki-core-doxygen-docker build #5527: 04STILL FAILING in 2 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5527/ [09:08:33] Project beta-code-update-eqiad build #239357: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239357/ [09:20:44] Project beta-code-update-eqiad build #239358: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239358/ [09:32:55] Project beta-code-update-eqiad build #239359: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239359/ [09:39:49] 10Continuous-Integration-Config, 10Project-Admins: Create phabricator tag to track CI blockers (#jenkins-failure) - https://phabricator.wikimedia.org/T218043 (10Lucas_Werkmeister_WMDE) Thanks! [09:45:06] Project beta-code-update-eqiad build #239360: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239360/ [09:57:18] Project beta-code-update-eqiad build #239361: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239361/ [10:05:58] 10Release-Engineering-Team (Kanban): Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10zeljkofilipin) [10:06:01] 10Release-Engineering-Team (Kanban), 10Jenkins: Evaluate Jenkins - https://phabricator.wikimedia.org/T218333 (10zeljkofilipin) 05Open→03Declined We're already using Jenkins, so no need to evaluate it. [10:06:14] Project mediawiki-core-doxygen-docker build #5528: 04STILL FAILING in 2 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5528/ [10:07:50] 10Release-Engineering-Team (Kanban): Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10zeljkofilipin) [10:07:52] 10Release-Engineering-Team (Kanban): Evaluate Tekton Pipeline - https://phabricator.wikimedia.org/T217912 (10zeljkofilipin) 05Open→03Declined After some initial evaluation, we have decided it does not meet our requirements. [10:09:29] Project beta-code-update-eqiad build #239362: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239362/ [10:18:59] Hi what is wrong with Gerrit? In last week it was down several times and it is down today too [10:21:31] Dvorapa: emergency maintenance [10:21:40] Project beta-code-update-eqiad build #239363: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239363/ [10:22:36] I see [10:24:44] 10Release-Engineering-Team (Kanban), 10Quibble, 10Patch-For-Review: Quibble should errors out when a git submodule fails - https://phabricator.wikimedia.org/T198980 (10hashar) a:03hashar Gerrit is not reachable but here is the patch: {F28420233} [10:24:48] 10Release-Engineering-Team (Kanban), 10Quibble, 10Patch-For-Review: Quibble should errors out when a git submodule fails - https://phabricator.wikimedia.org/T198980 (10hashar) p:05Triage→03High [10:33:51] Project beta-code-update-eqiad build #239364: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239364/ [10:46:02] Project beta-code-update-eqiad build #239365: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239365/ [10:58:13] Project beta-code-update-eqiad build #239366: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239366/ [11:06:12] Project mediawiki-core-doxygen-docker build #5529: 04STILL FAILING in 2 min 9 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5529/ [11:10:25] Project beta-code-update-eqiad build #239367: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239367/ [11:13:46] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Evaluate Phabricator Harbormaster - https://phabricator.wikimedia.org/T217901 (10zeljkofilipin) >>! In T217901#5033866, @Aklapper wrote: > @zeljkofilipin: - @20after4; + @mmodell. Thanks! [11:14:10] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Evaluate Phabricator Harbormaster - https://phabricator.wikimedia.org/T217901 (10zeljkofilipin) >>! In T217901#5034272, @mmodell wrote: > @zeljkofilipin Harbormaster can do more than that, however, not without some customization of the code (which isn'... [11:15:02] 10Release-Engineering-Team (Kanban): Evaluate Spinnaker - https://phabricator.wikimedia.org/T218335 (10zeljkofilipin) a:03zeljkofilipin [11:15:14] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Evaluate Spinnaker - https://phabricator.wikimedia.org/T218335 (10zeljkofilipin) [11:17:11] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: Work out how we are going to cut deploy branches without gerrit HTTP Token - https://phabricator.wikimedia.org/T218597 (10zeljkofilipin) >>! In T218597#5034575, @thcipriani wrote: > I've edited the setup section of t... [11:22:35] Project beta-code-update-eqiad build #239368: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239368/ [11:25:59] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) @greg T91454 has been resolved by @Krinkle, but I don't see it deployed. I've asked for clarification at T91454#5035445. [11:26:40] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: Work out how we are going to cut deploy branches without gerrit HTTP Token - https://phabricator.wikimedia.org/T218597 (10zeljkofilipin) a:03zeljkofilipin [11:34:47] Project beta-code-update-eqiad build #239369: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239369/ [11:35:40] 10Phabricator (Upstream), 10WMSE-Bug-Reporting-and-Translation-2019, 10Upstream: Hide archived projects on subproject page - https://phabricator.wikimedia.org/T218041 (10Peachey88) The simplest solution that already exists is that phab supports nesting subprojects, so you could have `WMSE → 2020 → Wikipedia-... [11:40:18] 10Phabricator (Upstream), 10WMSE-Bug-Reporting-and-Translation-2019, 10Upstream: Hide archived projects on subproject page - https://phabricator.wikimedia.org/T218041 (10Aklapper) Maybe showing first active subprojects, and then archived subprojects? Hiding might be more confusing? [11:40:59] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: Work out how we are going to cut deploy branches without gerrit HTTP Token - https://phabricator.wikimedia.org/T218597 (10mmodell) >>! In T218597#5034638, @Legoktm wrote: > Just to be clear, is the removal of the Ger... [11:42:04] 10Phabricator (Upstream), 10WMSE-Bug-Reporting-and-Translation-2019, 10Upstream: Hide archived projects on subproject page - https://phabricator.wikimedia.org/T218041 (10mmodell) @aklapper: I like the idea of simply sorting the active ones first. I'll see if that is difficult to implement. [11:46:58] Project beta-code-update-eqiad build #239370: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239370/ [11:59:09] Project beta-code-update-eqiad build #239371: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239371/ [12:06:12] Project mediawiki-core-doxygen-docker build #5530: 04STILL FAILING in 2 min 9 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5530/ [12:11:20] Project beta-code-update-eqiad build #239372: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239372/ [12:23:31] Project beta-code-update-eqiad build #239373: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239373/ [12:35:42] Project beta-code-update-eqiad build #239374: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239374/ [12:47:53] Project beta-code-update-eqiad build #239375: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239375/ [13:00:05] Project beta-code-update-eqiad build #239376: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239376/ [13:06:13] Project mediawiki-core-doxygen-docker build #5531: 04STILL FAILING in 2 min 9 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5531/ [13:07:37] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) It's train time now, but it's delayed because Gerrit is down. [13:12:16] Project beta-code-update-eqiad build #239377: 04STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239377/ [13:14:40] Yippee, build fixed! [13:14:40] Project beta-code-update-eqiad build #239378: 09FIXED in 2 min 24 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/239378/ [13:20:11] RECOVERY - zuul_service_running on contint1001 is OK: PROCS OK: 2 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-server [13:20:15] RECOVERY - zuul_merger_service_running on contint1001 is OK: PROCS OK: 1 process with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger [13:20:35] RECOVERY - zuul_gearman_service on contint1001 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 4730 [13:25:51] o/ i think yall are super busy but just in case [13:26:01] the eventgate-ci service pipeline build on tag job is faililng [13:26:07] https://integration.wikimedia.org/ci/job/service-pipeline-test-and-publish/72/console [13:26:11] zeljkof: will you be running the train today? [13:26:20] addshore: Yeah, he's gone for lunch [13:26:25] is the branch cut? [13:26:28] *checks* [13:26:30] I'm guessing not [13:26:34] With gerrit being down and everything :P [13:26:36] i think because of https://gerrit.wikimedia.org/r/#/c/integration/pipelinelib/+/492225/ [13:26:40] Reedy: indeed :P [13:26:56] i think mabye thcipriani is the right ping? (but maybe he is not online yet... :) ) [13:27:00] Reedy: zeljkof I have some things that I have been trying to merge itno master since mid last week that still havnt landed yet and should be in the branch [13:27:11] they are already deployed in prod [13:27:40] twentyafterfour: fyi your wikitech patch may need manually merged due to zuul not being on when you +2'd even though I ran recheck to try to get it to run through [13:28:14] (03PS1) 10Hashar: Properly abort when git submodule processing fails [integration/quibble] - 10https://gerrit.wikimedia.org/r/497474 (https://phabricator.wikimedia.org/T198980) [13:28:28] ottomata: yeah, I need to fix that, it is actually pushing and succeeding, it just fails the job due to the change you mentioned [13:28:57] bah, and why is https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-postgres-php70-docker/2552/ failing, bah [13:29:58] addshore: yes, train should start soon [13:30:04] Should I wait? [13:31:05] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [13:31:55] zeljkof: well, right now master wont cleanly merged them as CI is broken because of a commit in core that needs reverting to fix CI [13:32:15] but now CI for postgress is also broken, so the revert wont merge... [13:32:47] the patches i want in are already on the branch and deployed, perhaps i should just force merge them into master? [13:32:47] bah [13:33:04] Gotta love gerrit maint [13:33:30] zeljkof: or, we can just merge them on the branch once you make it? and leave master to slowly tick in [13:34:24] also, tbh, it is fine to not have it on test, but backporting it = extra effort, but could be fine, this is all just a bit meh! [13:35:59] PROBLEM - Puppet errors on integration-slave-docker-1053 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:37:09] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:37:52] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:38:49] (03PS3) 10Hashar: zuul: remove "check" pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/493188 (https://phabricator.wikimedia.org/T192217) [13:38:57] PROBLEM - Puppet errors on integration-slave-docker-1034 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:39:35] PROBLEM - Puppet errors on integration-slave-docker-1048 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:40:25] PROBLEM - Puppet errors on integration-slave-docker-1049 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:42:47] PROBLEM - Puppet errors on integration-slave-docker-1043 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:44:37] addshore: uh huh, no easy answers, I guess I need to start cutting the branch [13:45:50] zeljkof: jfdi :) [13:45:59] Stuff can be backported/cherry picked if needed [13:46:08] Reedy: yeah, that's the plan :) [13:50:22] thcipriani: ok thanks, do you have a timeline on a fix for that? [13:50:26] should I make a ticket? [13:50:37] it means i can't deploy :/ :) [13:51:07] ottomata: why does it mean that? your images are being created and pushed to the docker registry afaict [13:51:14] PROBLEM - Puppet errors on integration-slave-docker-1050 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:51:24] thcipriani: i look for them here [13:51:25] https://tools.wmflabs.org/dockerregistry/wikimedia/eventgate-ci/tags/ [13:51:33] i think they are removed when the job fails [13:51:53] or maybe not? [13:52:00] oh ya [13:52:01] Untagged: docker-registry.discovery.wmnet/wikimedia/eventgate-ci:v1.0.4-wmf3 [13:52:06] + docker rmi --force 915f3f206243 [13:52:13] that's just a local thing on contint1001 [13:52:18] hm ok [13:52:26] they should be in the registry still [13:52:32] PROBLEM - Puppet errors on integration-slave-docker-1052 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:52:39] is your ui just stale for some reason then ? :) [13:52:45] OH [13:52:46] the toollabs thing is cronned to run every 10 minutes or so [13:52:51] so yeah, probably stale :) [13:52:58] hmm [13:53:28] the easiest way to check would be to try to fetch the tag from the registry, I suppose [13:54:00] looking up docker registry http api... [13:54:26] PROBLEM - Puppet errors on integration-slave-docker-1041 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:54:59] docker pull docker-registry.wikimedia.org/wikimedia/eventgate-ci:[tag] [13:56:52] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:58:06] thcipriani: aye that'll work , i'm looking for an http api way to just check tho. [13:58:12] something like curl https://docker-registry.wikimedia.org/v2/wikimedia/eventgate-ci/tags [13:58:19] should work, but i'm not sure what is in the api docs yet... [13:58:29] oh! [13:58:30] /list [13:58:31] got it [13:58:38] curl https://docker-registry.wikimedia.org/v2/wikimedia/eventgate-ci/tags/list [13:58:39] yeehaw [13:58:41] thank you. ok [13:59:02] sorry for the confusion created by that patch :( [13:59:10] it's on my todo list to fix [13:59:17] PROBLEM - Puppet errors on integration-slave-docker-1037 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:59:19] (03CR) 10Hashar: [C: 03+2] "> Nice. Idea for later: a unit test for zuul layout to confirm all pipelines have a whitelist, or are for +2, or are for native git events" [integration/config] - 10https://gerrit.wikimedia.org/r/493188 (https://phabricator.wikimedia.org/T192217) (owner: 10Hashar) [13:59:25] PROBLEM - Puppet errors on integration-slave-jessie-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:59:31] but I was hoping it wasn't actually blocking anyone, just confusing them :) [13:59:42] haha it is true! [14:00:01] PROBLEM - Puppet errors on integration-slave-docker-1051 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [14:01:11] (03Merged) 10jenkins-bot: zuul: remove "check" pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/493188 (https://phabricator.wikimedia.org/T192217) (owner: 10Hashar) [14:01:13] !log Removed Zuul "check" pipeline | https://gerrit.wikimedia.org/r/#/c/integration/config/+/493188/ | T192217 [14:01:15] hashar: Failed to log message to wiki. Somebody should check the error logs. [14:01:15] T192217: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 [14:01:23] one less pipeline yeah [14:04:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 (10hashar) 05Open→03Resolved a:03hashar The `check` pipeline is gone! [14:06:39] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 (10Daimona) Nitpicky comment: the description "Jobs restricted to trusted users. Will vote +2." for the test pipeli... [14:07:19] (03PS5) 10Gehel: sonar: run sonar analysis as a pre-merge step [integration/config] - 10https://gerrit.wikimedia.org/r/487786 (https://phabricator.wikimedia.org/T215135) [14:09:15] (03CR) 10jerkins-bot: [V: 04-1] sonar: run sonar analysis as a pre-merge step [integration/config] - 10https://gerrit.wikimedia.org/r/487786 (https://phabricator.wikimedia.org/T215135) (owner: 10Gehel) [14:11:24] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) Cutting the branch. [14:13:14] Yippee, build fixed! [14:13:14] Project mediawiki-core-doxygen-docker build #5532: 09FIXED in 9 min 9 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/5532/ [14:15:27] 10Continuous-Integration-Infrastructure, 10MediaWiki-Documentation, 10Regression: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) At least PHP works since that generates https://doc.wikimedia.org/ The file `/srv/docroot/org/wikimedia/doc/mediawi... [14:19:35] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) Yes from http://php.net/manual/en/language.basic-syntax.phptags.php **Cha... [14:20:12] hashar: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/497466/ has +2 but jenkins isnt doing gate and submit [14:20:34] Zppix: please fill it as a task :) [14:21:20] Zppix: oh no sorry [14:21:38] Zppix: CI has been disabled this morning, so the patch needs a new Code-Review+2 [14:22:43] 10Release-Engineering-Team, 10User-Zppix: Jenkins is running Gate and Submit after Gerrit maint - https://phabricator.wikimedia.org/T218679 (10Zppix) [14:22:52] hashar: ugh ill close the task [14:23:13] 10Release-Engineering-Team, 10User-Zppix: Jenkins is running Gate and Submit after Gerrit maint - https://phabricator.wikimedia.org/T218679 (10Zppix) 05Open→03Invalid per IRC. [14:28:11] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) https://github.com/doxygen/doxygen/issues/3545 > PHP was not parsed proper... [14:46:03] 10Gerrit, 10Release-Engineering-Team: Grant reedy gerrit admin - https://phabricator.wikimedia.org/T218681 (10Reedy) [15:17:59] 10Gerrit: Create Gerrit Administrator right policy - https://phabricator.wikimedia.org/T218686 (10thcipriani) [15:20:00] 10Gerrit, 10Release-Engineering-Team: Grant reedy gerrit admin - https://phabricator.wikimedia.org/T218681 (10thcipriani) 05Open→03Resolved a:03thcipriani [15:26:14] (03Abandoned) 10Jdlrobson: Edit Project Config [extensions/MobileFrontend] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497459 (owner: 10Jdlrobson) [15:28:57] (03PS3) 10Krinkle: Disable 'release-quibble-vendor-mysql-php72-docker' job on wmf/ [integration/config] - 10https://gerrit.wikimedia.org/r/497437 [15:36:25] (03CR) 10Krinkle: [C: 03+2] Disable 'release-quibble-vendor-mysql-php72-docker' job on wmf/ [integration/config] - 10https://gerrit.wikimedia.org/r/497437 (owner: 10Krinkle) [15:37:56] (03Merged) 10jenkins-bot: Disable 'release-quibble-vendor-mysql-php72-docker' job on wmf/ [integration/config] - 10https://gerrit.wikimedia.org/r/497437 (owner: 10Krinkle) [15:43:38] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10Krinkle) Looks like all the Debian channels (jessie, stretch, buster, sid) are sti... [15:48:18] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497437 [15:48:18] Krinkle: Failed to log message to wiki. Somebody should check the error logs. [15:54:25] (03PS1) 10Paladox: Only allow trusted users access to refs* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497529 [15:56:21] (03PS1) 10Paladox: Only allow trusted users access to refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497530 [15:56:39] 10Release-Engineering-Team (Kanban), 10Release: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) [15:56:55] 10Release-Engineering-Team (Kanban), 10Release: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) [15:56:58] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) [15:57:06] 10Release-Engineering-Team (Kanban), 10Release: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) p:05Triage→03Unbreak! [15:58:22] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10greg) [15:58:39] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) [15:59:52] (03PS2) 10Paladox: Only allow trusted users access to refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497530 [16:08:22] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10Krinkle) A recent Gerrit upgrade removed support for the old `/r/p/` path for accessing repositories over HTTP(S). Any rep... [16:08:55] Krenair: it did? [16:09:18] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10Krinkle) [16:09:30] Oh, was it that super submodule thing? [16:10:26] ? [16:10:28] context? [16:10:33] Oh [16:10:36] Wrong ping sorry [16:10:43] Ment for Krinkle [16:11:13] paladox: Yes, as of a few weeks /r/p/ over https now redirects to a Gerrit search query. [16:11:37] It shouldent though [16:11:50] e.g. https://gerrit.wikimedia.org/r/p/VisualEditor/VisualEditor [16:11:54] Goes to https://gerrit.wikimedia.org/r/#/q/status%3Aopen+project%3AVisualEditor/VisualEditor [16:12:07] Wheras it used to be the same as https://gerrit.wikimedia.org/r/VisualEditor/VisualEditor.git [16:12:18] Ok [16:12:34] (which is 404 in a browser, but is a repo over Git) [16:12:57] I dont know if that change was intentional, but it did happen. Hence the UBN. [16:13:20] paladox: Can you check if other repositories are affected? [16:17:35] They are [16:17:44] At least /p/ redirects for me :( [16:17:55] Though anyways good we migrated [16:18:11] polygerrit breaks /p/ when using it to clone [16:18:20] Hence why I have a change to remove it from zuul [16:24:52] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) This didn't help: {P8231} [16:30:30] thcipriani: are you around? would you have a sec to hop in a hangout and explain something https://meet.google.com/xme-hwba-ksx re: gerrit? [16:34:24] * thcipriani joins [16:36:10] (03CR) 10C. Scott Ananian: "> > That follow up a discussion Cscott and I had on IRC an hour or so" [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:36:34] can i get a volunteer to merge a integration/config change for parsoid? [16:37:43] (03PS1) 10Paladox: Only allow trusted users to read refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 [16:38:02] legoktm is our usual sucker, but anyone who wants to avoid watching me re-learn how to deploy jenkins patches (the last one I merged was ~3 yrs ago) is welcome to help. ;) [16:38:21] now that config should work [16:38:41] https://gerrit.wikimedia.org/r/494802 <- this is the patch [16:38:49] i cannot belive gerrit has a registered group and anon group yet anon acks like registered :( [16:39:35] cscott: Yeah [16:39:49] (03PS5) 10Reedy: Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:40:00] ^ cscott: Presumably that one (just rebased it) [16:40:13] yup, that's the one, thanks [16:40:37] (03CR) 10Reedy: [C: 03+2] Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:41:10] i remember seeing that someone a bunch of php7.1 checks for core/extensions on friday, too, was going to look up who that was and see if they wanted to do parsoid as well ;) [16:42:32] (oh, that was Krinkle: https://gerrit.wikimedia.org/r/496688 ) [16:43:00] im going to be extreamly annoyed with gerrit if https://gerrit.wikimedia.org/r/497547 breaks ci again. [16:43:17] (03CR) 10jerkins-bot: [V: 04-1] Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:44:09] cscott: 16:43:12 MultipleInvalid: extra keys not allowed @ data['projects'][1376]['check'] [16:44:41] (03CR) 10jerkins-bot: [V: 04-1] Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:44:43] cscott: Want to split the patch... One for removing the PHP 7.1 (which we should be able to merge straight off) and one to add the check command? [16:45:28] Reedy: i'm not sure why jenkins is failing? but sure. [16:45:36] where is jnkins failing? [16:45:43] im currently investigating https://phabricator.wikimedia.org/T218702 [16:46:00] removing php 7.1 is the thing which is blocking us right now (because we need to add a library which is php>=7.2) [16:46:20] addshore: https://gerrit.wikimedia.org/r/#/c/integration/config/+/494802/ but i have no idea if it is related [16:46:34] feels unrelated [16:46:55] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) >>! In T218694#5036742, @Krinkle wrote: > A recent Gerrit upgrade removed support for the... [16:47:06] Reedy: MultipleInvalid: extra keys not allowed @ data['projects'][1376]['check'] <- that's the issue with 'check'? [16:47:12] yeah [16:47:18] It's not defined in JJB or something based on the error [16:47:33] So hence suggesting to just split it to make it easier/quicker [16:47:43] Can get the 7.1 removal done, and then work out the other after [16:48:14] addshore: https://gerrit.wikimedia.org/r/#/c/integration/config/+/494802/ but i have no idea if it is related [16:48:25] lol, up enter [16:51:15] (03PS6) 10Reedy: Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:51:15] ci works locally so nothing will break! [16:51:46] cscott: ^ I propose merging that to unblock you [16:51:52] (03PS1) 10Jforrester: .gitmodules: Update gerrit URI syntax, deprecated ages ago, now unsupported [integration/jenkins] - 10https://gerrit.wikimedia.org/r/497555 (https://phabricator.wikimedia.org/T218694) [16:53:01] (03CR) 10C. Scott Ananian: [C: 03+1] "The 'check' rule is causing validation errors:" [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:53:07] Reedy: works for me, thanks! [16:53:10] Reedy: i think hashar removed the check pipeline [16:53:14] (03CR) 10Reedy: [C: 03+2] Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:53:16] Reedy: indeed [16:53:20] have to announce it [16:53:21] That might explain it :) [16:53:24] ah [16:53:29] cscott: Ask hasharAway nicely how to add what you want :P [16:53:38] i was just trying to (gradually) move Parsoid closer to the pipeline for other PHP libraries [16:53:53] I guess explains why it works before, and rebase broke it [16:53:55] (I’m glad i caught this before you guys wasted time trying to find out) [16:53:57] so if other php libraries don't have a check pipeline, then parsoid doesn't need one either [16:54:08] cheers Zppix :) [16:54:14] No problem [16:54:49] thanks [16:55:32] (03Merged) 10jenkins-bot: Stop running php7.1 checks on Parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:55:55] i think i found my ci issue [16:55:57] (03CR) 10C. Scott Ananian: [C: 03+1] "(hashar clarified that the 'check' pipeline has been removed. so, since I was just trying to (gradually) move parsoid closer to the stand" [integration/config] - 10https://gerrit.wikimedia.org/r/494802 (https://phabricator.wikimedia.org/T216102) (owner: 10C. Scott Ananian) [16:55:59] reverting another core patch, yay [16:56:03] * addshore goes for tea [16:56:17] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/494802 [16:56:18] Reedy: Failed to log message to wiki. Somebody should check the error logs. [16:56:18] addshore: not one with my fingerprints on it, i hope? [16:56:24] cscott: nope [16:56:29] cscott: recheck should sort it [16:56:39] Reedy: thanks! [16:56:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 (10Zoranzoki21) >>! In T192217#5035989, @Daimona wrote: > Nitpicky comment: the description "Jobs restricted to tru... [16:57:04] If theres anything i could to help with our gerrit mishaps that weve had lately lmk [16:57:10] Do to* [17:09:54] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) I have send a bug to Debian but I have not received the automatic acknowle... [17:12:12] 10Beta-Cluster-Infrastructure, 10PHP 7.0 support, 10Patch-For-Review: Beta Cluster does not have php7.0-redis available - https://phabricator.wikimedia.org/T217938 (10Etonkovidova) 05Open→03Resolved Thanks! I can access betalabs db now. Resolving the ticket. [17:13:13] cscott: oh, my issue might be relating to some of your patches [17:13:19] cscott: per https://phabricator.wikimedia.org/T218702#5037051 [17:14:14] 10Continuous-Integration-Config, 10MediaWiki-extensions-Scribunto, 10Wikidata: [Task] Add Scribunto to extension-gate in CI - https://phabricator.wikimedia.org/T125050 (10Addshore) p:05High→03Unbreak! I'm going to mark this as unbreak now now, as this has wasted so many people hours over the past weeks /... [17:14:29] James_F: i just made https://phabricator.wikimedia.org/T125050 ubn as im so fed up :( [17:20:36] cscott: sorry, meant to ping you in here not in oeprations [17:22:15] * James_F hugs addshore. [17:22:21] :( [17:22:33] I'm not sure if it's blocked any more. [17:22:37] Or just not yet done. [17:23:08] zeljkof: how did / is the branch cut? [17:25:56] addshore: It didn't get cut because of issues. [17:26:01] oh [17:26:04] addshore: i believe z.eljkof is taking a break and making another attempt later, but there have been various complications. [17:26:07] addshore: https://phabricator.wikimedia.org/T218694 principally. [17:26:11] ack! [17:26:11] Yeah. [17:26:32] in that case, fi we fix wikibase CI and i merge all my patches i wont have to do any backports! [17:26:47] That'd be nice. ;-) [17:26:48] silver lining [17:26:54] And a change. ;-P [17:30:57] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) My local email server configuration was screwed up. The bug is: https://bu... [17:31:53] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10hashar) [17:33:57] (03Abandoned) 10Paladox: Only allow trusted users access to refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497530 (owner: 10Paladox) [17:34:02] (03Abandoned) 10Paladox: Only allow trusted users access to refs* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497529 (owner: 10Paladox) [17:37:59] (03CR) 10Paladox: [V: 03+1] "Verified locally that it works." [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 (owner: 10Paladox) [17:38:09] hashar: do you have time to re-review https://gerrit.wikimedia.org/r/c/mediawiki/core/+/496577/ ? I think there was a misunderstanding there about the purpose of the patch [17:39:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 (10EddieGP) >>! In T192217#5035980, @hashar wrote: > The `check` pipeline is gone! Thanks! It was my mistake to na... [17:42:25] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10Patch-For-Review, 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10Krinkle) >>! In T218694#5036831, @zeljkofilipin wrote: > This didn't help: > > {P... [17:42:45] (03CR) 10Krinkle: [C: 03+2] .gitmodules: Update gerrit URI syntax, deprecated ages ago, now unsupported [integration/jenkins] - 10https://gerrit.wikimedia.org/r/497555 (https://phabricator.wikimedia.org/T218694) (owner: 10Jforrester) [17:43:40] (03Merged) 10jenkins-bot: .gitmodules: Update gerrit URI syntax, deprecated ages ago, now unsupported [integration/jenkins] - 10https://gerrit.wikimedia.org/r/497555 (https://phabricator.wikimedia.org/T218694) (owner: 10Jforrester) [17:45:26] 10Release-Engineering-Team, 10Operations, 10Stashbot: Stashbot: Failed to log message to wiki. SAL can't write to wikitech.wikimedia.org - https://phabricator.wikimedia.org/T218708 (10hashar) [17:45:39] (03PS1) 10Jforrester: gate: Add Scribunto [integration/config] - 10https://gerrit.wikimedia.org/r/497574 (https://phabricator.wikimedia.org/T125050) [17:45:41] (03PS1) 10Jforrester: Drop `extension-quibble` for repos covered by `extension-gate` [integration/config] - 10https://gerrit.wikimedia.org/r/497575 [17:47:38] 10Release-Engineering-Team, 10Operations, 10Stashbot, 10Toolforge: Stashbot: Failed to log message to wiki. SAL can't write to wikitech.wikimedia.org - https://phabricator.wikimedia.org/T218708 (10hashar) I have looked at the bots doc on: https://wikitech.wikimedia.org/wiki/Tool:Stashbot https://wikitech... [17:47:51] (03CR) 10Krinkle: "How much time do its PHPUnit tests currently take on the last passing master/gate commit? If more than a few seconds, might be worth looki" [integration/config] - 10https://gerrit.wikimedia.org/r/497574 (https://phabricator.wikimedia.org/T125050) (owner: 10Jforrester) [17:48:18] (03CR) 10Krinkle: "(In the Jenkin build, the "Test Report" link offers a breakdown of time spent)" [integration/config] - 10https://gerrit.wikimedia.org/r/497574 (https://phabricator.wikimedia.org/T125050) (owner: 10Jforrester) [17:48:51] 10Release-Engineering-Team, 10Operations, 10Stashbot, 10Toolforge: Stashbot: Failed to log message to wiki. SAL can't write to wikitech.wikimedia.org - https://phabricator.wikimedia.org/T218708 (10bd808) [17:51:48] (03CR) 10Krinkle: "Is this meant to remove triggers for jobs on these repos? Looks like a few are. Which ones are intended or main motivation?" [integration/config] - 10https://gerrit.wikimedia.org/r/497575 (owner: 10Jforrester) [17:54:58] (03CR) 10Jforrester: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/497574 (https://phabricator.wikimedia.org/T125050) (owner: 10Jforrester) [17:56:47] tgr: will attempt to revisit https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/496577/ tonight. Sorry it is a bit of a mess here :/ [17:57:01] (not your change, but the situation in general is messy) [17:59:26] I can imagine [17:59:45] sorry for adding one more thing to the pile [18:00:06] but we were hoping to deploy that patch today [18:00:24] (03CR) 10Jforrester: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/497575 (owner: 10Jforrester) [18:11:05] 10Gerrit, 10Phabricator, 10Operations, 10Security-Team: Add gerrit.wikimedia.org to the Phabricator CSP - https://phabricator.wikimedia.org/T218308 (10jijiki) p:05Triage→03Normal [18:12:54] cscott: how is he run parser test fixing going? [18:21:57] 10Gerrit, 10Operations, 10Patch-For-Review: Convert Gerrit to use H2 as the database after 2.16 upgrade - https://phabricator.wikimedia.org/T211139 (10Paladox) p:05Normal→03High Per T200739#5034407 [18:23:00] 10Gerrit, 10Operations, 10serviceops: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10Paladox) 05Open→03Resolved Thank you @hashar. I think this issue is resolved for now. [18:24:19] 10Gerrit, 10Operations, 10serviceops, 10Patch-For-Review: Convert Gerrit to use H2 as the database - https://phabricator.wikimedia.org/T211139 (10Paladox) [18:33:35] (03PS1) 10Kosta Harlan: Add Force Radical to the whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/497582 [18:34:17] tgr: i will delegate to someone else from releng. I am just too tired to think properly at this hour :-( [18:35:31] thanks! [18:37:31] I have poked the back channel [18:40:19] (03PS2) 10Jforrester: Add WikibaseLexemeCirrusSearch extension [tools/release] - 10https://gerrit.wikimedia.org/r/497457 (https://phabricator.wikimedia.org/T216206) (owner: 10Smalyshev) [18:40:43] (03CR) 10Jforrester: [C: 03+2] "No need to make this depend on the repo being in its best state. Let's cut the Gordian knot." [tools/release] - 10https://gerrit.wikimedia.org/r/497457 (https://phabricator.wikimedia.org/T216206) (owner: 10Smalyshev) [18:41:57] (03Merged) 10jenkins-bot: Add WikibaseLexemeCirrusSearch extension [tools/release] - 10https://gerrit.wikimedia.org/r/497457 (https://phabricator.wikimedia.org/T216206) (owner: 10Smalyshev) [18:44:12] addshore: seems to be fixed now [18:44:25] ty cscott and James_F for +2ing those 2 things :) [18:44:30] im going to head off now :) [18:44:52] Always. [18:45:15] i'm watching a few builds complete to be sure, i hope i haven't jinxed things [18:51:39] * James_F grins. [18:58:10] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Upgrade ext-ast to 0.1.4 - https://phabricator.wikimedia.org/T218719 (10Daimona) [18:58:29] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Upgrade ext-ast to 0.1.4 - https://phabricator.wikimedia.org/T218719 (10Daimona) [18:59:53] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Have CI run seccheck tests - https://phabricator.wikimedia.org/T218721 (10Daimona) [19:00:06] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Have CI run seccheck tests - https://phabricator.wikimedia.org/T218721 (10Daimona) [19:04:02] 10Gerrit: Unable to push to a certain gerrit changeset due to "missing revisions" - https://phabricator.wikimedia.org/T218723 (10Krenair) [19:04:44] 10Gerrit: Unable to push to a certain gerrit changeset due to "missing revisions" - https://phabricator.wikimedia.org/T218723 (10Krenair) p:05Triage→03High Guessing there are more of these. [19:08:31] addshore: branch not cut [19:13:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Remove the "check" pipeline and Zuul's user-filter - https://phabricator.wikimedia.org/T192217 (10hashar) [19:18:50] 10Gerrit: Unable to push to a certain gerrit changeset due to "missing revisions" - https://phabricator.wikimedia.org/T218723 (10Krenair) a:03Volans @volans fixed mine. Are there others? [19:22:00] I intend to file the deployment-prep jessie -> stretch/buster migration task later today. [19:31:13] 10Beta-Cluster-Infrastructure: Figure out future for newly created deployment-prep jessie instances - https://phabricator.wikimedia.org/T218609 (10Krenair) [19:36:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Technical-Debt: Clear /srv/.git on contint1001 - https://phabricator.wikimedia.org/T149924 (10hashar) a:05hashar→03None [19:42:06] actually I'll just file what I have and do the real digging later [19:42:35] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:43:05] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:43:07] 10Beta-Cluster-Infrastructure: Figure out future for newly created deployment-prep jessie instances - https://phabricator.wikimedia.org/T218609 (10Krenair) [19:44:42] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:45:17] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:46:19] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:46:44] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [19:46:54] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10Jdforrester-WMF) Theoretically this is Resolved, but leav... [19:54:15] 10Continuous-Integration-Infrastructure, 10Operations-Software-Development, 10Patch-For-Review, 10cloud-services-team (Kanban): puppet broken on integration WMCS instances due to cumin/openstack Debian packages - https://phabricator.wikimedia.org/T218559 (10hashar) that is somehow due to `::profile::openst... [19:56:46] !log integration: sudo cumin --force '*' 'rm /etc/apt/sources.list.d/openstack-mitaka-jessie.list' # T218559 [19:56:48] hashar: Failed to log message to wiki. Somebody should check the error logs. [19:56:48] T218559: puppet broken on integration WMCS instances due to cumin/openstack Debian packages - https://phabricator.wikimedia.org/T218559 [19:56:50] (03PS1) 10Smalyshev: WikibaseLexemeCirrusSearch doesn't have browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/497600 [20:04:32] 10Continuous-Integration-Infrastructure, 10Operations-Software-Development, 10Patch-For-Review, 10cloud-services-team (Kanban): puppet broken on integration WMCS instances due to cumin/openstack Debian packages - https://phabricator.wikimedia.org/T218559 (10bd808) You have a conflicting pin of the python-p... [20:08:23] 10Continuous-Integration-Infrastructure: sury.org packages do not validate gpg - https://phabricator.wikimedia.org/T218735 (10hashar) [20:09:53] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [20:16:03] 10Continuous-Integration-Infrastructure: sury.org packages do not validate gpg - https://phabricator.wikimedia.org/T218735 (10hashar) In Puppet we have a key added via T144872 ` $ gpg --list-options show-keyring ./modules/contint/files/sury-php.gpg gpg: WARNING: no command supplied. Trying to guess what you mea... [20:16:50] 10Gerrit: Unable to push to a certain gerrit changeset due to "missing revisions" - https://phabricator.wikimedia.org/T218723 (10Volans) 05Open→03Resolved They should be all ok. Feel free to re-open in case of issues. [20:21:37] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: sury.org packages do not validate gpg - https://phabricator.wikimedia.org/T218735 (10hashar) Ok sury.org apt fetch seems fixed for now. [20:23:19] !log integration: sudo cumin --force '*' 'rm /etc/apt/preferences.d/jessie_mitaka_pinning_*' # T218559 [20:23:21] hashar: Failed to log message to wiki. Somebody should check the error logs. [20:23:21] T218559: puppet broken on integration WMCS instances due to cumin/openstack Debian packages - https://phabricator.wikimedia.org/T218559 [20:30:05] 10Continuous-Integration-Infrastructure, 10Operations, 10Packaging, 10Patch-For-Review: Upgrade jenkins-debian-glue to v0.20.0 - https://phabricator.wikimedia.org/T212774 (10hashar) I have cherry picked https://gerrit.wikimedia.org/r/495681/3 on the CI puppetmaster and I now have: ` $ apt-cache policy jen... [20:35:21] 10Continuous-Integration-Infrastructure, 10Operations, 10Packaging, 10Patch-For-Review: Upgrade jenkins-debian-glue to v0.20.0 - https://phabricator.wikimedia.org/T212774 (10hashar) Pending the puppet patch to be merged. Otherwise it is essentially solved :] [20:35:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Packaging, 10Upstream: jenkins-debian-glue should run the lintian version from cowbuilder instead of from host - https://phabricator.wikimedia.org/T186494 (10hashar) The patch made it in v0.20.0 and it has been / is being deplo... [20:36:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10Packaging, 10Patch-For-Review: Upgrade jenkins-debian-glue to v0.20.0 - https://phabricator.wikimedia.org/T212774 (10hashar) a:03hashar [20:36:43] 10Release-Engineering-Team (Kanban), 10Zuul, 10Patch-For-Review: Update zuul's GitPython dep - https://phabricator.wikimedia.org/T214739 (10hashar) a:05hashar→03None [20:37:57] 10Continuous-Integration-Infrastructure (Slipway), 10Release-Engineering-Team (Kanban), 10Lexicographical data, 10Wikidata, and 3 others: Migrate selenium-Wikibase-chrome selenium-WikibaseLexeme-chrome to Docker containers - https://phabricator.wikimedia.org/T210285 (10hashar) a:05hashar→03None Complet... [20:39:30] 10Continuous-Integration-Infrastructure (Slipway), 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Migrate DonationInterface CI job to Docker - https://phabricator.wikimedia.org/T203084 (10hashar) a:05hashar→03None There are patch aboves, including one... [20:40:05] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Zuul, 10Patch-For-Review, 10Upstream: Zuul get confused when querying a change that yields two entries - https://phabricator.wikimedia.org/T198968 (10hashar) p:05Triage→03High [20:40:29] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Testing, 10Quibble, 10User-zeljkofilipin: Quibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests - https://phabricator.wikimedia.org/T199116 (10hashar) a:05hashar→03None Sorry, I am definitely no... [20:41:17] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10HHVM, 10Patch-For-Review: hhvm systemd service on deployment-prep reports: hhvm.service: Ignoring invalid environment assignment 'RUN_AS_GROUP=www-data - https://phabricator.wikimedia.org/T209946 (10hashar) a:05hashar→... [20:42:23] (03CR) 10Jforrester: "Theoretically this should now be good to land." [integration/config] - 10https://gerrit.wikimedia.org/r/497322 (https://phabricator.wikimedia.org/T218553) (owner: 10Jforrester) [20:45:54] 10Continuous-Integration-Infrastructure, 10HHVM, 10Language-Team (Language-2019-January-March), 10Wikimedia-production-error (Shared Build Failure): Merge blocker: quibble-vendor-mysql-hhvm-docker in gate fails for most merges (exit status -11) - https://phabricator.wikimedia.org/T216689 (10Petar.petkovic) [20:52:29] 10Continuous-Integration-Infrastructure, 10Operations-Software-Development, 10Patch-For-Review, 10cloud-services-team (Kanban): puppet broken on integration WMCS instances due to openstack Debian packages - https://phabricator.wikimedia.org/T218559 (10Krenair) [20:59:04] !log create operations/software/gerrit/plugins/WikimediaWebSessions project T218739 [20:59:04] paladox: Failed to log message to wiki. Somebody should check the error logs. [21:00:24] !log hiding operations/software/gerrit/plugins/WikimediaWebSessions from normal users [21:00:24] paladox: Failed to log message to wiki. Somebody should check the error logs. [21:02:36] 10Release-Engineering-Team (Kanban), 10MediaWiki-Release-Tools, 10VisualEditor, 10MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), 10User-zeljkofilipin: fatal: Project not found: p/VisualEditor/VisualEditor - https://phabricator.wikimedia.org/T218694 (10zeljkofilipin) a:03zeljkofilipin [21:03:28] PROBLEM - Host integration-slave-docker-1046 is DOWN: CRITICAL - Host Unreachable (172.16.1.115) [21:24:16] RECOVERY - Puppet errors on integration-slave-docker-1037 is OK: OK: Less than 1.00% above the threshold [2.0] [21:25:03] RECOVERY - Puppet errors on integration-slave-docker-1051 is OK: OK: Less than 1.00% above the threshold [2.0] [21:26:52] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [2.0] [21:32:52] RECOVERY - Puppet errors on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [2.0] [21:33:01] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10HHVM, 10Patch-For-Review: hhvm systemd service on deployment-prep reports: hhvm.service: Ignoring invalid environment assignment 'RUN_AS_GROUP=www-data - https://phabricator.wikimedia.org/T209946 (10jijiki) p:05Triage→... [21:33:58] RECOVERY - Puppet errors on integration-slave-docker-1034 is OK: OK: Less than 1.00% above the threshold [2.0] [21:35:26] RECOVERY - Puppet errors on integration-slave-docker-1049 is OK: OK: Less than 1.00% above the threshold [2.0] [21:35:59] RECOVERY - Puppet errors on integration-slave-docker-1053 is OK: OK: Less than 1.00% above the threshold [2.0] [21:36:06] what is the status of train things, if anyone knows? [21:36:12] RECOVERY - Puppet errors on integration-slave-docker-1050 is OK: OK: Less than 1.00% above the threshold [2.0] [21:36:18] I tried to ask this earler but my connection is so horrible it might not have gotten sent [21:37:34] RECOVERY - Puppet errors on integration-slave-docker-1052 is OK: OK: Less than 1.00% above the threshold [2.0] [21:37:50] RECOVERY - Puppet errors on integration-slave-docker-1043 is OK: OK: Less than 1.00% above the threshold [2.0] [21:39:26] RECOVERY - Puppet errors on integration-slave-docker-1041 is OK: OK: Less than 1.00% above the threshold [2.0] [21:39:36] RECOVERY - Puppet errors on integration-slave-docker-1048 is OK: OK: Less than 1.00% above the threshold [2.0] [21:54:25] RECOVERY - Puppet errors on integration-slave-jessie-1004 is OK: OK: Less than 1.00% above the threshold [2.0] [22:00:07] (03PS2) 10Paladox: Only allow trusted users to read refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 [22:07:47] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) >>! In T206676#5036025, @zeljkofilipin wrote: > Cutting the branch. The branch has not been cut because of T218694. [22:08:39] (03CR) 10Jforrester: "Better commit message: https://phabricator.wikimedia.org/P8235" [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 (owner: 10Paladox) [22:09:04] (03PS3) 10Paladox: Create a new group, Trusted Users, who are allowed to read refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 [22:09:10] (03CR) 10Paladox: "> Better commit message: https://phabricator.wikimedia.org/P8235" [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 (owner: 10Paladox) [22:21:45] (03CR) 10Thcipriani: [V: 03+2 C: 03+2] Create a new group, Trusted Users, who are allowed to read refs/* [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/497547 (owner: 10Paladox) [22:23:16] (03PS5) 10Paladox: Docker: Add bazel image [integration/config] - 10https://gerrit.wikimedia.org/r/493638 [22:23:23] (03PS23) 10Paladox: Gerrit: Add CI for operations/software/gerrit (includes new docker image) [integration/config] - 10https://gerrit.wikimedia.org/r/493328 [22:24:02] (03PS3) 10Krinkle: Replace *-jsduck-* jobs with *-node10-docs-* ones (2) [integration/config] - 10https://gerrit.wikimedia.org/r/497322 (https://phabricator.wikimedia.org/T218553) (owner: 10Jforrester) [22:24:09] (03PS4) 10Krinkle: Replace *-jsduck-* jobs with *-node10-docs-* ones (2) [integration/config] - 10https://gerrit.wikimedia.org/r/497322 (https://phabricator.wikimedia.org/T218553) (owner: 10Jforrester) [22:24:23] (03CR) 10Krinkle: [C: 03+2] Replace *-jsduck-* jobs with *-node10-docs-* ones (2) [integration/config] - 10https://gerrit.wikimedia.org/r/497322 (https://phabricator.wikimedia.org/T218553) (owner: 10Jforrester) [22:25:58] (03Merged) 10jenkins-bot: Replace *-jsduck-* jobs with *-node10-docs-* ones (2) [integration/config] - 10https://gerrit.wikimedia.org/r/497322 (https://phabricator.wikimedia.org/T218553) (owner: 10Jforrester) [22:30:02] 10Beta-Cluster-Infrastructure, 10MediaWiki-Database, 10MediaWiki-Documentation: read only message for master DB down misleading - https://phabricator.wikimedia.org/T216496 (10RhinosF1) That should be better [22:33:15] 10Continuous-Integration-Config, 10phan-taint-check-plugin, 10Patch-For-Review: Have CI run seccheck tests - https://phabricator.wikimedia.org/T218721 (10Daimona) a:03Daimona [22:33:46] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Upgrade ext-ast to 0.1.4 - https://phabricator.wikimedia.org/T218719 (10Daimona) @Legoktm could you please take a look? [22:35:20] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [22:44:32] 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair) [22:47:44] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497322 / T218553 [22:47:46] Krinkle: Failed to log message to wiki. Somebody should check the error logs. [22:47:46] T218553: Wikibase CI appears to be broken - https://phabricator.wikimedia.org/T218553 [22:48:47] (03PS4) 10Krinkle: Drop extension-jsduck, no longer used [integration/config] - 10https://gerrit.wikimedia.org/r/497079 (owner: 10Jforrester) [22:49:23] (03Abandoned) 10Jforrester: Drop extension-jsduck, no longer used [integration/config] - 10https://gerrit.wikimedia.org/r/497079 (owner: 10Jforrester) [22:51:16] (03PS5) 10Jforrester: Drop jsduck jobs no longer used after I211c507f957675b [integration/config] - 10https://gerrit.wikimedia.org/r/496894 [22:51:58] (03CR) 10Jforrester: Drop jsduck jobs no longer used after I211c507f957675b (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/496894 (owner: 10Jforrester) [23:03:41] (03PS2) 10Jforrester: Drop `extension-quibble` for repos covered by `extension-gate` [integration/config] - 10https://gerrit.wikimedia.org/r/497575 [23:04:07] (03CR) 10Jforrester: "ping" [integration/config] - 10https://gerrit.wikimedia.org/r/480463 (https://phabricator.wikimedia.org/T204153) (owner: 10Jforrester) [23:05:59] (03CR) 10Jforrester: "Is this still needed? It didn't get merged, but java8-mjolnir (0.2.0) etc. were build in February for java8:0.5.0 and presumably that'd ha" [integration/config] - 10https://gerrit.wikimedia.org/r/485959 (owner: 10Legoktm) [23:18:47] 10Release-Engineering-Team (Kanban), 10Zuul: Evaluate Zuul v3 - https://phabricator.wikimedia.org/T218138 (10brennen) I started with the official [[ https://zuul-ci.org/docs/zuul/admin/quick-start.html | Quick-Start Installation and Tutorial ]], which is built around `docker-compose`. I spun up a Debian Stret... [23:20:00] 10Release-Engineering-Team (Kanban), 10Zuul: Evaluate Zuul v3 - https://phabricator.wikimedia.org/T218138 (10brennen) If anyone else would like to experiment with my demo installation, ping here or on IRC and I'll add your pubkey and write up quick SSH instructions.