[00:00:02] 10Release-Engineering-Team (Kanban), 10Developer Productivity, 10local-charts, 10Epic, 10Patch-For-Review: Automate LocalSettings.php creation for local-charts - https://phabricator.wikimedia.org/T217869 (10jeena) [00:07:43] Project beta-scap-eqiad build #242102: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242102/ [00:17:47] Project beta-scap-eqiad build #242103: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242103/ [00:27:41] Project beta-scap-eqiad build #242104: 04STILL FAILING in 8 min 32 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242104/ [00:30:33] (03CR) 10Legoktm: [C: 03+2] WikibaseLexemeCirrusSearch doesn't have browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/497600 (owner: 10Smalyshev) [00:32:27] (03Merged) 10jenkins-bot: WikibaseLexemeCirrusSearch doesn't have browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/497600 (owner: 10Smalyshev) [00:32:44] legoktm: thanks! [00:33:07] !log deploying https://gerrit.wikimedia.org/r/497600 [00:33:07] legoktm: Failed to log message to wiki. Somebody should check the error logs. [00:33:17] SMalyshev: you should be good now [00:37:58] Project beta-scap-eqiad build #242105: 04STILL FAILING in 8 min 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242105/ [00:47:58] Project beta-scap-eqiad build #242106: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242106/ [00:57:54] Project beta-scap-eqiad build #242107: 04STILL FAILING in 8 min 32 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242107/ [01:07:20] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Arlolra) [01:07:50] Project beta-scap-eqiad build #242108: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242108/ [01:17:55] Project beta-scap-eqiad build #242109: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242109/ [01:28:03] Project beta-scap-eqiad build #242110: 04STILL FAILING in 8 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242110/ [01:37:59] Project beta-scap-eqiad build #242111: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242111/ [01:47:51] Project beta-scap-eqiad build #242112: 04STILL FAILING in 8 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242112/ [01:57:55] Project beta-scap-eqiad build #242113: 04STILL FAILING in 8 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242113/ [02:07:47] Project beta-scap-eqiad build #242114: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242114/ [02:17:49] Project beta-scap-eqiad build #242115: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242115/ [02:19:47] (03PS1) 10Jack Phoenix: Remove my old email address from CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/497975 [02:21:14] (03PS1) 10Jack Phoenix: Re-enable ArticleFeedbackv5 CI integration [integration/config] - 10https://gerrit.wikimedia.org/r/497976 (https://phabricator.wikimedia.org/T218759) [02:27:48] Project beta-scap-eqiad build #242116: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242116/ [02:37:40] Project beta-scap-eqiad build #242117: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242117/ [02:47:38] Project beta-scap-eqiad build #242118: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242118/ [02:51:41] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Epic: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Tgr) [02:51:42] (03PS3) 10Kosta Harlan: [WIP] Add Parsoid to docker image and run for Selenium tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/497222 (https://phabricator.wikimedia.org/T218534) [02:52:15] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add Parsoid to docker image and run for Selenium tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/497222 (https://phabricator.wikimedia.org/T218534) (owner: 10Kosta Harlan) [02:56:27] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Tgr) [02:57:28] Project beta-scap-eqiad build #242119: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242119/ [03:07:11] Project beta-scap-eqiad build #242120: 04STILL FAILING in 8 min 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242120/ [03:17:07] Project beta-scap-eqiad build #242121: 04STILL FAILING in 8 min 35 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242121/ [03:27:02] Project beta-scap-eqiad build #242122: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242122/ [03:37:05] Project beta-scap-eqiad build #242123: 04STILL FAILING in 8 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242123/ [03:46:58] Project beta-scap-eqiad build #242124: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242124/ [03:56:55] Project beta-scap-eqiad build #242125: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242125/ [04:06:40] Project beta-scap-eqiad build #242126: 04STILL FAILING in 8 min 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242126/ [04:16:16] Project beta-scap-eqiad build #242127: 04STILL FAILING in 8 min 14 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242127/ [04:26:08] Project beta-scap-eqiad build #242128: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242128/ [04:36:11] Project beta-scap-eqiad build #242129: 04STILL FAILING in 8 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242129/ [04:45:51] Project beta-scap-eqiad build #242130: 04STILL FAILING in 8 min 19 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242130/ [04:50:59] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, 10good first bug: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Ammarpad) [04:55:46] Project beta-scap-eqiad build #242131: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242131/ [05:05:41] Project beta-scap-eqiad build #242132: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242132/ [05:15:33] Project beta-scap-eqiad build #242133: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242133/ [05:25:32] Project beta-scap-eqiad build #242134: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242134/ [05:35:31] Project beta-scap-eqiad build #242135: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242135/ [05:45:17] Project beta-scap-eqiad build #242136: 04STILL FAILING in 8 min 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242136/ [05:55:06] Project beta-scap-eqiad build #242137: 04STILL FAILING in 8 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242137/ [06:05:02] Project beta-scap-eqiad build #242138: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242138/ [06:14:57] Project beta-scap-eqiad build #242139: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242139/ [06:24:50] Project beta-scap-eqiad build #242140: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242140/ [06:34:40] Project beta-scap-eqiad build #242141: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242141/ [06:44:41] Project beta-scap-eqiad build #242142: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242142/ [06:50:56] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, 10good first bug: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Eisheeta) Hello If the task is available, can I work on this? Thank you! [06:54:47] Project beta-scap-eqiad build #242143: 04STILL FAILING in 8 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242143/ [07:04:48] Project beta-scap-eqiad build #242144: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242144/ [07:14:55] Project beta-scap-eqiad build #242145: 04STILL FAILING in 8 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242145/ [07:17:40] (03CR) 10D3r1ck01: "Awesome Jack, this was already made: I7b282fdc3db696b3eea777df4b9b3bb11ef15cf2. Thanks" [integration/config] - 10https://gerrit.wikimedia.org/r/497976 (https://phabricator.wikimedia.org/T218759) (owner: 10Jack Phoenix) [07:25:01] Project beta-scap-eqiad build #242146: 04STILL FAILING in 8 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242146/ [07:34:55] Project beta-scap-eqiad build #242147: 04STILL FAILING in 8 min 32 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242147/ [07:44:57] Project beta-scap-eqiad build #242148: 04STILL FAILING in 8 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242148/ [07:54:56] Project beta-scap-eqiad build #242149: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242149/ [08:03:39] (03CR) 10Ammarpad: [C: 04-1] "The user has been blocked as a sockpuppet on English Wikipedia, I don't think this should be merged." [integration/config] - 10https://gerrit.wikimedia.org/r/497582 (owner: 10Kosta Harlan) [08:04:51] Project beta-scap-eqiad build #242150: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242150/ [08:14:49] Project beta-scap-eqiad build #242151: 04STILL FAILING in 8 min 35 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242151/ [08:24:25] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10Legoktm) @hashar I built patched debs for buster at Project beta-scap-eqiad build #242152: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242152/ [08:29:22] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, 10good first bug: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Aklapper) Hi Eisheeta, [no need to ask](https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker#Communicate_that_you_work_on_a_task)... [08:34:35] Project beta-scap-eqiad build #242153: 04STILL FAILING in 8 min 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242153/ [08:44:28] Project beta-scap-eqiad build #242154: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242154/ [08:45:00] 10Continuous-Integration-Config, 10phan-taint-check-plugin: Upgrade php-ast to 0.1.4 - https://phabricator.wikimedia.org/T218719 (10Daimona) @Legoktm I checked all the remaining versions up to 1.2.6 and I can now confirm that we only need 0.1.5 for php-ast. 0.1.2 should be kept for the current version only. [08:54:24] Project beta-scap-eqiad build #242155: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242155/ [09:04:24] Project beta-scap-eqiad build #242156: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242156/ [09:14:12] Project beta-scap-eqiad build #242157: 04STILL FAILING in 8 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242157/ [09:15:45] Project beta-scap-eqiad build #242158: 04STILL FAILING in 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242158/ [09:28:52] YAY [09:29:10] 10:15:45 RuntimeError: Scap failed!: Call to mwscript eval.php stderr: groups: cannot find name for group ID 50062 [09:29:13] sounds interesting [09:33:00] Project beta-scap-eqiad build #242159: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242159/ [09:40:37] 10Gerrit, 10Release-Engineering-Team (Kanban): Bless Brennen with Gerrit administrator rights - https://phabricator.wikimedia.org/T218858 (10hashar) [09:41:53] 10Continuous-Integration-Infrastructure, 10Front-end-Standards-Group, 10MediaWiki-extensions-General, 10Core Platform Team Backlog (Designing), 10Services (designing): Decide whether we want the package-lock.json to commit or ignore - https://phabricator.wikimedia.org/T179229 (10Mvolz) @Jdforrester-WMF y... [09:42:53] Project beta-scap-eqiad build #242160: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242160/ [09:47:33] 10Diffusion, 10Repository-Admins, 10Wikidata: Diffusion: wikibase-termbox gerrit mirror connection time out - https://phabricator.wikimedia.org/T218780 (10Pablo-WMDE) FWIW the termbox mirror is now saying "Fully Imported" and HEAD is where we expect it to be. [09:52:58] Project beta-scap-eqiad build #242161: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242161/ [09:55:28] 10Continuous-Integration-Infrastructure, 10HHVM, 10Language-Team (Language-2019-January-March), 10Wikimedia-production-error (Shared Build Failure): Merge blocker: quibble-vendor-mysql-hhvm-docker in gate fails for most merges (exit status -11) - https://phabricator.wikimedia.org/T216689 (10hashar) I have... [09:57:32] addshore: yeah that cannot find name for gorup ID is a known issue. It is due to the WMCS LDAP server being overflowed/timing out [09:57:51] ironically, yesterday I had replied on the task that it looked fine now [09:58:13] 09:52:58 [55e12271489f4c0a4bc109a5] [no req] ExtensionDependencyError from line 293 of /srv/mediawiki-staging/php-master/includes/registration/ExtensionRegistry.php: FundraisingTranslateWorkflow requires Translate to be installed. [09:58:14] .... [10:02:39] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) [10:02:55] Project beta-scap-eqiad build #242162: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242162/ [10:04:26] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) Might be related to https://gerrit.wikimedia.org/r/#/c/operations/mediawi... [10:07:13] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) The first broken build was on March 20th 2019 at 20:59:13 which is before... [10:11:12] hashar: gotcha [10:11:33] addshore: it is another failure of doom https://phabricator.wikimedia.org/T218860 [10:11:36] no clue what is going [10:11:37] on [10:11:59] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) So it is broken and I have no idea what is going on sorry :( [10:12:56] Project beta-scap-eqiad build #242163: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242163/ [10:13:08] !log deployment-deploy01: sudo rm -fR /tmp/mw-cache-master | files were from Mar 15 10:21 [10:13:09] hashar: Failed to log message to wiki. Somebody should check the error logs. [10:15:34] 10Phabricator (Upstream), 10Upstream: drydock: translation typo on source: `chagne` - https://phabricator.wikimedia.org/T218447 (10MarcoAurelio) Upstream fix was imported to our install in 09f374b90c94 as 492b03628f19. The fix was still [[ https://translatewiki.net/wiki/Phabricator:phabricator-drydock-2eef34a0... [10:22:48] Project beta-scap-eqiad build #242164: 04STILL FAILING in 8 min 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242164/ [10:26:14] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) Wondering if that would cause the train to break when we next regenerate... [10:28:47] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10hashar) Note beta cluster can not update T218860 ` mwscript mergeMessageFileList.php --wiki="dewiktionary" --list-file="/srv/mediawiki... [10:29:13] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) [10:32:32] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) That also points to https://gerrit.wikimedia.org/r/#/c/mediawiki/extensio... [10:32:57] Project beta-scap-eqiad build #242165: 04STILL FAILING in 8 min 35 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242165/ [10:43:00] Project beta-scap-eqiad build #242166: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242166/ [10:44:20] (03PS1) 10Addshore: Allow wmde to "Add Patch Set" to others changes. [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498047 [10:50:13] 10Project-Admins: Replace tracking T40638 by new project tag for "Interface messages" - https://phabricator.wikimedia.org/T140077 (10Liuxinyu970226) @Nemo_bis: > That was supposed to be fixed at T140333. {F28435924} Really you say that this is fixed? [10:50:30] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) $wmgFundraisingTranslateWorkflow is only enabled on meta. What I guess i... [10:52:58] Project beta-scap-eqiad build #242167: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242167/ [10:54:23] (03PS2) 10Addshore: Allow wmde to "Add Patch Set" to others changes. [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498047 [10:56:20] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10hashar) A very basic attempt at reproducing the issue on my local setup: ` $ php... [10:57:56] (03CR) 10Ppchelko: [C: 03+1] Switch change-propagation to the pipeline (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/496387 (https://phabricator.wikimedia.org/T213193) (owner: 10Alexandros Kosiaris) [10:59:55] 10Release-Engineering-Team (Next), 10ChangeProp, 10Release Pipeline, 10serviceops, and 2 others: Migrate changeprop to kubernetes - https://phabricator.wikimedia.org/T213193 (10Pchelolo) [11:02:52] Project beta-scap-eqiad build #242168: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242168/ [11:12:29] (03PS1) 10Hashar: Allow project owners to rebase changes [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498049 [11:12:54] Project beta-scap-eqiad build #242169: 04STILL FAILING in 8 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242169/ [11:15:30] (03PS2) 10Hashar: Allow project owners to rebase changes [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498049 [11:17:20] (03CR) 10Hashar: [C: 04-1] "You should be able to add a new patchset as a project owner of the repository. One can try by pushing a change for review with:" [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498047 (owner: 10Addshore) [11:23:04] Project beta-scap-eqiad build #242170: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242170/ [11:24:27] 10Gerrit, 10Repository-Admins: Figure a way to semi or fully automat adding/removing submodules to mediawiki/{extensions|skins}.git - https://phabricator.wikimedia.org/T218866 (10MarcoAurelio) [11:24:35] 10Gerrit, 10Repository-Admins: Figure a way to semi or fully automat adding/removing submodules to mediawiki/{extensions|skins}.git - https://phabricator.wikimedia.org/T218866 (10MarcoAurelio) [11:27:05] hashar: https://phabricator.wikimedia.org/T199384 ? [11:29:52] 10Gerrit: Please rewrite sync-with-gerrit.py to use Gerrit REST API - https://phabricator.wikimedia.org/T194318 (10MarcoAurelio) [11:33:08] Project beta-scap-eqiad build #242171: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242171/ [11:43:05] Project beta-scap-eqiad build #242172: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242172/ [11:47:47] 10Gerrit: [admin] Group description amendment request - https://phabricator.wikimedia.org/T199384 (10hashar) 05Open→03Resolved a:03hashar Changed to `Gerrit project owners for the MwEmbedSupport extension` :) [11:47:50] hauskatze: done ! [11:48:06] hashar: thank you good sir [11:50:29] (03CR) 10Addshore: "This doesn't solve the main issue, which is people being able to work on the same change in repos that they do not own" [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498049 (owner: 10Hashar) [11:52:56] Project beta-scap-eqiad build #242173: 04STILL FAILING in 8 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242173/ [12:02:56] Project beta-scap-eqiad build #242174: 04STILL FAILING in 8 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242174/ [12:12:57] Project beta-scap-eqiad build #242175: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242175/ [12:23:30] Project beta-scap-eqiad build #242176: 04STILL FAILING in 9 min 8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242176/ [12:24:41] (03PS3) 10Hashar: Allow wmde to "Add Patch Set" to others changes. [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498047 (https://phabricator.wikimedia.org/T218869) (owner: 10Addshore) [12:25:06] (03CR) 10Hashar: [V: 03+2 C: 03+2] Allow wmde to "Add Patch Set" to others changes. [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498047 (https://phabricator.wikimedia.org/T218869) (owner: 10Addshore) [12:29:57] (03PS3) 10Hashar: Allow project owners/wmde to rebase changes [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498049 (https://phabricator.wikimedia.org/T218869) [12:33:29] Project beta-scap-eqiad build #242177: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242177/ [12:34:02] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10zeljkofilipin) [12:35:40] 10Release-Engineering-Team (Kanban): Evaluate ArgoCD - https://phabricator.wikimedia.org/T218827 (10zeljkofilipin) p:05Triage→03Normal [12:36:07] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10zeljkofilipin) [12:39:56] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Dzahn) @Eisheeta You could create a Gerrit user and review https://gerrit.wikimedia.org/r/c/operations/puppet/+/498057 if you like. Then there wil... [12:41:15] cloning over /p/ works for me. [12:43:30] Project beta-scap-eqiad build #242178: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242178/ [12:44:45] paladox: has-har fears r/p/ support will be dropped in the future [12:45:03] hauskatze i doin't think that's being dropped anytime soon [12:45:12] idk [12:45:16] at least no one mentioned it to me [12:46:04] when i reported /p/ breaks (ie there's a api over it that zuul uses) and since PolyGerrit added /p/ to https://github.com/GerritCodeReview/gerrit/blob/master/java/com/google/gerrit/httpd/raw/StaticModule.java#L74 [12:46:08] it's broke that api [12:46:11] but [12:46:21] . /p/ is completely optional [12:53:33] Project beta-scap-eqiad build #242179: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242179/ [13:03:31] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Nikitrain) I am working on Code search patch(s). [13:03:32] Project beta-scap-eqiad build #242180: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242180/ [13:05:11] (03CR) 10FR30799386: "> Patch Set 1: Code-Review-1" [integration/config] - 10https://gerrit.wikimedia.org/r/497582 (owner: 10Kosta Harlan) [13:11:39] (03CR) 10Alexandros Kosiaris: [C: 03+1] "\o/ thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/496387 (https://phabricator.wikimedia.org/T213193) (owner: 10Alexandros Kosiaris) [13:13:28] Project beta-scap-eqiad build #242181: 04STILL FAILING in 8 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242181/ [13:23:34] Project beta-scap-eqiad build #242182: 04STILL FAILING in 8 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242182/ [13:26:37] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) All wikis at 1.33.0-wmf.22 ([[ https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/498078 | gerrit ]], [[ htt... [13:27:24] (03Abandoned) 10Jack Phoenix: Re-enable ArticleFeedbackv5 CI integration [integration/config] - 10https://gerrit.wikimedia.org/r/497976 (https://phabricator.wikimedia.org/T218759) (owner: 10Jack Phoenix) [13:29:31] (03CR) 10Jack Phoenix: "LGTM but I think you could remove the comment about T159058 on L3095 since now that you're re-enabling CI, it is no longer relevant." [integration/config] - 10https://gerrit.wikimedia.org/r/497792 (https://phabricator.wikimedia.org/T218759) (owner: 10D3r1ck01) [13:32:34] (03CR) 10D3r1ck01: "> I think you could remove the comment about T159058 on L3095 since now that you're re-enabling CI, it is no longer relevant." [integration/config] - 10https://gerrit.wikimedia.org/r/497792 (https://phabricator.wikimedia.org/T218759) (owner: 10D3r1ck01) [13:33:01] (03PS3) 10D3r1ck01: Re-enable CI tests for ArticleFeedbackv5 extension [integration/config] - 10https://gerrit.wikimedia.org/r/497792 (https://phabricator.wikimedia.org/T218759) [13:33:38] Project beta-scap-eqiad build #242183: 04STILL FAILING in 8 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242183/ [13:37:09] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [13:43:38] Project beta-scap-eqiad build #242184: 04STILL FAILING in 8 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242184/ [13:53:44] Project beta-scap-eqiad build #242185: 04STILL FAILING in 8 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242185/ [13:59:08] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) [14:03:56] Project beta-scap-eqiad build #242186: 04STILL FAILING in 8 min 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242186/ [14:05:32] Project beta-scap-eqiad build #242187: 04STILL FAILING in 9.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242187/ [14:16:45] (03CR) 10Kosta Harlan: "> Ammarpad, the en.wiki template says that I have used multiple accounts ( an IP address )" [integration/config] - 10https://gerrit.wikimedia.org/r/497582 (owner: 10Kosta Harlan) [14:23:14] Project beta-scap-eqiad build #242188: 04STILL FAILING in 8 min 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242188/ [14:33:18] Project beta-scap-eqiad build #242189: 04STILL FAILING in 8 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242189/ [14:43:18] Project beta-scap-eqiad build #242190: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242190/ [14:46:15] twentyafterfour: fyi phab search is moving to es 6 starting today, if anything funny is reported that might be why [14:49:51] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10zeljkofilipin) [14:53:23] Project beta-scap-eqiad build #242191: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242191/ [14:54:13] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10thiemowmde) May I ask why all the patches uploaded so far actually **break** URLs that worked before? [14:54:55] Hi, I have an offboarding request--can I provide someone (twentyafterfour?) with a list of Phab tasks that I'd like to be unsubscribed from? Or maybe I have bulk edit permissions and don't realize it... [14:56:12] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] Re-enable CI tests for ArticleFeedbackv5 extension [integration/config] - 10https://gerrit.wikimedia.org/r/497792 (https://phabricator.wikimedia.org/T218759) (owner: 10D3r1ck01) [14:58:03] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] Remove my old email address from CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/497975 (owner: 10Jack Phoenix) [15:03:24] Project beta-scap-eqiad build #242192: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242192/ [15:05:48] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Aklapper) >>! In T218844#5044257, @thiemowmde wrote: > May I ask why all the patches uploaded so far actually **break** URLs that worked before?... [15:10:46] ebernhardson: I had done some prep for es6 and had to roll some of it back because it wasn't backwards compatible with 5 [15:11:01] but in general phab should be mostly ready [15:11:30] awight: sure I can do a bulk edit for you if you can't do it yourself [15:12:44] twentyafterfour: ty, no rush. I'm making a CSV of the tasks I'd like to unsubscribe from, is that easy to use from your side? [15:13:15] Project beta-scap-eqiad build #242193: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242193/ [15:13:46] awight: the easiest way is by a maniphest saved query I think [15:20:25] twentyafterfour: It doesn't really line up with project boundaries or anything, and I can't find a way to make a search for a list of given task IDs. Maybe I can use Conduit? [15:20:51] Project beta-update-databases-eqiad build #32586: 04FAILURE in 50 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32586/ [15:22:58] !log pruning images/containers on integration-slave-docker-1021 [15:22:59] hashar: Failed to log message to wiki. Somebody should check the error logs. [15:23:28] Project beta-scap-eqiad build #242194: 04STILL FAILING in 8 min 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242194/ [15:25:57] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Paladox) /p/ is completely optional. Cloning over /r/ and /r/p/ work. [15:26:10] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10thiemowmde) > What are the exact steps to see that breakage? […] I don't know. Just try following the link? In https://gerrit.wikimedia.org/r/#/... [15:31:24] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Aklapper) That specific link changes a Phab link and not a Gerrit link, so the patch needs improvement. [15:33:26] Project beta-scap-eqiad build #242195: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242195/ [15:35:01] awight: I can figure something out if you just make a list of task ids [15:35:17] I'm pretty close ;-) [15:35:17] echo '{"transactions": [{"type": "subscribers.remove", "value": ["T113428"]}]}' | arc call-conduit maniphest.edit | python -mjson.tool [15:35:18] T113428: CentralNotice: make-wmf-branch doesn't work for named extension deployment branches - https://phabricator.wikimedia.org/T113428 [15:35:23] for example [15:36:12] ah my user ID goes in "value" [15:39:25] echo '{"transactions": [{"type": "subscribers.remove", "value": ["PHID-USER-p6hvqn5njgnxuagekh4b"]}], "objectIdentifier": "T113428"}' | arc call-conduit maniphest.edit | python -mjson.tool [15:43:26] Project beta-scap-eqiad build #242196: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242196/ [15:50:24] awight: but it's such a bootiful task! [15:51:13] hehe, didn't mean to play favorites [15:51:57] On that note-- twentyafterfour is there a way to not produce one million wikibugs notifications for a bulk unsubscribe? [15:53:27] Project beta-scap-eqiad build #242197: 04STILL FAILING in 8 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242197/ [15:54:14] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Nikitrain) >>! In T218844#5044420, @Aklapper wrote: > That specific link changes a Phab link and not a Gerrit link, so the patch needs improvement... [15:55:05] awight wikibugs does not notify if you unsubscribe [15:55:13] but it does if you make any changes to the task [15:55:49] awight: we used to have a way to avoid any notification spam but that got broken. I think I can still configure a bulk job and silence it manually though [15:56:00] paladox: Perfect, thanks for the knowledge! [15:56:08] :) [15:56:17] I'll test one that I know goes to a channel... [15:59:42] cat awight-unsub.txt | (while read line; do echo "{\"transactions\": [{\"type\": \"subscribers.remove\", \"value\": [\"PHID-USER-p6hvqn5njgnxuagekh4b\"]}], \"objectIdentifier\": \"${line}\"}" | arc call-conduit maniphest.edit | python -mjson.tool; done) [15:59:46] Success! [16:03:30] Project beta-scap-eqiad build #242198: 04STILL FAILING in 8 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242198/ [16:03:36] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10thiemowmde) As far as I understand this ticket here, it is 1. exclusively about **Gerrit** links. 2. exclusively about links that are used to **cl... [16:05:27] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Nikitrain) got it! Thanks! I will take care :) [16:14:11] Project beta-scap-eqiad build #242199: 04STILL FAILING in 8 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242199/ [16:17:51] 10Beta-Cluster-Infrastructure, 10SDC General, 10Wikidata, 10Services (next): No jobs running on beta cluster - https://phabricator.wikimedia.org/T215339 (10Cparle) Just checking in to see if there's any movement on this ... [16:21:34] Yippee, build fixed! [16:21:35] Project beta-update-databases-eqiad build #32587: 09FIXED in 1 min 33 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32587/ [16:24:13] Project beta-scap-eqiad build #242200: 04STILL FAILING in 8 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242200/ [16:26:27] (03CR) 10Ammarpad: "> > Ammarpad, the en.wiki template says that I have used multiple" [integration/config] - 10https://gerrit.wikimedia.org/r/497582 (owner: 10Kosta Harlan) [16:26:35] (03PS4) 10Kosta Harlan: [WIP] Add Parsoid to docker image and run for Selenium tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/497222 (https://phabricator.wikimedia.org/T218534) [16:27:17] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add Parsoid to docker image and run for Selenium tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/497222 (https://phabricator.wikimedia.org/T218534) (owner: 10Kosta Harlan) [16:34:20] Project beta-scap-eqiad build #242201: 04STILL FAILING in 8 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242201/ [16:35:22] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10Jdforrester-WMF) Oh, yeah, you can't use proper dependencies for extensions that... [16:44:20] Project beta-scap-eqiad build #242202: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242202/ [16:46:19] (03PS4) 10Kosta Harlan: Remove requirement for properties file, import coverage if present [integration/config] - 10https://gerrit.wikimedia.org/r/494548 (https://phabricator.wikimedia.org/T215176) [16:54:23] Project beta-scap-eqiad build #242203: 04STILL FAILING in 8 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242203/ [16:54:59] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Tgr) Yeah. this is about links which will end up in somebody's git config as a git remote URL (so URL used for git clone, git remote add, .gitmodu... [16:56:34] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Tgr) >>! In T218844#5045107, @Tgr wrote: > there's a related task but I'm unable to find it That was {T163242}. [16:59:58] James_F: hello mind if assign to you ExtensionDependencyError FundraisingTranslateWorkflow requires Translate ?? [17:00:15] James_F: I barely understand what is going and can not babysit it tonight unfortunately [17:00:33] but would love to have it fixed to get beta-scap-eqiad job running again ;] [17:01:05] hashar: I'll deal, don't worry. [17:01:10] aweoms [17:01:13] it is a great bug [17:01:19] And yes, unbreaking beta-scap seems important. :-D [17:01:25] and probably we need some epic tasks to migrate all deployed extensions to use extension registry [17:01:29] I know wikibase has such a task [17:01:31] We have them. [17:01:36] ah [17:01:37] cool [17:01:37] We've been working on it since 2014. :-( [17:01:48] I love when folks already have filled tasks and that some are working on it [17:01:51] yeah 2014 .. [17:02:01] But yeah, Translate will be finished soon. [17:02:09] it is not like we have to fill for an IP or deliver a product by next quater to please our stock holders? :] [17:02:09] Wikibase is going to be a nightmare, unfortunately. [17:02:23] Indeed. :-) [17:02:47] Wikibase, maybe a task force can be set between folks knowing of the registry system and some people from wmde [17:03:03] or even fly people around for a quick sprint hackathon and get it done! :] [17:03:08] at least [17:03:15] we no more have the wikidata build [17:03:28] yay [17:03:39] we had an office party when that finally went away ^^ [17:04:21] Project beta-scap-eqiad build #242204: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242204/ [17:06:44] Lucas_WMDE: I can imagine! and again congrats on having getting rid of Wikidata! [17:06:49] (03CR) 10Kosta Harlan: "Adding Jeena and Brennen for review, as additional context as you look at what services are needed for tests in CI." [integration/quibble] - 10https://gerrit.wikimedia.org/r/497222 (https://phabricator.wikimedia.org/T218534) (owner: 10Kosta Harlan) [17:06:55] if you get a party for extension.json , I will fly to Berlin for sure [17:07:04] anyway, I am off, gotta head to a meeting [17:14:21] Project beta-scap-eqiad build #242205: 04STILL FAILING in 8 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242205/ [17:24:06] Project beta-scap-eqiad build #242206: 04STILL FAILING in 8 min 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242206/ [17:25:31] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Addshore) [17:25:37] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Addshore) [17:34:04] Project beta-scap-eqiad build #242207: 04STILL FAILING in 8 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242207/ [17:44:11] Project beta-scap-eqiad build #242208: 04STILL FAILING in 8 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242208/ [17:51:20] (03PS1) 10Krinkle: update documentation and descriptions of pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/498169 [17:51:24] Reedy: ^ [17:54:08] Project beta-scap-eqiad build #242209: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242209/ [17:55:46] (03CR) 10Reedy: [C: 03+1] update documentation and descriptions of pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/498169 (owner: 10Krinkle) [17:55:54] (03PS6) 10MarkAHershberger: Start work on docker-driven make to produce tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/495160 [18:03:22] (03CR) 10Krinkle: [C: 03+2] update documentation and descriptions of pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/498169 (owner: 10Krinkle) [18:03:59] Project beta-scap-eqiad build #242210: 04STILL FAILING in 8 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242210/ [18:05:29] (03Merged) 10jenkins-bot: update documentation and descriptions of pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/498169 (owner: 10Krinkle) [18:08:04] (03PS1) 10Umherirrender: [SearchExtraNS] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498178 [18:09:55] (03CR) 10MarkAHershberger: Start work on docker-driven make to produce tarballs (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/495160 (owner: 10MarkAHershberger) [18:11:30] (03PS1) 10Umherirrender: [MobileApp] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498182 [18:12:48] 10Release-Engineering-Team (Kanban): Evaluate ArgoCD - https://phabricator.wikimedia.org/T218827 (10dduvall) a:03dduvall [18:14:10] Project beta-scap-eqiad build #242211: 04STILL FAILING in 8 min 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242211/ [18:14:39] (03PS1) 10Umherirrender: [SubPageList3] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498185 [18:17:47] (03PS1) 10Umherirrender: [SubpageSortkey] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498187 [18:20:29] (03PS1) 10Umherirrender: [PerformanceInspector] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498189 [18:24:14] Project beta-scap-eqiad build #242212: 04STILL FAILING in 8 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242212/ [18:38:29] 10Release-Engineering-Team (Kanban), 10User-greg, 10Wikimedia-extension-review-queue: Investigate and make improvements to the extension review process - https://phabricator.wikimedia.org/T195244 (10greg) a:05Jrbranaa→03greg [18:42:09] Yippee, build fixed! [18:42:10] Project beta-scap-eqiad build #242213: 09FIXED in 16 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242213/ [18:42:29] 10Beta-Cluster-Infrastructure, 10Fundraising-Analysis, 10MediaWiki-extensions-Translate, 10Patch-For-Review: ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed. - https://phabricator.wikimedia.org/T218860 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF... [18:47:33] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Jdforrester-WMF) [18:50:14] greg added a subscriber: greg.Thu, Mar 21, 6:49 PM [18:50:14] Thanks for doing this! While I primarily use the tools SAL viewer (easier searching/etc for me) it's great to have this redundant on wikitech. [18:50:14] greg removed a subscriber: greg. [18:50:21] Phab needs a "drive by comment" mode [18:50:21] hello, can anyone help me explain this build failure? https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/497482/2 https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/19990/console [18:50:59] or this one, on the same patch: https://integration.wikimedia.org/ci/job/mwext-node10-rundoc-docker/70/console [18:51:08] they are very different but similarly nonsensical to me [18:51:58] 19:34:27 stderr: 'fatal: unable to access 'https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CheckUser/': The requested URL returned error: 400' [18:52:10] Isn't that the ongoing /r/p issue? paladox? [18:52:41] 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant: Vagrant has detected a configuration issue which exposes a vulnerability with the installed version of VirtualBox - https://phabricator.wikimedia.org/T210764 (10Tgr) [18:53:31] yeah i guess that's https://phabricator.wikimedia.org/T218844 [18:53:40] but i have no idea where that URL is defined [18:54:35] also, a few lines above that, it seemingly successfully clones from https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Babel ? [18:57:01] Computers suck [18:57:03] It "should" be defined in extensions.git, but I checked and that's fine. [18:57:11] It's probably cacheing. [18:57:16] Or.. [18:57:16] Reedy: :) yeah, I'm also really aware of my overload of subscriptions in phab and notification overload. I've greatly reduced my watched projects lately and committed notification bankruptcy [18:57:21] Aka blame other people. ;-) [18:57:26] git submodule sync is needed [18:57:35] Maybe we should add that to the "checkout the extensions" thing? [18:57:59] * Reedy files a task [18:58:05] why is it even cloning CheckUser [18:58:08] for testing VisualEditor [18:58:16] i am confused on multiple levels here [18:58:50] the 'integration/config' repo has a few results when grepping for "r/p/", but nothing that looks obviously connected [18:58:51] Dependancy tree? [18:58:53] Gated extension? [18:58:59] we don't depend on CheckUser though [18:59:01] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:59:03] MatmaRex: CU is in gate. [18:59:22] MatmaRex: So it's checked out for any extension or core patch. (As is VE.) [18:59:44] MatmaRex: It's a "if your code breaks CU's code then you don't get to go to space today" check. [19:00:06] Though CU itself doesn't have that much by way of unit tests, so it's not the best check. :-) [19:00:35] 10Continuous-Integration-Config, 10Gerrit: Run `git submodule sync` - https://phabricator.wikimedia.org/T218929 (10Reedy) [19:00:51] That should fix up these sorts of issues if we have more url changes in future [19:01:21] 10Continuous-Integration-Config, 10Gerrit: Run `git submodule sync` when updating extensions.git clones - https://phabricator.wikimedia.org/T218929 (10Reedy) [19:04:29] if no one has any idea how to fix this, i'm going to file an UBN and then go pout because i can't do work [19:05:20] maintenance-disconnect-full-disks build 56806 integration-slave-jessie-1002 (/srv: 99%): OFFLINE due to disk space [19:06:30] MatmaRex: The task I filed would fix it [19:06:34] The question is where it needs running [19:06:40] And whether that needs docker image updates and shizz [19:07:15] greg-g: I've just asked for the releng one to be fixed on wikitech too ;) [19:07:27] heh,sure :) [19:07:35] okay, i'm going to go pout without filing a task then [19:08:10] MatmaRex: Feel free to mark it high or UBN even [19:08:21] Or see if you can see where the right place to put the call is... [19:09:32] PROBLEM - Free space - all mounts on integration-slave-jessie-1002 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1002.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1002.diskspace._srv.byte_percentfree (<11.11%) [19:09:36] Reedy ohh [19:09:43] Reedy i see the issue now [19:09:49] p// [19:09:52] notice the slash [19:10:00] p/ [19:10:02] should work [19:10:09] 10Release-Engineering-Team (Kanban): Evaluate Tekton Pipeline - https://phabricator.wikimedia.org/T217912 (10dduvall) == Summary == Tekton is narrow in scope but it seems to do what it does well: It provides a coherent set of Custom Resource Definitions (CRD) necessary to get CI type workloads running on k8s ef... [19:10:39] The trailing slash breaks it? [19:10:40] srsly? [19:10:43] yes [19:11:07] wait [19:11:13] https://github.com/wikimedia/mediawiki-extensions/blob/master/.gitmodules#L527 [19:11:14] git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CheckUser/ [19:11:14] Cloning into 'CheckUser'... [19:11:19] it worked for me [19:12:02] 400 means a bad request [19:12:04] (03PS7) 10MarkAHershberger: Start work on docker-driven make to produce tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/495160 [19:12:19] Reedy it should have logged to logstash the error i think. [19:12:45] At least i can say /p/ is still supported. [19:13:02] though *may* break with gerrit 2.16 [19:13:18] since polygerrit now ocupy's that url for the dashboard :) [19:13:40] * paladox try's upstream [19:13:43] quibble/zuul.py [19:13:43] 53: git_base_url='https://gerrit.wikimedia.org/r/p', [19:13:47] do we just need to change that URL? [19:14:27] if it's going to break (or is breaking stuff)... [19:14:34] 10Continuous-Integration-Config, 10Gerrit: Run `git submodule sync` when updating extensions.git clones - https://phabricator.wikimedia.org/T218929 (10Legoktm) CI doesn't use mediawiki/extensions.git? [19:14:46] yup [19:14:54] Reedy /p/ is broken now [19:14:58] with 2.16 :( [19:15:04] fatal: https://gerrit.git.wmflabs.org/r/p/test/test/info/refs not valid: is this a git repository? [19:15:26] legoktm yup, removing /p is all you have to do :) [19:15:35] i already have the fix for integration/zuul ! [19:15:45] (03PS1) 10Legoktm: Drop /p from Gerrit clone URL [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) [19:16:09] (03CR) 10Paladox: [C: 03+1] Drop /p from Gerrit clone URL [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) (owner: 10Legoktm) [19:16:31] is this a UBN? [19:16:58] It's annoying... But everything isn't broken AFAIK [19:17:06] It's all because /p/ is now in https://github.com/GerritCodeReview/gerrit/blob/master/java/com/google/gerrit/httpd/raw/StaticModule.java#L74 [19:17:09] (03CR) 10Jforrester: [C: 03+1] "Oh right, yes, this is very much needed." [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) (owner: 10Legoktm) [19:18:50] I kind of forgot how to test quibble patches, so if its not UBN then I'll deal with that later [19:19:44] (03CR) 10MarkAHershberger: Start work on docker-driven make to produce tarballs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/495160 (owner: 10MarkAHershberger) [19:20:38] we can't merge anything in VisualEditor repo [19:20:41] Reedy yes it is :( (I tryed fixing it, but then upstream told me to stop using /p/ as a work around) [19:20:44] apparently because of this [19:20:47] so i'd call it UBN [19:20:55] This may be a bug in gerrit [19:21:00] since it's all of a sundden [19:21:03] *sudden [19:21:21] i can't even override jenkisn because somebody had the great idea to disable that [19:22:03] For that repo [19:22:04] ? [19:22:06] yes [19:22:13] I just V+2 and merged some branch patches in mw core :P [19:22:17] or maybe my account got downgraded [19:23:01] yeah, i still have the option to V+2 in other repos. just not in mediawiki/extensions/VisualEditor [19:23:34] and CI breaks regularly and it makes me mad every time [19:24:20] (03CR) 10Legoktm: [C: 03+2] "1+1=2" [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) (owner: 10Legoktm) [19:25:01] MatmaRex what's the url that's breaking for VS? [19:25:20] [19:51:02] 19:34:27 stderr: 'fatal: unable to access 'https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CheckUser/': The requested URL returned error: 400' [19:25:23] maintenance-disconnect-full-disks build 56810 integration-slave-jessie-1002: OFFLINE due to disk space [19:25:28] (03Merged) 10jenkins-bot: Drop /p from Gerrit clone URL [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) (owner: 10Legoktm) [19:25:29] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10dduvall) [19:25:31] 10Release-Engineering-Team (Kanban): Evaluate Tekton Pipeline - https://phabricator.wikimedia.org/T217912 (10dduvall) 05Open→03Resolved a:03dduvall [19:25:41] Reedy: try without /p/ ? [19:25:45] Reedy that url works for me [19:25:54] should work though [19:26:02] paladox: It doesn't work for CI [19:26:05] oh [19:26:06] I can't give you much more info than that [19:26:30] https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/19990/console and https://integration.wikimedia.org/ci/job/mwext-node10-rundoc-docker/70/console [19:26:33] hrm, it's important that we get rid of /p/, but that url should work. I checked on one of the integration machines and that clone works. [19:27:35] (03PS1) 10MarkAHershberger: Edit Project Config [extensions/CrawlableAllPages] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498207 [19:28:29] (03Abandoned) 10MarkAHershberger: Edit Project Config [extensions/CrawlableAllPages] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/498207 (owner: 10MarkAHershberger) [19:28:48] 400 indicates a bad request [19:29:03] gerrit may have logged the error [19:30:13] thcipriani: so should I be continuing with the quibble rollout or are you looking into why it isn't working? [19:30:17] I don't see anything in the error logs since it seems to think the problem is on the user end. [19:31:50] legoktm: I'm looking at some logs. Seems like the r/p/ thing is a bit of a red-herring (we just use that more often than /r/) but I do see 400s for /r/ as well [19:32:01] hmm [19:32:12] we should have logs on why Gerrit would respond with 400 right? [19:36:30] hrm, there is a slightly inscrutable jetty message for this [19:46:36] When did this issue start? [19:46:53] im not seeing any breaking changes here https://github.com/GerritCodeReview/gerrit/compare/v2.15.8...v2.15.12 [19:49:10] MatmaRex or Reedy ^^ [19:50:19] maintenance-disconnect-full-disks build 56815 integration-slave-jessie-1002: OFFLINE due to disk space [19:51:19] not sure, i first saw it today [19:51:30] ok [19:51:40] but i think we didn't merge any patches in that repo in the last few days [19:54:06] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 2 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10Paladox) p:05Triage→03High [20:02:11] I wonder if anyone is experencing the problem locally? If yes could you get the header please. [20:02:31] the error is comming from https://github.com/eclipse/jetty.project/blob/jetty-9.4.x/jetty-http/src/main/java/org/eclipse/jetty/http/HttpParser.java#L666 [20:15:14] maintenance-disconnect-full-disks build 56820 integration-slave-jessie-1002: OFFLINE due to disk space [20:16:58] 10Phabricator, 10Cloud-Services, 10Puppet, 10cloud-services-team (Kanban): puppet function ipresolve unable to look up instance on labs-puppetmaster - https://phabricator.wikimedia.org/T139011 (10GTirloni) [20:27:36] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Krinkle) [20:35:02] 10Beta-Cluster-Infrastructure, 10Patch-For-Review, 10Technical-Debt: Set up LVS in beta like prod - https://phabricator.wikimedia.org/T196662 (10GTirloni) https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Neutron_LVS [20:40:20] maintenance-disconnect-full-disks build 56825 integration-slave-jessie-1002: OFFLINE due to disk space [20:41:49] MatmaRex: Reedy I think we got the CI problem resolved, but if you see it again ping me please :) [20:41:55] well [20:42:00] the CI cloning problem anyway [20:42:02] :o [20:42:07] can't solve all the CI problems :P [20:42:09] thanks. let me try to +2 stuff again [20:42:22] k [20:44:06] (03PS2) 10Umherirrender: [MobileApp] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498182 [20:45:14] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Security, 10Upstream: Integrate MFA into Gerrit - https://phabricator.wikimedia.org/T198813 (10Legoktm) [20:56:04] Project beta-scap-eqiad build #242225: 04FAILURE in 8 min 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242225/ [20:57:11] (03CR) 10Legoktm: [C: 03+2] [SearchExtraNS] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498178 (owner: 10Umherirrender) [20:57:18] (03CR) 10Legoktm: [C: 03+2] [SubPageList3] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498185 (owner: 10Umherirrender) [20:57:22] (03CR) 10Legoktm: [C: 03+2] [SubpageSortkey] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498187 (owner: 10Umherirrender) [20:57:28] (03CR) 10Legoktm: [C: 03+2] [PerformanceInspector] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498189 (owner: 10Umherirrender) [20:57:39] (03CR) 10Legoktm: [C: 03+2] [MobileApp] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498182 (owner: 10Umherirrender) [20:58:48] (03Merged) 10jenkins-bot: [SearchExtraNS] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498178 (owner: 10Umherirrender) [20:58:50] (03Merged) 10jenkins-bot: [SubPageList3] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498185 (owner: 10Umherirrender) [21:00:01] (03Merged) 10jenkins-bot: [SubpageSortkey] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498187 (owner: 10Umherirrender) [21:00:04] (03Merged) 10jenkins-bot: [PerformanceInspector] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498189 (owner: 10Umherirrender) [21:00:08] (03Merged) 10jenkins-bot: [MobileApp] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498182 (owner: 10Umherirrender) [21:03:05] !log deploying https://gerrit.wikimedia.org/r/498178 https://gerrit.wikimedia.org/r/498185 https://gerrit.wikimedia.org/r/498187 https://gerrit.wikimedia.org/r/498189 https://gerrit.wikimedia.org/r/498182 [21:03:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:03:29] PROBLEM - Host integration-slave-docker-1046 is DOWN: CRITICAL - Host Unreachable (172.16.1.115) [21:03:55] (03CR) 10Jforrester: "❤️" [integration/config] - 10https://gerrit.wikimedia.org/r/498169 (owner: 10Krinkle) [21:05:13] maintenance-disconnect-full-disks build 56830 integration-slave-jessie-1002: OFFLINE due to disk space [21:06:51] Yippee, build fixed! [21:06:51] Project beta-scap-eqiad build #242226: 09FIXED in 9 min 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/242226/ [21:09:54] thcipriani: thanks, wmf-quibble-vendor-mysql-hhvm-docker is now passing, but mwext-node10-rundoc-docker is still failing with the same weird error as before [21:10:04] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/497482/2 https://integration.wikimedia.org/ci/job/mwext-node10-rundoc-docker/83/console [21:12:36] hrm. Looks like a compilation error. Missing libpng maybe? That's new... [21:22:48] libpng and pkg-config [21:23:03] and autoreconf [21:30:16] maintenance-disconnect-full-disks build 56835 integration-slave-jessie-1002: OFFLINE due to disk space [21:33:14] (03PS1) 10Legoktm: [EUCopyrightCampaign] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498247 [21:38:15] 10Continuous-Integration-Infrastructure, 10Phabricator (Upstream), 10MediaWiki-Documentation, 10Regression, 10Upstream: Doxygen search.php no longer works on doc.wikimedia.org - https://phabricator.wikimedia.org/T218233 (10Legoktm) stretch was easier than I expected, patched packages are at https://peopl... [21:39:35] integration/config resists grepping more than any other repo I've experienced. [21:40:27] (03PS1) 10Legoktm: [AdvancedSearch] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498249 [21:40:27] wowza: '{name}-node10-run{script}-docker' [21:41:07] mwext-node10-rundoc-docker is failing for flow over and over [21:41:13] first it was php71 [21:41:16] now this [21:41:23] jinxed [21:42:01] oh good. [21:42:01] and php71 passes [21:43:18] looks like an npm error but I don't understand well https://integration.wikimedia.org/ci/job/mwext-node10-rundoc-docker/88/console [21:48:39] "too far back". huh. [21:54:37] huh, wonder why this repo doesn't get hit by scap branch cleanup: https://gerrit.wikimedia.org/r/plugins/gitiles/VisualEditor/VisualEditor/+refs [21:55:03] Because it's not an extension? [21:55:14] maintenance-disconnect-full-disks build 56840 integration-slave-jessie-1002: OFFLINE due to disk space [21:55:23] yeah that's VE core [21:55:34] it's a submodule of the VE MW extension [21:55:38] I mean, if we're tagging, we should be removing, for sure [21:55:54] yeah [21:56:01] but it's also missing REL1_31 and REL1_32 [21:56:06] lol [21:56:08] I guess we don't recurse submodules as part of clean [22:09:36] * paladox is feeling excited that he finally got https://phabricator.wikimedia.org/T218739#5046540 working! [22:20:14] maintenance-disconnect-full-disks build 56845 integration-slave-jessie-1002: OFFLINE due to disk space [22:28:19] paladox, yay! and thanks for working on that! :) [22:28:29] your welcome :) [22:28:39] it 100% works in my testing :) [22:32:50] > The last Puppet run was at Thu Mar 21 19:25:38 UTC 2019 (186 minutes ago). [22:32:51] :| [22:33:18] legoktm see the cloud admin channel. [22:33:25] oh [22:33:35] !log legoktm@integration-slave-jessie-1002:/srv/jenkins-workspace/workspace$ sudo rm -rf * [22:33:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:34:06] !log onlined integration-slave-jessie-1002 [22:34:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:49:33] RECOVERY - Free space - all mounts on integration-slave-jessie-1002 is OK: OK: integration.integration-slave-jessie-1002.diskspace._mnt.byte_percentfree (No valid datapoints found) [22:50:09] 10Beta-Cluster-Infrastructure, 10Discovery-Search, 10MediaWiki-Search, 10Services (next): Beta Cluster search box displays unexisting pages as results - https://phabricator.wikimedia.org/T186993 (10EBernhardson) p:05Triage→03High [22:50:17] James_F: is https://phabricator.wikimedia.org/T218902 something you can backport in 10 minutes? :) [22:50:29] 10Beta-Cluster-Infrastructure, 10Discovery-Search, 10MediaWiki-Search, 10Services (next): Beta Cluster search box displays unexisting pages as results - https://phabricator.wikimedia.org/T186993 (10EBernhardson) Triaging as High, but i suspect if this is blocking SDoC it could perhaps be unbreak-now ? [22:52:02] greg-g: is there someone(s) in particular responsible for beta-cluster? In particular this task has been around since Feb: https://phabricator.wikimedia.org/T215339 [22:52:13] "No jobs running on beta cluster [22:52:13] " [22:52:25] ebernhardson: the question is more: who is responsible for the service in question in prod, they are responsible for it in beta :) [22:52:43] so, jobqueue.... [22:52:57] greg-g: ok, will poke them more. [22:52:59] thanks! [22:53:16] no explicit steward on dev/maint other than aaron as a "maintainer" [22:53:38] Isn't it "whoever runs it in prod"? :P [22:53:47] do I hear an echo? :P [22:53:47] * ebernhardson assigns reedy, he spoke up first [22:53:55] ebernhardson: Nope, Greg did :P [22:53:59] :P [22:54:03] * greg-g runs [22:58:16] greg-g: Sure. [22:58:17] * ebernhardson suspects this has been around so long because everyone looks at it and says not my responsiblity, including me [22:58:31] ebernhardson: usual problem :/ [22:58:34] James_F: thank you! [23:29:03] (03PS1) 10Legoktm: mediawiki-phan: Support new style phan for core [integration/config] - 10https://gerrit.wikimedia.org/r/498266 [23:29:45] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: operations-puppet-tests-stretch-docker not creating .tox/log ? - https://phabricator.wikimedia.org/T218962 (10greg) p:05Triage→03Unbreak! [23:29:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo configuration - https://phabricator.wikimedia.org/T218963 (10matmarex) p:05Triage→03Unbreak! [23:30:00] i filed this ^ about the problem i reported earlier [23:30:51] and i'm going to sleep [23:30:54] good luck [23:31:02] thanks MatmaRex [23:31:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Wikimedia-production-error (Shared Build Failure): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo con... - https://phabricator.wikimedia.org/T218963 [23:31:14] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Wikimedia-production-error (Shared Build Failure): operations-puppet-tests-stretch-docker not creating .tox/log ? - https://phabricator.wikimedia.org/T218962 (10greg) [23:32:31] (03CR) 10Legoktm: [C: 03+2] mediawiki-phan: Support new style phan for core [integration/config] - 10https://gerrit.wikimedia.org/r/498266 (owner: 10Legoktm) [23:33:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Wikimedia-production-error (Shared Build Failure): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo con... - https://phabricator.wikimedia.org/T218963 [23:33:34] Krinkle: Oops re. https://phabricator.wikimedia.org/T218963 [23:34:28] (03Merged) 10jenkins-bot: mediawiki-phan: Support new style phan for core [integration/config] - 10https://gerrit.wikimedia.org/r/498266 (owner: 10Legoktm) [23:34:53] !log rebuilding mediawiki-phan docker image for https://gerrit.wikimedia.org/r/498266 [23:34:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:38:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Wikimedia-production-error (Shared Build Failure): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo con... - https://phabricator.wikimedia.org/T218963 [23:39:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10Wikimedia-production-error (Shared Build Failure): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo con... - https://phabricator.wikimedia.org/T218963 [23:39:31] James_F: Looks like VE also rolled the dice on https://phabricator.wikimedia.org/T215562 [23:39:46] Revert that one for now, and add it to the list there. [23:39:55] I'm still going back and forth with npm customer support. [23:39:58] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Wikimedia-production-error (Shared Build Failure): operations-puppet-tests-stretch-docker not creating .tox/log ? - https://phabricator.wikimedia.org/T218962 (10greg) 05Open→03Invalid tox was a red-herring, this is why it failed:... [23:40:00] Lame. [23:40:12] But they seem to be too busy trying to blame it on eslint-config-wikimedia, which I doubt is relevant to this but oh well. [23:40:39] It might be our fault for using "interesting" packages upstream? [23:41:10] It's apparently got to do with the version of zlib that Debian 9 Stretch comes with [23:41:17] which breaks something in Node.js, and thus npm [23:41:53] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog): CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo configuration - https://phabricator.wikimedia.org/T218963 (10greg) sorry, wasn't sure :) [23:42:10] Ha. I'd be happy to bump to buster. ;-) [23:42:24] James_F: It's broken in all Debian versions afaik. [23:42:38] it'd be backported otherwise I think. [23:42:40] but not sure [23:42:55] https://packages.debian.org/buster/zlib1g [23:42:57] https://packages.debian.org/stretch/zlib1g [23:43:00] well, does look newer [23:43:02] But not stretch? Or just we magically never encountered it before? How would downgrading help? [23:43:05] but I don't know if or when it got fixed [23:43:15] well, I know it's fine on npm 3. [23:43:19] Happy to downgrade if it fixes a UBN, but… [23:43:29] James_F: I mean back to what VE had before. [23:43:31] the jsduck job [23:43:33] Internal pathway variances in node? Lovely. [23:43:57] Yeah, the big difference is npm 3 -> npm 6 and node 3 -> node 10. [23:44:02] Err. node 6. [23:44:09] oh it wasn't on jsduck but it was on npm3? [23:44:12] this particular job [23:44:17] yeah, revert to that then. [23:44:22] It's got to do with that, for sure. [23:44:44] same version of zlib on jessie and stretch, but somehow breaks node 10/npm6 but fine under npm3/node6 [23:44:52] Right. [23:45:16] It has to do with package-lock I think. When I recreated that file for Fresnel, it worked fine after that. [23:45:28] which is why Fresnel is the only repo on that list that does still use node 10 in CI [23:45:47] so might try to re-create it for VE and see if that passes. [23:45:55] Would it work if we deleted package-lock.json from the VE repo? [23:46:10] Yeah, delete the file, then use fresh-node10 to run npm-install and add that file instead. [23:46:26] Might not help, but if it does, it would narrow down what's going on [23:50:47] James_F: well, there goes that theory [23:51:04] 00:00:44.898 + echo 'No package-lock.json or npm-shrinkwrap.json detected, doing full install' [23:51:04] 00:00:44.898 + rm -rf node_modules [23:51:04] 00:00:44.898 No package-lock.json or npm-shrinkwrap.json detected, doing full install [23:51:04] 00:00:44.898 + npm install --no-progress [23:51:04] 00:00:56.630 npm ERR! invalid distance too far back [23:54:26] from what I saw jobs were doing that previously, which was weird after looking at the run.sh for that image [23:55:25] frex: https://integration.wikimedia.org/ci/job/mwext-node10-rundoc-docker/85/console