[01:01:12] 10Continuous-Integration-Infrastructure, 10Developer Productivity, 10MediaWiki-Core-Testing, 10Browser-Tests, 10Patch-For-Review: Consider moving browser based tests (Selenium and QUnit) to a non-voting pipeline - https://phabricator.wikimedia.org/T225248 (10MaxSem) >>! In T225248#5262721, @awight wrote:... [01:05:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 (10MaxSem) [01:05:43] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10Epic, 10phan: Enable mediawiki/mediawiki-phan-config on all Wikimedia-deployed repositories - https://phabricator.wikimedia.org/T224783 (10MaxSem) [01:53:54] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10matmarex) [02:19:18] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<44.44%) [06:44:20] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:26:29] (03PS1) 10Mainframe98: Archive the PureWikiDeletion extension [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) [10:37:45] hashar: so, this is strange CI failure, right? `RecursiveDirectoryIterator::__construct(../../tests/phan/stubs): failed to open dir: No such file or directory` https://integration.wikimedia.org/ci/job/mwext-php70-phan-docker/32902/console [10:38:57] zeljkof: sounds like a repo onfig issue :) [10:39:24] oh [10:39:28] ah, it's trying to run a php test but can't find a folder? [10:39:49] but it worked find in a previous run :/ https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseCirrusSearch/+/511127 [10:39:55] the phan config has some logic to find files at some hardcoded directories [10:40:11] the actual errors are listed in the xml checkstyle report below [10:40:50] 00:01:40.668 [10:40:51] ... [10:40:59] among others [10:41:12] wow [10:41:15] "\Wikibase\Search\Elastic\ElasticTermResult inherits Interface \CirrusSearch\Search\ResultsType declared at ../../extensions/CirrusSearch/includes/Search/ResultsType.php:25 which is also declared at ../../extensions/Wikibase/.phan/stubs/cirrussearch.php:275 [10:41:18] etc [10:41:20] how this got merged at all? [10:41:24] so the thing [10:41:31] the patch I'm working on doesn't even touch php [10:41:36] is Wikibase defines some stub classes under its .phan/stubs/cirrussearch.php [10:41:56] ah, something changed in wikibase, so this repo broke? [10:42:03] the build got trigger by extensions/WikibaseCirrusSearch which depends on both CirrusSearch and Wikibase [10:42:03] * zeljkof is confused [10:42:09] so since CirrusSearch define the concrete class [10:42:15] that conflicts with the stub classes from Wikibase [10:42:46] so something changed somewhere in the stack :] [10:43:50] zeljkof: that is for https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseCirrusSearch/+/511127/ isn't it ? [10:43:54] beautiful :) [10:44:10] hashar: yes [10:44:36] I usually download the raw console for the failed build and for the last successful build on that repo/branch [10:44:42] so digging in https://gerrit.wikimedia.org/r/#/q/project:mediawiki/extensions/WikibaseCirrusSearch [10:44:46] (list of open change) [10:44:58] there is a build on the change https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseCirrusSearch/+/517430/ which pass [10:45:00] and is from yesterday [10:45:26] so one can compare the failing build at https://integration.wikimedia.org/ci/job/mwext-php70-phan-docker/32902/ [10:45:44] with the https://integration.wikimedia.org/ci/job/mwext-php70-phan-docker/32786/ [10:46:44] vimdiff <(curl --compressed https://integration.wikimedia.org/ci/job/mwext-php70-phan-docker/32786/consoleText) <(curl --compressed https://integration.wikimedia.org/ci/job/mwext-php70-phan-docker/32902/consoleText) [10:46:44] :] [10:47:52] yeah, so there's not way I'm going to debug that :D [10:48:25] I'll do my thing and ask SMalyshev to fix the problem in the repo :) [10:48:44] bah [10:49:05] E_NOT_ENOUGH_TIME ;) [10:49:27] zeljkof: you might want to fill a task and subscribe Umherirrender :] [10:49:41] hashar: will do, thanks [10:52:11] zeljkof: root cause is Wikibase which upgraded phan (which is a good thing) [10:52:12] https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/494176/ [10:52:16] that introduces the stub files [10:53:13] so maybe you can reopen https://phabricator.wikimedia.org/T217018 and mention the conflict between Wikibase elasticsearch stub when a repo depends on both Wikibase and ElasticSearch itself [10:55:18] (03PS1) 10Hashar: operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 [10:55:30] (03CR) 10Hashar: [C: 03+2] operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 (owner: 10Hashar) [10:56:50] (03CR) 10jerkins-bot: [V: 04-1] operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 (owner: 10Hashar) [10:56:54] (03CR) 10jerkins-bot: [V: 04-1] operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 (owner: 10Hashar) [10:57:07] hashar: here it is T226005 [10:57:07] T226005: mwext-php70-phan-docker fails for WikibaseCirrusSearch - https://phabricator.wikimedia.org/T226005 [10:57:32] (03CR) 10Hashar: [C: 04-1] Archive the PureWikiDeletion extension (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [10:57:48] 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2019, 10Discovery-Search (Current work), 10Patch-For-Review, 10User-zeljkofilipin: The first Selenium test for WikibaseCirrusSearch - https://phabricator.wikimedia.org/T223774 (10zeljkofilipin) Blocked by T226005? [10:59:08] (03CR) 10Mainframe98: "> Patch Set 1: Code-Review-1" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [11:01:14] (03PS2) 10Hashar: operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 [11:03:32] (03PS2) 10Mainframe98: Archive the PureWikiDeletion extension [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) [11:04:09] (03CR) 10Mainframe98: Archive the PureWikiDeletion extension (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [11:06:11] (03CR) 10Hashar: [C: 03+2] operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 (owner: 10Hashar) [11:08:43] (03Merged) 10jenkins-bot: operations/debs/helm-diff make Debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/517618 (owner: 10Hashar) [11:18:47] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10Paladox) This seems like a bug, since you shouldn’t be able to remove a +2 after a change is merged, but it’s expected that you can remove votes after a change is merged. [11:32:12] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10Paladox) I’ve spoken with upstream who said that this is intentional due to the remove reviews right. [11:32:19] !log github: deleted https://github.com/wikimedia/mediawiki-extensions-PureWikiDeletion | T225991 [11:32:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:32:21] T225991: Archive the PureWikiDeletion extension - https://phabricator.wikimedia.org/T225991 [11:43:38] (03CR) 10MarcoAurelio: [C: 03+1] "Extension is now archived + code lgtm." [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [12:07:20] 10Gerrit, 10Operations, 10cloud-services-team, 10serviceops: Change /r/p/ to /r/ on all hosts (where https://gerrit.wikimedia.org/r/p/ exists) - https://phabricator.wikimedia.org/T222093 (10Paladox) p:05High→03Normal [12:08:35] 10Gerrit, 10Release-Engineering-Team, 10VPS-project-libraryupgrader: Re-enable use of Gerrit HTTP token to push patchsets - https://phabricator.wikimedia.org/T218750 (10Paladox) p:05Triage→03Normal [12:09:10] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10Reedy) >>! In T225979#5265026, @Paladox wrote: > This seems like a bug, since you shouldn’t be able to remove a +2 after a change is merged, but it’s expected that you can rem... [12:17:45] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10Paladox) @Reedy Yes, I thought it was a bug before I spoke with upstream. This is expected due to the “remove reviewer” right. Which I presume you and matmarex were granted.... [12:43:08] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10matmarex) @reedy Removing reviewers and reviews definitely has been possible for a while, but I am pretty sure it used to only be available on open changes. [12:55:32] 10Gerrit: Code-Review and Verified votes can be removed from changes after they are merged - https://phabricator.wikimedia.org/T225979 (10hashar) p:05Triage→03Low At least the reviews notes are still correct: ` $ fetch origin refs/notes/review:refs/notes/review $ git log -n1 --format=%N --notes=review a13c86... [13:01:36] (03CR) 10Alexandros Kosiaris: "> OK, pipeline test is installed for this repo and working. Success!" [integration/config] - 10https://gerrit.wikimedia.org/r/516517 (owner: 10Mholloway) [13:05:54] 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2019, 10Discovery-Search (Current work), 10Patch-For-Review, 10User-zeljkofilipin: The first Selenium test for WikibaseCirrusSearch - https://phabricator.wikimedia.org/T223774 (10zeljkofilipin) a:05zeljkofilipin→03None @Smalyshev I've done al... [13:16:49] 10Beta-Cluster-Infrastructure, 10RelEng-Archive-FY201718-Q1, 10Release-Engineering-Team (Kanban), 10media-storage, 10Patch-For-Review: deployment-ms-be03.deployment-prep and deployment-ms-be04.deployment-prep have high load / system CPU - https://phabricator.wikimedia.org/T160990 (10hashar) I have abando... [13:30:37] (03CR) 10Hashar: [C: 03+2] Archive the PureWikiDeletion extension [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [13:30:43] 10Phabricator, 10Developer-Advocacy (Apr-Jun 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10Aklapper) [13:31:34] (03CR) 10Hashar: [C: 03+2] zuul: tests jobs triggered for mediawiki/vendor.git [integration/config] - 10https://gerrit.wikimedia.org/r/517427 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:32:09] (03Merged) 10jenkins-bot: Archive the PureWikiDeletion extension [integration/config] - 10https://gerrit.wikimedia.org/r/517603 (https://phabricator.wikimedia.org/T225991) (owner: 10Mainframe98) [13:34:11] (03Merged) 10jenkins-bot: zuul: tests jobs triggered for mediawiki/vendor.git [integration/config] - 10https://gerrit.wikimedia.org/r/517427 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:35:29] 10Phabricator, 10Developer-Advocacy (Apr-Jun 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10Aklapper) Would it be possible for Security and stewards to take a look and evaluate Zulip in the next two weeks, to get an idea if that could work for you? (An... [13:35:32] (03CR) 10Hashar: [C: 03+2] "Doing that for now. We will see next how to address the case of tarballed repositories :-\" [integration/config] - 10https://gerrit.wikimedia.org/r/511866 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:35:42] (03CR) 10jerkins-bot: [V: 04-1] Use composer for Mediawiki REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/511866 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:36:28] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10AbuseFilter, 10CX-deployments, and 2 others: mediawiki/vendor REL1_* no longer ship dependencies for wmf extensions that are not in the mediawiki tarball - https://phabricator.wikimedia.org/T189560 (10RazeSoldier) The merger of this pa... [13:36:36] (03PS5) 10Hashar: Use composer for Mediawiki REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/511866 (https://phabricator.wikimedia.org/T189560) [13:37:04] anyone here have the ability to change page-properties on test wiki? [13:45:28] 10Phabricator, 10Developer-Advocacy (Apr-Jun 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10revi) Can I also be added to zulip for testing? (Email address (my phab handle)@(my phab handle).wiki) [13:49:22] (03CR) 10Hashar: [C: 03+2] Use composer for Mediawiki REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/511866 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:51:27] (03Merged) 10jenkins-bot: Use composer for Mediawiki REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/511866 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [13:56:29] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10AbuseFilter, 10CX-deployments, and 2 others: mediawiki/vendor REL1_* no longer ship dependencies for wmf extensions that are not in the mediawiki tarball - https://phabricator.wikimedia.org/T189560 (10hashar) I have send dummy changes... [13:57:47] 10Phabricator, 10Developer-Advocacy (Apr-Jun 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10sbassett) @revi - I think you can just sign up with an email, google or github account here: https://wikimedia.zulipchat.com/register/. Once you have an accoun... [14:05:49] 10Gerrit, 10Security: Cannot assign user name "XXX" to account ####; name already in use. - https://phabricator.wikimedia.org/T216605 (10akosiaris) >>! In T216605#5256926, @thcipriani wrote: > Thanks for the enthusiasm :) > > Could I get someone who is up-to-date on our LDAP setup (either @akosiaris or @bd808... [14:10:32] 10Continuous-Integration-Infrastructure, 10Zuul, 10Upstream, 10User-Addshore: Run less tests when a patch is in WIP mode in Gerrit - https://phabricator.wikimedia.org/T204125 (10hashar) I have checked the code, #zuul does not have support for Gerrit `wip` status. For a wip change, `gerrit query` yields: `... [14:26:41] (03PS1) 10Hashar: Bring back release-quibble-vendor jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517662 (https://phabricator.wikimedia.org/T189560) [14:27:23] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10AbuseFilter, 10CX-deployments, and 2 others: mediawiki/vendor REL1_* no longer ship dependencies for wmf extensions that are not in the mediawiki tarball - https://phabricator.wikimedia.org/T189560 (10hashar) I will mess a bit more wit... [14:32:46] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 2 others: TEC3:O3:O3.1:Q4 Goal - Move cpjobqueue, Wikidata Termbox SSR (new service), Kask (session storage service) and ORES (partially) through the production CD Pipeline - https://phabricator.wikimedia.org/T220398 (10akosiari... [14:32:54] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10Wikidata, and 5 others: Introduce wikidata termbox SSR to kubernetes - https://phabricator.wikimedia.org/T220402 (10akosiaris) 05Open→03Resolved a:03akosiaris ` curl -s -I -X GET 'http://termbox.discovery.wmnet:3030/termbox?editLink=%2Fe... [14:47:24] (03CR) 10Hashar: [C: 03+2] Bring back release-quibble-vendor jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517662 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [14:49:34] (03Merged) 10jenkins-bot: Bring back release-quibble-vendor jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517662 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [15:07:01] 10Phabricator, 10Developer-Advocacy (Apr-Jun 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10Aklapper) If it needs a bit more than to weeks that's also fine; I think we just need an arbitrary deadline so this task does not become neverending. Thanks a l... [15:07:14] 10Phabricator, 10Developer-Advocacy (Jul-Sep 2019): Re-evaluate our use of Phabricator Conpherence chat - https://phabricator.wikimedia.org/T127640 (10Aklapper) [15:23:44] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10hashar) [15:26:52] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10Jdforrester-WMF) We're currently using sury packages in `php71` and `quibble-stretch-php71`, `php72` and `quibble-stretch-php72`, `php73` and `qui... [15:29:20] (03Abandoned) 10Paladox: Use wikidata-query over wikidata-query-blazegraph [wikidata/query/blazegraph] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/512980 (owner: 10Paladox) [15:29:33] (03Restored) 10Paladox: Use wikidata-query over wikidata-query-blazegraph [wikidata/query/blazegraph] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/512980 (owner: 10Paladox) [15:30:14] (03PS4) 10Paladox: Update plugins to 2.15.13 [software/gerrit] (wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/507521 [15:38:04] (03PS1) 10Kosta Harlan: Codehealth: Allow broken coverage reports [integration/config] - 10https://gerrit.wikimedia.org/r/517681 [15:39:33] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10hashar) That is merely to have the MediaWiki tests to use the same package that Wikimedia is using. So I have scoped it to just php7.2 for now :-] [15:40:05] kostajh: Aha, that's what is broken about Wikibase. [15:40:18] hashar: :-((( That means yet more technical debt/cruft. [15:56:38] (03PS1) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [15:58:17] (03CR) 10jerkins-bot: [V: 04-1] docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [16:00:20] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10VPS-project-libraryupgrader: LibraryUpgrader CI normalisation tasks, June 2019 - https://phabricator.wikimedia.org/T225325 (10Mholloway) [16:16:09] (03CR) 10Jforrester: Update go-playground validator (031 comment) [blubber] - 10https://gerrit.wikimedia.org/r/516671 (https://phabricator.wikimedia.org/T215319) (owner: 10Thcipriani) [16:50:48] (03CR) 10Jforrester: [C: 03+2] Codehealth: Allow broken coverage reports [integration/config] - 10https://gerrit.wikimedia.org/r/517681 (owner: 10Kosta Harlan) [16:53:00] (03Merged) 10jenkins-bot: Codehealth: Allow broken coverage reports [integration/config] - 10https://gerrit.wikimedia.org/r/517681 (owner: 10Kosta Harlan) [16:54:00] !log Pushing new quibble-coverage docker image [16:54:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:54:47] jrbranaa: running a few minutes late to the data3 meeting [16:55:47] Kk. We’ll see if calendar/meet even works. It was down prior to my last meeting [16:58:44] James_F: thank you :) [16:59:06] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Implement pipeline config validation - https://phabricator.wikimedia.org/T225335 (10brennen) p:05Triage→03Normal a:03brennen Planning to pair on this with @thcipriani this coming Friday. [16:59:07] I’m retesting my patch for MW core and codehealth, if you want to try deploying that later [16:59:31] kostajh: Not pushed the new jjb job configs yet, sorry, having local issues. [17:02:12] !log Updated jjb jobs for mwext-codehealth-patch and mwext-codehealth-master-non-voting. [17:02:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:05:29] kostajh: Sounds good. [17:10:05] (03CR) 10Krinkle: docker: use php7.2 from SRE (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [17:11:22] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10Jdforrester-WMF) >>! In T226028#5265915, @hashar wrote: > That is merely to have the MediaWiki tests to use the same package... [17:13:09] Krinkle: I'd very much appreciate your thoughts on my proposed time targets for tests in T225972 [17:13:10] T225972: Establish performance budgets for tests of each area of code, and report (later, auto-skip) suites that take too long - https://phabricator.wikimedia.org/T225972 [17:18:40] kostajh: Looks like that broke the codehealth checks entirely; reverting the JJB bit. [17:18:54] James_F: JS/PHP seems fine, in the right ballpark anyway. For the browser test, I think per-test 1 s should suffice, 5 s at most. Our CI infra is hyperoptimised to not rely on disk IO for browser or web server stuff. It's really super fast, just look at Fresnel full page load times. [17:19:31] Krinkle: Yeah, I was thinking about UX workflows where you're waiting for animations to finish, but I guess we expect those timings to be over-ridden? [17:21:45] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, 10Epic: Establish performance budgets for tests of each area of code, and report (later, auto-skip) suites that take too long - https://phabricator.wikimedia.org/T225972 (10Jdforrester-WMF) [17:21:51] Yeah, we'll need to look at use cases as they come, but based on what we have now, I don't see any reason for 90% of tests to not be under 1s. [17:21:56] browser tests* [17:22:26] OK, have reduced the proposed limits. Next step is reporting on them generally. :-) [17:22:51] and yes, right now basically none of them are, almost exclusively due to use of MWBot and I suspect the screenshot/video saving stuff needs improving as well, that doesn't need to be blocking 'fs' [17:23:20] Right. [17:23:30] James_F: yeah looks like `-e` was passed as an argument to quibble rather than to `docker run`, oops [17:23:40] Yeah. [17:23:56] Is there a way to pass args todocker through that job template? [17:25:05] "options" is where we pass `--volume {...}` [17:25:34] Aha, I'll just patch that then. [17:26:45] (03PS1) 10Jforrester: jjb: Don't try to pass settings to docker to quibble itself, it breaks [integration/config] - 10https://gerrit.wikimedia.org/r/517697 [17:26:58] (03CR) 10Jforrester: [C: 03+2] jjb: Don't try to pass settings to docker to quibble itself, it breaks [integration/config] - 10https://gerrit.wikimedia.org/r/517697 (owner: 10Jforrester) [17:27:20] (03PS1) 10Kosta Harlan: Codehealth: Allow broken coverage reports [integration/config] - 10https://gerrit.wikimedia.org/r/517698 [17:27:49] Sorry kostajh. :-) [17:28:22] (03Abandoned) 10Kosta Harlan: Codehealth: Allow broken coverage reports [integration/config] - 10https://gerrit.wikimedia.org/r/517698 (owner: 10Kosta Harlan) [17:28:28] James_F: no worries, thanks! [17:29:12] (03Merged) 10jenkins-bot: jjb: Don't try to pass settings to docker to quibble itself, it breaks [integration/config] - 10https://gerrit.wikimedia.org/r/517697 (owner: 10Jforrester) [17:30:57] kostajh: Hm. somewhat confused by the health check failure here https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/PageTriage/+/516245/ [17:31:21] If I understand correctly, it thinks there's new code but no increase in coverage. I didn't add any php code though. [17:32:19] oh, its' seeing the moved JS files as new code. [17:32:28] right :( https://sonarcloud.io/component_measures?branch=516245&id=mediawiki-extensions-PageTriage&metric=new_lines_to_cover&view=list [17:33:55] PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 46.67% of data above the critical threshold [140.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:42:16] 10Continuous-Integration-Config, 10Release-Engineering-Team (Watching / External), 10Analytics, 10Patch-For-Review: Status of analytics/limn-*-data git repositories? - https://phabricator.wikimedia.org/T221064 (10Nuria) @fdans: I think we need to communicate to repo owners this move, would you please take... [17:52:35] RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 30.00% above the threshold [90.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:58:41] kostajh: Eurgh, yeah, that's a problem with my new -coverage docker image, I'm afraid. :-( [18:08:26] James_F: the mysql errors? [18:08:49] Yeah. Sorry, should have remembered. [18:09:27] Last week I managed to reproduce the error locally, but the /tmp logs get removed on exit. [18:23:30] James_F: should I create a revert patch? [18:24:22] Sorry, lost in debugging. Probably yes, unless you can spot a fix. :-( [18:25:10] Local debugging with the quibble-coverage job and running `mysql_install_db` directly works fine, but running `quibble` dies at the `mysql_install_db` step. [18:30:01] (03CR) 10Hashar: docker: use php7.2 from SRE (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [18:30:36] 10Continuous-Integration-Infrastructure: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10Jdforrester-WMF) [18:30:55] hashar: ^^ Help appreciated. :-) [18:32:22] James_F: eeeek [18:33:00] Yeah. :-( [18:33:03] for the tmp being erased, yeah the container has --rm I guess [18:33:07] so it is dished out entirely [18:33:16] It's one of the blockers to moving to php72. [18:33:18] without --rm, docker will keep the container around and it can thus be inspected somehow [18:33:19] or [18:33:34] No no, I'm interactively doing this inside the container. [18:33:47] you volume mount /tmp from your host : mkdir guest-tmp && sudo chown nobody:nogroup guest-tmp && docker run -v guest-tmp:/tmp [18:33:48] I'm still in the container. /tmp is blank. [18:33:50] something like that [18:33:51] ah [18:34:00] interactive yeah that is the 3rd option [18:34:11] drwxrwxrwt 1 root root 4096 Jun 18 18:29 tmp [18:34:25] (03PS2) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [18:34:35] do you have any more details in the output? [18:34:38] It's being touched, but I don't know if that's mysql writing to it and failing, or the logs getting written to it and then removed by something else. [18:35:05] I think last friday I found out the mariadb package to be a different version [18:35:20] https://www.irccloud.com/pastebin/DpGItRin/ [18:35:29] (03CR) 10jerkins-bot: [V: 04-1] docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [18:35:44] (03CR) 10Hashar: "They all build, but I would like to inspect/diff the containers to see what is being changed :-\" [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [18:35:54] -1 on my patch bah [18:36:07] sacrilège [18:36:11] The mariadb package is inherited from SRE's image, isn't it? [18:36:16] hold on [18:36:25] so mysql_install_db [18:36:37] from last week, it is apparently just a wrapper to mysqld --bootstrap [18:36:45] AssertionError: Image directory dockerfiles/php72 must have a 'control' file [18:36:46] bah [18:38:42] OH [18:38:44] hm [18:38:45] or [18:38:46] no [18:38:47] maybe [18:39:25] 10Continuous-Integration-Infrastructure: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10hashar) [18:39:39] (03PS3) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [18:39:47] (03CR) 10jerkins-bot: [V: 04-1] docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [18:40:13] James_F: the tmpdir is created by Quibble itself and it deletes it entirely when it ends [18:40:16] :-\ [18:40:59] Helpful. :-) [18:41:08] well that would be my assumption [18:41:48] class DatabaseServer(BackendServer): [18:41:57] def _init_rootdir(self, base_dir): [18:41:57] # Create a temporary data directory [18:42:01] self.log.debug('Root dir: %s' % self.rootdir) [18:42:08] I have skipped a bunch of code [18:42:24] and it is created by tempfile.TemporaryDirectory() [18:42:31] which is garbage collected by python automagically [18:44:36] (03PS4) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [18:44:53] James_F: which container is that ? [18:45:54] Local image of docker-registry.wikimedia.org/releng/quibble-coverage [18:47:44] Which is meant to replace quibble-stretch-php70 for coverage related things. [18:47:55] * hashar checks docker-registry.wikimedia.org/releng/quibble-coverage:0.0.31-2 [18:51:09] James_F: there is nothing logged :/ [18:51:31] Well, that's… unhelpful. [18:51:44] maybe we can just drop mysql support from CI ? [18:51:46] *grin* [18:51:51] If only. [18:52:20] But this is why having a single set of images for prod/dev/CI instead of three sets will save us time. Hopefully. [18:52:32] * James_F sighs. [18:52:42] yeah [18:52:47] that is the promise ultimately [18:54:17] docker-registry.wikimedia.org/releng/quibble-coverage seems to default to run as root [18:54:23] I guess it lacks a USER nobody ;D [18:54:45] Would that be an issue for this? [18:55:23] Hmm, maybe. [18:55:59] 10Continuous-Integration-Infrastructure: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10hashar) [18:56:08] James_F: yeah that is the issue [18:57:18] 10Release-Engineering-Team (Kanban): Set up a subdomain for Phame to enable caching - https://phabricator.wikimedia.org/T226044 (10JAufrecht) p:05Triage→03Normal [18:57:54] That's not inherited, but other things are? [18:58:08] 10Continuous-Integration-Infrastructure: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10hashar) Reproducible using: ` docker run --rm -it --entrypoint=bash docker-registry.wikimedia.org/releng/quibble-coverage:0.0.31-1 # rm -fR /tmp/mysqldata/mysql; m... [18:58:59] We also don't define an entrypoint… do we need to do that? [18:59:19] the ENTRYPOINT gets inherited from the parent image [18:59:27] But not USER nobody? [18:59:33] it is as well [18:59:41] Then… why does it break? [18:59:42] but dockerfiles/quibble-coverage/Dockerfile.template switches to user root [18:59:54] and does not switch back to user nobody [18:59:54] Oh, and doesn't switch back. Duh. [19:00:06] Want to write the patch or should I? [19:00:38] James_F: I will [19:00:47] Cool. [19:04:30] (03PS5) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [19:04:32] (03PS1) 10Hashar: docker: run releng/quibble-coverage as user nobody [integration/config] - 10https://gerrit.wikimedia.org/r/517713 (https://phabricator.wikimedia.org/T226040) [19:04:35] man [19:04:37] that is terrible [19:07:03] (03PS2) 10Hashar: docker: run releng/quibble-coverage as user nobody [integration/config] - 10https://gerrit.wikimedia.org/r/517713 (https://phabricator.wikimedia.org/T226040) [19:07:05] (03PS6) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [19:07:37] James_F: https://gerrit.wikimedia.org/r/#/c/integration/config/+/517713/ should do [19:09:16] * James_F nods. [19:09:49] * James_F tests locally. [19:10:38] Hmm, I still end up as root on an interactive session. [19:13:24] have you build it already? [19:13:26] James_F: ^ [19:15:29] * James_F re-prunes and re-builds. [19:17:34] $ docker run --rm -it --entrypoint=id docker-registry.wikimedia.org/releng/quibble-coverage:0.0.31-4 [19:17:34] uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup) [19:17:35] ;] [19:17:58] $ docker run --rm -it --entrypoint=id docker-registry.wikimedia.org/releng/quibble-coverage:0.0.31-3 [19:17:59] uid=0(root) gid=0(root) groups=0(root) [19:18:02] affreux [19:18:05] :) [19:18:08] tout à fait [19:18:19] exècration ! :D [19:18:32] *é [19:18:35] damn [19:18:45] * James_F grins. [19:19:09] I hate when file permissions suddenly change and you don't notice but when everything starts to fail [19:19:22] happened quite a lot on the beta cluster iirc [19:19:38] * James_F paints go-faster stripes on docker to build things quicker. [19:20:06] oh [19:20:15] (It's done ci-stretch/php/composer/node10/quibble-stretch and is now on sury-php.) [19:20:18] one trick is use a http proxy [19:21:03] in the docker-pkg.yaml config file https://doc.wikimedia.org/docker-pkg/index.html?highlight=proxy#configuration [19:21:07] http_proxy: will set the http proxy to be used for all docker-related operations. [19:21:28] So it borrows images from the registry rather than using local ones? [19:21:39] oh [19:21:40] no [19:21:58] I'm at the office, the Internet connection is not the slow bit. :-) [19:22:12] that is then passed to apt so that it can download through a proxy and thus potentially grab the huge .deb packages locally (over localhost) [19:22:19] ah yeah office might be fast [19:22:25] Oh, right. [19:22:27] to reuse images well [19:22:28] hmm [19:22:48] docker-pkg build --use-cache [19:22:50] * James_F twiddles thumbs. :-) [19:22:56] which instructs docker build to attempt to reuse images [19:23:00] the default when using docker build [19:23:19] so potentially it would reuse existing layers. That is good for quick development [19:23:33] but for building the final image, we run them without cache [19:23:38] Yes; I'm having it be slow because I deleted all local images. :-) [19:23:59] then [19:24:24] I will start ranting about how docker build as we use it is non deterministic and layers get invalidated for various reasons [19:24:37] or how I would love to be able to rebase layers on top of another chain [19:24:56] but wel [19:24:57] l [19:24:59] Yeah. [19:25:00] the USER nobody looks fine [19:26:06] Confirmed. [19:26:17] (03CR) 10Jforrester: [C: 03+2] docker: run releng/quibble-coverage as user nobody [integration/config] - 10https://gerrit.wikimedia.org/r/517713 (https://phabricator.wikimedia.org/T226040) (owner: 10Hashar) [19:26:37] (03CR) 10Hashar: [C: 03+1] "Seems good:" [integration/config] - 10https://gerrit.wikimedia.org/r/517713 (https://phabricator.wikimedia.org/T226040) (owner: 10Hashar) [19:26:54] then the usual dance [19:26:57] rebuild image [19:27:04] bump in jjb [19:27:04] Yeah. [19:27:07] assert [19:27:40] hashar: `git shortlog -sne HEAD~100..` ;-) [19:28:09] (03Merged) 10jenkins-bot: docker: run releng/quibble-coverage as user nobody [integration/config] - 10https://gerrit.wikimedia.org/r/517713 (https://phabricator.wikimedia.org/T226040) (owner: 10Hashar) [19:28:20] spaaaam [19:28:51] !log Pushing quibble-coverage:0.0.31-4 docker image. [19:28:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:31:38] (03PS1) 10Jforrester: jjb: Move all quibble-coverage jobs to a 0.0.31-4 [integration/config] - 10https://gerrit.wikimedia.org/r/517722 [19:31:49] (03PS2) 10Jforrester: jjb: Move all quibble-coverage jobs to a 0.0.31-4 [integration/config] - 10https://gerrit.wikimedia.org/r/517722 (https://phabricator.wikimedia.org/T225832) [19:32:08] (03CR) 10Hashar: [C: 03+1] jjb: Move all quibble-coverage jobs to a 0.0.31-4 [integration/config] - 10https://gerrit.wikimedia.org/r/517722 (https://phabricator.wikimedia.org/T225832) (owner: 10Jforrester) [19:32:17] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10Jdforrester-WMF) 05Open→03Resolved a:03hashar [19:32:31] * hashar looks at https://www.vesselfinder.com/fr [19:32:39] there is some boat using its horn since there is fog in my city [19:32:41] 10Continuous-Integration-Config, 10Quibble: Quibble fails at the mysql_install_db step on PHP72? - https://phabricator.wikimedia.org/T225840 (10Jdforrester-WMF) [19:32:43] 10Continuous-Integration-Config, 10Patch-For-Review: Move coverage tasks from PHP70 to PHP72 - https://phabricator.wikimedia.org/T225832 (10Jdforrester-WMF) [19:32:49] (it was all sunny and suddenly .. fog!) [19:32:52] 10Continuous-Integration-Config, 10Quibble: Quibble fails at the mysql_install_db step on PHP72? - https://phabricator.wikimedia.org/T225840 (10Jdforrester-WMF) [19:32:54] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: quibble-coverage docker image fails at mysql_install_db step - https://phabricator.wikimedia.org/T226040 (10Jdforrester-WMF) [19:33:34] Next I can do T225456. [19:33:35] T225456: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 [19:34:59] (03CR) 10Jforrester: [C: 03+2] jjb: Move all quibble-coverage jobs to a 0.0.31-4 [integration/config] - 10https://gerrit.wikimedia.org/r/517722 (https://phabricator.wikimedia.org/T225832) (owner: 10Jforrester) [19:37:24] quick request (hopefully): this is Isaac from Research. i'm testing some new changes to QuickSurveys on the beta wikis. this requires interface admin permissions to create pages like this: https://en.wikipedia.beta.wmflabs.org/wiki/MediaWiki:Reader-demographics-1-privacy [19:37:24] i have the necessary permission for english (reedy helped me last time: https://en.wikipedia.beta.wmflabs.org/wiki/Special:UserRights/Isaac) but i wanted to expand to include hebrew as well to make sure that right-to-left languages work as well. could someone help me get those permissions [19:37:24] (https://he.wikipedia.beta.wmflabs.org/wiki/%D7%9E%D7%99%D7%95%D7%97%D7%93:%D7%A0%D7%99%D7%94%D7%95%D7%9C_%D7%94%D7%A8%D7%A9%D7%90%D7%95%D7%AA_%D7%9E%D7%A9%D7%AA%D7%9E%D7%A9/Isaac) [19:37:49] * Reedy logs in [19:38:10] (03Merged) 10jenkins-bot: jjb: Move all quibble-coverage jobs to a 0.0.31-4 [integration/config] - 10https://gerrit.wikimedia.org/r/517722 (https://phabricator.wikimedia.org/T225832) (owner: 10Jforrester) [19:38:50] isaacj: Just admin? [19:38:58] !log Bumped jjb mwext-codehealth-*, mediawiki-*coverage*docker*, mwext*coverage*docker* jobs to use quibble-coverage:0.0.31-4 [19:39:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:39:05] As you don't have interface admin on en... [19:39:25] Reedy: yes, thanks, that worked last time (https://en.wikipedia.beta.wmflabs.org/wiki/Special:UserRights/Isaac) [19:39:34] Should be all good then [19:40:54] (I've granted them) [19:41:36] Reedy: ahh - i see the confusion. i'll let you know if it doesn't work for some reason but should be good. thanks! [19:41:43] sure! :) [19:48:19] (03PS7) 10Hashar: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) [19:49:21] (03CR) 10Hashar: [C: 03+2] "As a result some packages are no more provided by sury, at least:" [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [19:50:55] (03Merged) 10jenkins-bot: docker: use php7.2 from SRE [integration/config] - 10https://gerrit.wikimedia.org/r/517684 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [19:51:22] !log building containers for php7.2 from SRE # T226028 https://gerrit.wikimedia.org/r/#/c/integration/config/+/517684/ [19:51:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:51:25] T226028: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 [19:51:33] 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T220735 (10mmodell) [19:52:29] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10hashar) Will build the following images: * docker-registry.discovery.wmnet/releng/php72:0.2.0 * docker-registry.discovery.w... [20:03:41] kostajh: https://integration.wikimedia.org/ci/job/mwext-codehealth-master-non-voting/231/console broke with "java.lang.IllegalStateException: Fail to download plugin [cpp] into /cache/sonar/_tmp/fileCache1353896198680778665.tmp" – known issue or something I broke? [20:04:33] James_F: odd. Can you retry that build please? [20:05:02] Triggered as https://integration.wikimedia.org/ci/job/mwext-codehealth-master-non-voting/232/console [20:05:25] James_F: also, the MW core codehealth patch should be OK but would it be possible to test it with one patch before it's deployed everywhere? [20:06:11] kostajh: Not easily. We can just revert if it breaks. [20:06:41] James_F: ok, well it's up to you. I'm around for the next 25 minutes. We could try another day too. [20:07:12] (03PS1) 10Hashar: Bump jobs to use php7.2 from SRE instead of sury.org [integration/config] - 10https://gerrit.wikimedia.org/r/517733 (https://phabricator.wikimedia.org/T226028) [20:07:20] Hmm. https://integration.wikimedia.org/ci/job/mwext-codehealth-patch/548/console failed too. :-( [20:07:28] (Same issue.) [20:08:28] 10Continuous-Integration-Config: Move all CI generic tasks from PHP70 to PHP72 - https://phabricator.wikimedia.org/T225457 (10Jdforrester-WMF) [20:08:31] 10Continuous-Integration-Config, 10Patch-For-Review: Move coverage tasks from PHP70 to PHP72 - https://phabricator.wikimedia.org/T225832 (10Jdforrester-WMF) 05Open→03Resolved Upgraded and working: [x] [[https://integration.wikimedia.org/ci/job/mediawiki-phpunit-coverage-patch-docker/10206/console|mediawik... [20:09:02] (03CR) 10Jforrester: "Wikibase fix is landed." [integration/config] - 10https://gerrit.wikimedia.org/r/515293 (owner: 10Jforrester) [20:09:10] (03PS4) 10Jforrester: layout: Switch extension/skin-phan jobs from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515293 [20:09:21] (03CR) 10Jforrester: [C: 03+2] layout: Switch extension/skin-phan jobs from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515293 (owner: 10Jforrester) [20:09:52] 10Phabricator, 10Release-Engineering-Team (Kanban): Set up a subdomain for Phame to enable caching - https://phabricator.wikimedia.org/T226044 (10Aklapper) [20:10:50] James_F: hmm. Not sure, let me investigate it a bit [20:10:50] (03Merged) 10jenkins-bot: layout: Switch extension/skin-phan jobs from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515293 (owner: 10Jforrester) [20:11:12] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, and 4 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) >>! In T225730#5261530, @Krinkle wrote: > 2487ms to run AutoLoaderStructureTest:... [20:12:06] !log Reload Zuul to point extension-phan and skin-phan templates at php72 instead of php70 T225456 [20:12:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:12:08] T225456: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 [20:12:29] PROBLEM - Free space - all mounts on deployment-mwmaint01 is CRITICAL: CRITICAL: deployment-prep.deployment-mwmaint01.diskspace.root.byte_percentfree (<11.11%) [20:17:44] (03PS4) 10Jforrester: layout: Switch parsoid from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515294 [20:17:50] (03CR) 10Jforrester: [C: 03+2] layout: Switch parsoid from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515294 (owner: 10Jforrester) [20:18:00] (03PS4) 10Jforrester: layout: Switch MediaWiki from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515295 [20:20:12] (03Merged) 10jenkins-bot: layout: Switch parsoid from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515294 (owner: 10Jforrester) [20:22:03] !log Reload Zuul to point Parsoid's experimental phan job at php72 instead of php70 T225456 [20:22:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:22:05] T225456: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 [20:22:22] James_F: I am going to point to the new containers for php7.2 [20:22:29] RECOVERY - Free space - all mounts on deployment-mwmaint01 is OK: OK: All targets OK [20:22:59] hashar: Go for it. [20:23:23] (03PS1) 10Kosta Harlan: Codehealth: Copy sonar cache to place where castor can use it [integration/config] - 10https://gerrit.wikimedia.org/r/517739 [20:23:57] !log Switching jobs to SRE php7.2 package # T226028 [20:24:02] !log Switching jobs to SRE php7.2 package # T226028 https://gerrit.wikimedia.org/r/#/c/integration/config/+/517733/ [20:24:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:24:06] T226028: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 [20:24:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:24:09] (03CR) 10Hashar: [C: 03+2] Bump jobs to use php7.2 from SRE instead of sury.org [integration/config] - 10https://gerrit.wikimedia.org/r/517733 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [20:24:14] James_F: not really sure about why that plugin is failing. The above patch will help some in not requiring downloads of plugins on each build though. [20:24:31] kostajh: OK, will have a play and see if that fixes it. [20:24:50] Meantime, I've found that phan finds different errors when run on php72 than on php70, because of course it does. :-( [20:26:45] (03Merged) 10jenkins-bot: Bump jobs to use php7.2 from SRE instead of sury.org [integration/config] - 10https://gerrit.wikimedia.org/r/517733 (https://phabricator.wikimedia.org/T226028) (owner: 10Hashar) [20:27:13] James_F: would be nice to document those :) [20:28:28] hashar: It'd be nice to uninstall the disaster that is phan. [20:28:43] what is so wrong with it? [20:30:01] Bad output. Bad configuration. No documentation. Keeps breaking. Rolled out as a voting job to many repos without their maintainers being aware of it. [20:30:17] ther eis some doc on the wiki [20:30:26] the bad output, I know it can be fixed [20:30:28] Which is mostly "come to IRC and ask". [20:30:32] phan 0.x used to spurt to stdout [20:30:44] but later, it has an option to write the checkstyle.xml report to a specific file [20:30:52] which we can then process and show back as a "nice report" [20:31:16] the config and dependency madness with other extensions yeah hmm [20:31:17] *cough* [20:31:19] 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T220735 (10mmodell) [20:31:23] Yeah, but e.g. https://integration.wikimedia.org/ci/job/mwext-php72-phan-docker/11/console — apparently `` was fine on php70. [20:31:43] ;;-\ [20:31:56] who knows what kind of bug that could highlight [20:32:11] maybe wikidata folks will be able to dig into that [20:32:21] a possibility is that composer installed a different package somehow [20:32:27] Hopefully. Three days of breaking their repo with phan in a row isn't great. :-( [20:36:59] (03CR) 10Jforrester: [C: 03+2] layout: Switch MediaWiki from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515295 (owner: 10Jforrester) [20:38:32] (03Merged) 10jenkins-bot: layout: Switch MediaWiki from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515295 (owner: 10Jforrester) [20:40:06] !log Reload Zuul to point MediaWiki's phan jobs at php72 instead of php70 T225456 [20:40:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:40:08] T225456: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 [20:45:06] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 (10Jdforrester-WMF) OK, phan is moved over, and phan-sec-check is left to move over. [20:48:11] (03PS3) 10Jforrester: jjb: Drop mediawiki-core-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515296 [20:48:13] (03PS3) 10Jforrester: jjb: Drop job_mwext-php70-phan-docker/mwskin-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515297 [20:48:15] (03PS3) 10Jforrester: jjb: Drop php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515298 [20:48:28] (03CR) 10Jforrester: [C: 03+2] jjb: Drop mediawiki-core-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515296 (owner: 10Jforrester) [20:48:32] (03CR) 10Jforrester: [C: 03+2] jjb: Drop job_mwext-php70-phan-docker/mwskin-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515297 (owner: 10Jforrester) [20:48:36] (03CR) 10Jforrester: [C: 03+2] jjb: Drop php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515298 (owner: 10Jforrester) [20:50:45] (03Merged) 10jenkins-bot: jjb: Drop mediawiki-core-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515296 (owner: 10Jforrester) [20:50:48] (03Merged) 10jenkins-bot: jjb: Drop job_mwext-php70-phan-docker/mwskin-php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515297 (owner: 10Jforrester) [20:50:51] (03Merged) 10jenkins-bot: jjb: Drop php70-phan-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/515298 (owner: 10Jforrester) [20:50:57] nice spam :] [20:51:25] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10hashar) Should be good hopefully. [20:51:32] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10hashar) 05Open→03Resolved a:03hashar [21:00:47] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 (10Jdforrester-WMF) [21:01:47] 10Release-Engineering-Team (Kanban), 10phan-taint-check-plugin: Phan-taint-check-plugin not available for PHP > 7.0 - https://phabricator.wikimedia.org/T207344 (10Jdforrester-WMF) This is now the last remaining blocker to moving CI to newer versions of PHP (and dropping PHP support in MediaWiki). Prioritisatio... [21:08:31] (03CR) 10Brennen Bearnes: [V: 03+2 C: 03+2] "Merging after team discussion in this morning's SSD Sync." [releng/local-charts] - 10https://gerrit.wikimedia.org/r/514166 (https://phabricator.wikimedia.org/T223715) (owner: 10Brennen Bearnes) [21:20:10] (03PS1) 10Jforrester: parameter_functions: Stop working on php70-phan, they're gone [integration/config] - 10https://gerrit.wikimedia.org/r/517745 [21:20:12] (03PS1) 10Jforrester: dockerfiles: [mediawiki-phan-seccheck] Move from php70 to php72 [integration/config] - 10https://gerrit.wikimedia.org/r/517746 (https://phabricator.wikimedia.org/T225456) [21:20:14] (03PS1) 10Jforrester: jjb: Provide php72 versions of the phan-seccheck jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517747 (https://phabricator.wikimedia.org/T225456) [21:20:16] (03PS1) 10Jforrester: layout: Add php72 variants of phan-seccheck jobs as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) [21:20:18] (03PS1) 10Jforrester: layout: Drop php70-phan-seccheck entirely [integration/config] - 10https://gerrit.wikimedia.org/r/517749 (https://phabricator.wikimedia.org/T225456) [21:20:20] (03PS1) 10Jforrester: jjb: Drop php70-phan-seccheck jobs, unused [integration/config] - 10https://gerrit.wikimedia.org/r/517750 (https://phabricator.wikimedia.org/T225456) [21:20:29] (03CR) 10Jforrester: [C: 03+2] parameter_functions: Stop working on php70-phan, they're gone [integration/config] - 10https://gerrit.wikimedia.org/r/517745 (owner: 10Jforrester) [21:21:44] (03CR) 10Jforrester: [C: 04-1] "Blocked on T207344. :-(" [integration/config] - 10https://gerrit.wikimedia.org/r/517749 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:22:03] (03Merged) 10jenkins-bot: parameter_functions: Stop working on php70-phan, they're gone [integration/config] - 10https://gerrit.wikimedia.org/r/517745 (owner: 10Jforrester) [21:22:49] (03CR) 10Jforrester: [C: 03+2] Codehealth: Copy sonar cache to place where castor can use it [integration/config] - 10https://gerrit.wikimedia.org/r/517739 (owner: 10Kosta Harlan) [21:24:15] (03Merged) 10jenkins-bot: Codehealth: Copy sonar cache to place where castor can use it [integration/config] - 10https://gerrit.wikimedia.org/r/517739 (owner: 10Kosta Harlan) [21:33:42] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [mediawiki-phan-seccheck] Move from php70 to php72 [integration/config] - 10https://gerrit.wikimedia.org/r/517746 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:33:50] (03CR) 10Jforrester: [C: 03+2] jjb: Provide php72 versions of the phan-seccheck jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517747 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:33:59] (03CR) 10Jforrester: [C: 03+2] layout: Add php72 variants of phan-seccheck jobs as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:35:11] (03Merged) 10jenkins-bot: dockerfiles: [mediawiki-phan-seccheck] Move from php70 to php72 [integration/config] - 10https://gerrit.wikimedia.org/r/517746 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:35:49] !log Pushing docker image quibble-stretch-php72:0.3.3 [21:35:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:36:02] (03Merged) 10jenkins-bot: jjb: Provide php72 versions of the phan-seccheck jobs [integration/config] - 10https://gerrit.wikimedia.org/r/517747 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:36:04] (03CR) 10jerkins-bot: [V: 04-1] layout: Add php72 variants of phan-seccheck jobs as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:37:27] 10Release-Engineering-Team (Kanban), 10Developer Productivity, 10local-charts: Fix Install Script For Some Versions of Ubuntu - https://phabricator.wikimedia.org/T223715 (10brennen) 05Open→03Resolved a:03brennen [21:37:39] 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T220735 (10mmodell) [21:38:49] (03PS2) 10Jforrester: layout: Add php72 variants of phan-seccheck jobs as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) [21:39:12] (03CR) 10Jforrester: [C: 03+2] "…" [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:40:53] (03Merged) 10jenkins-bot: layout: Add php72 variants of phan-seccheck jobs as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/517748 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [21:41:37] Unable to find image 'docker-registry.wikimedia.org/releng/quibble-stretch-php72:0.0.31-5' locally [21:41:44] James_F: [21:42:00] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10Jdforrester-WMF) Doesn't look like this was built? ` jforrester@contint1001:~$ docker images | grep quibble-stretch-php72 d... [21:42:00] wmf-quibble-core-vendor-mysql-php72-docker [21:42:11] Yeah, just found that myself. [21:42:59] I suppose can keep the dockerfiles change for now, but undo the jjb change? [21:43:08] !log Zuul: Adding php72 variants of phan-seccheck jobs as experimental [21:43:11] or building hte images indeed [21:43:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:44:21] 10Continuous-Integration-Infrastructure, 10PHP 7.2 support, 10Patch-For-Review: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10Jdforrester-WMF) quibble-stretch-php72:0.0.31-5 build fails for me locally. [21:46:34] 10Continuous-Integration-Config, 10ORES, 10Scoring-platform-team: ORES wheels deployment repo Jenkins job should gate-and-submit - https://phabricator.wikimedia.org/T211041 (10Halfak) This is probably something that #releng can help us with when we get there. https://gerrit.wikimedia.org/r/#/admin/project... [21:47:35] Aha. [21:48:53] git-praise traces it to https://gerrit.wikimedia.org/r/#/c/integration/config/+/517733/ [21:49:01] (03PS1) 10Jforrester: Follow-up I853558b7: [quibble-stretch-php72] Also needs wikimedia-php72.list [integration/config] - 10https://gerrit.wikimedia.org/r/517754 [21:49:19] (03CR) 10Jforrester: [C: 03+2] Follow-up I853558b7: [quibble-stretch-php72] Also needs wikimedia-php72.list [integration/config] - 10https://gerrit.wikimedia.org/r/517754 (owner: 10Jforrester) [21:49:31] …praise, huh? :-) [21:50:09] git-value-neutral-attribute-to [21:50:40] As of last week, I'm working on an SVN project again (Wildfire Games, 0ad). [21:50:45] I've not missed SVN. [21:50:47] (03Merged) 10jenkins-bot: Follow-up I853558b7: [quibble-stretch-php72] Also needs wikimedia-php72.list [integration/config] - 10https://gerrit.wikimedia.org/r/517754 (owner: 10Jforrester) [21:50:53] My sympathies. [21:51:17] But, it's not too bad when combined with Arcanist and Phabricator. [21:51:24] I'd say it works better with SVN than with GIt. [21:51:28] !log Updating docker-pkg files on contint1001 to fix quibble-stretch-php72 [21:51:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:52:01] except of course for the CLI itself, everything is over the network, slow and with all the svn quirks [21:52:15] 10Gerrit, 10ORES, 10Scoring-platform-team: Write a cookbook for the workaround for getting LFS to gerrit - https://phabricator.wikimedia.org/T226055 (10Halfak) [21:52:23] 10Gerrit, 10ORES, 10Scoring-platform-team: Write a cookbook for the workaround for getting LFS to gerrit - https://phabricator.wikimedia.org/T226055 (10Halfak) p:05Triage→03High [21:52:40] pre-merge CR and CI and all. not bad :) [21:54:27] Krinkle: Any chance you do phan-seccheck stuff? T207344 is the last blocker to php72… [21:54:28] T207344: Phan-taint-check-plugin not available for PHP > 7.0 - https://phabricator.wikimedia.org/T207344 [21:54:43] sorry, I haven't touched that at all, no idea :/ [21:55:15] Me either. Oh well. [21:55:57] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10PHP 7.2 support: CI uses PHP7.2 packages from sury.org instead of SRE ones - https://phabricator.wikimedia.org/T226028 (10Jdforrester-WMF) https://gerrit.wikimedia.org/r/517754 fixes it. Fun. Merged, rebuilt the images, and php72... [21:59:15] !log Manually pruned some old quibble-coverage Docker images from contint1001. [21:59:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:00:12] 10Release-Engineering-Team, 10Release Pipeline, 10Wikibugs: Hide successful PipelineBot messages from wikibugs's feed, the same way we do for jenkins-bot - https://phabricator.wikimedia.org/T218442 (10Jdforrester-WMF) 05Open→03Resolved [22:01:11] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban): Move coverage tasks from PHP70 to PHP72 - https://phabricator.wikimedia.org/T225832 (10Jdforrester-WMF) [22:01:21] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (Kanban): Drop php55 testing support - https://phabricator.wikimedia.org/T224907 (10Jdforrester-WMF) [22:14:03] !log Apply https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/OATHAuth/+/496451/53/sql/mysql/patch-add_generic_fields.sql to centralauth on beta T218210 [22:14:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:14:07] T218210: SO878 Step 1: Refactor OATHAuth extension - https://phabricator.wikimedia.org/T218210 [22:16:38] 23:13:38 /tmp/jenkins3985707510268826110.sh: line 3: THING_SUBNAME: unbound variable [22:16:57] https://integration.wikimedia.org/ci/job/mwext-php70-phan-seccheck-docker/60001/console [22:17:03] 23:13:38 /tmp/jenkins3985707510268826110.sh: line 3: THING_SUBNAME: unbound variable [22:17:03] 23:13:38 ++ /usr/bin/env [22:17:03] 23:13:38 Build step 'Execute shell' marked build as failure [22:17:04] :( [22:17:42] James_F: ^ that you? [22:28:12] 10Continuous-Integration-Config: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Reedy) [22:28:22] 10Continuous-Integration-Config: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Reedy) p:05Triage→03Unbreak! [22:31:14] 10Continuous-Integration-Config: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Reedy) [22:34:21] Reedy: HMM. [22:36:07] I don't see how it is me. [22:36:29] But I certainly touched that area. [22:37:48] James_F you added https://github.com/wikimedia/integration-config/commit/5016e756284ff213fbe27da9b62b32b1e51e3639 [22:37:50] which touches THING_SUBNAME [22:37:52] bloody computers [22:38:13] oh [22:38:14] nvm [22:38:20] that's 72, not 70 [22:38:56] Correct, I didn't touch the mwext-php70-phan-seccheck-docker jobs. [22:39:15] Reedy: I could just disable all the seccheck jobs. ;-) [22:43:02] lol [22:43:24] that'll fix it :D [22:43:38] Seems to be passing into composer:0.1.9 correctly. [22:44:43] unfortunately, it's not obvious what those shell files include [22:45:04] Yeah, we have an overly-complicated set of docker images. [22:45:58] I mean, it's not a highly used value [22:46:06] But on line 3, nothing obvious [22:46:14] Yeah. [22:46:57] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Fresnel, 10Performance-Team: Switch Fresnel for MediaWiki jobs from php70 to php72 - https://phabricator.wikimedia.org/T220866 (10Jdforrester-WMF) [22:47:09] though, `cd /mediawiki/$THING_SUBNAME` seems suspect [22:47:30] but when you see jjb/mediawiki.yaml: options: '--env THING_SUBNAME="" --volume "$(pwd)"/src:/mediawiki' # FIXME container should use /src [22:47:36] cd /mediawiki/"" [22:47:56] No, it's '' or 'extension/Foo' or 'skin/Bar' or 'vendor'. [22:48:06] Hence "thing" rather than something more obvious. [22:48:22] $SOME_RANDOM_REPO_SUBNAME [22:49:37] but unbound does mean undefined [22:49:53] Yeah. [22:49:58] 10Continuous-Integration-Config: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Jdforrester-WMF) Locally, `docker run --rm --env THING_SUBNAME=extensions/OATHAuth -v /Users/jdforrester/Documents/code/vagrant/mediawiki:/mediawiki docker-registry.wikimedia.org/rele... [22:50:28] ^ if that passes.. it sounds like wherever isn't passing it in correctly [22:52:01] Yeah. [22:52:33] but being fine on 72 is odd too [22:52:49] As it's not obviously passed in as a param to the exec docker [22:52:49] That's not phan seccheck that passes, that's phan. [22:53:06] Phan seccheck is a totally different thing. Because of course it is. [22:53:18] (And Phan seccheck fatals of php72.) [22:53:21] Don't blame me for that one :P [22:53:31] I'm blaming your team given you wrote it. :-) [22:53:43] My team? Or bawolff? ;P [22:53:54] "To run against skins, you would set the THING_SUBNAME environment variable to [22:53:54] something like `skins/MonoBook`." [22:53:54] Is there a difference? ;-) [22:53:57] very helpful [22:53:58] Rude [22:54:01] I do some work too [22:54:32] https://github.com/wikimedia/integration-config/blob/d2a796002ee4850f5a8b87fc8fe2a8d641806bac/dockerfiles/mediawiki-phan-seccheck/README.md#L14 [22:54:43] That suggests the run should be passing it [22:55:09] But then we get to https://github.com/wikimedia/integration-config/blob/5016e756284ff213fbe27da9b62b32b1e51e3639/jjb/mediawiki.yaml#L182 [22:55:15] THING_SUBNAME="" [22:55:32] That's for mediawiki-core-php70-phan-seccheck-docker. [22:55:38] Where that's correct. [22:55:46] https://integration.wikimedia.org/ci/job/mwext-php70-phan-seccheck-docker/60004/console is broken [22:55:55] Yes. [22:56:15] so why does the example not show "" [22:56:54] What? [22:56:55] Nowhere on https://github.com/wikimedia/integration-config/blob/d2a796002ee4850f5a8b87fc8fe2a8d641806bac/dockerfiles/mediawiki-phan-seccheck/README.md does it say "" is acceptable [22:57:04] It doesn't say "to run for everything" [22:57:10] No, it won't. [22:57:12] it only lists to run for an extension (or skin) [22:57:23] Blame bawolff for not writing a better README. [22:57:25] So why aren't we passing the $THING (lol) name? [22:57:36] We clearly are everywhere else. [22:57:37] https://github.com/wikimedia/integration-config/blob/d2a796002ee4850f5a8b87fc8fe2a8d641806bac/dockerfiles/mediawiki-phan-seccheck/run.sh#L7 [22:57:49] THING_SUBNAME is used by almost all of our templated jobs. [23:00:24] we're very not consistent with shell syntax uses of " etc [23:00:44] Probably not. [23:02:10] OK, it's been an hour and I'm still no closer to working on what broke. [23:02:42] I can try to revert the jjb changes… [23:05:16] Reverting it is a no-op for the relevant jjb changes. [23:05:30] My only solution is to make them non-voting, but that's a bit sucky. [23:05:37] Reedy: What do you think? [23:06:18] So what excatly changed? [23:06:24] AFAICT, nothing. [23:06:26] That's the problem. [23:06:47] Or just numerous things together? [23:07:00] I added a similar job but for php72, and which isn't enabled as a voting job anywhere, and they don't cross-reference. [23:07:30] It's possible one of the general pushes to update jobs pushed an already-broken set-up in master into CI production? [23:07:42] But it'd have had to have been broken for months without anyone testing… [23:07:45] I think so, ye [23:07:46] s [23:07:51] If someone hasn't pushed everything... [23:07:52] Actually, that does sound plausible, doesn't it? [23:07:59] And then someone does [23:08:02] Or someone only makes a change locally [23:08:06] Yeah. [23:08:06] jjb will wipe it out [23:08:12] AFAIK we had this with a security fix [23:08:22] People completely unknowingly wiped it out [23:08:30] Because it didn't exist anywhere else, unless people patched locally [23:08:39] Which is where zuul deploy vs jjb wins massively [23:08:39] 10Release-Engineering-Team (Kanban), 10Security-Team, 10phan-taint-check-plugin: Phan-taint-check-plugin not available for PHP > 7.0 - https://phabricator.wikimedia.org/T207344 (10Jdforrester-WMF) [23:08:44] Right. [23:08:55] I'd concur that seems a likely idea [23:09:11] Someone has probably fixed it locally, and never comitted it [23:09:52] :( [23:10:02] I suspect Antoine might fix it in like 5 minutes [23:11:39] 10Continuous-Integration-Config: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Jdforrester-WMF) Working supposition is that someone made a local patch and pushed it via jjb without committing it ages ago, and it got accidentally wiped out when jjb seccheck jobs w... [23:11:52] Maybe they still have it locally ;P [23:12:11] Possibly legoktm does, yes. [23:12:33] Why didn't we ping him already? :P [23:14:42] He's been idle for four hours, I assumed he's AFK. [23:15:00] 10Continuous-Integration-Config: rsync: failed to set times on "/cache/." - https://phabricator.wikimedia.org/T226065 (10Huji) [23:15:01] he tweeted about an hour ago about something about docker [23:15:14] podman [23:15:41] 10Release-Engineering-Team (Kanban), 10Code-Stewardship-Reviews: Code Stewardship Review: Collection Extension - https://phabricator.wikimedia.org/T224922 (10Krinkle) [23:16:19] (03PS1) 10Jforrester: layout: Temporarily make all seccheck jobs non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/517764 (https://phabricator.wikimedia.org/T226064) [23:16:21] (03PS1) 10Jforrester: Revert "layout: Temporarily make all seccheck jobs non-voting" [integration/config] - 10https://gerrit.wikimedia.org/r/517765 (https://phabricator.wikimedia.org/T226064) [23:17:09] (03CR) 10Jforrester: [C: 03+2] layout: Temporarily make all seccheck jobs non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/517764 (https://phabricator.wikimedia.org/T226064) (owner: 10Jforrester) [23:17:46] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Jdforrester-WMF) [23:18:37] (03Merged) 10jenkins-bot: layout: Temporarily make all seccheck jobs non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/517764 (https://phabricator.wikimedia.org/T226064) (owner: 10Jforrester) [23:19:52] !log Making extension-seccheck non-voting on all 195 repos, hopefully temporarily. T226064 [23:19:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:19:55] T226064: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 [23:22:06] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Jdforrester-WMF) p:05Unbreak!→03High No longer blocking merges, but pretty bad. Pinging @Bawolff and @Legoktm who might... [23:22:40] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban): mwext-php70-phan-seccheck-docker blocking merges - https://phabricator.wikimedia.org/T226064 (10Reedy) [23:23:03] Reedy: The revert is still unmerged. ;-) [23:23:27] meh [23:25:25] Also, there's a bot that removes that for people now. [23:29:43] Reedy: Interesting, mwext-php72-phan-seccheck-docker (which is known-broken) gets further than mwext-php70-phan-seccheck-docker right now: [23:29:50] https://integration.wikimedia.org/ci/job/mwext-php72-phan-seccheck-docker/1/console vs. https://integration.wikimedia.org/ci/job/mwext-php72-phan-seccheck-docker/1/console [23:29:58] Err. vs. https://integration.wikimedia.org/ci/job/mwext-php70-phan-seccheck-docker/60011/console [23:34:25] (03PS1) 10Jforrester: jjb: Migrate wikibase-{client|repo}-docker from php70 to php72 [integration/config] - 10https://gerrit.wikimedia.org/r/517768 [23:34:27] (03PS1) 10Jforrester: jjb: Migrate docker-setup-mwext-for-phan from php70 to php72 [integration/config] - 10https://gerrit.wikimedia.org/r/517769 [23:37:45] 10Release-Engineering-Team (Kanban), 10Code-Stewardship-Reviews: Code Stewardship Review: Collection Extension - https://phabricator.wikimedia.org/T224922 (10Jrbranaa) Thanks @Aklapper and @Krinkle for the additional detail on this CSR.