[00:11:03] 10Gerrit, 10Cosmos: Update the master branch from REL1_35 on the Cosmos skin - https://phabricator.wikimedia.org/T264826 (10Universal_Omega) [00:47:19] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10EBernhardson) [00:54:23] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10ssastry) [01:01:25] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10Krinkle) cache-text, cache-upload, etcd, ircd, and memc are more certainly also still in use and we need at least one of each. For memc we can most likely size down, I don't think we added this many i... [02:02:29] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10Peachey88) [02:03:34] 10Beta-Cluster-Infrastructure: Do something about deployment-imagescaler01 - https://phabricator.wikimedia.org/T244642 (10Peachey88) Is this task still needed, according to {T257118}, This instance has been deleted? [02:23:59] 10Continuous-Integration-Config, 10Cosmos, 10User-Kizule: Enable CI in mediawiki/skins/Cosmos - https://phabricator.wikimedia.org/T264818 (10Kizule) >>! In T264818#6523670, @Universal_Omega wrote: > Seems to be resolved. @Universal_Omega Yes, so now you won't have to manually click on "submit". You will now... [02:31:45] (03CR) 10Zoranzoki21: Add Universal_Omega in CI allowlist (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/632583 (owner: 10Zoranzoki21) [02:31:58] 10Continuous-Integration-Config, 10Cosmos, 10User-Kizule: Enable CI in mediawiki/skins/Cosmos - https://phabricator.wikimedia.org/T264818 (10Universal_Omega) >>! In T264818#6523881, @Kizule wrote: >>>! In T264818#6523670, @Universal_Omega wrote: >> Seems to be resolved. > > @Universal_Omega Yes, so now you... [02:50:58] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10dpifke) [02:53:33] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10dpifke) [04:12:38] !log deployment-prep: removing cwdent, demon, mobrovac, and smalyshev from access (former employees with very little chance of using access anymore) [04:12:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [05:01:29] I wonder if 😂 still has access anywhere else [05:07:17] :) :) [05:07:38] We removed some of his access in production recently [05:16:54] hah, he's still in this channel :) [07:31:32] 10Phabricator, 10Developer-Advocacy: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Aklapper) p:05Triage→03Lowest [08:39:22] 10Gerrit, 10Cosmos, 10translatewiki.net: Add mediawiki/skins/Cosmos to translatewiki.net - https://phabricator.wikimedia.org/T264695 (10Universal_Omega) @Raymond I requested your review in the Gerrit change. [09:20:51] 10Gerrit: Cannot see my own changesets - https://phabricator.wikimedia.org/T264657 (10hashar) If I look at {F32374633} Timeout waiting for endpoint properties initialization: plugin zuul, endpoint related-changes-section Timeout waiting for endpoint properties initialization: plugin gerrit-theme, endpoint comm... [09:52:04] 10Beta-Cluster-Infrastructure, 10Cloud-VPS, 10Maps, 10Product-Infrastructure-Team-Backlog: [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins - https://phabricator.wikimedia.org/T263726 (10MSantos) @WDoranWMF I would like to ask you if this task could have some help from an SR... [10:45:12] (03PS1) 10Jbond: operations-puppet: bump version to update Gem file [integration/config] - 10https://gerrit.wikimedia.org/r/632689 [11:16:08] is it normal that the quibble step “INFO:backend.MySQL:Terminating MySQL” takes over a minute? [11:16:13] I’ve noticed this in two test runs now [11:16:44] (the latest being https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/62835/console) [11:29:20] 10Gerrit, 10Cosmos: Update the master branch from REL1_35 on the Cosmos skin - https://phabricator.wikimedia.org/T264826 (10Ciencia_Al_Poder) You go to gerrit project page https://gerrit.wikimedia.org/r/admin/repos/mediawiki/skins/Cosmos,branches and then there's a link to create new, with the initial revision... [11:46:54] 10Gerrit, 10Cosmos: Update the master branch from REL1_35 on the Cosmos skin - https://phabricator.wikimedia.org/T264826 (10Universal_Omega) 05Open→03Resolved a:03Universal_Omega >>! In T264826#6524711, @Ciencia_Al_Poder wrote: > You go to gerrit project page https://gerrit.wikimedia.org/r/admin/repos/me... [12:08:25] (03PS2) 10Jbond: operations-puppet: bump version to update Gem file [integration/config] - 10https://gerrit.wikimedia.org/r/632689 [14:00:28] 10Project-Admins, 10Discovery-Search: Do not dump WikibaseCirrusSearch codebase tasks into some #Discovery-Search team project tag only - https://phabricator.wikimedia.org/T263725 (10Gehel) WikibaseCirrusSearch isn't really a dedicated code base, it is a technical component part of CirrusSearch. It is unclear... [14:01:20] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10ci-test-error: parser tests running under phpunit are broken in extensions - https://phabricator.wikimedia.org/T248180 (10kostajh) p:05Triage→03High None of the patches for GrowthExperiments pass CI because of this. Does anyone have ideas on what ca... [14:12:03] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10ci-test-error: parser tests running under phpunit are broken in extensions - https://phabricator.wikimedia.org/T248180 (10thcipriani) >>! In T248180#6525280, @kostajh wrote: > None of the patches for GrowthExperiments pass CI because of this. Does anyon... [14:14:42] 10Gerrit, 10Cosmos, 10translatewiki.net: Add mediawiki/skins/Cosmos to translatewiki.net - https://phabricator.wikimedia.org/T264695 (10Universal_Omega) @Raymond I went ahead and merged it after consulting with some people on GitHub. It can be changed or reverted if you notice anything wrong with it. [14:17:27] 10Gerrit, 10Cosmos, 10translatewiki.net: Add mediawiki/skins/Cosmos to translatewiki.net - https://phabricator.wikimedia.org/T264695 (10Universal_Omega) [14:37:28] 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide: Deployment of latest Design Style Guide Gerrit clone doesn't seem to succeed - https://phabricator.wikimedia.org/T264894 (10Volker_E) [14:52:56] 10Beta-Cluster-Infrastructure, 10Cloud-VPS, 10Maps, 10Product-Infrastructure-Team-Backlog: [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins - https://phabricator.wikimedia.org/T263726 (10WDoranWMF) @MSantos @hnowlan will take a look next and help out! [15:35:03] (03PS3) 10Hashar: operations-puppet: bump version to update Gem file [integration/config] - 10https://gerrit.wikimedia.org/r/632689 (owner: 10Jbond) [15:35:26] (03CR) 10Hashar: "Looks like we need the puppet.git change (https://gerrit.wikimedia.org/r/c/operations/puppet/+/632687) to be merged first" [integration/config] - 10https://gerrit.wikimedia.org/r/632689 (owner: 10Jbond) [15:37:12] (03CR) 10Jbond: "> Patch Set 3:" [integration/config] - 10https://gerrit.wikimedia.org/r/632689 (owner: 10Jbond) [15:45:08] (03CR) 10Lars Wirzenius: "There's python3-etcd3 in Debian testing and unstable:" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/628796 (owner: 10Hashar) [15:53:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Discovery-Search, and 3 others: Add Java 11 Image to Config Docker Images - https://phabricator.wikimedia.org/T259646 (10zeljkofilipin) @hashar Yes, please... [16:12:03] (03PS3) 10Dduvall: Print logs on helm test [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/622429 (https://phabricator.wikimedia.org/T255596) (owner: 10Jeena Huneidi) [16:15:48] tgr_: good morning :] I slightly hijacked your mediawiki/core patch which adds logging for sessions renewal :] [16:16:00] (03CR) 10Dduvall: [C: 03+2] Print logs on helm test [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/622429 (https://phabricator.wikimedia.org/T255596) (owner: 10Jeena Huneidi) [16:16:01] went a bit spammy since I failed to locally fix phpcs errors [16:16:31] (03Merged) 10jenkins-bot: Print logs on helm test [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/622429 (https://phabricator.wikimedia.org/T255596) (owner: 10Jeena Huneidi) [16:19:01] hashar: thanks! the log output is an utter mess, but I'm not sure if that can be improved without fixing that SessionManager tries to create a session about ten times per request [16:19:21] something is probably broken there wrt internal caching [16:20:02] but, it only sends log events when cookies are output, so it does the job, just confusingly [16:20:28] ok so the triple entries are not related to your change [16:20:43] then I went to use the TestLogger->setCollect( true ); here and there, but I probably did it wrong [16:20:58] I went to use $this->logger , but maybe it should be $this->manager->logger [16:21:26] and I am not sure whether the collect should be turned on at the top of the test method or just before invoking a method being tested ;] [16:21:42] we should backport https://gerrit.wikimedia.org/r/c/mediawiki/core/+/630354 btw [16:21:53] (assuming the next train is wmf11 and not 13) [16:23:35] guess we should backport it to 10 (currently running) 11 (from last week) and 12 (not even commited yet bah) [16:25:17] Done. [16:25:27] it's a unit test so $this->logger is the right one, but in general I'd avoid testing log output, it just makes tests more fragile [16:26:39] especially since I don't think it's possible to predict the exact log lines in this case, it really depends on how much SessionManager repeats each action, so the test is little more than a snapshot of actual behavior [16:27:03] let me reduce those tests (and sorry for not fixing them yesterday, I ran out of time) [16:27:39] tgr_: no worries ;] I had time this morning to conitnue and what you have build! [16:31:13] I am +2ing the wmf.10 backport [16:37:24] going to clean the lettuce and turn on the oven for pizzas! [16:43:22] T264794 is too complicated to be a train blocker. I thought there was an easy way to do it, but that didn't work. [16:43:22] T264794: SessionManager should not emit Set-Cookies on session renewal - https://phabricator.wikimedia.org/T264794 [16:50:08] (03CR) 10Dduvall: [C: 03+2] PipelineStage: fix default chart names [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/631301 (https://phabricator.wikimedia.org/T264236) (owner: 10Jeena Huneidi) [16:50:45] (03Merged) 10jenkins-bot: PipelineStage: fix default chart names [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/631301 (https://phabricator.wikimedia.org/T264236) (owner: 10Jeena Huneidi) [16:55:39] tgr_: at least if we get more logs, that is probably a good enough unblocker ;) [16:57:27] 10Beta-Cluster-Infrastructure, 10Cloud-VPS, 10Maps, 10Product-Infrastructure-Team-Backlog: [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins - https://phabricator.wikimedia.org/T263726 (10hnowlan) This issue is caused by deployment-prep not doing group and user creation the s... [17:00:38] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10ci-test-error: parser tests running under phpunit are broken in extensions - https://phabricator.wikimedia.org/T248180 (10Jdforrester-WMF) Argh, sorry about this. Daniel's patch also caused T264849 in (the non-gated part of) Wikibase-land. :-( [17:02:19] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10ci-test-error: parser tests running under phpunit are broken in extensions - https://phabricator.wikimedia.org/T248180 (10Jdforrester-WMF) Hmm, no, this isn't an HTTP change? This is just bitrot from the new-ish Cite feature, it looks like? [17:09:43] tgr_: the log change looks good to me ( https://gerrit.wikimedia.org/r/c/mediawiki/core/+/632601 ) but maybe we could use another pair of eyes cause I don't quite know that code [17:11:11] sure [17:12:20] tgr_: and I am finishing an audio with the team. Then I will look at scap deploying the change to the HeaderCallback ( https://gerrit.wikimedia.org/r/c/mediawiki/core/+/632680 ) [17:13:26] also, we'll have to check log volume during deploying. AFAICS it can only log when a cookie is emitted, which is not that frequent. But it is hard to be sure ad WebResponse swallows setCookie calls that would not change anything. The first version of the patch logged on every anon request as apparently SessionManager tries to clear cookies then. [17:15:01] afk for a short while [17:20:37] zeljkof: about the sonarqube/java11 thing, we can do it sometime next week in the morning [17:20:54] zeljkof: I cant really dig into it [17:21:08] hashar: sure, when ever is a good time for you [17:21:18] I'll be all eyes and questions :) [17:22:58] zeljkof: that is all I wanted to say at the end of the meeting ;] [17:23:24] hashar: cool 👍 [17:30:23] (03CR) 10Dduvall: [C: 04-1] Support promote step in systemtests (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/628239 (owner: 10Jeena Huneidi) [17:33:56] (03CR) 10Jeena Huneidi: Support promote step in systemtests (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/628239 (owner: 10Jeena Huneidi) [17:41:11] (03CR) 10Lars Wirzenius: [V: 03+2 C: 03+2] "This is way too long to review in detail, but skimming indicates it seems fine. I looked at the test changes more carefully: they're our q" (033 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/631509 (https://phabricator.wikimedia.org/T260674) (owner: 10Ahmon Dancy) [17:47:47] (03CR) 10Vedmaka Wakalaka: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/632764 (owner: 10Vedmaka Wakalaka) [17:57:37] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)): Set up experimental Docker registry in integration project - https://phabricator.wikimedia.org/T262453 (10dduvall) a:03dduvall [17:57:57] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)): Set up experimental Docker registry in integration project - https://phabricator.wikimedia.org/T262453 (10dduvall) p:05Triage→03Medium [18:16:28] hashar: any thoughts about the remaining error on the log patch? can't tell if it is related [18:16:56] I tried to recover a screenshot from Jenkins but can't find any [18:19:06] doh, there's a "show more" button at the end that you have to click a dozen times to see all the artifacts [18:20:21] https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/62950/artifact/log/warning-for-anonymous-edits_is-not-shown-when-logged-in.mp4 [18:20:29] that didn't clarify much though [18:23:46] I see all other recent core patches are failing with similar but not-quite-the-same Wikibase Selenium errors [18:23:54] so not related to the logging patch [18:28:06] filed T264926 [18:28:06] T264926: Wikibase Selenium tests are flaky - https://phabricator.wikimedia.org/T264926 [18:32:02] tgr_: sorry I have missed all the above ! :) [18:34:21] tgr_: any leads as to who can review the other patches? [18:34:57] there is only one ATM. CPT could review that, if I can sort the Selenium thing out. [18:38:39] 10Release-Engineering-Team (Logspam), 10MassMessage, 10Malayalam-Sites, 10Wikimedia-production-error: MassMessage not working for delivery to Suwikiquote or Mlwikiquote: "CAS update failed on user_touched" - https://phabricator.wikimedia.org/T171003 (10BPirkle) Still seeing on wikiquote and wikibooks as of... [18:40:18] tgr_: and those seelenium tests have grown out of control :-\ [18:42:38] also apparently a core test run is taking ~35 mins. I remember it being slow, but not *that* slow... [18:53:12] tgr_: and eventually that passed [18:55:13] yeah I think whatever broke in Wikibase got fixed a few minutes ago [18:55:20] the other patches are now passing too [19:01:18] 10phan, 10ObjectFactory: Add phan annotations to ObjectFactory - https://phabricator.wikimedia.org/T264930 (10Daimona) [19:03:47] tgr_: so we just need a knowledgeable person to +2 ? [19:11:03] what are your plans re: the train, hashar? I can get the IP address change logging thing done today. It won't detect 100% when some browser suddenly changes the user, and it will be noisy, but a lot better than nothing. [19:11:43] do you want to retry wmf.11 after that? on Thursday? [19:11:56] so I was in the weekl train log triage meeting [19:12:06] and just had a quick chat with thcipriani about it [19:12:18] so I guess yes logging [19:12:26] and logging and loggin [19:12:51] and then push wmf.11 to some group. Which one to be determinated today ideally [19:13:03] and we do it tomorrow with the appropriate people being around [19:13:20] cause for today, it is probably too late to have the patch completed/reviewed/merged and deployed [19:14:39] tgr_: I was just reading over what you had re:tasks to complete, if you're happy with the state of logging then we're happy to try wmf.11 again and watch. Do you need any help with things you have in-flight? [19:15:24] thcipriani: looking for a reviewer for https://gerrit.wikimedia.org/r/c/mediawiki/core/+/632601 [19:18:09] I +1ed it cause it looks fine to me [19:18:25] then I havent touched mediawiki in ages and don't know much about the session code [19:21:01] In general, I'm not really happy about the situation but have no idea what else to log. Cached cookies are easier to log in ATS or Varnish, but we already do that (and it apparently doesn't always work). [19:21:29] so that might be the more important part, but well outside my area of competence [19:21:54] I'm poking around to find folks who might be able to review that core patch [19:23:46] it should be pretty easy, just adds some logging [19:23:53] really just needs a sanity check [19:29:23] thanks men. I am off going to have dinner and sleep :] [19:55:06] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10ci-test-error: parser tests running under phpunit are broken in extensions - https://phabricator.wikimedia.org/T248180 (10Jdforrester-WMF) OK, so, rechecks on the master branches of both Cite (where the error is coming from?) and ParserFunctions (finger... [20:02:27] thcipriani: does it need a core dev to review or just someone who's comfortable with php? I can give a +2 if needed. [20:02:49] (patch looks good to me) [20:04:58] oh nvm I see someone else +2 [20:22:32] (03PS8) 10HitomiAkane: layout.yaml: Add HitomiAkane to Jenkins whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/631228 (https://phabricator.wikimedia.org/T235286) [20:37:38] twentyafterfour: thanks for the offer though :) [21:01:21] 10Beta-Cluster-Infrastructure: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 (10RLazarus) >>! In T257118#6523783, @Krinkle wrote: > cache-text, cache-upload, etcd, ircd, and memc are most certainly also still in use and we need at least one of each. > > For memc we can most like... [21:11:53] 10MediaWiki-Codesniffer: Prohibit use of 'object' as type in docblocks - https://phabricator.wikimedia.org/T264948 (10Daimona) [21:22:46] 10phan, 10ObjectFactory: Add phan annotations to ObjectFactory - https://phabricator.wikimedia.org/T264930 (10Daimona) > It should be possible to use the template type from within a @phan-param with explicit array type. Update: [[https://phan.github.io/demo/?code=%3C%3Fphp%0A%0Aclass+A+%7B%0A%7D%0A%0A%2F**%0... [21:41:32] 10phan, 10ObjectFactory, 10Patch-For-Review: Add phan annotations to ObjectFactory - https://phabricator.wikimedia.org/T264930 (10Daimona) a:03Daimona [22:21:50] 10Phabricator (Upstream), 10Upstream: Wrong title in revision from recentchanges log - https://phabricator.wikimedia.org/T264951 (10Zero_Children) [23:06:20] (03PS1) 10Ahmon Dancy: Remove misleading comments about puppet [tools/train-dev] - 10https://gerrit.wikimedia.org/r/632826 [23:07:36] (03CR) 10Ahmon Dancy: Add app servers to train-dev (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/631509 (https://phabricator.wikimedia.org/T260674) (owner: 10Ahmon Dancy) [23:10:50] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Add app servers to train-dev - https://phabricator.wikimedia.org/T260674 (10dancy) 05Open→03Resolved [23:10:53] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Scap, 10MediaWiki Train Development Environment, and 3 others: Create a development & testing environment for scap + train deployments - https://phabricator.wikimedia.org/T255695 (10dancy) [23:48:07] (03CR) 10DannyS712: [C: 03+1] Enables CI for ExternalLinkConfirm extension [integration/config] - 10https://gerrit.wikimedia.org/r/632764 (owner: 10Vedmaka Wakalaka)