[00:53:04] 10Release-Engineering-Team-TODO, 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.15 deployment blockers - https://phabricator.wikimedia.org/T263181 (10thcipriani) 05Open→03Declined Week of data center switchover [07:24:32] (03CR) 10Hashar: [C: 03+2] Mark WikimediaApiPortal skin as in production [integration/config] - 10https://gerrit.wikimedia.org/r/633831 (owner: 10Cicalese) [07:25:45] (03Merged) 10jenkins-bot: Mark WikimediaApiPortal skin as in production [integration/config] - 10https://gerrit.wikimedia.org/r/633831 (owner: 10Cicalese) [07:43:12] 10Project-Admins: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10danshick-wmde) [07:45:05] 10Project-Admins: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10danshick-wmde) For WMDE-Technical-Writing the default assignee should be me. [07:54:43] 10Gerrit: Cannot see my own changesets - https://phabricator.wikimedia.org/T264657 (10hashar) 05Stalled→03Resolved a:03hashar @Amd989 thanks for coming back to us! Filing this task was definitely the appropriate thing to do, at the expense of some investigation on the side of our Gerrit, that at least see... [07:59:40] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10Kizule) 05Open→03Resolved a:03Kizule Hello @danshick-wmde, I've created projects which you requested: #WMDE - https://phabricator.wikimedia.org/project/view/5041/ #wmde-t... [08:00:05] Project mwcore-phpunit-coverage-master build #969: 04FAILURE in 5 hr 0 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/969/ [08:03:04] 10Release-Engineering-Team (CI & Testing services), 10MediaWiki-skins-WikimediaApiPortal: Create a coverage job for skins - https://phabricator.wikimedia.org/T265407 (10hashar) [08:03:54] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MediaWiki-skins-WikimediaApiPortal: Create a coverage job for skins - https://phabricator.wikimedia.org/T265407 (10hashar) [08:07:06] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10danshick-wmde) Thank you for the speedy action! I don't know what a Herald rule is, but I assume you're saying I'd need one for setting a default assignee. Will do -- [08:10:08] 10Release-Engineering-Team, 10Growth-Structured-Tasks, 10Growth-Team, 10serviceops: Move mwaddlink-query from github to gerrit - https://phabricator.wikimedia.org/T261403 (10kostajh) >>! In T261403#6538868, @MGerlach wrote: > @kostajh >> 1. take https://github.com/martingerlach/mwaddlink-query and move ut... [08:11:58] 10Release-Engineering-Team, 10Growth-Structured-Tasks, 10Growth-Team, 10serviceops: Move mwaddlink-query from github to gerrit - https://phabricator.wikimedia.org/T261403 (10kostajh) >>! In T261403#6540928, @thcipriani wrote: >> Second, should this go under `mediawiki/services/{service-name}`, or somewhere... [08:17:28] 10Project-Admins: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10danshick-wmde) [08:26:36] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10Kizule) [08:28:50] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10Kizule) > I don't know what a Herald rule is, but I assume you're saying I'd need one for setting a default assignee. @danshick-wmde Herald is something like bot, but it is int... [08:33:33] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MediaWiki-skins-WikimediaApiPortal: Create a coverage job for skins - https://phabricator.wikimedia.org/T265407 (10hashar) I had a very quick look at it, m... [08:36:16] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10danshick-wmde) Thanks @Kizule ! I opened https://phabricator.wikimedia.org/T265453 since you said I would need another Phab task for it. If I can set my own Herald rules, pleas... [08:39:54] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10Kizule) >>! In T265448#6541622, @danshick-wmde wrote: > Thanks @Kizule ! I opened https://phabricator.wikimedia.org/T265453 since you said I would need another Phab task for it... [09:08:00] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10danshick-wmde) Thank you, I tried it! [09:09:03] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10danshick-wmde) Maybe I've already succeeded in doing this thanks to @Kizule 's suggestions in T265453 ... [09:11:07] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10Kizule) 05Open→03Resolved a:03danshick-wmde This is done by @danshick-wmde in {H374} and works (tested with T265458). [09:25:18] 10Project-Admins: Proposal: create wmit-infrastructure - https://phabricator.wikimedia.org/T265461 (10valerio.bozzolan) [09:28:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Fundraising-Backlog, 10Patch-For-Review: JJB / zuul support for upgrading payments-wiki to php7.3 and MW 1.35 - https://phabricator.wikimedia.org/T261777 (... [09:31:43] (03CR) 10Kosta Harlan: [C: 03+1] layout: [AbuseFilter] Add extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/633297 (https://phabricator.wikimedia.org/T224990) (owner: 10Daimona Eaytoy) [09:50:51] 10Project-Admins: Proposal: create wmit-infrastructure - https://phabricator.wikimedia.org/T265461 (10Nemo_bis) Ok for me. We may migrate a few issues/discussions/todos currently "tracked" on our internal wiki. (For members: I mean https://wiki.wikimedia.it/wiki/Associazione:Sito .) [10:27:31] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10Aklapper) 05Resolved→03Open @danshick-wmde: Please do not do this and please disable this Herald rule. Auto-assigning is a classic anti-pattern in software development: All and any tasks get assigne... [10:31:00] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10Aklapper) (@Kizule: Please follow https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects and https://www.mediawiki.org/wiki/Phabricator/Project_management#Ty... [10:48:03] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10danshick-wmde) 05Open→03Invalid p:05Triage→03Lowest [10:58:38] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10danshick-wmde) I've done as you requested. I respectfully request that if Herald allows functionality that is rejected by organizational policy, that functionality should be disabled if possible or at le... [11:57:07] 10Phabricator-Bot-Requests: Create PatchDemoBot account - https://phabricator.wikimedia.org/T265418 (10matmarex) +1 [12:03:38] 10Phabricator: Set default assignee for WMDE-Technical-Writing - https://phabricator.wikimedia.org/T265453 (10Aklapper) Thanks; documented in https://www.mediawiki.org/w/index.php?title=Phabricator/Help/Herald_Rules&type=revision&diff=4171711&oldid=3719058 [12:27:40] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Doxygen, and 3 others: mwext-doxygen-publish Doxygen 1.8.18 fails for Wikibase.git - https://phabricator.wikimedia.org/T254465 (10hashar) The `debian/1.8.1... [12:42:52] 10Release-Engineering-Team, 10Growth-Structured-Tasks, 10Growth-Team, 10serviceops: Move mwaddlink-query from github to gerrit - https://phabricator.wikimedia.org/T261403 (10kostajh) >>! In T261403#6538868, @MGerlach wrote: >> 3. Growth engineers can work with you and Release Engineering to set up the Depl... [12:51:58] 10Phabricator-Bot-Requests: Create PatchDemoBot account - https://phabricator.wikimedia.org/T265418 (10Aklapper) 05Open→03Resolved a:03Aklapper @PatchDemoBot has been created. See P12992 for the API token. [12:53:20] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Aklapper) @urbanecm: I've added you. [13:42:40] 10Project-Admins, 10User-Kizule: New projects/tags: "WMDE" and "WMDE-Technical-Writing" - https://phabricator.wikimedia.org/T265448 (10Kizule) >>! In T265448#6541922, @Aklapper wrote: > (@Kizule: Please follow https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects and https://www.mediawiki.o... [14:15:48] 10Phabricator-Bot-Requests: Create PatchDemoBot account - https://phabricator.wikimedia.org/T265418 (10Esanders) Thanks [14:17:40] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10MW-on-K8s: Run stress tests on docker images infrastructure - https://phabricator.wikimedia.org/T264209 (10akosiaris) 1st obstacle found already. The push failed with '500 internal server error'. Logs in... [15:46:00] 10Project-Admins: Create addlink component - https://phabricator.wikimedia.org/T265346 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project has been created: #Add-Link. Feel free to adjust / correct the project description. Interested people are welcome to join the project as {icon users}... [15:50:11] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Fundraising-Backlog, 10Patch-For-Review: JJB / zuul support for upgrading payments-wiki to php7.3 and MW 1.35 - https://phabricator.wikimedia.org/T261777 (... [15:51:56] 10Project-Admins: Create addlink component - https://phabricator.wikimedia.org/T265346 (10MBinder_WMF) Based on conversations with the team, I assumed the Herald also needed updating, so I did so: https://phabricator.wikimedia.org/H26 [15:54:48] 10Project-Admins: Request for API-Portal project - https://phabricator.wikimedia.org/T265495 (10apaskulin) [16:15:10] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Fundraising-Backlog, 10Patch-For-Review: JJB / zuul support for upgrading payments-wiki to php7.3 and MW 1.35 - https://phabricator.wikimedia.org/T261777 (... [16:21:12] (03CR) 10Hashar: [C: 03+2] "Well done! I have created the Jenkins jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/633526 (https://phabricator.wikimedia.org/T265275) (owner: 10Karl Wettin (WMSE)) [16:22:35] (03Merged) 10jenkins-bot: Add publish pipeline to all Speechoid services [integration/config] - 10https://gerrit.wikimedia.org/r/633526 (https://phabricator.wikimedia.org/T265275) (owner: 10Karl Wettin (WMSE)) [16:23:33] (03CR) 10Hashar: "Zuul reloaded." [integration/config] - 10https://gerrit.wikimedia.org/r/633526 (https://phabricator.wikimedia.org/T265275) (owner: 10Karl Wettin (WMSE)) [16:24:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Wikispeech-Jobrunner, and 4 others: Publish Speechoid to docker repo - https://phabricator.wikimedia.org/T265275 (10hashar) a:03kalle I have deployed the... [16:31:32] ejegg: Are we good to land https://gerrit.wikimedia.org/r/c/integration/config/+/624791 now? (FundraisingEmailSubscribe master over to REL1_35 compat. instead of REL1_31.) [16:32:10] Almost James_F ! Just working with hashar to get the CI updated [16:32:22] thanks for your patience [16:33:45] Always. :-) [16:33:49] I did just cherry-pick it over to REL1_35 [16:33:52] I'm just keen to get it done. :-) [16:33:54] Yeah, I saw. [16:34:02] Is the master branch still used in a PHP 7.0 context? [16:34:39] Cut-over is always a pain; having a master branch that doesn't reflect production (or whatever) is a pain. [16:34:49] I can imagine it's quite stressful! [16:35:33] (03PS5) 10Jforrester: layout: [FundraisingEmailUnsubscribe] Make special FR REL1_35-php73 job voting [integration/config] - 10https://gerrit.wikimedia.org/r/624755 [16:36:04] (03PS4) 10Jforrester: layout: [FundraisingEmailUnsubscribe] Drop special FR REL1_31-php70 job [integration/config] - 10https://gerrit.wikimedia.org/r/624791 [16:44:32] (03CR) 10Jforrester: Remove fundraising/REL1_31 tests, point fr/1_35 to master (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/633799 (https://phabricator.wikimedia.org/T261777) (owner: 10Ejegg) [16:45:30] 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Scap: Make a way to build Scap .deb in Docker - https://phabricator.wikimedia.org/T265501 (10LarsWirzenius) [16:45:53] 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MediaWiki Train Development Environment: Make a way to test Scap in train-dev - https://phabricator.wikimedia.org/T265502 (10LarsWirzenius) [17:33:56] James_F: yeah, fortunately with the LTS we only have to do it every 2 years [17:34:28] Just working on clearing up a few more failing tests: https://integration.wikimedia.org/ci/job/quibble-donationinterface-REL1_35-php73-docker/10/console [17:37:47] * James_F nods. [17:41:46] Oh, to answer your question, we started sending traffic to the 1.35-upgraded (php7.3) payments-wiki servers last week, and just today have started upgrading the other set of servers [17:42:04] so we're past the point of no return, and will no longer use DonationInterface on php7.0 anywhere [17:42:26] that's why I'm trying to get those fixes merged back to master [18:12:55] !log Cherry-picking https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/634058 in beta [18:12:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:17:17] Project beta-scap-eqiad build #320969: 04FAILURE in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/320969/ [18:27:23] Yippee, build fixed! [18:27:23] Project beta-scap-eqiad build #320970: 09FIXED in 3 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/320970/ [18:48:58] (03PS1) 10Ahmon Dancy: ansible/install-php-packages.yml: Add php7.2-dba package [tools/train-dev] - 10https://gerrit.wikimedia.org/r/634061 [19:13:54] Project beta-scap-eqiad build #320975: 04FAILURE in 0.79 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/320975/ [19:25:30] (03CR) 10Mstyles: "recent codehealth jobs have been failing....I think it could be because of these changes. See https://gerrit.wikimedia.org/r/c/wikidata/qu" [integration/config] - 10https://gerrit.wikimedia.org/r/633558 (https://phabricator.wikimedia.org/T259646) (owner: 10Hashar) [19:27:29] Yippee, build fixed! [19:27:30] Project beta-scap-eqiad build #320976: 09FIXED in 3 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/320976/ [19:33:29] Yippee, build fixed! [19:33:29] Project mwcore-phpunit-coverage-master build #970: 09FIXED in 4 hr 33 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/970/ [19:41:41] (03CR) 10Hashar: [C: 04-1] "Thanks for the report Mstyles." [integration/config] - 10https://gerrit.wikimedia.org/r/633558 (https://phabricator.wikimedia.org/T259646) (owner: 10Hashar) [20:08:13] marxarelli: I'm seeing some weird formatting errors on Commons and wondering if it might be related to the group 1 deployment today. Are you the deployment person today? [20:09:34] kaldari: I am [20:10:28] If I go to any Commons pages while logged out, e.g. https://commons.wikimedia.org/wiki/Main_Page , the personal toolbar at the top is unstyled due to some missing CSS classes. Can you confirm in your browser? [20:10:58] for some reason it's only while logged out for me. [20:12:20] Looks bad on my phone in desktop mode but let me get back to my laptop [20:14:29] Other group1 wikis look fine though. Seems to be specific to commons. [20:18:16] kaldari: back. ok yeah, looks mucked up [20:18:22] unstyled [20:19:02] well, i can rollback [20:19:55] yeah, might need to. Sorry for the hassle. [20:20:21] I'll try to figure out what's causing it. [20:20:51] Is it possible to only roll-back Commons? [20:22:12] kaldari: the toolbar works for me, you sure it's not your client caching something? [20:22:31] we could but i'd rather rollback all of group1 in case it's happening elsewhere [20:22:46] Majavah: are you logged in? i'm seeing it as anonymous [20:23:00] marxarelli: yeah, logged out [20:23:29] https://usercontent.irccloud-cdn.com/file/cdnTObxV/Screen%20Shot%202020-10-14%20at%201.21.13%20PM.png [20:23:31] yes, and I've tried in 3 different browsers on 2 different machines, but could be related to HTML cached on a particular varnish server? [20:23:41] ah [20:25:48] yeah, with X-Wikimedia-Debug i don't see the issue [20:26:06] kaldari: ^ [20:26:34] marxarelli: would it be worth running scap again? [20:27:25] i suppose but i'm not sure what that'll do [20:27:35] me neither :) [20:27:36] are you thinking it's the blob store or something? [20:28:44] There's definitely some mal-formed HTML cached somewhere (either in varnish or whereever skin-output is cached on the servers). [20:30:40] it's the actual HTML that is bad, not the CSS (which I've never seen happen before). [20:31:36] ah interesting [20:31:41] i'm going to rollback group1 then [20:31:50] and give you and others time to inveestigate [20:31:57] uh oh, I'm seeing the same problem on Meta now too. [20:32:21] yeah, better roll back for now [20:40:04] kaldari: k. rolled back. i unfortunately have to step out for a short while (~ 1 hr) but i'll check back in asap [20:43:15] thanks for rollback marxarelli [20:43:53] oddly I saw the same output until I did a refresh and then it was gone [20:43:58] one meta and on commons :\ [20:45:08] ah, I see there's a task already https://phabricator.wikimedia.org/T265543 [20:51:35] marxarelli: thcipriani fix in. [20:51:41] When is wmf14 being deployed? [20:51:47] i'm trying to work out if the patch is needed there too [20:52:12] wmf.14? will be cut on Tuesday next week [20:52:37] okay great then provided this patch goes out today we should be fine (our TTL for cached HTML is < 7 days) [20:53:09] I saw a patch get merged, ready for backport? [20:54:05] yep [20:54:28] cool, once that merges I'll deploy and ping you in -operations when staged [20:55:05] just seeing scrollback here; i'm available to help if needed. [20:59:42] Sorry I missed all the excitement. I'm back now so if I can help out let me know [21:01:42] longma: no worries, you didn't miss anything yet I can handle the backport and then will gladly hand train back :) [21:03:03] okie dokie. So I'll deploy wmf.13 to group1 after the backport, right? [21:05:55] I think I may have seen another blocker come in? [21:06:54] but I'm not actually clear if it's a blocker for this train or not...looks like it's from wmf.9 [21:07:59] oh, I guess Jdlrobson filed that one, too, is https://phabricator.wikimedia.org/T263599 meant to block train as well? [21:11:10] that seems possible [21:16:21] thcipriani: yep syntaxHighlighterConfig [21:16:26] paste fail [21:16:26] https://gerrit.wikimedia.org/r/634002 [21:16:52] ^ thcipriani that's safe to merge (i cant as dont have +2) [21:17:14] Jdlrobson: could you check that one too if I backport? [21:17:22] thcipriani: yup [21:17:26] :) [21:17:33] not going anywhere