[00:19:32] note that git-sync-upstream is complaining that: "could not apply 62c8d7b22a... acme_chief: Update acme_chief::cert resource to fetch several cert versions" [00:19:39] on beta's puppetmaster [00:19:45] (and now I really am gone to bed) [00:20:39] thanks [00:22:52] that commit was changed and merged [00:22:56] * Krenair drops from cherry-picks [00:23:19] it rebases over origin/production cleanly now [00:51:48] the change has been merged upstream (already backported it!) [03:45:53] 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10phan: mwext-php70-phan-docker doesn't use composer cache - https://phabricator.wikimedia.org/T217479 (10Legoktm) 05Open→03Resolved \o/ [04:01:32] (03PS1) 10Krinkle: wmf: Rewrite update.sh based on npmjs.org tarball [integration/npm] - 10https://gerrit.wikimedia.org/r/498524 [04:01:38] (03PS1) 10Krinkle: Install 6.5.0 [integration/npm] - 10https://gerrit.wikimedia.org/r/498525 [04:02:08] legoktm: ^ [04:06:02] legoktm: if that LGTM, I'd like to then rebuild the node10 images and see if it makes a difference [04:06:21] Right now almost 50% of gerrit repos can't use node 10 due to this strange zlib bug. [04:06:31] (03PS2) 10Krinkle: wmf: Rewrite update.sh based on npmjs.org tarball [integration/npm] - 10https://gerrit.wikimedia.org/r/498524 (https://phabricator.wikimedia.org/T215562) [04:06:34] (03PS2) 10Krinkle: Install 6.5.0 [integration/npm] - 10https://gerrit.wikimedia.org/r/498525 [06:41:37] (03CR) 10Legoktm: [V: 03+2 C: 03+2] wmf: Rewrite update.sh based on npmjs.org tarball [integration/npm] - 10https://gerrit.wikimedia.org/r/498524 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [06:50:37] (03CR) 10Legoktm: [V: 03+2 C: 03+1] "LGTM whenever you're ready to roll it out." [integration/npm] - 10https://gerrit.wikimedia.org/r/498525 (owner: 10Krinkle) [07:50:31] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Tiven2240) [09:16:13] RECOVERY - Puppet staleness on deployment-db04 is OK: OK: Less than 1.00% above the threshold [3600.0] [09:55:59] PROBLEM - Puppet errors on deployment-cache-upload04 is CRITICAL: CRITICAL: 2.22% of data above the critical threshold [3.0] [09:59:20] PROBLEM - Puppet errors on deployment-cache-text05 is CRITICAL: CRITICAL: 8.99% of data above the critical threshold [3.0] [10:00:18] PROBLEM - Host Generic Beta Cluster is DOWN: check_ping: Invalid hostname/address - en.wikipedia.beta.wmflabs.org [10:01:40] ^ me [10:01:44] designate thing again [10:01:48] have a task for that problem [10:03:24] it's coming back [10:04:28] gotta go, laptop battery dying, will be back to sort out more puppet things later [10:05:24] RECOVERY - Host Generic Beta Cluster is UP: PING OK - Packet loss = 0%, RTA = 1.58 ms [11:21:05] 10Gerrit: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10Kelson) [11:23:43] 10Gerrit: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10Kelson) [11:25:57] RECOVERY - Puppet errors on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [2.0] [11:34:22] RECOVERY - Puppet errors on deployment-cache-text05 is OK: OK: Less than 1.00% above the threshold [2.0] [11:34:27] RECOVERY - Puppet errors on deployment-mx02 is OK: OK: Less than 1.00% above the threshold [2.0] [11:38:53] (03CR) 10Hashar: "I will most probably cut a new version of Quibble next week (and update CI jobs) ;]" [integration/quibble] - 10https://gerrit.wikimedia.org/r/498206 (https://phabricator.wikimedia.org/T218844) (owner: 10Legoktm) [11:43:08] (03PS1) 10D3r1ck01: Add alternative email (xsavitar.wiki@aol.com) to remove prev soon [integration/config] - 10https://gerrit.wikimedia.org/r/498634 [11:59:15] (03CR) 10D3r1ck01: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/498634 (owner: 10D3r1ck01) [13:37:10] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [3.0] [14:21:46] (03CR) 10Zfilipin: "> Patch Set 4: Code-Review+2" [integration/config] - 10https://gerrit.wikimedia.org/r/494548 (https://phabricator.wikimedia.org/T215176) (owner: 10Kosta Harlan) [14:42:09] 10Gerrit, 10Wikimedia-General-or-Unknown, 10Documentation, 10Epic, and 3 others: Update Gerrit /r/p/ links to /r/ - https://phabricator.wikimedia.org/T218844 (10zeljkofilipin) [16:08:52] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10DBA: MySQL database on deployment-db03 does not start due to InnoDB issue - https://phabricator.wikimedia.org/T216635 (10hashar) deployment-db03 is still around. And yes I guess you can delete it. For a better recovery, I guess we could at least g... [16:53:05] 10Gerrit, 10Repository-Admins: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10Zoranzoki21) [16:53:49] 10Gerrit: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10Paladox) [17:13:24] 10Gerrit, 10Cleanup, 10GitHub-Mirrors, 10Repository-Admins: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10hashar) [17:14:02] 10Gerrit, 10Cleanup, 10GitHub-Mirrors, 10Repository-Admins: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10hashar) [17:14:49] 10Gerrit, 10Cleanup, 10GitHub-Mirrors, 10Repository-Admins: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10hashar) I have adjusted the description by copy pasting from #cleanup > Fill an archive request. That leads to [[ https://phabricator.wikimedia.org/maniphest/tas... [18:49:11] (03CR) 10Krinkle: [C: 03+2] Install 6.5.0 [integration/npm] - 10https://gerrit.wikimedia.org/r/498525 (owner: 10Krinkle) [18:57:46] 10Continuous-Integration-Config, 10Gerrit: Run `git submodule sync` when updating extensions.git clones - https://phabricator.wikimedia.org/T218929 (10Jdforrester-WMF) See IRC discussion. Some of Quibble's repos are randomly still pointing at /r/p/ not /r/ so jobs are fatalling on launch. [18:59:00] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:02:44] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Jdforrester-WMF) [19:03:30] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.33.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T206676 (10Jdforrester-WMF) 05Open→03Resolved Train is fully deployed and appears stable. I've moved T218940 to be a blocker to next week's tr... [19:09:02] 10Phabricator (Upstream), 10Upstream: Support "pinning" tasks to the top of a column - https://phabricator.wikimedia.org/T103427 (10Aklapper) 05Open→03Resolved Indeed, closing as "works for me" now that grouping within project workboard columns is in place. [19:11:44] 10Phabricator (Upstream), 10Upstream: Phantom unread notifications when users are subscribed to objects they are not allowed to view - https://phabricator.wikimedia.org/T102653 (10Aklapper) 05Open→03Resolved Assuming this is fixed as per last comment (thanks for the followup!). Please reopen this task if... [19:11:46] 10Diffusion, 10Phabricator (Upstream), 10Upstream: File contents do not show up if content is not valid utf-8 - https://phabricator.wikimedia.org/T144672 (10Aklapper) 05Open→03Resolved Assuming this is fixed as per last comment and upstream https://secure.phabricator.com/T3788 (thanks for the followup!).... [19:24:14] (03CR) 10D3r1ck01: [C: 03+2] Updated .gitignore for .idea/ directories [tools/scap] - 10https://gerrit.wikimedia.org/r/494727 (owner: 10Effie Mouzeli) [19:27:29] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494727 (owner: 10Effie Mouzeli) [19:27:31] (03Merged) 10jenkins-bot: Updated .gitignore for .idea/ directories [tools/scap] - 10https://gerrit.wikimedia.org/r/494727 (owner: 10Effie Mouzeli) [19:29:31] (03CR) 10jenkins-bot: Updated .gitignore for .idea/ directories [tools/scap] - 10https://gerrit.wikimedia.org/r/494727 (owner: 10Effie Mouzeli) [19:40:06] 10Phabricator (2019-03-20): Moving tasks between columns on workboard changed task priority (when sorting tasks by priority) - https://phabricator.wikimedia.org/T136400 (10Peachey88) [19:58:01] (03PS1) 10Krinkle: dockerfiles: Update node10 images with npm 6.5.0 tarball [integration/config] - 10https://gerrit.wikimedia.org/r/498664 (https://phabricator.wikimedia.org/T215562) [20:07:08] RECOVERY - Puppet errors on deployment-db03 is OK: OK: Less than 1.00% above the threshold [2.0] [20:33:55] (03CR) 10Jforrester: [C: 03+1] dockerfiles: Update node10 images with npm 6.5.0 tarball [integration/config] - 10https://gerrit.wikimedia.org/r/498664 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [20:44:14] (03CR) 10Krinkle: [C: 03+2] "Took a while to get this to work locally due to https://www.mediawiki.org/wiki/Continuous_integration/Docker#Manifest_not_found - verified" [integration/config] - 10https://gerrit.wikimedia.org/r/498664 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [20:45:56] (03Merged) 10jenkins-bot: dockerfiles: Update node10 images with npm 6.5.0 tarball [integration/config] - 10https://gerrit.wikimedia.org/r/498664 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [20:52:40] !log Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/498664 / T215562 [20:52:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:52:44] T215562: npm 6 consistently fails with "Z_DATA_ERROR: invalid distance too far back" on some repos - https://phabricator.wikimedia.org/T215562 [20:54:43] (03PS1) 10Krinkle: jjb: update node10 jobs to docker image 0.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/498667 (https://phabricator.wikimedia.org/T215562) [21:08:07] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 7.87% of data above the critical threshold [3.0] [21:34:07] (03CR) 10Krinkle: [C: 03+2] "Recompiled and uploaded the 13 "*node10*" jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/498667 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [21:36:58] (03Merged) 10jenkins-bot: jjb: update node10 jobs to docker image 0.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/498667 (https://phabricator.wikimedia.org/T215562) (owner: 10Krinkle) [21:37:09] legoktm: Well, that didn't make any difference [21:37:18] same zlib error on Flow as before [21:37:23] :|| [21:38:04] 10Phabricator, 10Pywikibot: Add a new Herald rule for pywikibot - https://phabricator.wikimedia.org/T218798 (10Aklapper) >>! In T218798#5041658, @Dvorapa wrote: > I take care of adding #pywikibot to new pywikibot tasks since April 2018. This Harold rule would just do my job (and also back in time) automaticall... [21:38:17] legoktm: next is how do we build the nodejs package? [21:38:26] 10Phabricator, 10Pywikibot: Add a new Herald rule for pywikibot - https://phabricator.wikimedia.org/T218798 (10Aklapper) I've created H314. ("Security is not any of Software security bug" does not make sense here.) >>! In T218798#5041687, @Dvorapa wrote: > BTW there are cca [[https://phabricator.wikimedia.org... [21:38:28] I think they come from nodesource [21:39:00] 10Phabricator, 10Pywikibot: Make Herald add #Pywikibot to any tasks with a #Pywikibot-* project tag - https://phabricator.wikimedia.org/T218798 (10Aklapper) [21:39:23] 10Continuous-Integration-Config, 10Patch-For-Review, 10Upstream: npm 6 consistently fails with "Z_DATA_ERROR: invalid distance too far back" on some repos - https://phabricator.wikimedia.org/T215562 (10Krinkle) ` Status: Downloaded newer image for docker-registry.wikimedia.org/releng/node10-test:0.4.0 .. + n... [21:39:40] legoktm: btw, are you able to make sense of this comment? https://npm.community/t/zlib-invalid-distance-too-far-back/5577 [21:39:59] probably worth asking Moritz https://phabricator.wikimedia.org/T203239#4587938 [21:40:40] uhhh [21:40:55] are they just suggesting we upgrade to debian testing? [21:42:29] Krinkle: no, I don't really understand the comment. why can't they say exactly which version of zlib works? [21:43:28] legoktm: component means nodesource? [21:44:29] I'm trying to find where "component/node10" is defined. [21:45:10] I guess it's not an automatic mirror like for debian [21:45:20] no, component is just something we created [21:45:34] So it's probably not defined anywhere other than that some people have agreed to pull stuff from nodesource and push it under that name in our repo [21:45:40] k [21:45:47] yes [21:45:56] though I'd double check with Moritz that we're not rolling our own nodejs packages [21:45:59] would the signature be signed or metadata'ed in some way to determine the source? [21:46:08] the package be signed* [21:46:16] or would we always re-sign ourselves? [21:46:18] https://tools.wmflabs.org/apt-browser/stretch-wikimedia/component/node10/ [21:46:30] given the +wmf2 in the version numbers, we've patched it somehow [21:46:40] so maybe not from nodesource then [21:47:33] https://github.com/wikimedia/operations-debs-nodejs is unused [21:47:38] https://tools.wmflabs.org/sal/log/AWeD_CXdA1BDhGjCcqIN [21:47:42] Trying to find a change log [21:48:40] apt changelog nodejs [21:51:16] Reading package lists... Done [21:51:16] root@694d449e7e2f:/src# apt changelog nodejs [21:51:16] E: Failed to fetch changelog:/nodejs.changelog Changelog unavailable for nodejs=10.4.0~dfsg-1+wmf2 [21:51:39] poking around in https://apt.wikimedia.org/wikimedia/dists/stretch-wikimedia/component/node10/ [21:51:48] Maintainer: Debian Javascript Maintainers [21:51:54] Krinkle: https://paste.fedoraproject.org/paste/F59FL14Lua1aam0C3dOmPA/raw [21:52:06] so I guess its forked from https://tracker.debian.org/news/964195/accepted-nodejs-1040dfsg-1-source-into-experimental/ [21:52:32] I added the deb-src line: [21:52:33] root@0f4dd2b3a4bd:~# cat /etc/apt/sources.list.d/stretch-node10.list [21:52:33] deb http://apt.wikimedia.org/wikimedia stretch-wikimedia component/node10 [21:52:33] deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia component/node10 [21:52:37] then ran # apt update [21:52:44] and then you can run "apt source nodejs" [21:52:54] and it'll unpack the debian tree used to create the package for you [21:52:58] 10Continuous-Integration-Config, 10Operations, 10Operations-Software-Development, 10Patch-For-Review, 10cloud-services-team (Kanban): Flake8 for python files without extension in puppet repo - https://phabricator.wikimedia.org/T144169 (10GTirloni) [21:53:30] legoktm: hm.. so googling kapouer@melix.org nodejs shows they do nodesource [21:53:34] so yeah, seems like that's what we're based on [21:53:54] they're also the debian maintainer of nodejs AIUI [21:54:01] Any particular reason this isn't a git repo? [21:54:01] https://tracker.debian.org/pkg/nodejs [21:54:13] that's a question for Moritz :) [21:54:22] k :) [21:54:33] I don't know what's "normal" but I thought most debs were in gerrit [21:56:28] 10Phabricator (Upstream), 10Upstream: [Send Message] button on person page is shown to users without permission, showing error message - https://phabricator.wikimedia.org/T91438 (10Aklapper) 05Open→03Invalid Indeed, closing as obsolete. Thanks for the followup! [22:00:47] 10Phabricator, 10Pywikibot: Make Herald add #Pywikibot to any tasks with a #Pywikibot-* project tag - https://phabricator.wikimedia.org/T218798 (10Dvorapa) >>! In T218798#5050852, @Aklapper wrote: > I've created H314. ("Security is not any of Software security bug" does not make sense here.) > >>>! In T218798... [22:02:04] legoktm: hm.. right [22:02:07] so might not be nodesource [22:02:42] Yeah, matches changelog from https://packages.debian.org/buster/nodejs [22:02:47] and https://launchpad.net/debian/+source/nodejs/10.4.0~dfsg-1 [22:03:15] But zlib1g varies between buster and stretch [22:03:16] https://packages.debian.org/buster/zlib1g [22:03:18] https://packages.debian.org/stretch/zlib1g [22:03:34] does that mean it could be our fault? [22:03:59] Like, something depends on the new zlib behaviour [22:04:15] the old zlib works fine with npm 3, so it's not "broken" broken. but it is broken with node 10 / npm 6. [22:09:04] I suppose, if they've said that node10 is incompatible with older zlib [22:09:09] then we'd also need to backport zlib or something [22:09:49] legoktm: well, nodesource offers all node versions for all distros pretty much [22:10:00] I'm trying to find out if they "did" something to make node10 build on stretch [22:10:14] https://deb.nodesource.com/node_10.x/dists/stretch/main/ [22:20:10] 10Phabricator, 10Pywikibot: Make Herald add #Pywikibot to any tasks with a #Pywikibot-* project tag - https://phabricator.wikimedia.org/T218798 (10Aklapper) 05Open→03Resolved a:03Aklapper Added #Pywikibot to 163 tasks by using `@Phabricator_maintenance`. Resolving task. [22:20:30] 10Phabricator, 10Pywikibot, 10Proposal, 10Pywikibot-RfCs: RFC: Add pywikibot-core to all new pywikibot-script/module/class tasks automatically - https://phabricator.wikimedia.org/T170090 (10Dvorapa) 05Stalled→03Resolved a:03Aklapper [22:22:25] 10Gerrit, 10Cleanup, 10GitHub-Mirrors, 10Repository-Admins: openzim gerrit repo should be archived - https://phabricator.wikimedia.org/T219060 (10Dinoguy1000) @hashar How much of that is needed just to archive this single Gerrit repo, though? It would be helpful if you could take a bit of time to go throug... [22:32:30] 10Phabricator, 10Pywikibot: Make Herald add #Pywikibot to any tasks with a #Pywikibot-* project tag - https://phabricator.wikimedia.org/T218798 (10Dvorapa) Many thanks! :) [23:22:26] https://integration.wikimedia.org/ci/job/operations-puppet-tests-stretch-docker/8596/console [23:22:36] 23:14:39 + git fetch --quiet zuul refs/zuul/production/Zca217914017e476b84b91484c2016189 [23:22:38] 23:14:39 fatal: Could not read from remote repository. [23:22:42] 23:14:39 Please make sure you have the correct access rights [23:22:44] 23:14:39 and the repository exists. [23:22:48] not a one-off either [23:30:10] hmm [23:30:18] is that a internal zuul ref? [23:30:26] not a ref stored in gerrit? [23:30:26] (03PS1) 10Legoktm: [MiniInvite] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498680 [23:30:53] but it seems like it couldn't even talk to the remote [23:31:13] (03CR) 10Legoktm: [C: 03+2] [EUCopyrightCampaign] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498247 (owner: 10Legoktm) [23:31:18] (03CR) 10Legoktm: [C: 03+2] [AdvancedSearch] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498249 (owner: 10Legoktm) [23:31:25] (03CR) 10Legoktm: [C: 03+2] [MiniInvite] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498680 (owner: 10Legoktm) [23:32:40] (03Merged) 10jenkins-bot: [EUCopyrightCampaign] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498247 (owner: 10Legoktm) [23:32:47] (03Merged) 10jenkins-bot: [AdvancedSearch] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498249 (owner: 10Legoktm) [23:33:14] (03Merged) 10jenkins-bot: [MiniInvite] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498680 (owner: 10Legoktm) [23:33:43] !log deploying https://gerrit.wikimedia.org/r/498247 https://gerrit.wikimedia.org/r/498249 https://gerrit.wikimedia.org/r/498680 [23:33:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:41:53] (03PS1) 10Legoktm: [Drafts] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498681 [23:41:55] (03CR) 10Legoktm: [C: 03+2] [Drafts] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498681 (owner: 10Legoktm) [23:41:57] (03PS1) 10Legoktm: [QuizGame] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498682 [23:41:59] (03CR) 10Legoktm: [C: 03+2] [QuizGame] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498682 (owner: 10Legoktm) [23:42:01] (03PS1) 10Legoktm: [SpamDiffTool] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498683 [23:42:03] (03CR) 10Legoktm: [C: 03+2] [SpamDiffTool] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498683 (owner: 10Legoktm) [23:44:01] (03Merged) 10jenkins-bot: [Drafts] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498681 (owner: 10Legoktm) [23:44:04] (03Merged) 10jenkins-bot: [QuizGame] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498682 (owner: 10Legoktm) [23:44:07] (03Merged) 10jenkins-bot: [SpamDiffTool] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/498683 (owner: 10Legoktm) [23:44:24] !log deploying https://gerrit.wikimedia.org/r/498681 https://gerrit.wikimedia.org/r/498682 https://gerrit.wikimedia.org/r/498683 [23:44:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:47:54] (03PS1) 10Legoktm: Fix typo in publish pipeline description [integration/config] - 10https://gerrit.wikimedia.org/r/498684 [23:53:25] (03CR) 10Krinkle: [C: 03+1] "thx" [integration/config] - 10https://gerrit.wikimedia.org/r/498684 (owner: 10Legoktm) [23:58:48] (03PS1) 10Legoktm: Whitelist more phan annotations [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/498686