[00:19:50] 10Release-Engineering-Team (Kanban), 10Scap, 10Scoring-platform-team, 10Patch-For-Review: Support git-lfs - https://phabricator.wikimedia.org/T180627#4144828 (10awight) It's reasonable that git might keep copies of the file around, to make it possible to switch branches for example without incurring huge b... [07:03:03] !log Replaced integration-slave-docker-1001 to get 4GB of RAM [07:03:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:20:59] (03PS2) 10Hashar: Switch mediawiki/core and vendor to quibble [integration/config] - 10https://gerrit.wikimedia.org/r/427697 [07:22:44] (03CR) 10Hashar: [C: 032] Switch mediawiki/core and vendor to quibble [integration/config] - 10https://gerrit.wikimedia.org/r/427697 (owner: 10Hashar) [07:24:01] (03Merged) 10jenkins-bot: Switch mediawiki/core and vendor to quibble [integration/config] - 10https://gerrit.wikimedia.org/r/427697 (owner: 10Hashar) [07:29:09] (03PS1) 10Hashar: Allow quibble on non master branches [integration/config] - 10https://gerrit.wikimedia.org/r/427857 [07:30:39] (03PS1) 10Hashar: Adjust tests after 01e1932b7b [integration/config] - 10https://gerrit.wikimedia.org/r/427858 [07:32:20] (03CR) 10Hashar: [C: 032] Adjust tests after 01e1932b7b [integration/config] - 10https://gerrit.wikimedia.org/r/427858 (owner: 10Hashar) [07:33:11] (03PS1) 10Hashar: Drop Quibble php5.5 from REL1_31 [integration/config] - 10https://gerrit.wikimedia.org/r/427859 [07:33:41] (03Merged) 10jenkins-bot: Adjust tests after 01e1932b7b [integration/config] - 10https://gerrit.wikimedia.org/r/427858 (owner: 10Hashar) [07:34:24] (03CR) 10Hashar: [C: 032] Drop Quibble php5.5 from REL1_31 [integration/config] - 10https://gerrit.wikimedia.org/r/427859 (owner: 10Hashar) [07:35:44] (03Merged) 10jenkins-bot: Drop Quibble php5.5 from REL1_31 [integration/config] - 10https://gerrit.wikimedia.org/r/427859 (owner: 10Hashar) [07:58:17] (03CR) 10Hashar: "> Core's phpcs times out if just run through composer (which is why we had this bespoke job) – and quibble tries the composer route and th" [integration/config] - 10https://gerrit.wikimedia.org/r/427698 (owner: 10Hashar) [07:58:37] 10Deployments, 10Release-Engineering-Team (Kanban), 10Operations, 10Patch-For-Review, 10Release: Deploy Scap 3.8.0 to production - https://phabricator.wikimedia.org/T192124#4145207 (10fgiunchedi) >>! In T192124#4144670, @demon wrote: > That was part of that commit. I was kinda following the example set b... [08:00:22] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10MW-1.30-release-notes, 10MW-1.27-release-notes, and 4 others: Phaseout CI mediawiki config / extensions_load.txt to load extensions - https://phabricator.wikimedia.org/T189567#4145208 (10hashar) I havent worked on th... [08:03:51] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4145209 (10hashar) The jobs for mediawiki/core without extensions have been migrated to use... [08:04:37] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#3855599 (10hashar) a:03hashar [08:06:24] 10Continuous-Integration-Infrastructure (shipyard): CI docker build should use a git cache - https://phabricator.wikimedia.org/T175968#4145216 (10hashar) 05Open>03declined docker build doesn't support mounting a volume. For CI the git cache is on the host which is then mounted at run time with `-v /srv/git:... [08:26:50] !log replaced integration-slave-docker-1010 and integration-slave-docker-1011 with bigger instances (4GB RAM) [08:26:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:27:09] !log Created integration-slave-docker-1016 and integration-slave-docker-1017 (2G RAM / 2 executors ) [08:27:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:38:56] !log Cleaned integration-slave-docker-1005 disk (deleting workspace and all docker images) [08:38:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:39:22] 10Deployments, 10Release-Engineering-Team (Kanban), 10Operations, 10Patch-For-Review, 10Release: Deploy Scap 3.8.0 to production - https://phabricator.wikimedia.org/T192124#4145433 (10mmodell) [09:39:56] 10Deployments, 10Release-Engineering-Team (Kanban), 10Operations, 10Patch-For-Review, 10Release: Deploy Scap 3.8.0 to production - https://phabricator.wikimedia.org/T192124#4128785 (10mmodell) >>! In T192124#4145207, @fgiunchedi wrote: > I don't know about conftool but AFAICS scap doesn't contain archite... [13:51:30] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427906 [13:51:33] (03CR) 10QChris: [V: 031 C: 032] Allow “Gerrit Managers” to import history [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427906 (owner: 10QChris) [13:51:49] (03CR) 10QChris: [V: 032 C: 032] Allow “Gerrit Managers” to import history [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427906 (owner: 10QChris) [13:53:22] (03PS1) 10QChris: Import done. Revoke import grants [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427908 [13:53:25] (03CR) 10QChris: [V: 031 C: 032] Import done. Revoke import grants [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427908 (owner: 10QChris) [13:53:39] (03CR) 10QChris: [V: 032 C: 032] Import done. Revoke import grants [wikidata/query/LDFServer] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/427908 (owner: 10QChris) [14:01:15] 10Gerrit: Update gerrit to 2.14.8 - https://phabricator.wikimedia.org/T192648#4145889 (10Paladox) [14:04:22] 10Gerrit: Update gerrit to 2.14.8 - https://phabricator.wikimedia.org/T192648#4145914 (10Paladox) [14:11:07] Is there something special I should/can do with scap to remove a file? Will sync-file work? [14:11:37] Nope [14:11:39] Sync the parent dir [14:11:47] so the whole dblist folder [14:12:54] * andrewbogott tries [14:17:39] It's gone from https://noc.wikimedia.org/conf/ :) [14:29:33] 10Phabricator, 10Project-Admins: Replace all "tracking" tasks with tags/projects (if unbounded) or relabel them to outcome tasks (if bounded) - https://phabricator.wikimedia.org/T192655#4146000 (10Jdforrester-WMF) [14:32:42] 10Project-Admins: Replace tracking bug T24710 by new project tag "IIS" - https://phabricator.wikimedia.org/T102505#4146029 (10Jdforrester-WMF) [14:32:45] 10Project-Admins: Replace tracking bug T22257 by new project tag "SQLite" - https://phabricator.wikimedia.org/T102503#4146030 (10Jdforrester-WMF) [14:32:47] 10Project-Admins: Replace tracking bug T21719 by new project tag "HTML5" - https://phabricator.wikimedia.org/T102502#4146031 (10Jdforrester-WMF) [14:32:49] 10Project-Admins: Replace tracking bug T18456 by new project tag "GIF" - https://phabricator.wikimedia.org/T102501#4146032 (10Jdforrester-WMF) [14:32:51] 10Project-Admins: Replace tracking bug T16123 by new project tag "AJAX" - https://phabricator.wikimedia.org/T102499#4146033 (10Jdforrester-WMF) [14:32:53] 10Project-Admins: Replace tracking bug T5646 by new project tag "Feeds" - https://phabricator.wikimedia.org/T102495#4146035 (10Jdforrester-WMF) [14:32:55] 10Project-Admins: Replace tracking bug T11767 by new project tag "MSSQL" - https://phabricator.wikimedia.org/T102497#4146034 (10Jdforrester-WMF) [14:32:58] 10Project-Admins: Replace tracking bug T2209 by new project tag "HTML" - https://phabricator.wikimedia.org/T102492#4146036 (10Jdforrester-WMF) [14:33:00] 10Project-Admins: Replace tracking bug T76447 by new project tag "Wiktionary-fr" - https://phabricator.wikimedia.org/T101948#4146037 (10Jdforrester-WMF) [14:33:02] 10Project-Admins: replace schema change tracking task T51188 by project - https://phabricator.wikimedia.org/T97412#4146038 (10Jdforrester-WMF) [14:33:04] 10Phabricator, 10Project-Admins: Replace all "tracking" tasks with tags/projects (if unbounded) or relabel them to outcome tasks (if bounded) - https://phabricator.wikimedia.org/T192655#4146024 (10Jdforrester-WMF) [14:38:40] (03CR) 10Jforrester: [C: 031] mediawiki/core phpcs job and docker container [integration/config] - 10https://gerrit.wikimedia.org/r/427698 (owner: 10Hashar) [15:03:36] "beta-scap-eqiad" seems broken… is that known (couldn't find a ticket) [15:03:41] https://integration.wikimedia.org/ci/job/beta-scap-eqiad/ [15:04:45] tin looks like it's being crap [15:08:23] deployment-tinfoil.ducktape.eqiad … [15:09:15] I think it just requires switching off and back on [15:09:24] For something I forgot the name of [15:09:30] In Jenkins [15:11:11] Yeah, I think there's documentation for this case before [15:11:42] https://phabricator.wikimedia.org/T72597 [15:13:31] hoo: fixed, possibly [15:13:34] https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update-eqiad/ is now running [15:13:38] cool :) [15:14:04] Btw, I'm currently digging through the sites-table related ticket mess [15:14:16] I like how we replaced them with something else in 2015 [15:14:19] except we didn't [15:14:38] xD [15:14:44] temporary hacks, yo [15:15:11] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Jenkins, 10Patch-For-Review, 10Upstream: Jenkins Gearman plugin has deadlock on executor threads (was: Beta Cluster stopped receiving code updates (beta-update-dat... - https://phabricator.wikimedia.org/T72597#4146162 [15:27:31] (03CR) 10Krinkle: Prohibit PHP's vanilla execution (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/423030 (owner: 10MaxSem) [15:53:34] woo hoo, the admin page in polygerrit is now on mobile from gerrit 2.16 / 3.0 :) [16:07:07] 10Phabricator, 10Project-Admins: Replace all "tracking" tasks with tags/projects (if unbounded) or relabel them to outcome tasks (if bounded) - https://phabricator.wikimedia.org/T192655#4146253 (10Aklapper) General +1, but I'd love to kill a good bunch of irrelevant tracking tasks, as I don't like categorizati... [16:12:48] 10Gerrit: New Gerrit UI doesn't work well with browser addons like GhostText - https://phabricator.wikimedia.org/T192665#4146270 (10Anomie) [16:13:00] looks like gerrit now supports elasticsearch [16:13:06] https://twitter.com/esuohesrupd/status/987313966547456001 [16:14:36] 10Project-Admins: Replace tracking bug T2209 by new project tag "HTML" - https://phabricator.wikimedia.org/T102492#1366015 (10Krinkle) I'm not sure even "HTML validity" would be useful to track. Just as much as "PHP" isn't useful, it's too generic and not actionable in a way that others aren't already. Bugs rel... [16:16:02] thcipriani: hi, I'd have to deploy https://gerrit.wikimedia.org/r/#/c/427927/ should I do this now or wait til monday? [16:16:03] 10Gerrit: New Gerrit UI doesn't work well with browser addons like GhostText - https://phabricator.wikimedia.org/T192665#4146288 (10Anomie) [16:18:02] dcausse: I'm guessing the train isn't going to go out today... thcipriani ? [16:18:08] 10Gerrit, 10Upstream: New Gerrit UI doesn't work well with browser addons like GhostText - https://phabricator.wikimedia.org/T192665#4146298 (10Paladox) filled upstream at https://bugs.chromium.org/p/gerrit/issues/detail?id=8811 [16:18:10] dcausse: we can backport this now, wmf.30 is on testwikis only so it's fairly low risk. Also, this was the only blocker for rolling wmf.30 forward so otherwise wmf.30 is fairly well-tested otherwise, so testing in group1 should also be ok if you can verify the patch. [16:19:49] thcipriani: it's causing few errors on group1 wikis, I'm fine either ways (deploy now or wait for monday) [16:21:42] I like thcipriani's idea, low risk and good to get us on solid footing for Monday [16:22:10] ok deploying if no one objects [16:22:44] dcausse: works for me, wmf.30 is only on group0 wikis at the moment, is it possible to verify patch there? [16:23:12] thcipriani: wmf.30 seems to be on group1 as well [16:23:52] https://tools.wmflabs.org/versions/ is wrong at the moment, only advisorswiki (the first wiki in the list) is on wmf.30 [16:23:59] ah ok [16:24:09] all other group1 wikis are on wmf.29 https://en.wikibooks.org/wiki/Special:Version [16:24:30] so looking at the logs yes I can verify that these errors go away [16:24:43] deploying [16:24:46] +1 [16:27:56] 10Gerrit: Gerrit's New UI reply dialog ignores Ctrl-Z - https://phabricator.wikimedia.org/T192666#4146326 (10Anomie) [16:31:30] 10Gerrit, 10Upstream: Gerrit's New UI reply dialog ignores Ctrl-Z - https://phabricator.wikimedia.org/T192666#4146364 (10Paladox) https://bugs.chromium.org/p/gerrit/issues/detail?id=8812 [17:03:32] 10Release-Engineering-Team (Watching / External), 10Operations, 10Parsoid, 10Patch-For-Review: Provide an archive endpoint for older Parsoid debs (on releases.wikimedia.org or elsewhere) - https://phabricator.wikimedia.org/T150672#4146464 (10Dzahn) 05Open>03Resolved 12:43 < subbu> mutante, success http... [17:30:35] !log rebase integration-puppetmaster01:/var/lib/puppet/git conflicting on https://gerrit.wikimedia.org/r/#/c/348236/ [17:30:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:36:54] 10Phabricator, 10Project-Admins: Replace all "tracking" tasks with tags/projects (if unbounded), relabel them to outcome tasks (if bounded), or kill them entirely (if pointless) - https://phabricator.wikimedia.org/T192655#4146726 (10Jdforrester-WMF) [19:16:56] 10Phabricator, 10Project-Admins: Replace all "tracking" tasks with tags/projects (if unbounded), relabel them to outcome tasks (if bounded), or kill them entirely (if pointless) - https://phabricator.wikimedia.org/T192655#4146000 (10greg) There was a bit of work on this before and the #tracking project's workb... [20:25:25] (03CR) 10Thcipriani: "Working on this today, and I realized you'll need to either create the mathoid namespace or override the namespace in the chart. This pass" [integration/config] - 10https://gerrit.wikimedia.org/r/425936 (https://phabricator.wikimedia.org/T188935) (owner: 10Dduvall) [20:32:23] Hm.. beta is *still* sending password reminders via wiki@wikimedia.org. [20:32:34] Or a regression? [20:39:55] 10Beta-Cluster-Infrastructure: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4146974 (10Krinkle) [20:40:12] Hm.. couldn't find an existing task, but I feel like was definitely mentioned/known years ago [20:40:17] But just never reported? [20:40:40] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Operations: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4146985 (10Krinkle) [20:40:50] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Operations: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4146974 (10Krinkle) [21:18:14] 10Release-Engineering-Team (Kanban), 10Scap, 10Operations: mwscript rebuildLocalisationCache.php takes 40 minutes - https://phabricator.wikimedia.org/T191921#4147059 (10Krinkle) @thcipriani Is this with or without translation cache (TC) and JIT? [21:19:00] 10RelEng-Archive-FY201718-Q1, 10Release, 10Train Deployments: 1.30.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T170636#4147061 (10Krinkle) [21:27:44] 10Release-Engineering-Team (Kanban), 10Scap, 10Operations: mwscript rebuildLocalisationCache.php takes 40 minutes - https://phabricator.wikimedia.org/T191921#4147070 (10thcipriani) This uses the value from `/etc/hhvm/php.ini`: `hhvm.jit = false`. Played with `-vEval.Jit=1` yesterday and it was quite a bit s... [21:29:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Patch-For-Review, 10Wikimedia-log-errors (Jenkins Failure): Broken quibble jobs fail all mediawiki commits - https://phabricator.wikimedia.org/T192576#4147075 (10Krinkle) 05Open>03Resolved a:03Krinkle [21:29:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Patch-For-Review, 10Wikimedia-log-errors (Jenkins Failure): Broken quibble jobs fail all mediawiki commits - https://phabricator.wikimedia.org/T192576#4143521 (10Krinkle) a:05Krinkle>03hashar [21:32:10] !log deployment-kafka04: full disk, deleted /var/log/syslog.1 (3G) to make interactive sessions work again [21:32:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:32:14] yes, it was THAT bad [21:34:15] I guess there's no automatic stuff to prevent that from happening [21:34:45] It's called logrotate and works great. [21:35:13] Except when kafka is broken for days, logs every error into syslog and slowly eats up all the disk space. [21:35:35] ah, I heard about logrotate [21:35:59] I guess that doesn't fix the jobqueue failures :) [21:43:02] Short measurement shows that syslog grows by about 15K per minute. That'd be ~0,93G/h. So disk will be full again in about three hours. [21:43:36] Uhmm no, wait. [21:47:15] Output of du already is in K, that means it grows about 15M per minute. [21:48:23] So yeah, it'll be full again in 3h. [22:00:49] 10Beta-Cluster-Infrastructure, 10MW-1.32-release-notes (WMF-deploy-2018-04-24 (1.32.0-wmf.1)), 10Patch-For-Review, 10Puppet: deployment-prep has jobqueue issues - https://phabricator.wikimedia.org/T192473#4147110 (10EddieGP) [22:04:06] is there already logrotate that is puppetized? [22:04:16] then you can change the config to rotate faster [22:04:28] by just changing a number [22:04:48] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Operations: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4146974 (10MarcoAurelio) Maybe we should change it to `wiki@wikimedia.beta.wmflabs.org`. Does that address need to exist? [22:07:59] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Operations: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4147125 (10Dzahn) [22:10:48] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Operations: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4146974 (10Dzahn) >>! In T192686#4147116, @MarcoAurelio wrote: > Maybe we should change it to `wiki@wikim... [22:12:55] mutante: well, I guess we can have the address created for beta then :) [22:13:12] I guess this is modified on CommonSettings-labs.php iirc [22:14:28] yep: $wgPasswordSender = 'wiki@wikimedia.org'; [22:14:39] I guess we should setup one for labs [22:19:13] Hauskatze: agree, CommonSettings-labs sounds about right [22:19:23] ack, a config issue [22:19:47] though not sure about mailservers in beta either [22:19:56] but yea, just change it [22:20:00] the sender that is [22:20:27] created a patch to use $wgPasswordSender = 'wiki@wikimedia.beta.wmflabs.org'; [22:20:44] https://phabricator.wikimedia.org/T192686 [22:20:48] eww [22:20:57] https://gerrit.wikimedia.org/r/428026 [22:25:21] wikimedia.beta.wmflabs.org has address 208.80.155.135 [22:25:28] 135.128-25.155.80.208.in-addr.arpa domain name pointer instance-deployment-cache-text04.deployment-prep.wmflabs.org. [22:25:34] 135.128-25.155.80.208.in-addr.arpa domain name pointer beta.wmflabs.org. [22:25:45] so that would be the expected actual sender [22:26:07] +1 to to the patch though, i think its' good in itself that it's "something not pro" [22:26:10] prod [22:26:57] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Operations, 10Patch-For-Review: Beta Cluster sends password reset mails with prod address - https://phabricator.wikimedia.org/T192686#4147140 (10MarcoAurelio) With regards to my patch above, we should investigate if t... [22:28:30] .arpa? isn't that a military thing? [22:30:23] mutante: so wiki@... is good or would the sender see those numbers and letters as sender instead? [22:30:31] *sender/receiver [22:31:25] Hauskatze: that shows how the Internet started from the military . yes [22:31:34] arpanet [22:31:42] when I had hair in my head [22:31:43] :P [22:31:46] https://en.wikipedia.org/wiki/ARPANET [22:32:28] Hauskatze: the sender can be fake unless specific measures are taken to prevent it [22:32:47] dmarc [22:32:50] that said, dunno about setup of mail servers in "labs" [22:32:55] or which they use [22:32:58] without checking [22:33:13] well, with 70+ instances called deployment-.* I guess no one will bother [22:33:19] maybe worth searching for existing tickets related to email from labs [22:33:27] or historic ones that are closed [22:35:08] Hauskatze: so here is the config in production: [22:35:20] 117 wiki: :blackhole: This mail address sends out automated messages, please do not reply. [22:35:30] it's exim [22:35:46] i am out for now though [22:36:20] but that :blackhole: thing is right for this kind of sender address [22:36:42] so the address should be :blackhole:@wikimedia.beta.wmflabs.org ? [22:37:06] no, the exim config on the server that handles mail for beta.wmflabs.org [22:37:14] should use the "blackhole" feature from http://www.exim.org/exim-html-3.20/doc/html/spec_23.html [22:37:31] so that the wiki@ alias is a blackhole [22:37:36] * Hauskatze is starting to get nervous not understanding :) [22:37:45] anyway, it's late, time to sleep :) [22:37:55] it just means that the mail server is throwing it away properly [22:38:08] when people reply to mail from wiki@ [22:38:10] ah, k [22:38:26] and exim is the specific mail server software [22:38:34] I understnad now [22:38:50] ok, cool. cya later, i need to change locations [22:39:14] me too, from desk to bed ;) [22:55:36] 10Release-Engineering-Team (Kanban), 10Scap, 10Operations: mwscript rebuildLocalisationCache.php takes 40 minutes - https://phabricator.wikimedia.org/T191921#4147196 (10Krinkle) @thcipriani Hm.. these are seconds though, as opposed to minutes. Is there something different about these commands? I'm asking bec... [22:58:06] 10Release-Engineering-Team (Kanban), 10Scap, 10Operations: mwscript rebuildLocalisationCache.php takes 40 minutes - https://phabricator.wikimedia.org/T191921#4147197 (10thcipriani) >>! In T191921#4147196, @Krinkle wrote: > @thcipriani Hm.. these are seconds though, as opposed to minutes. Is there something d... [23:09:17] Krinkle: I think the $wgEmergencyContact is for noc@wikimedia [23:10:01] it'll use the one from CommonSettings.php if none is set in CommonSettings-labs.php if I understand the file hiearchy rightly [23:10:22] what it is sent or who watches that address, that I don't know [23:20:09] I know who watches it, and it's used for a bunch of non-wiki things. [23:20:13] But nothig in core uses it [23:20:26] anyway, I found what it's used for, mostly within extensions, and it seems fine for that to be set to the prod one.