[00:07:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL [03:47:09] 06Release-Engineering-Team, 10ArchCom-RfC, 06Developer-Relations, 06WMF-Legal, and 6 others: Create formal process for CREDITS files - https://phabricator.wikimedia.org/T139300#2826209 (10bd808) 05Open>03Resolved [04:46:04] PROBLEM - Puppet run on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [05:21:05] RECOVERY - Puppet run on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [06:48:52] Project selenium-Wikibase » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #191: 04FAILURE in 2 hr 8 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/191/ [08:42:54] 10Continuous-Integration-Config, 10MediaWiki-Unit-tests, 05MW-1.29-release-notes, 13Patch-For-Review, and 2 others: MediaWikiTestCaseTest (and possibly other tests) are not being run by Jenkins - https://phabricator.wikimedia.org/T151081#2826406 (10hashar) Thanks Adam. Looks like it is covered now :-} [09:15:10] !log cherry-pick of https://gerrit.wikimedia.org/r/#/c/323517 to deployment-puppetmaster02 to test [09:15:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:35:05] (03PS1) 10Hashar: dib: apache::mod::php5 solely for Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/323803 [09:39:06] (03CR) 10Hashar: [C: 032] dib: apache::mod::php5 solely for Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/323803 (owner: 10Hashar) [09:39:30] !log Regenerated Nodepool image for Trusty. It no more includes apache::mod::php5 which broke the build and is not needed on Trusty ( https://gerrit.wikimedia.org/r/323803 ) [09:39:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:39:56] (03Merged) 10jenkins-bot: dib: apache::mod::php5 solely for Jessie [integration/config] - 10https://gerrit.wikimedia.org/r/323803 (owner: 10Hashar) [09:48:03] !log Image ci-trusty-wikimedia-1480326016 in wmflabs-eqiad is ready [09:48:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:48:08] Yippee, build fixed! [13:48:08] Project selenium-VisualEditor » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #227: 09FIXED in 4 min 7 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/227/ [13:52:27] 06Release-Engineering-Team, 06Developer-Relations (Oct-Dec-2016), 07Documentation: Merge Wikimedia's "Deployment checklist for new extensions" doc pages - https://phabricator.wikimedia.org/T142081#2827095 (10Aklapper) 05Open>03Resolved Well, one week is over, and the "merging" (what this task is about) i... [14:06:42] 10scap, 10Icinga, 06Operations: expose hosts in maintenance state so we can prevent scap from running on them - https://phabricator.wikimedia.org/T100777#2827126 (10hashar) +#scap there might be another task already [14:17:46] (03PS1) 10Hashar: test: mark test_zuul_project_in_gerrit as a qa test [integration/config] - 10https://gerrit.wikimedia.org/r/323831 [14:37:38] (03CR) 10Hashar: [C: 032] test: mark test_zuul_project_in_gerrit as a qa test [integration/config] - 10https://gerrit.wikimedia.org/r/323831 (owner: 10Hashar) [14:38:27] (03Merged) 10jenkins-bot: test: mark test_zuul_project_in_gerrit as a qa test [integration/config] - 10https://gerrit.wikimedia.org/r/323831 (owner: 10Hashar) [14:47:42] (03PS1) 10Hashar: integration-config-qa archive tox logs [integration/config] - 10https://gerrit.wikimedia.org/r/323836 [15:37:33] hashar hi, i found a way to support skins and ext deps [15:37:34] https://gerrit.wikimedia.org/r/#/c/323540/ [15:37:42] Tests pass. [15:38:13] Also i found a bug, Microsoft Edge does not display Cannot Merge on the change but chrome does, which is strange [15:38:16] (gerrit) [15:40:19] I have no idea how to report this to microsoft as the patches will likly be merged before they get to it. So they wont see what i see [15:40:19] (03PS1) 10Hashar: test_zuul_coverage replace extdistribution with gerrit [integration/config] - 10https://gerrit.wikimedia.org/r/323840 [15:41:00] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827340 (10zeljkofilipin) [15:43:08] (03CR) 10Hashar: "Kunal, that is a follow up on a test suite you have added. Made it to use Gerrit as a reference instead of MediaWiki extensions distributo" [integration/config] - 10https://gerrit.wikimedia.org/r/323840 (owner: 10Hashar) [15:43:09] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827341 (10zeljkofilipin) 05Open>03Resolved a:03zeljkofilipin Looks like all tests failed because of a timeout. All tests are passing now, so I am resolving the task. Pl... [15:44:26] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:05:17] (03PS1) 10Hashar: (WIP) test: simplify zuul_project_in_gerrit (WIP) [integration/config] - 10https://gerrit.wikimedia.org/r/323845 [16:23:53] 10Beta-Cluster-Infrastructure, 10Gerrit, 06Operations, 13Patch-For-Review: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#2827415 (10ArielGlenn) A git pull on tin for ​operations/dumps.git worked just fine. Maybe knock that off the list? [16:29:54] 10Gerrit, 06Operations, 07Beta-Cluster-reproducible, 13Patch-For-Review: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#2827430 (10Krenair) [16:30:19] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Flake8 for python files without extension in puppet repo - https://phabricator.wikimedia.org/T144169#2827435 (10hashar) `gfind . -type f ! -path '*/.git/*' -a ! -name '*.*' -exec file --mime {} +|grep x-python|cut -d\ -f1` sho... [16:31:11] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2827438 (10hashar) 170 files based on `gfind . -type f ! -path '*/.git/*' -a ! -name '*.*' -exec file --mime {} +|grep x-shell|cut -d\ -f1` ``` ./fil... [16:36:12] 10Deployment-Systems, 03Scap3, 13Patch-For-Review: Update Debian Package for Scap3 - https://phabricator.wikimedia.org/T127762#2827450 (10thcipriani) 05stalled>03Open @fgiunchedi could you update the scap packages to v.3.4.0-1? Changelog: {P4523} -- Thanks! [16:37:24] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827464 (10Tobi_WMDE_SW) Yeah, seems to work again. While the tests failed because of a timeout, the timeout was caused because something was wrong with mediawiki setup on the... [16:39:47] PROBLEM - Puppet run on deployment-restbase01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [16:46:09] 10Continuous-Integration-Config, 10Analytics-Cluster: Add CI job Oozie XML stylesheet validation for the analytics/refinery repository - https://phabricator.wikimedia.org/T147072#2827492 (10MarcoAurelio) 05declined>03Open p:05Normal>03Lowest [16:48:16] 10Continuous-Integration-Config, 10Analytics-Cluster: Add CI job Oozie XML stylesheet validation for the analytics/refinery repository - https://phabricator.wikimedia.org/T147072#2827504 (10MarcoAurelio) I've reopened because I plan to investigate how to add this job in ligh of recent commits I've submitted to... [16:51:40] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827541 (10zeljkofilipin) Looks like there was a problem with the skin. Maybe @hashar will know. [16:57:32] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827559 (10hashar) That one is quite obvious, the Vector skin is not cloned. I guess we should always inject it in the `mwext-mw-selenium` job. [17:01:00] 10Browser-Tests-Infrastructure, 07Browser-Tests: Browser tests fail on different extensions - https://phabricator.wikimedia.org/T151710#2827569 (10hashar) Sorry I have spoke to fast. mediawiki/skins/Vector is actually cloned it but that is done AFTER install.php and thus the installer does not automatically in... [17:07:38] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Flake8 for python files without extension in puppet repo - https://phabricator.wikimedia.org/T144169#2827575 (10Volans) @hashar thanks for the update, this is still on my radar, but I didn't had time to work on/drive it yet [17:14:15] (03PS1) 10Hashar: test: support state of Gerrit repositories [integration/config] - 10https://gerrit.wikimedia.org/r/323856 [17:19:46] RECOVERY - Puppet run on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0] [18:16:38] 10scap, 06Operations: Decide on /var/lib vs /home as locations of homedir for mwdeploy - https://phabricator.wikimedia.org/T86971#980532 (10demon) I see zero reason we can't move the homedir to /var/lib. [18:31:23] So… anyone have an idea about running "npm install" as part of the jsduck test for VE so we can move some dependencies to that? [18:33:16] 10Continuous-Integration-Infrastructure: Spike: Evaluate experimental Docker based CI w/ scap builds - https://phabricator.wikimedia.org/T150501#2827947 (10dduvall) [18:33:17] 10Continuous-Integration-Infrastructure: Install and configure Jenkins Docker plugin - https://phabricator.wikimedia.org/T150505#2827943 (10dduvall) 05Open>03declined The plugin ended up providing very little value over a small shell script, and had some intractable issues related to shared volume and effect... [18:40:54] 06Release-Engineering-Team, 10MediaWiki-General-or-Unknown, 05MW-1.23-release: 1.23.14 is really 1.23.13 - https://phabricator.wikimedia.org/T136484#2827981 (10demon) 05Open>03declined 1.23.13 / 1.23.14 aren't the latest releases for 1.23.x anymore (that's 1.23.15). I'm closing declined now since it's no... [18:43:07] 03Scap3 (Scap3-Adoption-Phase1), 10releng-201516-q4: [keyresult] Migrate remaining trebuchet deployed services - https://phabricator.wikimedia.org/T129290#2828006 (10thcipriani) [18:43:09] 10Deployment-Systems, 03Scap3 (Scap3-Adoption-Phase1), 10scap, 10Sentry: Deploy sentry with scap3 - https://phabricator.wikimedia.org/T129155#2828005 (10thcipriani) 05Open>03Invalid [18:51:09] (Krinkle might have an idea? Not urgent though.) [18:51:39] James_F: What kind of dependencies? [18:51:55] Only the .json config for jsduck and the .js files need to exist for jsduck test to pass [18:51:56] Krinkle: OOjs and OOjs UI. [18:52:23] 03Scap3 (Scap3-Adoption-Phase1), 10Wikimedia-Wikimania-Scholarships, 15User-bd808: Deploy scholarships with scap3 - https://phabricator.wikimedia.org/T129134#2828112 (10bd808) a:03bd808 [18:52:29] Krinkle: And jsduck needs to "see" all of those (or have a huge amount of content in external.js). [18:52:37] James_F: VE standalone? [18:52:43] Yes. [18:53:43] James_F: That means the non-dist demo won't work anymore without npm-install first and such things [18:53:53] Krinkle: Yes. [18:54:15] And we dont' run npm-install for submodules, so we'd have to make assumptions not do npm-depend on things that mediawiki doesn't provide [18:54:19] Anyway, yeah, it can be done. [18:54:22] Krinkle: Same as OOUI's demo folder. [18:54:25] We already do it for doc publish [18:54:29] and coverage publish [18:54:33] because of grunt:copy [18:54:39] for lib/ and src/ [18:54:43] into demo/ [18:54:52] should be fine [19:00:26] (03PS1) 10Jforrester: [VisualEditor] Run a custom jsduck script that does npm first [integration/config] - 10https://gerrit.wikimedia.org/r/323872 [19:04:28] 03Scap3 (Scap3-Adoption-Phase1), 10Wikimedia-Wikimania-Scholarships, 15User-bd808: Deploy scholarships with scap3 - https://phabricator.wikimedia.org/T129134#2828184 (10bd808) a:05bd808>03Niharika @Niharika said she would like to take a shot at this to learn a bit about #scap3 and prod deployments. Hopef... [19:05:25] thcipriani: ^ if you know of an existing conversion from trebuchet to scap3 for a simple repo like scholarships that would be helpful. [19:05:56] I think the only goofy thing that we do there that will need some help is a puppet provided config file that get placed in the repo [19:06:39] I think that can be fixed with a dangling simlink or maybe better yet a small change in the app itself to look for the file somewhere in /etc [19:06:49] yup, I can take look. I can't think of similar repos off hand. [19:07:10] ieg, but it's not converted yet either ;) [19:07:22] yeah, looking in /etc would be ideal. I know we're enforcing some things that trebuchet did not in regards to puppetized repos. [19:07:28] I know there was a patch for that one at some point [19:07:32] ...ieg [19:07:46] T129154 [19:08:09] its the other php prod app I wrote [19:08:22] yeap [19:08:34] I meant that there was a patch for ieg -> scap3 at some point :) [19:08:54] but it may be deceiving... [19:13:02] (03CR) 10Jforrester: "I'm not quite sure I know what I'm doing here." [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [19:20:33] (03PS1) 10Chad: make-release: remove offline mode, it's weird [tools/release] - 10https://gerrit.wikimedia.org/r/323877 [19:20:48] (03CR) 10jenkins-bot: [V: 04-1] [VisualEditor] Run a custom jsduck script that does npm first [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [19:38:20] (03CR) 10Chad: [C: 032] make-release: remove offline mode, it's weird [tools/release] - 10https://gerrit.wikimedia.org/r/323877 (owner: 10Chad) [19:45:50] (03CR) 10Jforrester: "Needed for I246e529c, I23f1e093, and I79f25504." [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [19:47:31] Yipee i managed to get phabricator installed on labs [19:47:35] with only these errors [19:47:54] https://phabricator.wikimedia.org/P4526 [19:47:59] twentyafterfour ^^ [20:04:11] paladox: We talked about that failure the other day--happens because of a dependency mismatch. [20:04:17] I think twentyafterfour wrote a patch for it [20:10:58] 06Release-Engineering-Team, 03releng-201617-q2, 05MW-1.28-release: Release MediaWiki 1.28 - https://phabricator.wikimedia.org/T148087#2828472 (10demon) 05Open>03Resolved [20:25:44] (03CR) 10Hashar: "Ah neat. Will take over from there! Almost good :]" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [20:33:46] (03PS1) 10Hashar: Lower verbosity of bundle install for some jobs [integration/config] - 10https://gerrit.wikimedia.org/r/323898 [20:36:15] (03CR) 10Hashar: [C: 032] Lower verbosity of bundle install for some jobs [integration/config] - 10https://gerrit.wikimedia.org/r/323898 (owner: 10Hashar) [20:37:40] (03Merged) 10jenkins-bot: Lower verbosity of bundle install for some jobs [integration/config] - 10https://gerrit.wikimedia.org/r/323898 (owner: 10Hashar) [20:42:38] (03PS1) 10Hashar: Reuse bundle-nodepool macro [integration/config] - 10https://gerrit.wikimedia.org/r/323900 [20:42:49] (03CR) 10Hashar: [C: 032] Reuse bundle-nodepool macro [integration/config] - 10https://gerrit.wikimedia.org/r/323900 (owner: 10Hashar) [20:44:03] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicized but not fixed - https://phabricator.wikimedia.org/T151805#2828622 (10Kghbln) [20:45:34] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828641 (10Kghbln) [20:46:31] (03PS2) 10Hashar: [VisualEditor] Run a custom jsduck script that does npm first [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [20:46:37] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828622 (10Reedy) For number 6... IT's already on all branches as of Aug 23 https://gerrit.wikimedia.org/r/#/q/Ic929a385fa81c27cbc6ac3... [20:47:39] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828663 (10Reedy) Same for number 4 https://gerrit.wikimedia.org/r/#/q/Ib767de853a37099305db20529378fa756ee1bdfe [20:47:43] (03Merged) 10jenkins-bot: Reuse bundle-nodepool macro [integration/config] - 10https://gerrit.wikimedia.org/r/323900 (owner: 10Hashar) [20:48:14] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828666 (10Reedy) And number 3 https://gerrit.wikimedia.org/r/#/q/If024398d7bd4b578ad7f8c74367787f5b19eb9d7 [20:48:25] (03CR) 10Hashar: "Rebased and slightly adjusted. Can you confirm we just need to 'npm install' and that 'npm test' is not needed? :)" (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [20:48:36] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828667 (10Reedy) And number 2 https://gerrit.wikimedia.org/r/#/q/If024398d7bd4b578ad7f8c74367787f5b19eb9d7 [20:48:38] James_F: VisualEditor / Jsduck I have polished your attempt :] [20:48:52] James_F: not sure whether we should run 'npm test' before jsduck [20:49:08] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828668 (10Reedy) And number 1 https://gerrit.wikimedia.org/r/#/q/I4e112a9e918df9fe78b62c311939239b483a21f5 Proposing to close as... [20:51:18] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828706 (10Reedy) 05Open>03Invalid See also https://lists.wikimedia.org/pipermail/mediawiki-announce/2016-August/000195.html [20:51:28] Reedy: :) [20:51:34] Heh. [20:51:42] that was fun to watch :) [20:51:47] I was WTF-ing for a moment [20:52:04] * greg-g goes to take a late lunch [20:55:19] hashar: `npm it` would be good. [20:55:30] James_F: neeat :] [20:55:37] (Thank you.) [20:55:38] James_F: your patch was almost perfect! [20:55:51] Gosh. [20:56:54] just had a slight and understandable confusion between builders and jobs [20:57:37] refreshed the job [20:57:55] https://integration.wikimedia.org/ci/job/visualeditor-jsduck-jessie/961/console :D [21:00:31] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828787 (10Kghbln) @Reedy A I get it, so the fixes ware already released. On cannot really tell from the tagging if it is the case or n... [21:01:51] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828804 (10Reedy) >>! In T151805#2828787, @Kghbln wrote: > @Reedy A I get it, so the fixes ware already released. On cannot really tell... [21:01:55] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828805 (10Kghbln) @Reedy Yeah, indeed invalid. Cool! Thanks a lot! [21:02:18] (03CR) 10Jforrester: [C: 031] [VisualEditor] Run a custom jsduck script that does npm first (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [21:03:53] (03CR) 10Hashar: [C: 032] "Did a recheck on https://gerrit.wikimedia.org/r/#/c/323884/" [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [21:04:21] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828840 (10Kghbln) I meant the issue tagging here on phab. [21:04:54] you can now set the mysql user/pass for phabricator in hiera [21:05:07] (03Merged) 10jenkins-bot: [VisualEditor] Run a custom jsduck script that does npm first [integration/config] - 10https://gerrit.wikimedia.org/r/323872 (owner: 10Jforrester) [21:05:37] so one more step for labs testing unlocked [21:05:47] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828849 (10Kghbln) Ah, there is a task assigned indicating the point release. [21:06:56] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828850 (10Kghbln) It has been a long day so senior moments involved, too. [21:07:59] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828853 (10Kghbln) You may have a jolly good laugh now. [21:11:06] James_F: solved! congratulations [21:14:10] hashar: Thank you. [21:38:12] 06Release-Engineering-Team, 05Release, 05Security, 07Security-Core: Security issues publicly announced, fixed but not released? - https://phabricator.wikimedia.org/T151805#2828622 (10hashar) I rather have one ask questions and get an answer than someone not bothering to ask when we could well have missed r... [22:30:01] (03PS2) 10Chad: make-release: remove offline mode, it's weird [tools/release] - 10https://gerrit.wikimedia.org/r/323877 [22:30:03] (03CR) 10Chad: [V: 032] make-release: remove offline mode, it's weird [tools/release] - 10https://gerrit.wikimedia.org/r/323877 (owner: 10Chad) [23:18:20] Why isn't syncing a file syncing it to /srv/mediawiki on tin? [23:18:50] PHP Notice: Undefined variable: wmgElevateDefaultPasswordPolicy in /srv/mediawiki/wmf-config/CommonSettings.php on line 366 [23:19:13] it's right on staging.. [23:20:36] sync-common fixed it