[00:17:30] <wikibugs_>	 10Continuous-Integration-Config, 10MediaWiki-Core-Tests: PHPUnit entry point 'PHPUnit_TextUI_Command' not found during mediawiki-phpunit-coverage-patch job - https://phabricator.wikimedia.org/T196323#4252507 (10Legoktm)
[02:43:46] <wmf-insecte>	 Project beta-scap-eqiad build #210399: 04FAILURE in 0.51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/210399/
[02:50:05] <greg-g>	 ^ looks like it failed to acquire the lock file, but the lock file isn't there anymore, it'll probably pass next run
[02:59:05] <wmf-insecte>	 Yippee, build fixed!
[02:59:05] <wmf-insecte>	 Project beta-scap-eqiad build #210400: 09FIXED in 5 min 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/210400/
[06:03:34] <shinken-wm>	 PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<50.00%)
[07:07:26] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10ContentTranslation, 10ContentTranslation-Deployments, and 2 others: Add Backport Repository to Jenkins Package Build - https://phabricator.wikimedia.org/T196037#4252718 (10Reedy)
[07:08:35] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK
[07:17:52] <wikibugs_>	 10Continuous-Integration-Infrastructure, 10Operations, 10SRE-Access-Requests, 10Patch-For-Review: Add Reedy to contint-docker group - https://phabricator.wikimedia.org/T196192#4252742 (10MoritzMuehlenhoff) p:05Triage>03Normal Patch looks fine, but this request needs to pass the three day waiting/review...
[07:25:59] <wikibugs_>	 (03CR) 10Reedy: [C: 032] Automatically fix `@gropu` to `@group` [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/437151 (owner: 10Legoktm)
[07:27:10] <wikibugs_>	 (03Merged) 10jenkins-bot: Automatically fix `@gropu` to `@group` [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/437151 (owner: 10Legoktm)
[07:27:45] <wikibugs_>	 (03CR) 10jenkins-bot: Automatically fix `@gropu` to `@group` [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/437151 (owner: 10Legoktm)
[08:28:01] <wikibugs_>	 10Continuous-Integration-Config, 10ContentTranslation, 10ContentTranslation-Deployments, 10Jenkins, 10Language-2018-Apr-June: Add Backport Repository to Jenkins Package Build - https://phabricator.wikimedia.org/T196037#4252888 (10hashar)
[09:37:03] <wikibugs_>	 (03PS1) 10Hashar: Readd git plugin to VE screenshot job [integration/config] - 10https://gerrit.wikimedia.org/r/437205 (https://phabricator.wikimedia.org/T193883)
[09:37:16] <hashar>	 zeljkof:  https://gerrit.wikimedia.org/r/437205 Readd git plugin to VE screenshot job       :]]
[10:21:13] <wikibugs_>	 (03PS1) 10Hashar: Migrate GoogleGeocode to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/437211 (https://phabricator.wikimedia.org/T183512)
[10:23:14] <hashar>	 zeljkof: running webdriver.io on all extensions is definitely a huge plus. It catches issues that phpunit does not.  Example: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php70-docker/5230/artifact/log/should-be-creatable.png  :]
[10:23:25] <wikibugs_>	 (03CR) 10Hashar: [C: 032] Migrate GoogleGeocode to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/437211 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar)
[10:25:37] <wikibugs_>	 (03Merged) 10jenkins-bot: Migrate GoogleGeocode to Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/437211 (https://phabricator.wikimedia.org/T183512) (owner: 10Hashar)
[10:34:53] <wikibugs_>	 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4253213 (10hashar)
[11:40:54] <zeljkof>	 hashar: yeah! :D
[12:57:30] <wikibugs_>	 (03PS1) 10Hashar: [Graph] Add non voting QUnit job [integration/config] - 10https://gerrit.wikimedia.org/r/437233 (https://phabricator.wikimedia.org/T151262)
[12:58:46] <wikibugs_>	 (03CR) 10Zfilipin: [C: 032] Readd git plugin to VE screenshot job [integration/config] - 10https://gerrit.wikimedia.org/r/437205 (https://phabricator.wikimedia.org/T193883) (owner: 10Hashar)
[12:59:18] <wikibugs_>	 (03CR) 10Hashar: [C: 032] [Graph] Add non voting QUnit job [integration/config] - 10https://gerrit.wikimedia.org/r/437233 (https://phabricator.wikimedia.org/T151262) (owner: 10Hashar)
[13:00:24] <wikibugs_>	 (03Merged) 10jenkins-bot: Readd git plugin to VE screenshot job [integration/config] - 10https://gerrit.wikimedia.org/r/437205 (https://phabricator.wikimedia.org/T193883) (owner: 10Hashar)
[13:01:00] <wikibugs_>	 (03Merged) 10jenkins-bot: [Graph] Add non voting QUnit job [integration/config] - 10https://gerrit.wikimedia.org/r/437233 (https://phabricator.wikimedia.org/T151262) (owner: 10Hashar)
[13:03:45] <wikibugs_>	 (03PS1) 10Hashar: Use PHP7 for mext-qunit-non-voting-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/437237
[13:19:28] <wikibugs_>	 (03PS1) 10Hashar: Rename QUnit non voting job [integration/config] - 10https://gerrit.wikimedia.org/r/437240
[13:19:42] <wikibugs_>	 (03CR) 10Hashar: [C: 032] Rename QUnit non voting job [integration/config] - 10https://gerrit.wikimedia.org/r/437240 (owner: 10Hashar)
[13:21:22] <wikibugs_>	 (03Merged) 10jenkins-bot: Rename QUnit non voting job [integration/config] - 10https://gerrit.wikimedia.org/r/437240 (owner: 10Hashar)
[13:57:37] <shinken-wm>	 PROBLEM - Puppet errors on deployment-jobrunner03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[14:47:03] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10MW-1.31-release-notes (WMF-deploy-2018-04-03 (1.31.0-wmf.28)), 10Patch-For-Review, 10User-zeljkofilipin: Video recording for Selenium tests in Node.js - https://phabricator.wikimedia.org/T179188#4253965 (10zeljkofilipin) a:05zeljkofilipin>03None Not working on it...
[14:48:44] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10VisualEditor, 10Patch-For-Review, 10User-zeljkofilipin: `FATAL: java.io.IOException: java.nio.file.NoSuchFileException: ...build/screenshots.yml` for language-screenshots-VisualEditor - https://phabricator.wikimedia.org/T193883#4182477 (10zeljkofilipin) a:03zeljkofil...
[14:49:09] <Krinkle>	 greg-g: Hm.. did something change between now and Friday regarding Phabricator not allowing .mov uploads?
[14:49:53] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10CirrusSearch, 10Discovery, 10Discovery-Search, and 2 others: selenium-CirrusSearch-jessie does not run any tests - https://phabricator.wikimedia.org/T193244#4163850 (10zeljkofilipin) a:05zeljkofilipin>03None
[14:54:59] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10VisualEditor, 10Patch-For-Review, 10User-Ryasmeen, 10User-zeljkofilipin: `FATAL: java.io.IOException: java.nio.file.NoSuchFileException: ...build/screenshots.yml` for language-screenshots-VisualEditor - https://phabricator.wikimedia.org/T193883#4253988 (10zeljkofilip...
[14:56:51] <greg-g>	 Krinkle not that I know of but I'll ask Mukunda. He wasn't planning to dry a deployment last week (he was on vacation).
[14:57:21] <Krinkle>	 It's currently rejecting .mov uploads, with an error asking the site administrator to consult the manual on how to configure file uploads :)
[15:02:10] <paladox>	 Krinkle how big is the file?
[15:02:23] <Krinkle>	 paladox: File size is not the issue.
[15:02:38] <Reedy>	 Or if it is, it's crappy error handling :P
[15:02:49] <paladox>	 Krinkle https://github.com/wikimedia/puppet/blob/production/modules/phabricator/data/fixed_settings.yaml#L40
[15:03:13] <Krinkle>	 It's not the issue because the same file uploaded yesterday cannot be uploaded again, and I am able to upload a PNG that is larger than this video.
[15:03:29] <paladox>	 ok
[15:18:51] <wikibugs_>	 (03PS1) 10Ema: Enable debian-glue for libvmod-re2 [integration/config] - 10https://gerrit.wikimedia.org/r/437270 (https://phabricator.wikimedia.org/T180329)
[15:21:29] <wmf-insecte>	 Project beta-update-databases-eqiad build #25948: 04FAILURE in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25948/
[15:27:45] <shinken-wm>	 PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[15:33:42] <shinken-wm>	 PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[16:21:37] <wmf-insecte>	 Project beta-update-databases-eqiad build #25949: 04STILL FAILING in 1 min 36 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25949/
[16:33:21] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10VisualEditor, 10User-zeljkofilipin: language-screenshots-VisualEditor fails with `stderr: warning: failed to remove cache` - https://phabricator.wikimedia.org/T196374#4254436 (10zeljkofilipin) a:03zeljkofilipin
[16:52:53] <wikibugs_>	 10Phabricator: Decide how to distribute phabricator badges - https://phabricator.wikimedia.org/T196390#4254467 (10Framawiki) p:05Triage>03Low
[16:55:05] <wikibugs_>	 10Phabricator: Decide how to distribute phabricator badges - https://phabricator.wikimedia.org/T196390#4254467 (10zeljkofilipin) The first step could be a draft of the proposed rules in task description. So far, I have awarded a few badges without any process to a few people, including to myself. 😅
[17:11:04] <wikibugs_>	 10Release-Engineering-Team, 10Scap, 10JADE, 10Wikimedia-General-or-Unknown: JADE submodule on tin is dirty - https://phabricator.wikimedia.org/T195343#4254520 (10hashar) scap does lint json. That file is a test file which causes an invalid syntax error on purpose.  See also T188914
[17:21:40] <wmf-insecte>	 Project beta-update-databases-eqiad build #25950: 04STILL FAILING in 1 min 38 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25950/
[17:22:58] <wikibugs_>	 10Scap, 10Navigation-Popups: scap fails on .eslintrc.json: invalid JSON - https://phabricator.wikimedia.org/T188914#4023698 (10Legoktm) If it doesn't adhere to the strict JSON spec, it shouldn't have a .json extension :(
[17:24:39] <wikibugs_>	 10Scap, 10Page-Previews: scap fails on .eslintrc.json: invalid JSON - https://phabricator.wikimedia.org/T188914#4254588 (10Legoktm)
[17:25:28] <wikibugs_>	 10Scap, 10Page-Previews: scap fails on .eslintrc.json: invalid JSON - https://phabricator.wikimedia.org/T188914#4023698 (10Legoktm) 05Open>03Resolved a:03mmodell Fixed in {2d6bd5f4b78935ccbd23b0bdebb163ae21488367}.
[17:29:52] <wikibugs_>	 10Release-Engineering-Team, 10JADE, 10Wikimedia-General-or-Unknown, 10Patch-For-Review: JADE submodule on tin is dirty - https://phabricator.wikimedia.org/T195343#4254609 (10Legoktm) I think scap's behavior is correct. We should never deploy invalid JSON files. In this case this isn't supposed to be used a...
[17:59:06] <wikibugs_>	 10Phabricator: Decide how to distribute phabricator badges - https://phabricator.wikimedia.org/T196390#4254665 (10Aklapper) > Couldn't we imagine that they would be distributed automatically, like StackOverflow? Is there an upstream feature request about this? Either https://secure.phabricator.com/maniphest/quer...
[18:02:27] <Yaron>	 Can anyone explain why this patch was rejected? https://gerrit.wikimedia.org/r/#/c/437249/
[18:02:39] <Yaron>	 I don't understand the error message ("Invalid modifier on function getMainHTML. in ./includes/forminputs/PF_DateInput.php on line -1").
[18:04:00] <Reedy>	 -1 makes it sound rather broken
[18:04:24] <Yaron>	 Yes, it doesn't appear to be 100% on the level.
[18:05:52] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Set up webperf node in Beta Cluster - https://phabricator.wikimedia.org/T195314#4254691 (10Krinkle)
[18:06:13] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review: Set up webperf node in Beta Cluster - https://phabricator.wikimedia.org/T195314#4222943 (10Krinkle)
[18:06:33] <wikibugs_>	 10Phabricator: Decide how to distribute phabricator badges - https://phabricator.wikimedia.org/T196390#4254467 (10Ebe123) There does not seem to be.
[18:11:03] <wikibugs_>	 10Release-Engineering-Team, 10JADE, 10Wikimedia-General-or-Unknown, 10Patch-For-Review: JADE submodule on tin is dirty - https://phabricator.wikimedia.org/T195343#4254698 (10mmodell) Agreed the file shouldn't be named .json
[18:11:41] <shinken-wm>	 PROBLEM - Puppet errors on deployment-webperf11 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[18:12:43] <shinken-wm>	 PROBLEM - Puppet errors on deployment-deploy-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[18:14:08] <legoktm>	 Yaron: static function parseDate( $date ) {{
[18:14:11] <legoktm>	 there are two {{
[18:17:26] <Yaron>	 @legoktm - ah, thanks!
[18:21:42] <wmf-insecte>	 Project beta-update-databases-eqiad build #25951: 04STILL FAILING in 1 min 41 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25951/
[18:39:22] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[18:48:01] <wikibugs_>	 10Phabricator, 10WMF-Legal: Support Google and Github accounts for authenticating in Phab to ease volunteer contribution sign up - https://phabricator.wikimedia.org/T169007#4254834 (10Krenair) 05Open>03declined >>! In T169007#3422258, @Fjalapeno wrote: > I would hope we have some trust for Github and Googl...
[19:18:19] <wikibugs_>	 (03CR) 10Hashar: [C: 032] Enable debian-glue for libvmod-re2 [integration/config] - 10https://gerrit.wikimedia.org/r/437270 (https://phabricator.wikimedia.org/T180329) (owner: 10Ema)
[19:20:29] <wikibugs_>	 (03Merged) 10jenkins-bot: Enable debian-glue for libvmod-re2 [integration/config] - 10https://gerrit.wikimedia.org/r/437270 (https://phabricator.wikimedia.org/T180329) (owner: 10Ema)
[19:21:32] <wmf-insecte>	 Project beta-update-databases-eqiad build #25952: 04STILL FAILING in 1 min 31 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25952/
[19:25:03] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4254932 (10greg)
[19:27:43] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4254978 (10Anomie) That should have been cleaned up by update.php running maintenance/deduplicateArchiveRevId.php, which should have happened shortly after {1c7bf1baf} was merged on Ma...
[19:29:19] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4254989 (10Anomie) >>! In T196401#4254978, @Anomie wrote: > Any idea how that didn't happen?  Or more correctly, how that somehow managed to "run" without cleaning up the duplicates?
[19:32:34] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4254998 (10greg) No idea, and our Jenkins logs don't go back that far :(
[19:36:35] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4254932 (10Reedy) ``` reedy@deployment-tin:~$ mwscript maintenance/deduplicateArchiveRevId.php --wiki=enwikivoyage ...Update 'DeduplicateArchiveRevId' already logged as completed. reed...
[19:37:02] <Reedy>	 !log running foreachwiki maintenance/deduplicateArchiveRevId.php --force T196401
[19:37:04] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[19:37:04] <stashbot>	 T196401: beta-update-databases failing - https://phabricator.wikimedia.org/T196401
[19:37:32] <greg-g>	 Reedy: weird
[19:38:08] <Reedy>	 Yup
[19:38:15] <Reedy>	 It's not even like the script was committed broken, thenf ixed
[19:38:17] <greg-g>	 thanks though!
[19:39:11] <Reedy>	 most of the rest look like they don't need changes
[19:41:09] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10MediaWiki-Database: beta-update-databases failing - https://phabricator.wikimedia.org/T196401#4255010 (10Reedy) 05Open>03Resolved a:03Reedy ```lines=25 reedy@deployment-tin:~$ foreachwiki maintenance/deduplicateArchiveRevId.php --force ------------------------------------...
[19:41:49] <shinken-wm>	 PROBLEM - SSH on integration-slave-docker-1015 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[19:46:01] <wmf-insecte>	 Yippee, build fixed!
[19:46:02] <wmf-insecte>	 Project beta-update-databases-eqiad build #25953: 09FIXED in 3 min 37 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/25953/
[19:47:20] <greg-g>	 weee
[19:47:50] <wikibugs_>	 10Beta-Cluster-Infrastructure: Puppet failure at deployment-snapshot01 - https://phabricator.wikimedia.org/T196198#4255034 (10Krenair) 05Open>03Resolved a:03Krenair I fixed this by adding some stuff to hiera during some much more major puppet work around that host
[20:03:14] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar), 10Puppet: Define scap::sources in a way that is shared between prod and beta - https://phabricator.wikimedia.org/T196034#4255104 (10Imarlier)
[20:11:41] <shinken-wm>	 RECOVERY - SSH on integration-slave-docker-1015 is OK: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u4 (protocol 2.0)
[20:37:32] <mdholloway>	 !log Deployed to BC: [mobileapps/deploy@276ea43]: Update mobileapps to f579f0d
[20:37:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[20:55:04] <wikibugs_>	 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, 10Patch-For-Review: [EPIC] Migrate Mediawiki jobs from Nodepool to Docker - https://phabricator.wikimedia.org/T183512#4255218 (10hashar)
[21:03:50] <wikibugs_>	 10Release-Engineering-Team (Next), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: scap support for git-lfs - https://phabricator.wikimedia.org/T181855#4255246 (10awight)
[21:07:02] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10Scap, 10Scoring-platform-team, 10Patch-For-Review: Support git-lfs - https://phabricator.wikimedia.org/T180627#4255268 (10awight)
[21:07:05] <wikibugs_>	 10Release-Engineering-Team (Next), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: scap support for git-lfs - https://phabricator.wikimedia.org/T181855#4255270 (10awight)
[21:24:02] <wikibugs_>	 10Scap: Track scap syncs that are part of a given SWAT window - https://phabricator.wikimedia.org/T193311#4255322 (10thcipriani) p:05Triage>03Normal We looked into this at the RelEng offsite a bit. There are a few options here:  1. We could require deployers to start, e.g., `scap swat start` before they are...
[21:25:11] <wikibugs_>	 10Scap: Document scap swat command - https://phabricator.wikimedia.org/T196411#4255329 (10thcipriani) p:05Triage>03Normal
[21:28:58] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10Scap, 10Patch-For-Review, 10Scoring-platform-team (Current): Support git-lfs in scap - https://phabricator.wikimedia.org/T180627#4255367 (10awight) p:05Normal>03High
[21:29:54] <wikibugs_>	 10Release-Engineering-Team (Next), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: scap support for git-lfs - https://phabricator.wikimedia.org/T181855#4255372 (10awight) I tried to do a deployment to ores2001 today, running three passes thus: 1. Deploy the master code (d77e52c) in case LFS works...
[21:33:58] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10Scap, 10Patch-For-Review, 10Scoring-platform-team (Current): Support git-lfs in scap - https://phabricator.wikimedia.org/T180627#4255382 (10awight) I tried to do a deployment to ores2001 today, running three passes thus: 1. Deploy the master code (d77e52c) in case LFS...
[21:35:14] <wikibugs_>	 10Release-Engineering-Team (Kanban), 10Release Pipeline: Build Math extension container on Postmerge - https://phabricator.wikimedia.org/T196414#4255383 (10thcipriani) p:05Triage>03Normal
[21:38:28] <shinken-wm>	 PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[21:42:09] <shinken-wm>	 PROBLEM - SSH on integration-slave-docker-1014 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[22:01:57] <shinken-wm>	 RECOVERY - SSH on integration-slave-docker-1014 is OK: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u4 (protocol 2.0)
[22:10:48] <wikibugs_>	 10Release-Engineering-Team, 10JADE, 10Wikimedia-General-or-Unknown, 10Patch-For-Review: JADE submodule on tin is dirty - https://phabricator.wikimedia.org/T195343#4255444 (10Legoktm) 05Open>03Resolved a:03Legoktm Will be fixed for the next deployment branch.
[22:18:27] <shinken-wm>	 RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:30:28] <wikibugs_>	 10Release-Engineering-Team, 10JADE, 10Wikimedia-General-or-Unknown, 10Patch-For-Review: JADE submodule on tin is dirty - https://phabricator.wikimedia.org/T195343#4224100 (10awight) @Legoktm Great fix, thank you!
[22:30:28] <shinken-wm>	 PROBLEM - Host deployment-puppetmaster02 is DOWN: CRITICAL - Host Unreachable (10.68.21.200)
[22:38:25] <wikibugs_>	 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4255531 (10awight) Closing, our plan is simple: * Get deployment working with a single LFS file, in the `submodules/assets` directory....
[22:38:54] <wikibugs_>	 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: ORES should use a git large file plugin for storing serialized binaries - https://phabricator.wikimedia.org/T171619#4255536 (10awight)
[22:39:03] <wikibugs_>	 10Gerrit, 10ORES, 10Operations, 10Patch-For-Review, 10Scoring-platform-team (Current): Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#4255532 (10awight) 05Open>03Resolved a:03awight
[22:41:32] <wikibugs_>	 10Scap, 10Operations, 10Packaging, 10Patch-For-Review: Install git-lfs client (at least on scap targets & masters) - https://phabricator.wikimedia.org/T180628#4255542 (10awight) Confirmed that git-lfs is installed on deploy1001, removing blocking task dependency.
[22:42:07] <wikibugs_>	 10Beta-Cluster-Infrastructure, 10ORES, 10Scoring-platform-team: Reimage deployment-ores01 as Stretch - https://phabricator.wikimedia.org/T189790#4255545 (10awight)
[22:42:15] <wikibugs_>	 10Scap, 10Operations, 10Packaging, 10Patch-For-Review: Install git-lfs client (at least on scap targets & masters) - https://phabricator.wikimedia.org/T180628#4255544 (10awight)
[23:26:17] <wikibugs_>	 10Phabricator, 10LDAP: Having difficulty logging into Phabricator via LDAP when multiple accounts returned for username - https://phabricator.wikimedia.org/T138672#4255611 (10SMcCandlish) Update: I can login to Phabricator by one but not both of the available methods. I can login to Wikitech, but I cannot logi...