[00:07:25] <shinken-wm>	 PROBLEM - Host deployment-cumin is DOWN: CRITICAL - Host Unreachable (172.16.5.1)
[00:21:04] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10HHVM, 10Upstream: hhvm aborts on extensions/CirrusSearch/tests/phan/stubs/intl.php blocking deployment sync - https://phabricator.wikimedia.org/T221814 (10Smalyshev)
[00:22:11] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10HHVM, 10Upstream: hhvm aborts on extensions/CirrusSearch/tests/phan/stubs/intl.php blocking deployment sync - https://phabricator.wikimedia.org/T221814 (10Smalyshev) hhvm backtrace: {P8435}
[01:15:37] <thcipriani>	 mutante: not sure about the ssl update change or whether or not it will break any existing clients, would prefer to deploy when releng is more available (next week) unless it's urgent
[01:20:23] <mutante>	 thcipriani: ok, i removed it from the slot
[02:42:27] <wikibugs>	 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10Wikidata, and 5 others: Introduce wikidata termbox SSR to kubernetes - https://phabricator.wikimedia.org/T220402 (10mobrovac) >>! In T220402#5136129, @Tarrow wrote: > We are indeed using service-runner; I don't think this provides /_info or /?...
[03:53:42] <wikibugs>	 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair)
[04:18:18] <wikibugs>	 (03PS1) 10Jdlrobson: Drop Selenium Ruby jobs from Jenkins on MobileFrontend [integration/config] - 10https://gerrit.wikimedia.org/r/506343 (https://phabricator.wikimedia.org/T177260)
[05:08:53] <shinken-wm>	 PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<50.00%)
[06:43:53] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK
[09:04:13] <wikibugs>	 10Scap (Scap3-MediaWiki-MVP), 10MediaWiki-Configuration: SCAP should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php - https://phabricator.wikimedia.org/T125678 (10Ladsgroup) Some parts of the problem: T125678
[10:17:20] <wikibugs>	 (03Abandoned) 10Pablo Grass (WMDE): termbox: build production variant in CI [integration/config] - 10https://gerrit.wikimedia.org/r/501339 (https://phabricator.wikimedia.org/T220107) (owner: 10Pablo Grass (WMDE))
[11:52:28] <wikibugs>	 10Project-Admins, 10MediaWiki-extensions-TEI: Create mediawiki-extensions-TEI component - https://phabricator.wikimedia.org/T221772 (10Tpt) Thank you!
[12:44:11] <wikibugs>	 (03CR) 10Jforrester: [C: 03+1] Move security alerts to security-admin-feed@lists.wikimedia.org [integration/config] - 10https://gerrit.wikimedia.org/r/506313 (owner: 10Brian Wolff)
[12:44:30] <wikibugs>	 (03CR) 10Jforrester: [C: 03+1] Drop Selenium Ruby jobs from Jenkins on MobileFrontend [integration/config] - 10https://gerrit.wikimedia.org/r/506343 (https://phabricator.wikimedia.org/T177260) (owner: 10Jdlrobson)
[12:48:28] <wikibugs>	 (03CR) 10Hashar: "I just clone repositories respecting the Gerrit hierarchy, but most people do not. So yeah having an environment variable would definitely" [integration/config] - 10https://gerrit.wikimedia.org/r/504437 (owner: 10Hashar)
[13:33:05] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (Backlog), 10Readers-Web-Backlog: CI: Minerva PHPUnit tests should be included in shared extension gate job - https://phabricator.wikimedia.org/T202030 (10hashar) Hi, I actually looked at adding Minerva but eventually there is not much support to ad...
[13:36:27] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "I have updated the php-composer-security-docker job :)  Thank you!" [integration/config] - 10https://gerrit.wikimedia.org/r/506313 (owner: 10Brian Wolff)
[13:38:36] <wikibugs>	 (03Merged) 10jenkins-bot: Move security alerts to security-admin-feed@lists.wikimedia.org [integration/config] - 10https://gerrit.wikimedia.org/r/506313 (owner: 10Brian Wolff)
[13:40:47] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Drop Selenium Ruby jobs from Jenkins on MobileFrontend [integration/config] - 10https://gerrit.wikimedia.org/r/506343 (https://phabricator.wikimedia.org/T177260) (owner: 10Jdlrobson)
[13:42:15] <wikibugs>	 (03Merged) 10jenkins-bot: Drop Selenium Ruby jobs from Jenkins on MobileFrontend [integration/config] - 10https://gerrit.wikimedia.org/r/506343 (https://phabricator.wikimedia.org/T177260) (owner: 10Jdlrobson)
[13:42:33] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [RelatedArticles] Add Disambiguator as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/506220 (owner: 10Umherirrender)
[13:42:43] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [Popups] Add Gadgets as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/506217 (owner: 10Umherirrender)
[13:44:00] <wikibugs>	 (03Merged) 10jenkins-bot: [RelatedArticles] Add Disambiguator as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/506220 (owner: 10Umherirrender)
[13:44:14] <wikibugs>	 (03Merged) 10jenkins-bot: [Popups] Add Gadgets as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/506217 (owner: 10Umherirrender)
[13:57:41] <wikibugs>	 (03PS1) 10Tpt: Make TEI extension depends on Math [integration/config] - 10https://gerrit.wikimedia.org/r/506419
[14:44:26] <shinken-wm>	 PROBLEM - Host deployment-ms-be03 is DOWN: CRITICAL - Host Unreachable (172.16.5.51)
[14:45:20] <shinken-wm>	 PROBLEM - Host deployment-ms-be04 is DOWN: CRITICAL - Host Unreachable (172.16.4.129)
[15:09:48] <jdlrobson>	 is beta cluster backed up? A few patches I'm waiting on don't seem to have gone through yet (despite being merged an hour ago)
[15:10:33] <jdlrobson>	 Reedy: just saw https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/506453/ but not understanding it. Is this related?
[15:10:43] <Reedy>	 jdlrobson: https://phabricator.wikimedia.org/T221860
[15:10:58] <Reedy>	 That is failing on completely core patches
[15:11:13] <jdlrobson>	 I don't understand. Why would that be passing in Minerva but not elsewhere?
[15:11:25] <jdlrobson>	 sorry rephrase - why is that passing in MobileFrontend build but not in core
[15:11:29] <Reedy>	 Not sure
[15:12:04] <jdlrobson>	 Can you link me to an example in core
[15:12:28] <jdlrobson>	 ah found https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/506421/
[15:12:29] <Reedy>	 https://phabricator.wikimedia.org/T221860
[15:12:33] <Reedy>	 ffs, yeah that one
[15:13:18] <Reedy>	 jdlrobson: What're you missing from beta? Looks like code updates are working fine
[15:13:29] <jdlrobson>	 1:12 INFO:backend.DevWebServer:[Thu Apr 25 14:51:12 2019] 127.0.0.1:33472 [200]: //api.php?format=json 14:51:12 Unhandled rejection Error: badtoken: Invalid CSRF token.
[15:16:11] <jdlrobson>	 Reedy: are you sure it's effecting ALL jenkins builds? I just ask as I see that's touching the API and tokens.
[15:16:26] <jdlrobson>	 the test that's failing is creating an account
[15:16:31] <wikibugs>	 10Phabricator, 10WMNO-General: Disable Phabricator account WMNOjorid - https://phabricator.wikimedia.org/T221863 (10jhsoby-WMNO)
[15:16:42] <Reedy>	 I've not seen any other core patches
[15:16:46] <jdlrobson>	 is it possible account creation is impacted by the patches you are trying to land?
[15:16:57] <Reedy>	 Shouldn't be
[15:17:04] <Reedy>	 IT's only changing log out
[15:17:21] <jdlrobson>	 (i see https://gerrit.wikimedia.org/r/506386 is passing for example)
[15:17:38] <wikibugs>	 10Phabricator, 10WMNO-General: Disable Phabricator account WMNOjorid - https://phabricator.wikimedia.org/T221863 (10Reedy) 05Open→03Resolved a:03Reedy Done
[15:19:37] <jdlrobson>	 The failure in https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/506429/ is not related to this test either
[15:20:00] <jdlrobson>	 it could just be selenium "quibbling" due to too many runs at the same time
[15:20:18] <wikibugs>	 10Release-Engineering-Team, 10Librarization, 10Quibble, 10TechCom: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T220723 (10Krinkle) a:05Krinkle→03None Re-triaging for TechCom –  This is first and foremost a feature request for th...
[15:20:26] <jdlrobson>	 https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php70-docker/18163/consoleFull is an issue with the rollback link
[15:20:27] <wikibugs>	 10Release-Engineering-Team, 10Librarization, 10Quibble: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T220723 (10Krinkle)
[15:22:32] <jdlrobson>	 The revert is also failing... https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/506453/ ... Reedy looks like a problem with the CI pipeline to me
[15:22:45] <Reedy>	 With something relating to echo
[15:22:56] <Reedy>	 uh, flow
[15:24:43] <jdlrobson>	 > What're you missing from beta? Looks like code updates are working fine < on this subject.. what I'm noticing is MobileFrontend is stuck on 5e9cf6c but 65d7f7beb8ad8b0 has been the master commit for over an hour
[15:26:05] <Reedy>	 Last scap just finished...
[15:26:07] * Reedy watches https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/244108/console
[15:28:00] <Reedy>	 commit 99b105bfd8c2c70a064ed0c1d91f1f0cad45c068
[15:28:00] <Reedy>	 Author: Jdlrobson <jrobson@wikimedia.org>
[15:28:00] <Reedy>	 Date:   Thu Apr 25 14:05:58 2019 +0000
[15:28:00] <Reedy>	     Update git submodules
[15:28:00] <Reedy>	     
[15:28:01] <Reedy>	     * Update MobileFrontend from branch 'master'
[15:28:03] <Reedy>	       to 65d7f7beb8ad8b0fb7d204251ee73a58f71e6451
[15:28:14] <Reedy>	 Lets see if https://integration.wikimedia.org/ci/job/beta-scap-eqiad/246633/console pushes it out
[15:29:47] <wikibugs>	 10Phabricator, 10WMNO-General: Make #WMNO-General a subproject of #WMNO - https://phabricator.wikimedia.org/T221864 (10jhsoby-WMNO)
[15:30:22] <Reedy>	 jdlrobson: FWIW, I don't see any mention of "logout" in the selenoum dir of mw core
[15:30:28] <jdlrobson>	 yeh me neither
[15:30:51] <jdlrobson>	 but i'm seeing other selenium failures without the token error so it could be something fundamentally wrong with the Selenium job
[15:31:00] <Reedy>	 flaky tests? never
[15:31:23] <Reedy>	 It just seemed very co-incidental tests failing on that patch was merged
[15:31:26] <jdlrobson>	 https://gerrit.wikimedia.org/r/506421 is passing now!
[15:31:36] <jdlrobson>	 (after a recheck)
[15:31:41] <jdlrobson>	 Selenium flakes all the time :/
[15:31:42] <Reedy>	 ffs
[15:32:07] <Reedy>	 lets see if it'll submit it this time
[15:32:33] <jdlrobson>	 I find in particular it flakes when there's lots of jobs running in parallel
[15:32:35] <jdlrobson>	 i have no idea why
[15:35:10] <jdlrobson>	 crossing fingers!
[15:38:31] <Reedy>	 MF still seems stuck on beta
[15:38:47] <wmf-insecte>	 Project beta-scap-eqiad build #246634: 04FAILURE in 0.67 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/246634/
[15:39:12] <Reedy>	 lol
[15:41:10] <Reedy>	 16:37:26 15:37:26 deployment-deploy01.deployment-prep.eqiad.wmflabs failed to update opcache: 
[15:41:14] <Reedy>	 I wonder if that explains the issue
[15:49:42] * paladox adds mobile searching to PolyGerrit https://gerrit-review.googlesource.com/c/gerrit/+/221732
[15:53:00] <Reedy>	 jdlrobson: Looks like your code is there, but the git cache isn't updating Special:Version for some reason
[15:54:15] <jdlrobson>	 im finishing for the day now Reedy, but looks like these should go through
[15:54:24] <jdlrobson>	 will look first thing tomorrow
[15:54:28] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage - https://phabricator.wikimedia.org/T221860 (10Jdlrobson) I don't think this is specifically a pro...
[15:54:48] <wmf-insecte>	 Yippee, build fixed!
[15:54:49] <wmf-insecte>	 Project beta-scap-eqiad build #246635: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/246635/
[15:55:11] <wikibugs>	 10Release-Engineering-Team (Kanban), 10MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), and 2 others: Enable a MobileFrontend daily Selenium build - https://phabricator.wikimedia.org/T220659 (10Jdlrobson) https://integration.wikim...
[15:59:00] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Make TEI extension depends on Math [integration/config] - 10https://gerrit.wikimedia.org/r/506419 (owner: 10Tpt)
[16:00:48] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [GuidedTour] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506206 (owner: 10Umherirrender)
[16:00:58] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [ExternalGuidance] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505834 (owner: 10Umherirrender)
[16:01:03] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [FeaturedFeeds] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505838 (owner: 10Umherirrender)
[16:01:08] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [GlobalBlocking] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505843 (owner: 10Umherirrender)
[16:01:16] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [GlobalCssJs] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505845 (owner: 10Umherirrender)
[16:01:21] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [Graph] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506202 (owner: 10Umherirrender)
[16:01:26] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Parsoid, 10Jenkins: composer-package-php73-docker seems to fail often on Parsoid builds - https://phabricator.wikimedia.org/T221872 (10cscott)
[16:02:05] <wikibugs>	 (03Merged) 10jenkins-bot: Make TEI extension depends on Math [integration/config] - 10https://gerrit.wikimedia.org/r/506419 (owner: 10Tpt)
[16:02:17] <wikibugs>	 (03Merged) 10jenkins-bot: [GuidedTour] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506206 (owner: 10Umherirrender)
[16:02:32] <wikibugs>	 (03Merged) 10jenkins-bot: [ExternalGuidance] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505834 (owner: 10Umherirrender)
[16:02:35] <wikibugs>	 (03Merged) 10jenkins-bot: [FeaturedFeeds] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505838 (owner: 10Umherirrender)
[16:02:57] <wikibugs>	 (03Merged) 10jenkins-bot: [GlobalBlocking] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505843 (owner: 10Umherirrender)
[16:03:09] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [PageViewInfo] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506212 (owner: 10Umherirrender)
[16:03:11] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [ParsoidBatchAPI] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506214 (owner: 10Umherirrender)
[16:03:13] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [PoolCounter] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506216 (owner: 10Umherirrender)
[16:03:57] <wikibugs>	 (03Merged) 10jenkins-bot: [GlobalCssJs] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505845 (owner: 10Umherirrender)
[16:03:59] <wikibugs>	 (03Merged) 10jenkins-bot: [Graph] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506202 (owner: 10Umherirrender)
[16:04:05] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [WikimediaMessages] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505440 (owner: 10Umherirrender)
[16:04:11] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [Campaigns] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505443 (owner: 10Umherirrender)
[16:04:17] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [BetaFeatures] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505448 (owner: 10Umherirrender)
[16:04:23] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [FundraisingTranslateWorkflow] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505841 (owner: 10Umherirrender)
[16:04:40] <wikibugs>	 (03Merged) 10jenkins-bot: [PageViewInfo] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506212 (owner: 10Umherirrender)
[16:04:58] <wikibugs>	 (03Merged) 10jenkins-bot: [ParsoidBatchAPI] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506214 (owner: 10Umherirrender)
[16:05:04] <wikibugs>	 (03Merged) 10jenkins-bot: [PoolCounter] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506216 (owner: 10Umherirrender)
[16:09:50] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] [WikimediaMessages] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505440 (owner: 10Umherirrender)
[16:11:50] <wikibugs>	 (03Merged) 10jenkins-bot: [Campaigns] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505443 (owner: 10Umherirrender)
[16:12:27] <wikibugs>	 (03Merged) 10jenkins-bot: [BetaFeatures] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505448 (owner: 10Umherirrender)
[16:13:38] <wikibugs>	 (03Merged) 10jenkins-bot: [FundraisingTranslateWorkflow] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505841 (owner: 10Umherirrender)
[16:16:09] <wikibugs>	 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 5 others: Introduce kask session storage service to kubernetes - https://phabricator.wikimedia.org/T220401 (10Eevans) >>! In T220401#5134204, @akosiaris wrote: > @Eevans @Clarakosi chart has been merged and is published. The on...
[16:18:23] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] [WikimediaMessages] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505440 (owner: 10Umherirrender)
[16:20:42] <wikibugs>	 (03Merged) 10jenkins-bot: [WikimediaMessages] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/505440 (owner: 10Umherirrender)
[16:34:51] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage - https://phabricator.wikimedia.org/T221860 (10Reedy) https://gerrit.wikimedia.org/r/#/c/mediawiki...
[16:35:35] <Krenair>	 !log shutting down deployment-ms-fe02 and deployment-poolcounter04 T218729
[16:35:37] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[16:35:38] <stashbot>	 T218729: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729
[16:37:28] <wikibugs>	 10Beta-Cluster-Infrastructure: Migrate away from Debian Jessie to Debian Stretch - https://phabricator.wikimedia.org/T218729 (10Krenair)
[16:38:18] <Krenair>	 ... huh, well that didn't go as expected
[16:39:48] <Krenair>	 looks like I missed a rather important ms-fe02 user
[16:40:28] <shinken-wm>	 PROBLEM - Host deployment-ms-fe02 is DOWN: CRITICAL - Host Unreachable (172.16.5.66)
[16:40:45] <shinken-wm>	 PROBLEM - Host deployment-poolcounter04 is DOWN: CRITICAL - Host Unreachable (172.16.5.58)
[16:41:01] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage - https://phabricator.wikimedia.org/T221860 (10Reedy) https://gerrit.wikimedia.org/r/505683 is fai...
[16:41:41] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage - https://phabricator.wikimedia.org/T221860 (10Reedy) https://gerrit.wikimedia.org/r/505648 too...
[16:42:10] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs - https://phabricator.wikimedia.org/T221860 (10Reedy)
[16:42:25] <Krenair>	 okay, moved
[16:42:28] <Krenair>	 now it looks ok
[16:48:36] <Krenair>	 originals but not thumbs
[16:51:56] <Krenair>	 Looks like thumbor got the change of ms-fe hosts but did not restart
[16:52:03] <Krenair>	 I tried restarting it but it does not want to come back
[16:54:47] <Krenair>	 oh it cares about poolcounter too, lovely
[17:01:35] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Kanban): Calendar shows "Unhandled exception error: Expected a single result!" - https://phabricator.wikimedia.org/T220241 (10mmodell)
[17:02:09] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Kanban): Calendar shows "Unhandled exception error: Expected a single result!" - https://phabricator.wikimedia.org/T220241 (10mmodell) This is fixed in d7744d7e9bd0 but I haven't deployed it because I don't want to do a phab deploy from hotel wifi.
[17:11:05] <Reedy>	 God CI is slow
[17:28:45] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: On deployment-prep scap cache_git_info takes 12 minutes (that is too slow) - https://phabricator.wikimedia.org/T204762 (10hashar) To the best I could tell when I investigated it: * that seems to be a regression from end of July 2018,...
[17:42:15] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Thumbor: thumbor on deployment-imagescaler03 does not want to start with firejail private-dev rule - https://phabricator.wikimedia.org/T221879 (10Krenair)
[17:45:26] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Thumbor: libpng problems on deployment-imagescaler02 - https://phabricator.wikimedia.org/T221880 (10Krenair)
[17:54:38] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: On deployment-prep scap cache_git_info takes 12 minutes (that is too slow) - https://phabricator.wikimedia.org/T204762 (10hashar) I hacked scap to output some logging info to better show the cache_git_info progress https://integratio...
[18:00:46] <wikibugs>	 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Request Sauce Labs access for niedzielski - https://phabricator.wikimedia.org/T206358 (10Niedzielski) 05Resolved→03Open @zeljkofilipin, I'm reopening this task as Sauce Labs now says "Your trial period has ended." /cc @greg who is listed on wiki as...
[18:10:50] <Krenair>	 Reedy, https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/OpenStackManager/+/506477/ approved at 1620, merged 1808
[18:10:51] <wmf-insecte>	 Project beta-scap-eqiad build #246647: 15ABORTED in 6 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/246647/
[18:11:02] <Reedy>	 yup
[18:11:22] <Reedy>	 queue is empty though now almost
[18:14:19] <hashar>	 because people force merge patches which breaks the flow :-\
[18:23:29] <wikibugs>	 10Phabricator, 10Project-Admins, 10Tool-extjsonuploader: Move #tool-extjsonuploader to a subproject of #tools - https://phabricator.wikimedia.org/T221540 (10Tgr)
[18:33:38] <wikibugs>	 (03PS1) 10Umherirrender: [SandboxLink] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506504
[18:37:49] <wikibugs>	 (03PS1) 10Umherirrender: [RSS] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506507
[18:45:54] <wikibugs>	 (03PS1) 10Umherirrender: [Popups] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506513
[18:49:50] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: On deployment-prep scap cache_git_info takes 12 minutes (that is too slow) - https://phabricator.wikimedia.org/T204762 (10Krinkle) @hashar Perhaps an ENV variable is interfering? Try with `sudo -i` instead of `sudo -s` and maybe also...
[18:49:53] <wikibugs>	 (03PS1) 10Umherirrender: [RelatedArticles] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506521
[18:52:34] <wikibugs>	 (03PS1) 10Umherirrender: [QuickSurveys] Add phan [integration/config] - 10https://gerrit.wikimedia.org/r/506523
[18:58:25] <wikibugs>	 (03PS1) 10Umherirrender: [Score] Add TimedMediaHandler as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/506524
[19:21:11] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs - https://phabricator.wikimedia.org/T221860 (10D3r1ck01) https://gerrit.wikimedia.org/r/c/mediaw...
[19:40:32] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs - https://phabricator.wikimedia.org/T221860 (10Krinkle) >>! In T221860#513...
[20:09:52] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10Patch-For-Review, 10Wikimedia-production-error (Shared Build Failure): AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs - https://phabricator.wikimedia.org/T221860 (10Krinkle) @Jdlrobson This is...
[21:57:41] <shinken-wm>	 PROBLEM - Citoid on deployment-sca01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[22:02:18] <shinken-wm>	 PROBLEM - Citoid on deployment-sca02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[22:02:38] <shinken-wm>	 PROBLEM - Content Translation Server on deployment-sca02 is CRITICAL: connect to address 172.16.5.112 and port 8080: Connection refused
[22:06:59] <shinken-wm>	 PROBLEM - Content Translation Server on deployment-sca01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[22:07:10] <shinken-wm>	 RECOVERY - Citoid on deployment-sca02 is OK: HTTP OK: HTTP/1.1 200 OK - 921 bytes in 0.027 second response time
[22:08:05] <shinken-wm>	 PROBLEM - Mathoid on deployment-mathoid is CRITICAL: connect to address 172.16.5.73 and port 10042: Connection refused
[22:16:49] <shinken-wm>	 RECOVERY - Content Translation Server on deployment-sca01 is OK: HTTP OK: HTTP/1.1 200 OK - 904 bytes in 0.030 second response time
[22:17:37] <shinken-wm>	 RECOVERY - Content Translation Server on deployment-sca02 is OK: HTTP OK: HTTP/1.1 200 OK - 904 bytes in 0.026 second response time
[22:19:10] <shinken-wm>	 PROBLEM - Citoid on deployment-sca02 is CRITICAL: connect to address 172.16.5.112 and port 1970: Connection refused
[22:22:29] <shinken-wm>	 RECOVERY - Citoid on deployment-sca01 is OK: HTTP OK: HTTP/1.1 200 OK - 921 bytes in 0.026 second response time
[22:22:51] <shinken-wm>	 PROBLEM - Content Translation Server on deployment-sca01 is CRITICAL: connect to address 172.16.5.13 and port 8080: Connection refused
[22:28:36] <shinken-wm>	 PROBLEM - Content Translation Server on deployment-sca02 is CRITICAL: connect to address 172.16.5.112 and port 8080: Connection refused
[22:28:43] <shinken-wm>	 PROBLEM - Citoid on deployment-sca01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[22:33:04] <shinken-wm>	 RECOVERY - Mathoid on deployment-mathoid is OK: HTTP OK: HTTP/1.1 200 OK - 925 bytes in 0.029 second response time
[22:33:29] <shinken-wm>	 RECOVERY - Citoid on deployment-sca01 is OK: HTTP OK: HTTP/1.1 200 OK - 921 bytes in 0.029 second response time
[22:33:36] <shinken-wm>	 RECOVERY - Content Translation Server on deployment-sca02 is OK: HTTP OK: HTTP/1.1 200 OK - 904 bytes in 0.028 second response time
[23:04:57] <wikibugs>	 10Continuous-Integration-Config, 10MediaWiki-extensions-Translate: Translate coverage publish step fails with: Incorrect whitelist config, no code coverage will be generated. - https://phabricator.wikimedia.org/T221913 (10hashar)
[23:07:57] <wikibugs>	 10Release-Engineering-Team, 10MobileFrontend, 10Browser-Tests, 10MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), and 2 others: AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs - https://phabricator.wikimedia.org/T221860 (10hashar) Thank you for the quick ''fix'' :]
[23:10:00] <wikibugs>	 10Continuous-Integration-Config, 10Patch-For-Review: 'recheck' on a CR+2 patch should trigger gate-and-submit, not test - https://phabricator.wikimedia.org/T105474 (10hashar) Seems that feature is now working. Potentially because when commenting `recheck` the event emitted by Gerrit now carries the Code-Review...