[00:01:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [00:02:26] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10incubator.wikimedia.org, 10I18n: Allow creating an independent "incubator wiki" instead of hosting all new wikis in one Incubator wiki with prefixes - https://phabricator.wikimedia.org/T228745 (10Gnangarra) Suggest the dom... [01:16:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [02:09:48] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<50.00%) [02:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [04:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [05:01:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:26:45] 10Phabricator, 10Regression: Cannot access project workboards: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10Aklapper) [06:27:36] 10Phabricator, 10Regression: Cannot access project workboards or login after phabricator upgrade. - https://phabricator.wikimedia.org/T230969 (10Joe) [06:27:39] 10Phabricator, 10Regression: Cannot access project workboards or log in: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10Aklapper) Cannot reproduce the problem (but maybe some work has taken place in the last 5 hours that I wasn't online?). [06:27:42] 10Phabricator, 10Regression: Cannot access project workboards or log in: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10Cyberpower678) [06:28:08] 10Phabricator, 10Regression: Cannot access project workboards or log in: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10Cyberpower678) Yes. This appears to be fixed now. [06:28:16] 10Phabricator, 10Regression: Cannot access project workboards or log in: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10Marostegui) @Aklapper the workboards just got fixed by @mmodell ` [06:25:10] ok workboards are fixed ` [06:28:22] 10Phabricator, 10Regression: Cannot access project workboards or log in: Failed to load class or interface "PhabricatorWorkboardViewState" - https://phabricator.wikimedia.org/T230969 (10mmodell) 05Open→03Resolved a:03mmodell this is fixed by {db569a45a8da0ecd1e91eb69c7b8e0d4e5448a39} [06:28:50] 10Phabricator, 10Regression: Cannot access project workboards or log in after phabricator upgrade - https://phabricator.wikimedia.org/T230969 (10Joe) [06:28:54] 10Phabricator, 10Regression: Cannot access project workboards or log in after phabricator upgrade - https://phabricator.wikimedia.org/T230969 (10Aklapper) Heh. Nice timing (and good morning from here). Thanks for the quick fix! [06:30:50] 10Phabricator: Add contact opprotunity if login does not work - https://phabricator.wikimedia.org/T230972 (10RolandUnger) [06:44:13] 10Phabricator, 10Regression: Cannot access project workboards or log in after phabricator upgrade - https://phabricator.wikimedia.org/T230969 (10mmodell) For the record, what I had to do was execute `arc liberate` inside the phabricator source tree, e.g. `/srv/phab/phabricator` but this had to be done locally... [06:44:45] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:11:48] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10Quibble, and 2 others: Quibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests - https://phabricator.wikimedia.org/T199116 (10hashar) >>! In... [07:30:45] (03PS1) 10Kosta Harlan: jjb: [mwext-codehealth*] Pass ZUUL_PROJECT as argument [integration/config] - 10https://gerrit.wikimedia.org/r/531600 [07:40:45] kostajh: ZUUL_PROJECT should be recognized by Quibble and added to the list of repos to clone :-\ [07:41:14] eg: ZUUL_PROJECT=mediawiki/extensions/Foobar quibble [07:41:19] that should clonse Foobar [07:41:46] hashar: in my testing locally, passing e.g. `quibble --run=phpunit-unit mediawiki/extensions/GrowthExperiments` causes EXT_DEPENDENCIES to be ignored, and only GrowthExperiments to be cloned, which is what I am aiming for [07:49:24] hashar: since the code health job is running unit tests only now, we don't need all the dependencies cloned, and that should speed up both the initial cloning step and the composer install step. [08:06:42] hmm [08:07:38] kostajh: I gotta reproduce. but EXT_DEPENDENCIES content should alaways be added to the list of repos to clone [08:08:10] * hashar tries [08:11:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [08:14:38] (03CR) 10Hashar: "Content of EXT_DEPENDENCIES is always added to the list of repositories to clone. Passing some other projects as arguments does not magic" [integration/config] - 10https://gerrit.wikimedia.org/r/531600 (owner: 10Kosta Harlan) [08:15:24] (03CR) 10Hashar: "(or atlernatively: use releng/zuul-cloner to clone the repositories we need, then run quibble with --skip-zuul which bypass the logic to " [integration/config] - 10https://gerrit.wikimedia.org/r/531600 (owner: 10Kosta Harlan) [08:16:11] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10mmodell) >>! In T226044#5429736, @JAufrecht wrote: > The i... [08:36:04] (03Abandoned) 10Hashar: zuul: periodic pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/376740 (https://phabricator.wikimedia.org/T93414) (owner: 10Hashar) [08:38:42] (03CR) 10Hashar: [C: 04-1] layout: Migrate users of bespoke -tox-docker jobs to generic one [integration/config] - 10https://gerrit.wikimedia.org/r/526749 (owner: 10Jforrester) [08:39:16] (03CR) 10Hashar: [C: 04-1] "That merges all those projects in the same shared dependent queue :( See also my comment on the parent change https://gerrit.wikimedia.org" [integration/config] - 10https://gerrit.wikimedia.org/r/526750 (owner: 10Jforrester) [08:41:31] 10Phabricator, 10Developer-Advocacy: Decrease issue with assignee field set for years without progress (aka cookie licking) - https://phabricator.wikimedia.org/T228575 (10Quiddity) a:03Aklapper [08:43:04] 10Continuous-Integration-Config: Include svgmin into the CI processes - https://phabricator.wikimedia.org/T229763 (10hashar) [08:45:08] 10Continuous-Integration-Config, 10MediaViewer: Include svgmin into the CI processes - https://phabricator.wikimedia.org/T229763 (10hashar) [08:46:31] 10Continuous-Integration-Config, 10MediaViewer: Include svgmin into the CI processes - https://phabricator.wikimedia.org/T229763 (10hashar) [08:46:44] * hashar triages https://phabricator.wikimedia.org/tag/jenkins/ [08:57:11] 10Continuous-Integration-Config, 10Jenkins: Typo on https://integration.wikimedia.org/ci/ - https://phabricator.wikimedia.org/T191078 (10hashar) 05Stalled→03Resolved a:03hashar That should be have been reported to #upstream #jenkins maintainers at https://issues.jenkins-ci.org/ Anyway I have tried again... [08:58:36] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata Query UI, 10Patch-For-Review: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831 (10hashar) [08:58:48] 10Continuous-Integration-Config: Setup CI for mediawiki/extensions/OAuth2.git - https://phabricator.wikimedia.org/T230977 (10MarcoAurelio) [08:59:04] 10Continuous-Integration-Config, 10CPT Initiatives (OAuth 2.0): Setup CI for mediawiki/extensions/OAuth2.git - https://phabricator.wikimedia.org/T230977 (10MarcoAurelio) [09:01:17] 10Continuous-Integration-Config: Setup CI for mediawiki/extensions/OAuth2.git - https://phabricator.wikimedia.org/T230977 (10MarcoAurelio) Looks a different 2.0 thing. [09:06:41] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata Query UI, 10Patch-For-Review, 10User-Addshore: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831 (10hashar) 05Stalled→03Resolved a:03Addshore The PhantomJS issue was due to some missing lib... [09:07:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Jenkins: JENKINS-2111 path sanitization ineffective when using legacy Workspace Root Directory - https://phabricator.wikimedia.org/T213956 (10hashar) p:05Triage→03Low [09:10:38] (03PS1) 10MarcoAurelio: Add mediawiki/extensions/OAUth2 to Zuul [integration/config] - 10https://gerrit.wikimedia.org/r/531665 (https://phabricator.wikimedia.org/T230977) [09:11:36] (03PS2) 10MarcoAurelio: Add mediawiki/extensions/OAUth2 to Zuul [integration/config] - 10https://gerrit.wikimedia.org/r/531665 (https://phabricator.wikimedia.org/T230977) [09:12:29] (03CR) 10MarcoAurelio: "Please verify just in case we need to use .*-composer templates, instead; or if the re" [integration/config] - 10https://gerrit.wikimedia.org/r/531665 (https://phabricator.wikimedia.org/T230977) (owner: 10MarcoAurelio) [09:14:43] 10Continuous-Integration-Config, 10LibUp, 10Test-Coverage: Publish labs/libraryupgrader test coverage report - https://phabricator.wikimedia.org/T229619 (10MarcoAurelio) [09:22:55] 10Phabricator: Add contact opprotunity if login does not work - https://phabricator.wikimedia.org/T230972 (10Aklapper) 05Open→03Invalid The frontpage already explains where to bring up questions: https://www.mediawiki.org/wiki/Talk:Phabricator/Help [09:23:43] 10Gerrit, 10Release-Engineering-Team-TODO (201908), 10Gerrit-Privilege-Requests, 10MediaWiki-Gerrit-Group-Requests, 10User-zeljkofilipin: Cannot amend Gerrit changeset in MediaWiki Core repo (due to not being in addPatchSet?) - https://phabricator.wikimedia.org/T230916 (10hashar) The ability to add a pat... [09:23:47] 10Gerrit, 10Release-Engineering-Team-TODO (201908), 10Gerrit-Privilege-Requests, 10MediaWiki-Gerrit-Group-Requests, 10User-zeljkofilipin: Cannot amend Gerrit changeset in MediaWiki Core repo (due to not being in addPatchSet?) - https://phabricator.wikimedia.org/T230916 (10zeljkofilipin) [09:24:51] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10Gerrit-Privilege-Requests, and 2 others: Cannot amend Gerrit changeset in MediaWiki Core repo (due to not being in addPatchSet?) - https://phabricator.wikimedia.org/T230916 (10hashar) [09:28:14] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10Gerrit-Privilege-Requests, and 2 others: Cannot amend Gerrit changeset in MediaWiki Core repo (due to not being in addPatchSet?) - https://phabricator.wikimedia.org/T230916 (10hashar) 05Open→03Resolv... [09:43:22] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Developer-Advocacy: Tech Talks Proposal 2019: new CI system candidate demonstration - https://phabricator.wikimedia.org/T220695 (10srodlund) a:03srodlund @greg @larsWirzenius Are you able to give me an estimate for when... [09:48:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Jenkins: JENKINS-2111 path sanitization ineffective when using legacy Workspace Root Directory - https://phabricator.wikimedia.org/T213956 (10hashar) https://integration.wikimedia.org/ci/configure has: | Home directory | `/var/lib/j... [09:53:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Jenkins: JENKINS-2111 path sanitization ineffective when using legacy Workspace Root Directory - https://phabricator.wikimedia.org/T213956 (10hashar) The directory comes from a java system parameter used to start Jenkins: ` -Djenkins... [09:55:47] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Jenkins: JENKINS-2111 path sanitization ineffective when using legacy Workspace Root Directory - https://phabricator.wikimedia.org/T213956 (10hashar) [10:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [10:15:57] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10Wikimedia-General-or-Unknown, 10Epic, and 2 others: Enable mediawiki/mediawiki-phan-config on all Wikimedia-deployed repositories - https://phabricator.wikimedia.org/T224783 (10Aklapper) [10:20:35] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Developer-Advocacy: Tech Talks Proposal 2019: new CI system candidate demonstration - https://phabricator.wikimedia.org/T220695 (10srishakatux) a:05srodlund→03None [10:48:21] PROBLEM - Free space - all mounts on deployment-mediawiki-07 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki-07.diskspace.root.byte_percentfree (<100.00%) [11:12:06] (03PS2) 10Kosta Harlan: jjb: [mwext-codehealth*] Set ZUUL_DEPENDENCIES to ZUUL_PROJECT [integration/config] - 10https://gerrit.wikimedia.org/r/531600 [11:14:07] (03CR) 10Kosta Harlan: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/531600 (owner: 10Kosta Harlan) [11:27:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins: JENKINS-2111 path sanitization ineffective when using legacy Workspace Root Directory - https://phabricator.wikimedia.org/T213956 (10hashar) 05Open→03Resolved a:03ha... [11:28:26] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Zuul: Zuul should not start gate-and-submit for WIP patch sets - https://phabricator.wikimedia.org/T201744 (10hashar) [11:28:54] 10Continuous-Integration-Infrastructure, 10Wikidata: php-lint in wikibase times out - https://phabricator.wikimedia.org/T198493 (10hashar) [11:31:24] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201908): CI is missing a template for "Skins without HHVM" - https://phabricator.wikimedia.org/T230602 (10hashar) a:05Jdforrester-WMF→03Osnard @Osnard introduced the Zuul template `skin-quibble-composer-nohhvm`. [11:31:30] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201908): CI is missing a template for "Skins without HHVM" - https://phabricator.wikimedia.org/T230602 (10hashar) 05Open→03Resolved [11:46:20] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T220744 (10alaa_wmde) [11:47:55] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Jenkins jobs not running after pushing to gerrit for Jpita user - https://phabricator.wikimedia.org/T231003 (10zeljkofilipin) [11:48:20] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Jenkins jobs not running after pushing to gerrit for Jpita user - https://phabricator.wikimedia.org/T231003 (10zeljkofilipin) p:05Triage→03Normal [11:50:15] 10Phabricator: Add contact opportunity if login does not work - https://phabricator.wikimedia.org/T230972 (10Aklapper) [11:51:25] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Jenkins jobs not running after pushing to gerrit for Jpita user - https://phabricator.wikimedia.org/T231003 (10zeljkofilipin) @hashar can you help again, please? 😁 [12:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [13:29:46] (03CR) 10Hashar: [C: 03+2] Add mediawiki/extensions/OAUth2 to Zuul [integration/config] - 10https://gerrit.wikimedia.org/r/531665 (https://phabricator.wikimedia.org/T230977) (owner: 10MarcoAurelio) [13:31:07] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T220744 (10Tarrow) [13:31:17] (03Merged) 10jenkins-bot: Add mediawiki/extensions/OAUth2 to Zuul [integration/config] - 10https://gerrit.wikimedia.org/r/531665 (https://phabricator.wikimedia.org/T230977) (owner: 10MarcoAurelio) [13:47:51] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Scap, 10serviceops, and 3 others: Enhance MediaWiki deployments for support of php7.x - https://phabricator.wikimedia.org/T224857 (10jijiki) @thcipriani When a deployer run scap pull on mwdebug, they got `sudo: [/... [14:11:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [14:17:35] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MobileFrontend, 10Documentation, 10Readers-Web-Backlog (Tracking): Migrate documentation generation to Node 10.15.2 from node 6.11.0 - https://phabricator.wikimedia.org/T230841 (10Jdlrobson) [14:18:27] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MobileFrontend, 10Documentation, 10Readers-Web-Backlog (Tracking): Migrate documentation generation to Node 10.15.2 from node 6.11.0 - https://phabricator.wikimedia.org/T230841 (10Jdlrobson) [14:25:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T220745 (10zeljkofilipin) a:03zeljkofilipin [14:39:50] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T220745 (10zeljkofilipin) [14:42:43] 10Continuous-Integration-Config, 10Release-Engineering-Team (Watching / External), 10Analytics, 10Patch-For-Review: Status of analytics/limn-*-data git repositories? - https://phabricator.wikimedia.org/T221064 (10Nuria) [15:02:07] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T220744 (10zeljkofilipin) 05Open→03Resolved [15:28:28] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Other: OAuth2: problems with CI - https://phabricator.wikimedia.org/T231022 (10MarcoAurelio) [15:37:00] 10Continuous-Integration-Config, 10MediaWiki-extensions-Other: OAuth2: problems with CI - https://phabricator.wikimedia.org/T231022 (10hashar) [15:40:34] (03PS1) 10MarcoAurelio: WIP: Switch OAuth2 to use `extension-quibble-composer-nohhvm`. [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) [15:41:14] 10Continuous-Integration-Config, 10User-MarcoAurelio: Setup CI for mediawiki/extensions/OAuth2.git - https://phabricator.wikimedia.org/T230977 (10MarcoAurelio) 05Open→03Resolved a:03MarcoAurelio Done here. Some problems being sorted out at T231022. [15:42:13] (03PS2) 10MarcoAurelio: WIP: Switch OAuth2 to use `extension-quibble-composer-nohhvm`. [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) [15:42:34] (03CR) 10MarcoAurelio: "Ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) (owner: 10MarcoAurelio) [15:45:31] (03PS3) 10MarcoAurelio: Switch OAuth2 to use `extension-quibble-composer-nohhvm`. [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) [15:45:47] but where is marco aurelio ! :] [15:45:48] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T220745 (10zeljkofilipin) [15:46:20] (03CR) 10Hashar: [C: 03+2] "Excellent :-]" [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) (owner: 10MarcoAurelio) [15:47:53] (03Merged) 10jenkins-bot: Switch OAuth2 to use `extension-quibble-composer-nohhvm`. [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) (owner: 10MarcoAurelio) [15:53:38] (03CR) 10MarcoAurelio: "> Excellent :-]" [integration/config] - 10https://gerrit.wikimedia.org/r/531715 (https://phabricator.wikimedia.org/T231022) (owner: 10MarcoAurelio) [15:54:27] hashar: bonjour, have you reloaded zuul to deploy 531715? I see nix in the releng log :) [15:56:28] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T220745 (10hashar) [15:59:04] hauskatze: nop sorry. But reloaded now! [15:59:22] hauskatze: I went side tracked by some other duty eek ;] [15:59:34] hashar: perfect, retesting then [15:59:59] hauskatze: also is phan actually configured on that repo? [16:00:26] phan-taint-check-plugin [16:00:32] sounds valid :-] [16:00:49] not .phan, but they surely will [16:01:04] if they do not add it in some time we can remove it from the config [16:01:42] running ci on a dummy change [16:05:34] better ) [16:06:24] no clue why parallel-lint fails though [16:06:56] probably because no file was there? [16:06:58] idk [16:07:23] on a 'true' change: https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php72-docker/3119/console [16:07:29] sounds like phpcs [16:07:41] I'll be back later [16:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [16:25:07] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10Usmanmuhd) I also have to another patch (https://gerrit.wikimedia.org/r/#/c/research/article-recommender/deploy/+/527571/) merged. The deadli... [16:36:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T220745 (10zeljkofilipin) [16:55:23] Project beta-scap-eqiad build #263580: 04FAILURE in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263580/ [17:05:21] Project beta-scap-eqiad build #263581: 04STILL FAILING in 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263581/ [17:15:14] Project beta-scap-eqiad build #263582: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263582/ [17:16:12] 10Release-Engineering-Team, 10Product-Analytics: Create a repository and user for Product Analytics Oozie jobs? - https://phabricator.wikimedia.org/T230743 (10fdans) Tagging Release Engineering to create the repo. example https://github.com/wikimedia/wikimedia-discovery-analytics [17:25:24] Project beta-scap-eqiad build #263583: 04STILL FAILING in 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263583/ [17:26:08] Is someone aware of the 'unresolved setup issues' in phab showing? (it may be mid swat for it and i dont realize it) [17:27:02] it also may be something we dont care about.... i dont really wanna talk about the actual error in a public channel [17:27:11] twentyafterfour: ^ ? [17:28:09] robh: it's not particularly serious issue I can hide it to avoid confusion [17:28:21] i assumed after reading it wasnt something we care about [17:28:28] but i didnt wanna not say anything =] [17:28:35] pretty sure only phab admins see it? [17:28:44] yeah only phab admins see it [17:28:48] * twentyafterfour marked it as ignored [17:28:57] yeah, since its such a small group who can see it i wanted to check =] [17:35:23] Project beta-scap-eqiad build #263584: 04STILL FAILING in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263584/ [17:37:21] 10Release-Engineering-Team, 10MinervaNeue, 10MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), 10Readers-Web-Backlog (Tracking), and 2 others: Flaky wdio test: MinervaNeue/history_steps "Page diff Added and removed content" - https://phabricator.wikimedia.org/T229031 (10zeljkofilipin) [17:42:49] 10Release-Engineering-Team, 10MinervaNeue, 10MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), 10Readers-Web-Backlog (Tracking), and 2 others: Flaky wdio test: MinervaNeue/history_steps "Page diff Added and removed content" - https://phabricator.wikimedia.org/T229031 (10zeljkofilipin) >>! In T229031#5366700, @Jdl... [17:45:22] Project beta-scap-eqiad build #263585: 04STILL FAILING in 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263585/ [17:55:20] Project beta-scap-eqiad build #263586: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263586/ [18:05:15] Project beta-scap-eqiad build #263587: 04STILL FAILING in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263587/ [18:11:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [18:15:22] Project beta-scap-eqiad build #263588: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263588/ [18:25:18] Project beta-scap-eqiad build #263589: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263589/ [18:35:15] Project beta-scap-eqiad build #263590: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263590/ [18:43:11] ^ Duplicate entry 'pam-messages:apihelp-query+allpages-example-b' for key 'PRIMARY' [18:45:23] Project beta-scap-eqiad build #263591: 04STILL FAILING in 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263591/ [18:55:22] Project beta-scap-eqiad build #263592: 04STILL FAILING in 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263592/ [19:05:21] Project beta-scap-eqiad build #263593: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263593/ [19:15:13] Project beta-scap-eqiad build #263594: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263594/ [19:16:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [10.0] [19:18:42] (03PS3) 10Jeena Huneidi: Use alias for mediawiki-dev chart [releng/local-charts] - 10https://gerrit.wikimedia.org/r/530245 [19:19:52] (03PS4) 10Jeena Huneidi: Use alias for mediawiki-dev chart [releng/local-charts] - 10https://gerrit.wikimedia.org/r/530245 [19:24:43] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201908), 10Operations, and 2 others: Prepare Phame to support heavy traffic for a Tech Department blog - https://phabricator.wikimedia.org/T226044 (10JAufrecht) > __Wikimedia Foundation__ (https://wikimediafo... [19:25:13] Project beta-scap-eqiad build #263595: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263595/ [19:35:18] Project beta-scap-eqiad build #263596: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263596/ [19:45:16] Project beta-scap-eqiad build #263597: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263597/ [19:55:19] Project beta-scap-eqiad build #263598: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263598/ [20:05:15] Project beta-scap-eqiad build #263599: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263599/ [20:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [20:15:15] Project beta-scap-eqiad build #263600: 04STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263600/ [20:25:19] Project beta-scap-eqiad build #263601: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263601/ [20:35:21] Project beta-scap-eqiad build #263602: 04STILL FAILING in 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263602/ [20:45:23] Project beta-scap-eqiad build #263603: 04STILL FAILING in 56 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263603/ [20:55:21] Project beta-scap-eqiad build #263604: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263604/ [20:56:45] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10Aklapper) @Usmanmuhd: Who exactly set a deadline on what exactly? [21:01:39] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10leila) @Aklapper Usmanmuhd is a GSoC student and they have a deadline to finish their project with us. Aklapper: Nschaaf doesn't work in WM... [21:03:11] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10leila) [21:05:17] Project beta-scap-eqiad build #263605: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263605/ [21:05:31] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10Aklapper) >>! In T230859#5432633, @leila wrote: > @Aklapper Usmanmuhd is a GSoC student and they have a deadline to finish their project with... [21:15:21] Project beta-scap-eqiad build #263606: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263606/ [21:25:19] Project beta-scap-eqiad build #263607: 04STILL FAILING in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263607/ [21:35:20] Project beta-scap-eqiad build #263608: 04STILL FAILING in 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263608/ [21:41:42] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10bd808) Proxy +2 applied per @bmansurov's +1s and comments on the patches. We really need to examine the Foundation off-boarding blanket +2 re... [21:45:07] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10leila) @bd808 really appreciate it (and agreed). [21:45:11] Project beta-scap-eqiad build #263609: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263609/ [21:45:15] 10Release-Engineering-Team, 10Recommendation-API, 10Research, 10Google-Summer-of-Code (2019): Merge the patch for GSoC - https://phabricator.wikimedia.org/T230859 (10leila) 05Open→03Resolved [21:55:14] Project beta-scap-eqiad build #263610: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263610/ [22:05:17] Project beta-scap-eqiad build #263611: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263611/ [22:08:12] Project beta-scap-eqiad build #263612: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263612/ [22:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [22:15:14] Project beta-scap-eqiad build #263613: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263613/ [22:25:13] Project beta-scap-eqiad build #263614: 04STILL FAILING in 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263614/ [22:35:11] Project beta-scap-eqiad build #263615: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263615/ [22:45:13] Project beta-scap-eqiad build #263616: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263616/ [22:55:15] Project beta-scap-eqiad build #263617: 04STILL FAILING in 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263617/ [23:05:18] Project beta-scap-eqiad build #263618: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263618/ [23:10:01] Project beta-scap-eqiad build #263619: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263619/ [23:13:11] Project beta-scap-eqiad build #263620: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263620/ [23:15:25] Project beta-scap-eqiad build #263621: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263621/ [23:19:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Discovery-Search: quibble-vendor-mysql-hhvm-docker for WikibaseCirrusSearch takes over 40 minutes - https://phabricator.wikimedia.org/T222757 (10Smalyshev) Ok I am getting multipl... [23:21:26] Project beta-scap-eqiad build #263622: 04STILL FAILING in 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263622/ [23:25:10] Project beta-scap-eqiad build #263623: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263623/ [23:27:58] Project beta-scap-eqiad build #263624: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263624/ [23:35:19] Project beta-scap-eqiad build #263625: 04STILL FAILING in 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263625/ [23:44:30] Project beta-scap-eqiad build #263626: 04STILL FAILING in 0.53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263626/ [23:55:14] Project beta-scap-eqiad build #263627: 04STILL FAILING in 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/263627/ [23:59:23] 10Beta-Cluster-Infrastructure: No merged changes are deployed on betacluster - https://phabricator.wikimedia.org/T231058 (10Urbanecm) [23:59:50] 10Beta-Cluster-Infrastructure: No merged changes are deployed on betacluster - https://phabricator.wikimedia.org/T231058 (10Urbanecm) p:05Triage→03Unbreak! Beta cluster is quite important => bumping to UBN!