[01:45:25] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [01:51:22] (03CR) 10GeoffreyT2000: [C: 031] "I accept your request to add me to the list." [integration/config] - 10https://gerrit.wikimedia.org/r/443433 (owner: 10Gergő Tisza) [03:41:18] !log Shut off deployment-mediawiki06 from Horizon (but not yet deleted) - T192996 [03:41:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [03:41:23] T192996: Delete deployment-mediawiki06 - https://phabricator.wikimedia.org/T192996 [03:45:21] 10Phabricator, 10Cleanup: Vandalism on Phabricator: Undo changes made (2018-07-01) - https://phabricator.wikimedia.org/T198552 (10MusikAnimal) The bot has re-ran through most if not all tasks and restored the triage level. In the process it fixed a few other things (missing tags, subscribers, etc.) that the bo... [03:49:07] !log Create deployment-webperf12 as equivalent of webperf1002/webperf2002 in prod (T195312, T194390) [03:49:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [03:49:12] T194390: EQIAD & CODFW: 1 VM in each data center for xhprof/xhgui/other profiling tools - https://phabricator.wikimedia.org/T194390 [03:49:12] T195312: Move flame graphs hosting from mwlog1001 to webperf-2 and enable in Beta Cluster - https://phabricator.wikimedia.org/T195312 [03:55:11] PROBLEM - Puppet errors on deployment-webperf12 is CRITICAL: CRITICAL: 83.33% of data above the critical threshold [0.0] [04:22:42] twentyafterfour: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/432533/ is abandoned but still applied to beta, and seems to conflict with origin/HEAD of ops/puppet, thus making it no longer update automatically due to merge conflict. [04:31:21] !log Setting up puppetmaster/cerf for deployment-webperf12 (T195312) [04:31:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [04:31:26] T195312: Move flame graphs hosting from mwlog1001 to webperf-2 and enable in Beta Cluster - https://phabricator.wikimedia.org/T195312 [04:40:15] RECOVERY - Puppet errors on deployment-webperf12 is OK: OK: Less than 1.00% above the threshold [0.0] [05:39:24] (03PS2) 10Subramanya Sastry: [Parsoid] Stop injecting MwEmbedHandler as a dependency, it's a no-op [integration/config] - 10https://gerrit.wikimedia.org/r/443002 (owner: 10Jforrester) [05:39:33] (03CR) 10Subramanya Sastry: [C: 031] [Parsoid] Stop injecting MwEmbedHandler as a dependency, it's a no-op [integration/config] - 10https://gerrit.wikimedia.org/r/443002 (owner: 10Jforrester) [06:28:40] PROBLEM - Puppet errors on deployment-zotero01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:29:03] 10Release-Engineering-Team, 10Performance-Team, 10MW-1.32-release-notes (WMF-deploy-2018-06-26 (1.32.0-wmf.10)), 10Patch-For-Review: Saving timings up 50% since 2018-06-28 20:53 - https://phabricator.wikimedia.org/T198483 (10jcrespo) Actually, I see it now on normal levels, since 2018-07-02 20:00: https://... [07:08:39] RECOVERY - Puppet errors on deployment-zotero01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:17] (03PS1) 10Veeru: Modify access rules [extensions/examples] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/443557 [07:26:54] (03Abandoned) 10Veeru: Modify access rules [extensions/examples] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/443557 (owner: 10Veeru) [07:30:15] (03PS1) 10MeetGadhiya: Insert the description of the change.s [extensions/Cargo] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/443560 [07:55:57] 10Release-Engineering-Team, 10Performance-Team, 10MW-1.32-release-notes (WMF-deploy-2018-06-26 (1.32.0-wmf.10)), 10Patch-For-Review: Saving timings up 50% since 2018-06-28 20:53 - https://phabricator.wikimedia.org/T198483 (10Tgr) >>! In T198483#4392950, @jcrespo wrote: > Actually, I see it now on normal le... [08:35:55] 10Phabricator: Create more Phabricator administrators - https://phabricator.wikimedia.org/T198578 (10Aklapper) 05Open>03declined >>! In T198578#4375222, @zhuyifei1999 wrote: > Redundant to T198576? Yes, I'd say so. (If other functionality apart from disabling user accounts was the intention to create this t... [08:39:37] (03PS1) 10Hashar: docker: add libssl-dev to releng/tox-labs-striker [integration/config] - 10https://gerrit.wikimedia.org/r/443570 (https://phabricator.wikimedia.org/T198076) [08:41:02] (03CR) 10Hashar: [C: 032] docker: add libssl-dev to releng/tox-labs-striker [integration/config] - 10https://gerrit.wikimedia.org/r/443570 (https://phabricator.wikimedia.org/T198076) (owner: 10Hashar) [08:42:42] (03Merged) 10jenkins-bot: docker: add libssl-dev to releng/tox-labs-striker [integration/config] - 10https://gerrit.wikimedia.org/r/443570 (https://phabricator.wikimedia.org/T198076) (owner: 10Hashar) [08:44:19] !log Building docker container releng/tox-labs-striker to add libssl-dev | T198076 [08:44:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:44:23] T198076: CI tests failing for labs/striker due to "upstream library changes and loose specification of the dependencies in the requirements.txt file" - https://phabricator.wikimedia.org/T198076 [09:15:32] (03PS1) 10Hashar: (WIP) Quibble job for release tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/443578 [10:45:43] (03PS2) 10Hashar: (WIP) Quibble job for release tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/443578 (https://phabricator.wikimedia.org/T197469) [10:57:12] 10Release-Engineering-Team, 10Performance-Team, 10MW-1.32-release-notes (WMF-deploy-2018-06-26 (1.32.0-wmf.10)), 10Patch-For-Review: Saving timings up 50% since 2018-06-28 20:53 - https://phabricator.wikimedia.org/T198483 (10aaron) Judging from https://performance.wikimedia.org/xenon/svgs/daily/2018-06-26.... [10:57:55] 10Continuous-Integration-Config, 10MediaWiki-extensions-Translate, 10TimedMediaHandler, 10Patch-For-Review: CI no more inject extensions dependencies on REL branches - https://phabricator.wikimedia.org/T197933 (10hashar) [11:18:03] (03PS3) 10Hashar: (WIP) Quibble job for release tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/443578 (https://phabricator.wikimedia.org/T197469) [11:20:13] (03CR) 10jerkins-bot: [V: 04-1] (WIP) Quibble job for release tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/443578 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [11:23:14] (03PS4) 10Hashar: (WIP) Quibble job for release tarballs [integration/config] - 10https://gerrit.wikimedia.org/r/443578 (https://phabricator.wikimedia.org/T197469) [12:41:02] (03PS1) 10Hashar: Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) [12:45:04] (03PS2) 10Hashar: Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) [12:49:12] (03PS3) 10Hashar: Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) [12:51:20] (03CR) 10jerkins-bot: [V: 04-1] Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [12:53:45] (03PS4) 10Hashar: Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) [12:55:51] (03CR) 10Hashar: [C: 032] Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [12:57:29] (03CR) 10jerkins-bot: [V: 04-1] Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [12:58:31] (03CR) 10Hashar: [C: 032] "I forgot to deploy the jobs in Jenkins :/" [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [13:01:17] (03Merged) 10jenkins-bot: Gated extensions now use Quibble on REL branches [integration/config] - 10https://gerrit.wikimedia.org/r/443604 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [13:11:41] (03PS1) 10Hashar: Gated extensions were no more running tests on REL [integration/config] - 10https://gerrit.wikimedia.org/r/443614 (https://phabricator.wikimedia.org/T197469) [13:13:35] (03CR) 10Hashar: [C: 032] Gated extensions were no more running tests on REL [integration/config] - 10https://gerrit.wikimedia.org/r/443614 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [13:15:02] (03Merged) 10jenkins-bot: Gated extensions were no more running tests on REL [integration/config] - 10https://gerrit.wikimedia.org/r/443614 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [13:30:51] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q3, 10Epic, and 2 others: Migrate gated extensions jobs to Quibble/Docker - https://phabricator.wikimedia.org/T197469 (10hashar) [14:14:46] (03PS1) 10Hashar: Remove legacy jobs from gated extensions [integration/config] - 10https://gerrit.wikimedia.org/r/443636 (https://phabricator.wikimedia.org/T197469) [14:17:07] (03PS2) 10Hashar: Remove legacy jobs from gated extensions [integration/config] - 10https://gerrit.wikimedia.org/r/443636 (https://phabricator.wikimedia.org/T197469) [14:19:14] (03CR) 10Hashar: [C: 032] Remove legacy jobs from gated extensions [integration/config] - 10https://gerrit.wikimedia.org/r/443636 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [14:20:39] (03Merged) 10jenkins-bot: Remove legacy jobs from gated extensions [integration/config] - 10https://gerrit.wikimedia.org/r/443636 (https://phabricator.wikimedia.org/T197469) (owner: 10Hashar) [14:22:36] good [14:22:41] more stuff migrated to Quibble \o/ [14:43:58] almost anyway... [15:14:56] (03CR) 10Subramanya Sastry: [C: 031] Test Parsoid's version of Poem extension's parserTests against MediaWiki [integration/config] - 10https://gerrit.wikimedia.org/r/442002 (owner: 10Arlolra) [16:09:37] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:42:11] (03CR) 10Thcipriani: [V: 032 C: 032] Provide Groovy library for invoking Blubber/Docker [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/442233 (https://phabricator.wikimedia.org/T196940) (owner: 10Dduvall) [17:24:20] hasharAway seems tests still fail for tmh just a different error this time [17:24:21] https://integration.wikimedia.org/ci/job/release-quibble-vendor-mysql-hhvm-docker/19/console [17:24:26] 17:23:35 Fatal error: Class undefined: getID3 in /workspace/src/extensions/TimedMediaHandler/handlers/ID3Handler/ID3Handler.php on line 15 [17:24:27] and [17:24:33] 17:23:35 Destructor threw an object exception: exception 'MWException' with message 'TestApiUploadVideo::tearDown() must call parent::tearDown()' in /workspace/src/tests/phpunit/MediaWikiTestCase.php:134 [17:25:00] the first one should depend on the composer dep, but dosen't seem to because composer is adding it into mw vendor folder instead of timedmediahandler's [17:25:36] waring, if phab logs stupid api request errors, I've been busy maknig some [17:25:50] I mean we're talking about a small handful, maybe 30 in several hours, but still [17:30:34] apergos: spammer [17:31:05] it me [17:31:34] trying to mock up something that wil tell me which tasks (it will have to wlak through all 4000 of them) still have some vandalism nureverted [17:31:38] *unreverted [17:32:01] sure got to know more about phab conduit than I ever wanted... [18:46:42] 10Phabricator: Delete H33 - https://phabricator.wikimedia.org/T198745 (10Mholloway) [18:53:03] 10Phabricator, 10User-greg: Delete H33 - https://phabricator.wikimedia.org/T198745 (10greg) 05Open>03Resolved a:03greg See T95932 for history. Team changes are not automatically propagated to Phabricator Herald rules :P Rule archived. [18:54:10] Hauskatze: :) [18:55:05] * Hauskatze hides [19:01:13] greg-g: is anyone in particular doing active maintenance of Scap? [19:04:08] a few people :) [19:04:13] (I know they're gone) [19:33:41] 10Scap, 10Patch-For-Review: update-interwiki-cache failed: 'Namespace' object has no attribute 'force' - https://phabricator.wikimedia.org/T196642 (10MarcoAurelio) [20:05:10] 10Scap, 10Patch-For-Review: update-interwiki-cache failed: 'Namespace' object has no attribute 'force' - https://phabricator.wikimedia.org/T196642 (10Reedy) [20:05:25] 10Scap, 10Patch-For-Review: update-interwiki-cache failed: 'Namespace' object has no attribute 'force' - https://phabricator.wikimedia.org/T196642 (10Reedy) [20:34:33] 10Gerrit: no jenkins-bot actions are triggered for my user at gerrit.wikimedia.org, project:pywikibot/core - https://phabricator.wikimedia.org/T198665 (10MarcoAurelio) Mpaa's account strangely does not have any email address when searching for it. That might be the reason? I get `owner:"Mpaa (1065)"` instead of... [21:02:51] 10Continuous-Integration-Config, 10Gerrit, 10Jenkins: Jenkins bot ignores the patches submitted by Mpaa - https://phabricator.wikimedia.org/T198573 (10Paladox) [21:04:30] no_justification: you there? [21:06:08] 10Continuous-Integration-Config, 10Gerrit, 10Jenkins: Jenkins bot ignores the patches submitted by Mpaa - https://phabricator.wikimedia.org/T198573 (10Paladox) >>! In T198573#4370182, @MarcoAurelio wrote: > @Mpaa's gerrit account does not seem to have an email address associated to it (`owner:"Mpaa (1065)"`... [21:11:23] (03CR) 10C. Scott Ananian: [C: 031] "Yeah, that's probably a good idea." [integration/config] - 10https://gerrit.wikimedia.org/r/442004 (owner: 10Arlolra) [21:18:46] twentyafterfour: if you're around, could you take look at beta puppetmaster and adjust or un-pick https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/432533/ - it's currently causing a rebase conflict, and subsequently the puppetmaster has been stale since about 3 days. [21:24:59] PROBLEM - Puppet errors on deployment-webperf11 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [21:25:01] PROBLEM - Puppet errors on deployment-memc05 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [21:27:21] PROBLEM - Puppet errors on deployment-restbase01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:28:20] PROBLEM - Puppet errors on deployment-conf03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:33:25] PROBLEM - Puppet errors on deployment-kafka-main-1 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:34:35] PROBLEM - Puppet errors on deployment-etcd-01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [21:35:43] PROBLEM - Puppet errors on deployment-memc07 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:40:24] PROBLEM - Puppet errors on deployment-sca01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:40:26] PROBLEM - Puppet errors on deployment-changeprop is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [21:42:55] PROBLEM - Puppet errors on deployment-certcentral-testclient02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [21:42:55] PROBLEM - Puppet errors on deployment-elastic05 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [21:44:51] PROBLEM - Puppet errors on deployment-aqs02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:44:51] PROBLEM - Puppet errors on deployment-cassandra3-02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [21:44:55] PROBLEM - Puppet errors on deployment-kafka-jumbo-1 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [21:46:40] PROBLEM - Puppet errors on deployment-certcentral02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [21:47:08] Krinkle: is this shinken-wm related to your message above? [21:47:54] PROBLEM - Puppet errors on deployment-elastic06 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:48:28] PROBLEM - Puppet errors on deployment-chromium01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [21:48:30] PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [21:48:46] !log cleaned up cherry-pick conflict to try to fix puppet [21:48:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:49:57] PROBLEM - Puppet errors on deployment-mediawiki06 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [21:59:58] RECOVERY - Puppet errors on deployment-memc05 is OK: OK: Less than 1.00% above the threshold [0.0] [22:02:53] RECOVERY - Puppet errors on deployment-elastic05 is OK: OK: Less than 1.00% above the threshold [0.0] [22:09:36] RECOVERY - Puppet errors on deployment-etcd-01 is OK: OK: Less than 1.00% above the threshold [0.0] [22:09:48] RECOVERY - Puppet errors on deployment-aqs02 is OK: OK: Less than 1.00% above the threshold [0.0] [22:09:48] RECOVERY - Puppet errors on deployment-cassandra3-02 is OK: OK: Less than 1.00% above the threshold [0.0] [22:09:54] RECOVERY - Puppet errors on deployment-kafka-jumbo-1 is OK: OK: Less than 1.00% above the threshold [0.0] [22:09:59] RECOVERY - Puppet errors on deployment-webperf11 is OK: OK: Less than 1.00% above the threshold [0.0] [22:10:25] RECOVERY - Puppet errors on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0] [22:10:27] RECOVERY - Puppet errors on deployment-changeprop is OK: OK: Less than 1.00% above the threshold [0.0] [22:10:43] RECOVERY - Puppet errors on deployment-memc07 is OK: OK: Less than 1.00% above the threshold [0.0] [22:10:57] :) [22:11:10] !log rebased stuff on deployment-puppetmaster and ran puppet everywhere through cumin [22:11:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:11:39] RECOVERY - Puppet errors on deployment-certcentral02 is OK: OK: Less than 1.00% above the threshold [0.0] [22:11:52] and with that I'm going to sleep, it's too hot to concentrate [22:12:23] RECOVERY - Puppet errors on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0] [22:12:56] RECOVERY - Puppet errors on deployment-certcentral-testclient02 is OK: OK: Less than 1.00% above the threshold [0.0] [22:12:56] RECOVERY - Puppet errors on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0] [22:13:18] RECOVERY - Puppet errors on deployment-conf03 is OK: OK: Less than 1.00% above the threshold [0.0] [22:13:24] RECOVERY - Puppet errors on deployment-kafka-main-1 is OK: OK: Less than 1.00% above the threshold [0.0] [22:13:31] RECOVERY - Puppet errors on deployment-tin is OK: OK: Less than 1.00% above the threshold [0.0] [22:13:31] RECOVERY - Puppet errors on deployment-chromium01 is OK: OK: Less than 1.00% above the threshold [0.0] [22:14:55] RECOVERY - Puppet errors on deployment-mediawiki06 is OK: OK: Less than 1.00% above the threshold [0.0] [22:15:39] Krenair: me too, though thankfully we're having not-so-hot days here [22:16:00] these days at least, a week ago or so it was like fire [22:45:44] Krenair: Hm.. which conflict was it? [22:46:17] Ah, you solved both. [22:46:19] Thanks [23:50:25] (03CR) 10Srishakatux: "Hello! Could you share what this commit is addressing?" [extensions/Cargo] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/443560 (owner: 10MeetGadhiya)