[00:05:01] Project beta-update-databases-eqiad build #18715: 15ABORTED in 45 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/18715/ [00:17:58] Yippee, build fixed! [00:17:59] Project selenium-Flow » firefox,beta,Linux,BrowserTests build #463: 09FIXED in 1 min 57 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/463/ [00:22:22] 10MediaWiki-Codesniffer: Incorrect indenting moving {} onto newline - https://phabricator.wikimedia.org/T171695#3473325 (10Reedy) [01:13:54] 10Continuous-Integration-Config, 10Release-Engineering-Team (Watching / External), 10MediaWiki-Core-Tests, 10MediaWiki-Parser, and 4 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3473406 (10Krinkle) 05Open>03Resolved... [04:02:12] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<55.56%) [05:23:48] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3473690 (10demon) For checking in, yes without issue (that's all client side). However fetch/pull requires rsync access to fatten the blobs... [06:27:46] Krinkle: around? [07:07:11] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:14:54] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:28:19] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10MinervaNeue, 10Operations: The mobile-frontend-placeholder message is not updated in din.wikipedia.org - https://phabricator.wikimedia.org/T171711#3473889 (10Amire80) [08:42:28] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10MinervaNeue, 10Operations: The mobile-frontend-placeholder message is not updated in din.wikipedia.org - https://phabricator.wikimedia.org/T171711#3473906 (10KartikMistry) It seems LocalisationUpdate is failing. See: https://tools.wmf... [08:43:37] PROBLEM - Puppet errors on integration-slave-jessie-android is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [08:45:25] PROBLEM - Puppet errors on integration-puppetmaster01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [08:53:23] PROBLEM - Puppet errors on webperformance is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [08:53:33] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:54:43] PROBLEM - Puppet errors on jenkinstest is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:54:49] PROBLEM - Puppet errors on integration-saltmaster is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [08:55:03] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:55:45] PROBLEM - Puppet errors on integration-slave-docker-1002 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [08:57:40] PROBLEM - Puppet errors on saucelabs-01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [09:00:25] PROBLEM - Puppet errors on castor02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [09:01:22] bah [09:01:59] Data retrieved from Integration/host/integration-puppetmaster01 is String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 [09:02:23] PROBLEM - Puppet errors on integration-r-lang-01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [09:04:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): integration puppetmaster yield String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 - https://phabricator.wikimedia.org/T171712#3473914 (10hashar) [09:04:53] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): integration puppetmaster yield String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 - https://phabricator.wikimedia.org/T171712#3473926 (10hashar) p:05Triage>03High a:03hashar [09:05:49] PROBLEM - Puppet errors on saucelabs-02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [09:06:13] PROBLEM - Puppet errors on saucelabs-03 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [09:07:37] PROBLEM - Puppet errors on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [09:07:45] PROBLEM - Puppet errors on integration-publishing is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [09:07:53] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [09:09:49] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): integration puppetmaster yield String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 - https://phabricator.wikimedia.org/T171712#3473944 (10hashar) ``` $ ./hiera_lookup -v --fqdn=`hostname -f` discovery::app_routes ... DEBUG:... [09:09:52] PROBLEM - Puppet errors on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [09:12:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): integration puppetmaster yield String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 - https://phabricator.wikimedia.org/T171712#3473946 (10hashar) a:05hashar>03None [09:14:55] PROBLEM - Puppet errors on integration-slave-docker-1003 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [09:15:25] PROBLEM - Puppet errors on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [09:18:33] elukey: there will potentially be less connections to redis jobrunner next week ( mw/core https://gerrit.wikimedia.org/r/#/c/356119/ ) [09:24:10] !log deployment-cache-upload04 deployment-cache-text04 upgraded logster 0.0.10-1~jessie1 -> 0.0.10-2~jessie1 - T171318 [09:24:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:24:14] T171318: logster should not resolve statsd's IP every time it sends a metric - https://phabricator.wikimedia.org/T171318 [09:25:25] !log deployment-tin deleting temporary l10n cache from July 19th 20:09 at /tmp/scap_l10n_3608512748 1.5G [09:25:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:26:35] hashar: \o/ [09:35:14] 10Continuous-Integration-Config: zeus.cfg - https://phabricator.wikimedia.org/T171716#3474011 (10Younes2001) [09:39:17] elukey: and I think I spammed you about jobrunner deployment yesterday. scap does not quite support yet all the use cases of jobrunner [09:39:26] but that is definitely known to releng (well to the scap developers at least) [09:43:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: integration puppetmaster yield String, not Hash or nil at /etc/puppet/manifests/realm.pp:51 - https://phabricator.wikimedia.org/T171712#3474038 (10hashar) 05Open>03Resolved a:03Joe [09:50:22] RECOVERY - Puppet errors on integration-puppetmaster01 is OK: OK: Less than 1.00% above the threshold [0.0] [09:50:24] RECOVERY - Puppet errors on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [09:52:22] RECOVERY - Puppet errors on integration-r-lang-01 is OK: OK: Less than 1.00% above the threshold [0.0] [09:52:41] RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [09:52:47] RECOVERY - Puppet errors on integration-publishing is OK: OK: Less than 1.00% above the threshold [0.0] [09:53:35] RECOVERY - Puppet errors on integration-slave-jessie-android is OK: OK: Less than 1.00% above the threshold [0.0] [09:53:35] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [09:54:39] RECOVERY - Puppet errors on jenkinstest is OK: OK: Less than 1.00% above the threshold [0.0] [09:54:49] RECOVERY - Puppet errors on integration-saltmaster is OK: OK: Less than 1.00% above the threshold [0.0] [09:54:55] RECOVERY - Puppet errors on integration-slave-docker-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [09:55:04] RECOVERY - Puppet errors on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [09:55:25] RECOVERY - Puppet errors on castor02 is OK: OK: Less than 1.00% above the threshold [0.0] [09:55:45] RECOVERY - Puppet errors on integration-slave-docker-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [09:55:47] RECOVERY - Puppet errors on saucelabs-02 is OK: OK: Less than 1.00% above the threshold [0.0] [09:56:13] RECOVERY - Puppet errors on saucelabs-03 is OK: OK: Less than 1.00% above the threshold [0.0] [09:57:35] RECOVERY - Puppet errors on integration-slave-trusty-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [09:57:54] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [09:58:22] RECOVERY - Puppet errors on webperformance is OK: OK: Less than 1.00% above the threshold [0.0] [09:59:50] RECOVERY - Puppet errors on integration-slave-trusty-1004 is OK: OK: Less than 1.00% above the threshold [0.0] [10:20:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Performance-Team, 10WebPageTest, 10Patch-For-Review: Where to trigger WebPageTest jobs? - https://phabricator.wikimedia.org/T166756#3306680 (10hashar) p:05Triage>03Normal [10:46:44] (03Abandoned) 10Hashar: parallel 2 -> 64 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/362359 (owner: 10Hashar) [11:00:15] (03CR) 10Hashar: [C: 031] "Sent as a pull request to https://github.com/jenkinsci/android-emulator-plugin/pull/69" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [11:04:53] (03CR) 10Hashar: [C: 031] "Sent as a PR https://github.com/jenkinsci/android-emulator-plugin/pull/70" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366484 (owner: 10Mholloway) [11:06:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3474263 (10hashar) [11:12:03] (03CR) 10Zfilipin: [C: 031] "My concern is that this will lead to tests being fixed more slowly, but if it's something that would help Reading team, go ahead." [integration/config] - 10https://gerrit.wikimedia.org/r/365933 (https://phabricator.wikimedia.org/T94684) (owner: 10Hashar) [11:13:35] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Browser test jobs should use xUnit publisher instead of Junit - https://phabricator.wikimedia.org/T94684#3474279 (10zeljkofilipin) Related to {T94212}? [11:16:40] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10Patch-For-Review, 10User-zeljkofilipin: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3474282 (10zeljkofilipin) a:05Jdlrobson>03zeljkofilipin All blocking tasks are resolved.... [11:17:38] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspace/wikimedia-fundrais... - https://phabricator.wikimedia.org/T171724#3474287 [11:18:23] PROBLEM - Puppet errors on deployment-ms-be03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:19:28] (03CR) 10Zfilipin: [C: 032] Add unstable status to browser tests jobs [integration/config] - 10https://gerrit.wikimedia.org/r/365933 (https://phabricator.wikimedia.org/T94684) (owner: 10Hashar) [11:20:14] PROBLEM - Puppet errors on deployment-ms-be04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:20:42] (03Merged) 10jenkins-bot: Add unstable status to browser tests jobs [integration/config] - 10https://gerrit.wikimedia.org/r/365933 (https://phabricator.wikimedia.org/T94684) (owner: 10Hashar) [11:21:12] PROBLEM - Puppet errors on deployment-sca02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:23:18] (03CR) 10Zfilipin: "Deployed:" [integration/config] - 10https://gerrit.wikimedia.org/r/365933 (https://phabricator.wikimedia.org/T94684) (owner: 10Hashar) [11:24:33] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [11:24:59] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog): Accommodate flaky tests flapping - https://phabricator.wikimedia.org/T94212#3474310 (10zeljkofilipin) [11:25:01] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Browser test jobs should use xUnit publisher instead of Junit - https://phabricator.wikimedia.org/T94684#1170069 (10zeljkofilipin) 05Open>03Resolved Resolved by https://gerrit.wikim... [11:25:34] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog), 10User-zeljkofilipin: Accommodate flaky tests flapping - https://phabricator.wikimedia.org/T94212#1157641 (10zeljkofilipin) [11:26:03] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [11:27:47] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Accommodate flaky tests flapping - https://phabricator.wikimedia.org/T94212#3474334 (10zeljkofilipin) 05Open>03Resolved a:03zeljkofilipin [[ https://gerrit.wikimedia.org/r/#/c/365933/ | 365933 ]] (Add unstable st... [11:31:24] !log realign installed debian packages on integration-slave-jessie-1001 and integration-slave-jessie-1002 - T171724 [11:31:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:31:28] T171724: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspace/wikimedia-fundraising-civicrm/src/wikimedia/fundraising/civicrm-buildkit/bin/amp/src/Amp/Database/MySQL.php on line 58 - https://phabricator.wikimedia.org/T171724 [11:36:03] RECOVERY - Puppet errors on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [11:36:28] (03CR) 10Zfilipin: [C: 032] Do not run mwext-mw-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367674 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [11:37:54] (03Merged) 10jenkins-bot: Do not run mwext-mw-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367674 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [11:39:34] !log Reloading Zuul to deploy 723ab49039714ae2507ef43a94884de1a6bb347e [11:39:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:40:36] (03CR) 10Zfilipin: "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/367674 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [11:44:34] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [11:45:27] (03PS3) 10Zfilipin: Run mediawiki-core-qunit-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367413 (https://phabricator.wikimedia.org/T164024) [11:45:33] (03CR) 10jerkins-bot: [V: 04-1] Run mediawiki-core-qunit-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367413 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [11:50:29] (03PS4) 10Zfilipin: Run mediawiki-core-qunit-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367413 (https://phabricator.wikimedia.org/T164024) [11:55:35] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [12:00:04] (03CR) 10Zfilipin: [C: 032] Run mediawiki-core-qunit-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367413 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [12:00:57] (03Merged) 10jenkins-bot: Run mediawiki-core-qunit-selenium-jessie for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367413 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [12:04:46] !log Reloading Zuul to deploy 79781d8b08a6dbd4d2e2b5554fbb04154c0acf48 [12:04:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:07:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspace/wikimedia-fundrais... - https://phabricator.wikimedia.org/T171724#3474393 [12:15:36] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [12:44:39] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [12:49:32] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [12:53:22] (03PS1) 10Zfilipin: Do not run mwgate-rake-jessie job for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367885 (https://phabricator.wikimedia.org/T164024) [12:55:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspace/wikimedia-fundrais... - https://phabricator.wikimedia.org/T171724#3474628 [13:01:45] (03CR) 10Zfilipin: [C: 032] Do not run mwgate-rake-jessie job for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367885 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [13:02:34] (03Merged) 10jenkins-bot: Do not run mwgate-rake-jessie job for RelatedArticles [integration/config] - 10https://gerrit.wikimedia.org/r/367885 (https://phabricator.wikimedia.org/T164024) (owner: 10Zfilipin) [13:10:31] (03CR) 10Mholloway: "Thanks, hashar! I'll chime in there." [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366484 (owner: 10Mholloway) [13:19:42] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [13:21:57] (03CR) 10Mholloway: [C: 031] "By the way, hashar, I'm not sure if you've heard but Niedzielski is transitioning to the mobile web team and probably doesn't have much ti" [integration/config] - 10https://gerrit.wikimedia.org/r/366908 (https://phabricator.wikimedia.org/T150623) (owner: 10Hashar) [13:24:33] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0] [13:24:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3474658 (10Mholloway) Screenshots need updating again as a result of 31aedab. I'll get to that s... [13:39:08] !log Reloading Zuul to deploy 8787b4bbe75691e35792864434f8e09f3aaf6e53 [13:39:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:45:06] 10Continuous-Integration-Config, 10Wikipedia-Android-App-Backlog: Remove xdummy config stuff if no longer needed - https://phabricator.wikimedia.org/T171535#3474754 (10Mholloway) p:05Triage>03Low [13:45:20] Project selenium-VisualEditor » firefox,beta,Linux,BrowserTests build #471: 04FAILURE in 1 min 19 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/471/ [13:55:42] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [14:11:15] (03PS3) 10Hashar: Move wikimedia-fundraising-civicrm to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/319331 [14:13:05] (03PS4) 10Hashar: Move wikimedia-fundraising-civicrm to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/319331 (https://phabricator.wikimedia.org/T119140) [14:13:32] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/worksp... - https://phabricator.wikimedia.org/T171724#3474819 [14:13:34] 10Continuous-Integration-Config, 10Continuous-Integration-Scaling, 10Release-Engineering-Team, 10releng-201516-q3, and 2 others: [keyresult] Migrate as many misc CI jobs as possible to Nodepool - https://phabricator.wikimedia.org/T119140#3474818 (10hashar) [14:25:18] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10media-storage: nscd does not cache localhost causing high CPU usage when localhost is often resolved - https://phabricator.wikimedia.org/T171745#3474846 (10hashar) [14:25:38] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10media-storage, 10Patch-For-Review: deployment-ms-be03.deployment-prep and deployment-ms-be04.deployment-prep have high load / system CPU - https://phabricator.wikimedia.org/T160990#3117645 (10hashar) I have extracted the `nscd` part to a... [14:28:09] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations, 10media-storage, 10Patch-For-Review: nscd does not cache localhost causing high CPU usage when localhost is often resolved - https://phabricator.wikimedia.org/T171745#3474869 (10hashar) For #beta-cluster-infrastructure the... [14:35:17] (03CR) 10Hashar: [C: 032] "That has proven to fix things for us thus merging it in Gerrit. There is a PR filled upstream and we will catch up with whatever they come" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [14:35:27] (03CR) 10Hashar: [C: 032] "That has proven to fix things for us thus merging it in Gerrit. There is a PR filled upstream and we will catch up with whatever they come" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366484 (owner: 10Mholloway) [14:35:45] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [14:38:54] (03Merged) 10jenkins-bot: Update system image component name construction for APIs >= 25 [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [14:38:56] (03Merged) 10jenkins-bot: [Test] Stop forcing QEMU 1 use with the "-engine classic" flag [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366484 (owner: 10Mholloway) [14:51:43] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [15:03:32] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/worksp... - https://phabricator.wikimedia.org/T171724#3474977 [15:07:38] 10Release-Engineering-Team (Kanban), 10MobileFrontend, 10Reading-Web-Backlog, 10RelatedArticles, and 2 others: [EPIC] Port Selenium tests from Ruby to Node.js on Reading Web extensions - https://phabricator.wikimedia.org/T162256#3474983 (10zeljkofilipin) [15:07:40] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), and 2 others: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3474982 (10zeljkofilipin) 05Open>03Resolved [15:13:45] RainbowSprinkles upstream have now started to move to get rid of drafts (finally) :). Changes created through the webui will be wips. [15:14:17] so we will need to get the bot at some point to ignore wips otherwise there will be alot noice as wips are public [15:18:05] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Analytics-Kanban, 10Analytics-Wikistats: Set up continuous integration for wikistats 2.0 UI - https://phabricator.wikimedia.org/T170458#3475013 (10Milimetric) Just to keep the archives happy, yes, we're using Differential for code review. [15:20:56] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475026 (10Halfak) > but it's not undoable. What? We do github based deploys in our Cloud VPS cluster (not beta) using [fabric](http://w... [15:24:07] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Performance-Team, 10WebPageTest, 10Patch-For-Review: Where to trigger WebPageTest jobs? - https://phabricator.wikimedia.org/T166756#3475035 (10hashar) 05Open>03Resolved [15:25:29] (03PS4) 10Hashar: android: drop -no-audio [integration/config] - 10https://gerrit.wikimedia.org/r/366908 (https://phabricator.wikimedia.org/T150623) [15:25:48] (03PS2) 10Hashar: Bump Android periodic test to android-25 [integration/config] - 10https://gerrit.wikimedia.org/r/366965 (https://phabricator.wikimedia.org/T150623) [15:25:53] Well the gerrit urls now look really wierd. /me likes the polygerrit ones now as they keep the old format :) [15:26:10] /c//+/ [15:29:04] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475056 (10bd808) >>! In T171619#3475026, @Halfak wrote: > AFAICT, there's no good option for doing scap-based deploys in Cloud VPS (outside... [15:29:54] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10MinervaNeue, 10Operations, 10Reading-Web-Backlog: The mobile-frontend-placeholder message is not updated in din.wikipedia.org - https://phabricator.wikimedia.org/T171711#3475060 (10bmansurov) [15:31:45] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [15:37:23] 10Continuous-Integration-Infrastructure (Little Steps Sprint), 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: For MediaWiki extensions, merge rubocop inside mwext-mw-selenium-jessie - https://phabricator.wikimedia.org/T164479#3234955 (10zeljkofilipin) a:03zeljkofilipin [15:39:37] (03CR) 10Hashar: [C: 032] "refreshed the job and it works :)" [integration/config] - 10https://gerrit.wikimedia.org/r/366908 (https://phabricator.wikimedia.org/T150623) (owner: 10Hashar) [15:39:43] (03CR) 10Hashar: [C: 032] "refreshed the job and it works :)" [integration/config] - 10https://gerrit.wikimedia.org/r/366965 (https://phabricator.wikimedia.org/T150623) (owner: 10Hashar) [15:40:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3475076 (10hashar) The job is now running with android-25 and QEMU 2 which make it run tasks in p... [15:41:17] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3475077 (10hashar) 05Open>03Resolved [15:45:15] (03Merged) 10jenkins-bot: android: drop -no-audio [integration/config] - 10https://gerrit.wikimedia.org/r/366908 (https://phabricator.wikimedia.org/T150623) (owner: 10Hashar) [15:45:52] (03Merged) 10jenkins-bot: Bump Android periodic test to android-25 [integration/config] - 10https://gerrit.wikimedia.org/r/366965 (https://phabricator.wikimedia.org/T150623) (owner: 10Hashar) [15:46:16] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475091 (10Halfak) Gotcha. Maybe we could stick that in our `ores-staging` project. @demon, if that makes things easier for you, we can... [15:48:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Create a Jenkins job that runs Echo RSpec tests daily - https://phabricator.wikimedia.org/T171753#3475095 (10zeljkofilipin) [15:55:19] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog: apps-android-wikipedia-periodic-test takes a while to fetch screenshots - https://phabricator.wikimedia.org/T171754#3475126 (10hashar) [15:55:56] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: deployment-imagescaler01 has no mwdeploy user - https://phabricator.wikimedia.org/T166013#3475143 (10Dzahn) 05Resolved>03Open [15:58:53] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475148 (10Halfak) I talked to @Paladox -- who offered that we could make use of phab-tin. Would it be better to set up our own deployment... [15:59:05] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Release-Engineering-Team (Kanban), 10Patch-For-Review: deployment-imagescaler01 has no mwdeploy user - https://phabricator.wikimedia.org/T166013#3475151 (10Dzahn) 05Open>03Resolved [16:00:21] (03PS1) 10XZise: [FEAT] Rewrite commit message validator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/367907 [16:01:38] (03CR) 10XZise: "This is a reimplementation/rebase of I788ae0d1 for the new repository." [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/367907 (owner: 10XZise) [16:02:08] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog: apps-android-wikipedia-periodic-test takes a while to fetch screenshots - https://phabricator.wikimedia.org/T171754#3475174 (10hashar) [16:02:56] (03Abandoned) 10XZise: [FEAT] Rewrite commit message validator [integration/jenkins] - 10https://gerrit.wikimedia.org/r/247056 (owner: 10XZise) [16:04:46] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog: apps-android-wikipedia-periodic-test takes a while to fetch screenshots - https://phabricator.wikimedia.org/T171754#3475178 (10hashar) [16:06:18] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog: apps-android-wikipedia-periodic-test takes a while to fetch screenshots - https://phabricator.wikimedia.org/T171754#3475126 (10hashar) The Facebook repo defines a `pullScreenshotsFromDirectory` maybe it wo... [16:14:48] (03PS2) 10XZise: [FEAT] Rewrite commit message validator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/367907 [16:15:34] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:21:03] 10Release-Engineering-Team, 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475205 (10awight) @demon We're fine with deploying from WMF production repos, I'm sure we can figure something out to push mirrored code or... [16:28:08] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<44.44%) [16:30:57] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10MinervaNeue, 10Operations, 10Reading-Web-Backlog (Tracking): The mobile-frontend-placeholder message is not updated in din.wikipedia.org - https://phabricator.wikimedia.org/T171711#3475232 (10Jdlrobson) [16:33:36] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475235 (10greg) (Just doing some project management, don't worry about the "watching" bit, we'll just create a (sub)t... [16:35:39] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: The mobile-frontend-placeholder message is not updated in din.wikipedia.org - https://phabricator.wikimedia.org/T171711#3475238 (10Amire80) Removing reading tags. It's most likely an ops issue with LU. [16:40:05] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475257 (10greg) [16:40:13] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3473889 (10greg) p:05Triage>03High [16:40:45] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Services (next), 10User-Joe: puppet dependency loop on deployment-sca hosts - https://phabricator.wikimedia.org/T171173#3475262 (10greg) [16:41:46] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3473889 (10Dzahn) likely caused by https://gerrit.wikimedia.org/r/#/c/255958/ follow-up to fix it was uploa... [16:43:52] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475273 (10greg) >>! In T171711#3475263, @Dzahn wrote: > likely caused by https://gerrit.wikimedia.org/r/#/c... [16:45:23] 10Release-Engineering-Team, 10MediaWiki-extensions-LocalisationUpdate, 10Operations: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475277 (10greg) a:03Reedy [16:45:38] (03PS6) 10MacFan4000: rm DeepSea [integration/config] - 10https://gerrit.wikimedia.org/r/367041 (https://phabricator.wikimedia.org/T171385) [16:45:41] 10Scap, 10MediaWiki-extensions-LocalisationUpdate: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3473889 (10greg) [16:49:21] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: ORES should use git-fat for wheel deployments - https://phabricator.wikimedia.org/T171619#3475303 (10demon) >>! In T171619#3475026, @Halfak wrote: >> but it's not undoable. > > What? > Bad choice of word... [16:51:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3475309 (10hashar) >>! In T150623#3474658, @Mholloway wrote: > Screenshots need updating again as... [16:52:15] 10Scap, 10MediaWiki-extensions-LocalisationUpdate: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475311 (10Reedy) I'm amused it took a week for people to notice. This is always the case... [16:52:41] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Git-fat should support pulling from production or labs - https://phabricator.wikimedia.org/T171758#3475312 (10awight) [16:53:43] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/worksp... - https://phabricator.wikimedia.org/T171724#3475326 [16:55:37] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:55:37] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Git-fat should support pulling from production or labs - https://phabricator.wikimedia.org/T171758#3475334 (10demon) Just wanting to clear up a little bit of FUD, since there seems to be this misconception th... [16:56:50] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Git-fat should support pulling from production or labs - https://phabricator.wikimedia.org/T171758#3475335 (10Halfak) What does *better* look like? What would you do if you had the time to do it? [16:59:22] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475357 (10awight) [16:59:48] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475312 (10awight) @demon Thanks! I've updated the description. [17:00:18] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Scoring-platform-team-Backlog: Simplify git-fat support for pulling from both production and labs - https://phabricator.wikimedia.org/T171758#3475365 (10demon) The idea I was noodling the other day was kind of a generic rsync service that... [17:03:13] 10Scap, 10MediaWiki-extensions-LocalisationUpdate: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475383 (10Amire80) >>! In T171711#3475311, @Reedy wrote: > I'm amused it took a week for people to notice. This is always the case... Sorry, n... [17:13:46] Project beta-code-update-eqiad build #165647: 04FAILURE in 45 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/165647/ [17:23:48] Yippee, build fixed! [17:23:49] Project beta-code-update-eqiad build #165648: 09FIXED in 47 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/165648/ [17:40:04] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [18:06:22] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [18:07:29] 10Continuous-Integration-Config, 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: Android SDK is suddenly failing to auto-install, blocking tests from being executed - https://phabricator.wikimedia.org/T171562#3475614 (10Mholloway) Works as expected with 26.0.0 again... https://gerrit.wikimedia.org/r/#/... [18:09:03] 10Scap, 10MediaWiki-extensions-LocalisationUpdate: l10nupdate failing with "git pull of extensions failed" since July 19th - https://phabricator.wikimedia.org/T171711#3475616 (10greg) >>! In T171711#3475383, @Amire80 wrote: > Noticing a failure of a command in the log, on the other hand, should ideally be noti... [18:58:13] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: rebuildTermSqlIndex.php causing beta-update-databases-eqiad to timeout and abort - https://phabricator.wikimedia.org/T168036#3353770 (10greg) >>! In T168036#3374325, @WMDE-leszek wrote: > @Reedy sorry for being silent here... [19:08:22] aude: Heya, about? Curious what the status is on T171370 / https://gerrit.wikimedia.org/r/#/c/367328/ [19:08:23] T171370: ERROR: "LBFactory::getEmptyTransactionTicket: WikiPageUpdater::injectRCRecords does not have outer scope" - https://phabricator.wikimedia.org/T171370 [19:11:30] Yippee, build fixed! [19:11:30] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #23: 09FIXED in 22 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/23/ [19:13:44] 10Release-Engineering-Team (Kanban), 10Wikimedia-Developer-Summit, 10User-greg: Facilitate Wikidev'17 main topic "How to manage our technical debt" - https://phabricator.wikimedia.org/T147937#3475821 (10greg) [19:29:49] 10MediaWiki-Codesniffer: Undefined offset: 1 on codesniffer FunctionCommentSniff - https://phabricator.wikimedia.org/T171776#3475940 (10Umherirrender) [19:50:52] oh wow polygerrit has a emojie selector now. [19:53:27] paladox: 👍🏾 [19:53:32] :) [19:54:32] (though that's in master not on stable-2.14) [20:04:22] RainbowSprinkles https://groups.google.com/forum/#!topic/repo-discuss/V9IgnEDKQE8 heh [20:04:28] 3.x will remove reviewdb code [20:08:19] * RainbowSprinkles yawns [20:15:42] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [20:26:19] 10Release-Engineering-Team, 10Africa-Wikimedia-Developers, 10Composer: Setup a Composer Repository (Packagist) for MediaWiki Extensions - https://phabricator.wikimedia.org/T170897#3476102 (10hashar) 05Open>03declined I found this task randomly. We had a request for comment and talked about using compose... [20:29:07] damn they propose polygerrit as the default ui for gerrit 2.15. Luckly there's configs to make gwtui the default ui. [20:33:26] * paladox starts the disccusion on opposing polygerrit as default ui in 2.15 until it gains inline edit :) [20:42:57] Project selenium-RelatedArticles » chrome,beta-desktop,Linux,BrowserTests build #480: 04FAILURE in 2.6 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-desktop,PLATFORM=Linux,label=BrowserTests/480/ [20:42:58] Project selenium-RelatedArticles » chrome,beta-mobile,Linux,BrowserTests build #480: 04FAILURE in 2.1 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-mobile,PLATFORM=Linux,label=BrowserTests/480/ [20:43:49] 10Release-Engineering-Team (Kanban), 10MobileFrontend, 10Reading-Web-Backlog, 10RelatedArticles, and 2 others: [EPIC] Port Selenium tests from Ruby to Node.js on Reading Web extensions - https://phabricator.wikimedia.org/T162256#3476118 (10Jdlrobson) [20:43:51] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), and 2 others: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3476116 (10Jdlrobson) 05Resolved>03Open https://integr... [20:46:32] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [20:50:43] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [21:03:01] 10Beta-Cluster-Infrastructure: hhvm error logs on beta showing no errors when cluster down - https://phabricator.wikimedia.org/T151171#3476235 (10greg) [21:04:27] oh wow the repo is going to be huge with notedb https://gerrit.googlesource.com/homepage/+/md-pages/docs/Notedb.md [21:07:03] 10MediaWiki-Codesniffer: Provide Codesniffer rules to enforce "short" type definitions (int/bool, not integer/boolean) - https://phabricator.wikimedia.org/T145162#3476250 (10Umherirrender) The sniff should also check case insensitive and with a colon after it, have seen "@return Boolean: Some comment" in extensi... [21:26:34] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0] [21:26:54] 10Beta-Cluster-Infrastructure: hhvm error logs on beta showing no errors when cluster down - https://phabricator.wikimedia.org/T151171#3476267 (10greg) 05stalled>03Invalid Closing, feel free to reopen if still valid/more information is provided. [21:27:19] is it normal, that deployment-tin has a process of update.php for two wikis which have CPU time > 14hs? [21:33:01] Wouldn't be the first time [21:33:14] Some badly coded maintenance script [21:34:32] Mmm [21:34:37] I think I'll just kill them [21:34:39] /srv/mediawiki-staging/multiversion/MWScript.php update.php --wiki=cawiki --quick [21:34:41] didn't take long [21:35:14] !log kill two long running update.php jobs on deployment-tin [21:35:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:36:19] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #24: 04FAILURE in 48 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/24/ [21:39:45] Reedy: thx :) [21:41:00] looks like the load is decreasing too [21:41:21] Sagan: Oh, I bet I know [21:41:34] I bet those were spawned when PopulatePPSortKey was buggy [21:41:41] So they got stuck in an infinite loop [21:41:52] (y) :D [21:42:35] when I look at nagf, looks like there was this kind of higher load was there since 12 AM [21:49:07] Project selenium-MinervaNeue » firefox,beta,Linux,BrowserTests build #24: 04FAILURE in 1 hr 1 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/24/ [22:04:55] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10MathSearch: mwext-testextension-zend should load extension mathsearch after math - https://phabricator.wikimedia.org/T117659#3476311 (10greg) [22:05:02] 10Continuous-Integration-Config, 10MathSearch: mwext-testextension-zend should load extension mathsearch after math - https://phabricator.wikimedia.org/T117659#3476313 (10greg) a:05hashar>03None [22:55:14] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), 10User-zeljkofilipin: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3476505 (10Jdlrobson) [22:55:41] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), and 2 others: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3218631 (10Jdlrobson) [22:55:46] 10Release-Engineering-Team (Kanban), 10Reading-Web-Backlog, 10RelatedArticles, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), 10User-zeljkofilipin: Rewrite Related pages browser tests in Node.js - https://phabricator.wikimedia.org/T164024#3218631 (10Jdlrobson) [22:55:55] 10Continuous-Integration-Infrastructure: Jobs with Node 6 should also have npm 3 - https://phabricator.wikimedia.org/T161861#3476509 (10Jdforrester-WMF) How hard would it be to get a modern version of `npm` in CI? [22:58:28] Krinkle: Given that I'm running node 7/npm 5 I'm just going to have endless issues with dirtied repos and complaints, I guess. :-( [23:03:24] James_F: Could add it to your global gitignore (~/.config/git/ignore) [23:03:52] https://github.com/Krinkle/dotfiles/blob/master/hosts/KrinkleMac/config/git/ignore [23:03:54] Only works if everyone like me only works on WMF projects. [23:04:25] James_F: Well, you can still git-add it anywhere as needed. And git status still complains about changes to tracked files [23:04:34] it just means you won't get dirty about a new untracked file anymore. [23:04:54] Which.. is exactly what you want, except when creating a new git repo maybe. [23:05:00] Yeah. [23:05:20] We need a WMF-wide all-repos shared .gitignore somehow.