[00:09:16] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:44:15] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [00:53:47] PROBLEM - puppet last run on contint1001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [01:10:18] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [01:22:57] RECOVERY - puppet last run on contint1001 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [01:45:16] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [03:23:42] 10Continuous-Integration-Config, 10VisualEditor, 10VisualEditor-ContentEditable, 10Patch-For-Review: VisualEditor IME test failing in CI on pull-through, but not locally or in the source repo? - https://phabricator.wikimedia.org/T176453#3630671 (10dchan) As I've noted in gerrit, the Filibuster dump at patc... [03:36:17] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [04:02:33] PROBLEM - Puppet errors on deployment-memc05 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [04:16:17] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [04:37:33] RECOVERY - Puppet errors on deployment-memc05 is OK: OK: Less than 1.00% above the threshold [0.0] [05:37:16] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [05:45:49] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [10.0] [06:57:18] PROBLEM - Puppet errors on deployment-kafka01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [07:12:18] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [07:45:34] 10Gerrit: Switch to mariadb java connector once we upgrade to gerrit 2.14 - https://phabricator.wikimedia.org/T176164#3630790 (10Marostegui) [07:45:36] (03CR) 10Hashar: "We no more have any trusty instances/images. I guess we can have a php55 docker images nowadays?" [integration/config] - 10https://gerrit.wikimedia.org/r/347130 (owner: 10Thcipriani) [07:45:49] 10Gerrit: Switch to mariadb java connector once we upgrade to gerrit 2.14 - https://phabricator.wikimedia.org/T176164#3615775 (10Marostegui) Removed the DBA project, but will remain subscribed just in case something is needed from us. [07:48:04] (03CR) 10Hashar: [C: 032] "https://gerrit.wikimedia.org/r/#/c/377529/ got merged" [integration/config] - 10https://gerrit.wikimedia.org/r/379875 (https://phabricator.wikimedia.org/T174972) (owner: 10Hashar) [07:48:20] 10Continuous-Integration-Infrastructure (phase-out-trusty), 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Package php modules for Zend 5.5 on Jessie - https://phabricator.wikimedia.org/T174972#3630794 (10hashar) 05Open>03Resolved \O/ [07:48:22] 10Continuous-Integration-Infrastructure (phase-out-trusty), 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Migrate PHP5.5 jobs from Trusty to Jessie - https://phabricator.wikimedia.org/T161882#3630796 (10hashar) [07:49:08] (03Merged) 10jenkins-bot: dib: php5.5 packages are now in contint::packages::php [integration/config] - 10https://gerrit.wikimedia.org/r/379875 (https://phabricator.wikimedia.org/T174972) (owner: 10Hashar) [07:51:22] (03CR) 10Hashar: [C: 032] Make unit tests voting [integration/config] - 10https://gerrit.wikimedia.org/r/379776 (owner: 10Umherirrender) [07:52:36] (03CR) 10Hashar: [C: 04-1] PoC: Port existing Trusty jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/347130 (owner: 10Thcipriani) [07:52:57] (03Merged) 10jenkins-bot: Make unit tests voting [integration/config] - 10https://gerrit.wikimedia.org/r/379776 (owner: 10Umherirrender) [07:54:50] (03CR) 10Hashar: [C: 032] dockerfile: for phan do not run composer as root [integration/config] - 10https://gerrit.wikimedia.org/r/379303 (owner: 10Hashar) [07:56:23] (03Merged) 10jenkins-bot: dockerfile: for phan do not run composer as root [integration/config] - 10https://gerrit.wikimedia.org/r/379303 (owner: 10Hashar) [07:56:28] (03Abandoned) 10Hashar: dib: produce Docker image [integration/config] - 10https://gerrit.wikimedia.org/r/375018 (owner: 10Hashar) [07:57:00] (03CR) 10Hashar: [C: 032] Archive Extension:WikiTwidget [integration/config] - 10https://gerrit.wikimedia.org/r/379754 (https://phabricator.wikimedia.org/T148826) (owner: 10MarcoAurelio) [07:58:06] (03Merged) 10jenkins-bot: Archive Extension:WikiTwidget [integration/config] - 10https://gerrit.wikimedia.org/r/379754 (https://phabricator.wikimedia.org/T148826) (owner: 10MarcoAurelio) [08:15:50] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [09:02:03] PROBLEM - Puppet errors on deployment-trending01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [09:12:14] !log added deployment-mediawiki07 to deployment-prep (stretch-based app server, WIP) [09:12:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:14:34] 10Continuous-Integration-Config, 10VisualEditor, 10VisualEditor-ContentEditable, 10Patch-For-Review: VisualEditor IME test failing in CI on pull-through, but not locally or in the source repo? - https://phabricator.wikimedia.org/T176453#3630935 (10dchan) OK, I have tested a rewritten version of ve.ce.Test... [09:34:59] PROBLEM - Puppet errors on deployment-mediawiki07 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [09:37:21] 10Continuous-Integration-Config, 10VisualEditor, 10VisualEditor-ContentEditable, 10Patch-For-Review: VisualEditor IME test failing in CI on pull-through, but not locally or in the source repo? - https://phabricator.wikimedia.org/T176453#3630973 (10dchan) a:03dchan [09:38:18] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [09:54:58] RECOVERY - Puppet errors on deployment-mediawiki07 is OK: OK: Less than 1.00% above the threshold [0.0] [10:04:23] (03PS1) 10Hashar: dib: use a profile to install arcanist [integration/config] - 10https://gerrit.wikimedia.org/r/380458 [10:04:39] (03CR) 10Hashar: "Depends on puppet patch https://gerrit.wikimedia.org/r/#/c/379726/" [integration/config] - 10https://gerrit.wikimedia.org/r/380458 (owner: 10Hashar) [10:07:25] (03PS1) 10Hashar: dib: use a profile to install zuul [integration/config] - 10https://gerrit.wikimedia.org/r/380460 [10:17:07] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [10:32:44] (03PS1) 10Hashar: dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 [10:43:17] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [10:43:59] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins, 10User-MarcoAurelio: Deprecate unmaintained/inactive WikiTwidget extension - https://phabricator.wikimedia.org/T148826#3631174 (10MarcoAurelio) 05Open>03Resolved Now really done. [10:44:30] 10Release-Engineering-Team (Kanban), 10Cleanup, 10Repository-Admins, 10User-MarcoAurelio: Deprecate unmaintained/inactive WikiTwidget extension - https://phabricator.wikimedia.org/T148826#3631181 (10MarcoAurelio) [10:52:08] RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [10:53:20] (03PS1) 10Hashar: build: bump puppet to 3.8.5 [integration/config] - 10https://gerrit.wikimedia.org/r/380478 [11:02:32] (03CR) 10Hashar: [C: 032] build: bump puppet to 3.8.5 [integration/config] - 10https://gerrit.wikimedia.org/r/380478 (owner: 10Hashar) [11:03:37] (03Merged) 10jenkins-bot: build: bump puppet to 3.8.5 [integration/config] - 10https://gerrit.wikimedia.org/r/380478 (owner: 10Hashar) [11:07:37] (03PS1) 10Hashar: build: merge rake and tox-dib jobs [integration/config] - 10https://gerrit.wikimedia.org/r/380481 [11:12:11] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:14:12] 10Release-Engineering-Team (Kanban), 10Cleanup, 10Repository-Admins, 10User-MarcoAurelio: Deprecate unmaintained/inactive WikiTwidget extension - https://phabricator.wikimedia.org/T148826#3631289 (10Aklapper) Thanks!! [11:17:59] (03CR) 10Hashar: [C: 032] build: merge rake and tox-dib jobs [integration/config] - 10https://gerrit.wikimedia.org/r/380481 (owner: 10Hashar) [11:20:04] (03Merged) 10jenkins-bot: build: merge rake and tox-dib jobs [integration/config] - 10https://gerrit.wikimedia.org/r/380481 (owner: 10Hashar) [11:20:11] JFTR, deployment-mediawiki07 might see puppet failures for a while, it'll take a bit until all those puppet manifests are adapted to stretch [11:20:26] (03PS1) 10Hashar: dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 [11:21:34] (03CR) 10jerkins-bot: [V: 04-1] dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 (owner: 10Hashar) [11:24:52] (03PS2) 10Hashar: dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 [11:24:54] (03PS1) 10Hashar: fix integration-config-dib-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/380486 [11:27:41] (03CR) 10jerkins-bot: [V: 04-1] dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 (owner: 10Hashar) [11:33:54] (03CR) 10Hashar: [C: 032] fix integration-config-dib-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/380486 (owner: 10Hashar) [11:35:37] (03Merged) 10jenkins-bot: fix integration-config-dib-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/380486 (owner: 10Hashar) [11:36:58] (03PS3) 10Hashar: dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 [11:37:00] (03PS1) 10Hashar: integration-config-dib-jessie was not cloning the repo [integration/config] - 10https://gerrit.wikimedia.org/r/380488 [11:37:59] PROBLEM - App Server Main HTTP Response on deployment-mediawiki07 is CRITICAL: HTTP CRITICAL: HTTP/1.1 404 Not Found - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 392 bytes in 0.008 second response time [11:38:01] (03CR) 10jerkins-bot: [V: 04-1] dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 (owner: 10Hashar) [11:45:58] PROBLEM - Puppet errors on deployment-mediawiki07 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [11:48:09] (03CR) 10Hashar: [C: 032] integration-config-dib-jessie was not cloning the repo [integration/config] - 10https://gerrit.wikimedia.org/r/380488 (owner: 10Hashar) [11:50:03] (03PS4) 10Hashar: dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 [11:50:25] (03Merged) 10jenkins-bot: integration-config-dib-jessie was not cloning the repo [integration/config] - 10https://gerrit.wikimedia.org/r/380488 (owner: 10Hashar) [11:50:53] (03PS2) 10Addshore: Add all wikidata build exts to make-wmf-branch [tools/release] - 10https://gerrit.wikimedia.org/r/378771 (https://phabricator.wikimedia.org/T173940) [11:53:20] (03CR) 10Hashar: [C: 032] dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 (owner: 10Hashar) [11:57:23] (03Merged) 10jenkins-bot: dib: add puppet-lint [integration/config] - 10https://gerrit.wikimedia.org/r/380484 (owner: 10Hashar) [12:27:30] (03CR) 10Addshore: [C: 032] Refactor 'operations-puppet-tests-docker' into macros for easy reuse [integration/config] - 10https://gerrit.wikimedia.org/r/379959 (owner: 10Legoktm) [12:28:42] (03Merged) 10jenkins-bot: Refactor 'operations-puppet-tests-docker' into macros for easy reuse [integration/config] - 10https://gerrit.wikimedia.org/r/379959 (owner: 10Legoktm) [12:30:28] !log Reloading Zuul to deploy Refactor 'operations-puppet-tests-docker' into macros for easy reuse [integration/config] - https://gerrit.wikimedia.org/r/379959 [12:30:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:32:25] (03PS4) 10Addshore: fabric: Add command to pull docker image on all hosts [integration/config] - 10https://gerrit.wikimedia.org/r/378783 [12:32:29] (03PS5) 10Addshore: fabric: Add command to pull docker image on all hosts [integration/config] - 10https://gerrit.wikimedia.org/r/378783 [12:36:38] !log addshore@integration-saltmaster:~$ sudo salt -v '*slave-docker*' cmd.run 'sudo docker rmi wmfreleng/operations-puppet:0.0.1 wmfreleng/operations-puppet:0.1.0' [12:36:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:41:19] 10Continuous-Integration-Infrastructure (shipyard): Some docker slave still have old containers using old images - https://phabricator.wikimedia.org/T176623#3631625 (10Addshore) [13:09:19] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [13:12:39] PROBLEM - Puppet errors on integration-puppetmaster01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [13:36:40] (03PS1) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [13:38:41] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [13:41:39] (03PS2) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [13:43:14] (03PS1) 10Hashar: Expand JJB macros in conditional-step [integration/jenkins-job-builder] - 10https://gerrit.wikimedia.org/r/380504 [13:43:44] (03Abandoned) 10Hashar: Expand JJB macros in conditional-step [integration/jenkins-job-builder] - 10https://gerrit.wikimedia.org/r/380504 (owner: 10Hashar) [13:43:51] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [13:44:20] (03PS3) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [13:47:00] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [13:47:32] Yippee, build fixed! [13:47:35] Project selenium-VisualEditor » firefox,beta,Linux,BrowserTests build #533: 09FIXED in 3 min 30 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/533/ [13:52:38] RECOVERY - Puppet errors on integration-puppetmaster01 is OK: OK: Less than 1.00% above the threshold [0.0] [13:56:27] (03PS4) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [14:04:33] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [14:14:16] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [14:29:42] 10Release-Engineering-Team, 10MobileFrontend, 10Operations, 10Readers-Web-Backlog (Tracking): Diff page produces 503 on first visit - https://phabricator.wikimedia.org/T176637#3632060 (10Jdlrobson) [14:31:59] 10Release-Engineering-Team, 10MobileFrontend, 10Operations, 10Readers-Web-Backlog (Tracking): Diff page produces 503 on first visit - https://phabricator.wikimedia.org/T176637#3632079 (10Jdlrobson) [14:41:49] 10Release-Engineering-Team, 10Readers-Web-Backlog, 10RelatedArticles, 10Browser-Tests, and 2 others: Automated browser tests cannot create pages on the Beta Cluster as anonymous user in RelatedArticles tests - https://phabricator.wikimedia.org/T176315#3632103 (10Jdlrobson) @phuedx this requires expertise o... [15:00:28] (03PS5) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [15:02:21] (03PS2) 10Hashar: dib: use a profile to install arcanist [integration/config] - 10https://gerrit.wikimedia.org/r/380458 [15:05:12] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Developer-Relations (Jul-Sep 2017), 10User-zeljkofilipin: WebdriverIO tech talk - https://phabricator.wikimedia.org/T171852#3632209 (10zeljkofilipin) Started converting wdio e-mails to blog posts: - [[ https://phabricator.wikimedia.org... [15:09:15] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Ruby, 10User-zeljkofilipin: Announce Selenium Ruby framework deprecation on appropriate mailing lists (QA, engineering, wikitech-l) - https://phabricator.wikimedia.org/T173488#3632243 (10zeljkofilipin) Started converting wdio e-mails to... [15:09:50] Yippee, build fixed! [15:09:50] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #137: 09FIXED in 18 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/137/ [15:10:40] 10Continuous-Integration-Config, 10VisualEditor, 10VisualEditor-ContentEditable, 10Patch-For-Review, 10User-Ryasmeen: VisualEditor IME test failing in CI on pull-through, but not locally or in the source repo? - https://phabricator.wikimedia.org/T176453#3632259 (10Jdforrester-WMF) 05Open>03Resolved [15:13:17] (03PS2) 10Hashar: dib: use a profile to install zuul [integration/config] - 10https://gerrit.wikimedia.org/r/380460 [15:13:18] (03PS2) 10Hashar: dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 [15:14:48] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [15:19:23] (03CR) 10jerkins-bot: [V: 04-1] dib: use a profile to install arcanist [integration/config] - 10https://gerrit.wikimedia.org/r/380458 (owner: 10Hashar) [15:21:53] :( [15:25:43] Project selenium-MobileFrontend » firefox,beta,Linux,BrowserTests build #574: 04FAILURE in 3 min 43 sec: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/574/ [15:27:46] (03CR) 10jerkins-bot: [V: 04-1] dib: use a profile to install zuul [integration/config] - 10https://gerrit.wikimedia.org/r/380460 (owner: 10Hashar) [15:28:50] (03CR) 10jerkins-bot: [V: 04-1] dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 (owner: 10Hashar) [15:34:04] PROBLEM - Free space - all mounts on deployment-kafka01 is CRITICAL: CRITICAL: deployment-prep.deployment-kafka01.diskspace.root.byte_percentfree (<100.00%) [15:37:35] 10Release-Engineering-Team, 10Readers-Web-Backlog, 10RelatedArticles, 10Browser-Tests, and 2 others: Automated browser tests cannot create pages on the Beta Cluster as anonymous user in RelatedArticles tests - https://phabricator.wikimedia.org/T176315#3632361 (10phuedx) 👍 Just so long as there's an explana... [15:39:41] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10Ruby, 10User-zeljkofilipin: Announce Selenium Ruby framework deprecation on appropriate mailing lists (QA, engineering, wikitech-l) - https://phabricator.wikimedia.org/T173488#3632364 (10zeljkofilipin) September reminder: https://phabri... [15:44:51] (03PS6) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [15:47:57] (03Abandoned) 10Thcipriani: PoC: Port existing Trusty jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/347130 (owner: 10Thcipriani) [15:55:17] Yippee, build fixed! [15:55:17] Project selenium-MinervaNeue » firefox,beta,Linux,BrowserTests build #138: 09FIXED in 28 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/138/ [16:14:13] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T174358#3632448 (10mmodell) a:05demon>03mmodell [16:14:15] (03CR) 10jerkins-bot: [V: 04-1] dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 (owner: 10Hashar) [16:16:34] (03PS7) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [16:16:36] (03PS1) 10Hashar: Capture tox logs via a full tree traversal [integration/config] - 10https://gerrit.wikimedia.org/r/380542 [16:16:37] 10Gerrit, 10MediaWiki-Vagrant, 10Patch-For-Review: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#3632479 (10bd808) >>! In T152801#3628733, @Byrnedj12 wrote: > 2) in support/Vagrantfile-extra.rb: > uncomment lines 19-20 > uncomment line 28, set the m... [16:33:01] 10Deployment-Systems, 10Release-Engineering-Team (Kanban): Automate the recurring management of wikitech:Deployments and phab:#train_deployments - https://phabricator.wikimedia.org/T114488#3632528 (10mmodell) No progress per-se as I was shaving yaks all week. [16:38:01] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Next), 10User-zeljkofilipin: Add createAccount method to nodemw - https://phabricator.wikimedia.org/T173505#3632542 (10greg) [16:38:05] 10Release-Engineering-Team (Next), 10VisualEditor, 10User-Ryasmeen, 10User-zeljkofilipin: LanguageScreenshotBot trying to edit a non-existent page without signing in - https://phabricator.wikimedia.org/T162454#3632544 (10greg) [16:39:20] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/worksp... - https://phabricator.wikimedia.org/T171724#3632546 [16:39:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Next): Figure out the package conflict between libtidy-dev from sury and hhvm-tidy - https://phabricator.wikimedia.org/T169008#3632548 (10greg) [16:39:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2017, 10Wikimedia-Logstash, and 2 others: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#3632549 (10greg) [16:42:08] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Wikidata-Sprint: Trigger Jenkins jobs on patchset uploaded by the user chrisneuroth - https://phabricator.wikimedia.org/T173831#3632555 (10greg) 05Open>03Resolved a:03hashar [16:43:44] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.30.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T170637#3632562 (10demon) 05Open>03Resolved [16:45:20] 10Continuous-Integration-Infrastructure (phase-out-trusty), 10Release-Engineering-Team (Kanban), 10Patch-For-Review: Remove trusty from Nodepool, WMCS, and clean out puppet - https://phabricator.wikimedia.org/T175696#3632570 (10hashar) 05Open>03Resolved [16:48:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10DNS, 10Operations, and 2 others: CI: operations-dns-lint broken due to missing Maxmind DB file - https://phabricator.wikimedia.org/T175864#3632579 (10greg) a:05hashar>03None Just waiting on puppet merge, unassigning antoine. [16:52:07] 10Release-Engineering-Team (Watching / External), 10Contributors-Team, 10MobileFrontend, 10Operations, and 2 others: Diff page produces 503 on first visit - https://phabricator.wikimedia.org/T176637#3632602 (10greg) Per Dev/Maintainers adding the Contributors team as maintainers of the diff view. [16:52:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10DNS, 10Operations, and 2 others: CI: operations-dns-lint broken due to missing Maxmind DB file - https://phabricator.wikimedia.org/T175864#3632606 (10Dzahn) I would have merged https://gerrit.wikimedia.org/r/#/c/377986/ but it... [16:52:35] 10Gerrit, 10Release-Engineering-Team (Next), 10Operations: Gerrit is failing to start gerrit-ssh on gerrit2001 - https://phabricator.wikimedia.org/T176532#3632607 (10greg) [16:53:45] 10Release-Engineering-Team (Next), 10Readers-Web-Backlog, 10RelatedArticles, 10Browser-Tests, and 2 others: Automated browser tests cannot create pages on the Beta Cluster as anonymous user in RelatedArticles tests - https://phabricator.wikimedia.org/T176315#3632617 (10greg) [16:58:54] CI seems starved of jessie nodes again. :-( [16:58:57] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Next), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspac... - https://phabricator.wikimedia.org/T171724#3632628 [16:59:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Next), 10Wikimedia-Hackathon-2017, 10Wikimedia-Logstash, and 2 others: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#3632630 (10greg) a:05hashar>03None [17:03:32] (03CR) 10Hashar: dockerfile: port build script to python (037 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/379775 (owner: 10Hashar) [17:03:41] (03PS2) 10Hashar: dockerfile: port build script to python [integration/config] - 10https://gerrit.wikimedia.org/r/379775 [17:03:58] thcipriani: thanks for the friday review of the dockerfile/build.py port [17:04:08] I missed the email notification :D [17:05:22] (03CR) 10Hashar: "I renamed a few variables here and there, introduced a few more. I tried to disambiguate what is a "dockerfile" which really was used f" [integration/config] - 10https://gerrit.wikimedia.org/r/379775 (owner: 10Hashar) [17:06:13] (03PS3) 10Hashar: dib: use a profile to install arcanist [integration/config] - 10https://gerrit.wikimedia.org/r/380458 [17:06:18] (03PS3) 10Hashar: dib: use a profile to install zuul [integration/config] - 10https://gerrit.wikimedia.org/r/380460 [17:06:23] (03PS3) 10Hashar: dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 [17:07:09] !log Greg is now a contint-admin [17:07:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:07:30] (03PS1) 10Dduvall: WIP service pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/380551 [17:08:34] mutante: thanks :) [17:09:23] yw :) [17:09:29] (03PS8) 10Hashar: dib: add some spec test [integration/config] - 10https://gerrit.wikimedia.org/r/380502 [17:10:17] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [17:11:09] (03PS3) 10Addshore: Low prio queue for libraryupdater [integration/config] - 10https://gerrit.wikimedia.org/r/380307 [17:11:22] (03PS1) 10Hashar: Remove .* from .gitignore [integration/config] - 10https://gerrit.wikimedia.org/r/380553 [17:11:29] hashar: ^^ thoughts? :D [17:11:54] (03CR) 10Addshore: [C: 031] Remove .* from .gitignore [integration/config] - 10https://gerrit.wikimedia.org/r/380553 (owner: 10Hashar) [17:12:16] addshore: TS;HR: it is missing a bunch of context :] [17:12:24] (Too Short; Havent Reviewed) [17:12:58] what is the libraryupdater? If that is to mass spam stuff to composer.json/package.json , yeah that sounds like a good idea [17:12:59] :D [17:13:07] hashar: yes, thats the one [17:13:20] and whenever it runs it blocks up the test queue for about 5+ hours [17:13:26] though there is a way to send for review and directly +2 [17:13:33] chad might know [17:13:54] I dont think we want to do that for it really, they should probably have reviews [17:13:59] legoktm: might have another opinion? [17:18:07] (03CR) 10Hashar: [C: 031] "You would have to duplicate every tests pipelines in Zuul :( But since the precedence can only be set per pipeline ..." [integration/config] - 10https://gerrit.wikimedia.org/r/380307 (owner: 10Addshore) [17:18:26] dinner time! [17:18:50] 10Gerrit, 10Jenkins: Jenkins-bot lating with adding verifies in patches on gerrit - https://phabricator.wikimedia.org/T176546#3632737 (10greg) 05Open>03Invalid There's a misunderstanding of what those numbers mean. The time it takes to actually run the tests is only one part of the total time until the te... [17:18:58] addshore: also the dockerfile build script is ported to python via https://gerrit.wikimedia.org/r/#/c/379775/ [17:19:04] addshore: might want to try it out :] [17:19:37] (03CR) 10Addshore: "I think the gate-submit pipeline should be 'okay'." [integration/config] - 10https://gerrit.wikimedia.org/r/380307 (owner: 10Addshore) [17:20:18] hasharDinner: I will, I saw but was waiting for you to finish the comments :D [17:20:25] ;] [17:20:42] and today I have missed Tyler review so the amendment comes late to the party [17:20:49] out for real [17:24:06] tata! [17:24:43] 10Gerrit, 10Jenkins: Jenkins-bot lating with adding verifies in patches on gerrit - https://phabricator.wikimedia.org/T176546#3632772 (10Zoranzoki21) >>! In T176546#3632737, @greg wrote: > There's a misunderstanding of what those numbers mean. > > The time it takes to actually run the tests is only one part o... [17:27:01] addshore, hashar: bumps that only touch composer.json are auto +2d. Everything else needs to be reviewed by a human. I'm okay putting it in a low priority queue, but I already time it for weekend nights to avoid impacting people [17:27:55] Still impacts silly people like Amir1 and I that do stuff on the weekend :p [17:28:26] legoktm: are they instantly +2ed or does it wait for test to run? Or does it literally just submit? [17:29:23] Instantly +2 so it goes into gate and submit right away but it still ends up in the test queue regardless [17:34:49] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Parser, 10Readers-Web-Backlog (Tracking): Templates rendering as links on beta cluster - https://phabricator.wikimedia.org/T173576#3533588 (10demon) Yes, it's controlled through a combination of several settings: final page size, expans... [17:47:59] (03CR) 10jerkins-bot: [V: 04-1] dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 (owner: 10Hashar) [18:15:17] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [18:36:15] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [18:51:08] no_justification: TIL ['a' 'b', 'c'] is valid Python [18:51:18] returns ['ab', 'c'] [18:51:40] Yes [18:51:46] So did I [18:52:07] I was getting `rm-fR` as my resulting command [18:57:11] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Operations: Reimage gerrit2001 as stretch - https://phabricator.wikimedia.org/T168562#3633014 (10Dzahn) a:03Dzahn [18:57:19] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Operations: Reimage gerrit2001 as stretch - https://phabricator.wikimedia.org/T168562#3368352 (10Dzahn) p:05Lowest>03Normal [18:59:02] 10Gerrit, 10Release-Engineering-Team (Next), 10Operations: Gerrit is failing to start gerrit-ssh on gerrit2001 - https://phabricator.wikimedia.org/T176532#3633018 (10Dzahn) We'll do T168562 and reinstall this box with stretch to, ideally, kill 2 birds with one stone. Confirm if this issue goes away and see i... [19:25:21] thcipriani: puppet is broken on deployment-aqs01-03, can you have a look? [19:25:47] andrewbogott: sure. don't know a whole lot about that node... [19:25:54] you created it :) [19:25:59] oh :) [19:26:13] that doesn't sounds like me :P [19:27:13] > Invalid parameter rpc_address on Class[Cassandra] [19:27:15] * thcipriani digs [19:31:15] hrm, seems to be related to https://gerrit.wikimedia.org/r/#/c/372124/ but that patch seems to have changed hieradata... [19:34:38] 10Continuous-Integration-Config, 10Continuous-Integration-Scaling, 10Release-Engineering-Team, 10releng-201516-q3, and 2 others: [keyresult] Migrate as many misc CI jobs as possible to Nodepool - https://phabricator.wikimedia.org/T119140#3633101 (10Ejegg) [19:34:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Next), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: wikimedia-fundraising-civicrm fails with Call to a member function getDriver() on null in phar:///srv/jenkins-workspace/workspac... - https://phabricator.wikimedia.org/T171724#3633098 [19:46:18] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [19:48:00] (03PS4) 10Hashar: dib: use a profile to for the worker_localhost [integration/config] - 10https://gerrit.wikimedia.org/r/380467 [19:49:02] yeah hmm [19:49:10] that is a 10 changes dependency tree ! [19:53:08] my crazy hiera command: RUBYLIB=/var/lib/puppet/lib hiera profile::cassandra::settings ::labsproject=deployment-prep ::hostname=deployment-aqs01.deployment-prep.eqiad.wmflabs --debug -c /etc/puppet/hiera.yaml seems to reveal that it is, in fact, looking at the yaml file https://gerrit.wikimedia.org/r/#/c/372124/38/hieradata/labs/deployment-prep/common.yaml [19:53:46] thcipriani: you can use puppet/util/hiera_lookup as well [19:53:52] remember to use the labs tree [19:53:53] but! Notice: {"instances"=>{}, "rack"=>"rack1", "seeds"=>["deployment-aqs01.deployment-prep.eqiad.wmflabs", "deployment-aqs02.deployment-prep.eqiad.wmflabs", "deployment-aqs03.deployment-prep.eqiad.wmflabs"], "additional_jvm_opts"=>["-javaagent:/srv/deployment/prometheus/jmx_exporter/lib/jmx_prometheus_javaagent-0.8-20170117.190412-1.jar=7800:/etc/cassandra/prometheus_jmx_exporter.yaml"], [19:53:55] "cluster_name"=>"Analytics Query Service Storage Beta", "listen_address"=>"10.68.18.237", "logstash_host"=>"deployment-logstash2.deployment-prep.eqiad.wmflabs", "rpc_address"=>"10.68.18.237", "target_version"=>"2.2"} [19:54:22] chasemp: ah, neat. I've been using that weird heira lookup command for quite a while :) [19:54:51] depending on if you are looking for something role() specified it could have different behavior I'm not sure [20:04:43] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins: Archive Extension:Phalanx - https://phabricator.wikimedia.org/T176665#3633249 (10MarcoAurelio) [20:05:08] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins: Archive Extension:Phalanx - https://phabricator.wikimedia.org/T176665#3633264 (10MarcoAurelio) [20:05:25] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins, 10User-MarcoAurelio: Archive Extension:Phalanx - https://phabricator.wikimedia.org/T176665#3633279 (10MarcoAurelio) a:03MarcoAurelio [20:07:34] can someone please activate https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/Phalanx again and give me +2 so I can archive it? [20:07:38] see task [20:09:55] tabbycat: done [20:10:04] thanks :) [20:11:58] greg-g: do you think I can be added to https://gerrit.wikimedia.org/r/#/admin/groups/252,members so I can do the stuff myself? [20:12:50] if not, I'll do as always: patch and spam people all day long until it gets done :D [20:19:33] $ git review [20:19:35] Could not connect to gerrit. [20:19:36] Enter your gerrit username: [20:19:38] wtf... [20:22:43] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins, 10Patch-For-Review, 10User-MarcoAurelio: Archive Extension:Phalanx - https://phabricator.wikimedia.org/T176665#3633321 (10MarcoAurelio) [20:30:32] 10Release-Engineering-Team, 10Cleanup, 10Repository-Admins, 10Patch-For-Review, 10User-MarcoAurelio: Archive Extension:Phalanx - https://phabricator.wikimedia.org/T176665#3633329 (10MarcoAurelio) [20:36:06] RECOVERY - Puppet errors on deployment-aqs02 is OK: OK: Less than 1.00% above the threshold [0.0] [20:37:00] RECOVERY - Puppet errors on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:40:51] RECOVERY - Puppet errors on deployment-aqs03 is OK: OK: Less than 1.00% above the threshold [0.0] [21:00:31] (03Draft1) 10MarcoAurelio: Typo fix in docs [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/380637 [21:00:34] (03PS2) 10MarcoAurelio: Typo fix in docs [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/380637 [21:03:00] tabbycat: git review -s (for setup) ? [21:03:56] mutante: it happened to me on some ocasions after cloning for the first time a repo. I just make git review fail with 'publickey' and then I try again, and it works :) [21:04:01] I could submit it [21:04:21] did "git init" just in case there was a problem with .git [21:04:24] but no success [21:04:29] but its resolved now [21:06:53] tabbycat use plain git :) [21:07:11] what is that? [21:07:17] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [21:07:19] tabbycat git pull [21:07:21] git push [21:07:25] git [21:07:38] paladox: git review is not plain git? :) [21:07:43] nope [21:07:47] git-review is a tool [21:08:22] it has problems with ssh. [21:08:26] git push is a bit of a problem since, afaik, I'd have to git push HEAD:refs/for/master/T1234 ? [21:08:33] oh wait i meant http. [21:08:49] tabbycat ah yep [21:08:58] appart from that infrequent incident, git review works fine for me [21:09:02] git push HEAD:refs/for/master/T1234