[00:55:41] PROBLEM - Puppet staleness on integration-dev is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [03:17:58] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #824: FAILURE in 35 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/824/ [03:26:42] 10Gerrit-Migration, 10Gitblit-Deprecate, 3releng-201516-q1, 10Diffusion: [keyresult] Allow cloning of Phabricator hosted git repositories - https://phabricator.wikimedia.org/T128#1657344 (10greg) >>! In T128#1657143, @hashar wrote: >>>! In T128#1620386, @mmodell wrote: >> Current status: cloning over https... [03:33:30] 10Beta-Cluster, 10pywikibot-core, 5Patch-For-Review: Link.langlinkUnsafe does not work on Beta-Cluster wikis - https://phabricator.wikimedia.org/T112006#1657346 (10greg) a:3Krenair [04:31:19] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #568: FAILURE in 39 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/568/ [06:39:23] RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK [07:24:35] 10Deployment-Systems, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 10Unplanned-Sprint-Work, and 2 others: is shown instead of the actual message on Special:ContentTranslation in ... - https://phabricator.wikimedia.org/T112964#1657430 [08:00:57] (03PS4) 10Hashar: Update math tests [integration/config] - 10https://gerrit.wikimedia.org/r/228285 (owner: 10Paladox) [08:02:01] (03CR) 10Hashar: [C: 032] Update math tests [integration/config] - 10https://gerrit.wikimedia.org/r/228285 (owner: 10Paladox) [08:03:58] (03Merged) 10jenkins-bot: Update math tests [integration/config] - 10https://gerrit.wikimedia.org/r/228285 (owner: 10Paladox) [08:23:47] (03CR) 10Zfilipin: [C: 04-2] WIP Fix WrongPass MediawikiApi::LoginError error for VisualEditor Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/236540 (https://phabricator.wikimedia.org/T111510) (owner: 10Zfilipin) [08:25:09] (03Abandoned) 10Hashar: WIP Fix WrongPass MediawikiApi::LoginError error for VisualEditor Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/236540 (https://phabricator.wikimedia.org/T111510) (owner: 10Zfilipin) [08:51:07] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 5Patch-For-Review: Factor out ruby stuff from contint puppet manifests - https://phabricator.wikimedia.org/T110865#1657529 (10zeljkofilipin) To make things simpler, we should move all jobs from ubuntu precise to trusty, that will reduc... [08:52:05] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 5Patch-For-Review: Factor out ruby stuff from contint puppet manifests - https://phabricator.wikimedia.org/T110865#1657530 (10zeljkofilipin) Antoine suggested to take a look here http://packages.ubuntu.com/search?keywords=ruby [08:52:12] 10Browser-Tests, 10QuickSurveys, 5Patch-For-Review, 3Reading-Web-Sprint-56-Four Lions: QA: Setup browser tests on beta cluster so we can share test articles - https://phabricator.wikimedia.org/T112204#1657531 (10phuedx) [239158](https://gerrit.wikimedia.org/r/239158) has a +1 from me – and I've also learne... [09:48:22] (03CR) 10Spage: "Whatever! The current Blueprint testing is non-existent, surely my patch brings it up to parity with the Vector skin." [integration/config] - 10https://gerrit.wikimedia.org/r/226542 (owner: 10Paladox) [09:50:26] Yippee, build fixed! [09:50:26] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #825: FIXED in 37 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/825/ [10:05:47] PROBLEM - Puppet staleness on deployment-eventlogging02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [10:29:09] 10Continuous-Integration-Config: [Regression] Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657646 (10hashar) On gallium the directory `/srv/org/wikimedia/integration/cover/$DOC_PROJECT` was apparently created on Sep 12 18:10:28. [10:41:14] 10Continuous-Integration-Config: [Regression] Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657653 (10hashar) Found it! The job [[ https://integration.wikimedia.org/ci/job/phpunit-coverage-publish/42/ | phpunit-coverage-publish #42 ]] has been t... [10:41:32] 10Continuous-Integration-Config: [Regression] Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657654 (10hashar) p:5High>3Normal Lowering priority, it is an artifact. [11:16:00] Do adding someone to WMF LDAP need a Phabricator ticket or something else? [11:16:05] Err. Does. [11:35:44] (03CR) 10Zfilipin: "Removing myself from reviewers, I am not familiar with this, I can not really provide good feedback." [integration/config] - 10https://gerrit.wikimedia.org/r/239189 (https://phabricator.wikimedia.org/T93727) (owner: 10Paladox) [11:36:44] (03CR) 10Zfilipin: "Removing myself from reviewers, I am not familiar with this, I can not really provide good feedback." [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [11:51:27] James_F, I think ostriches, Reedy and robla do it by request [11:51:40] Krenair: Aha, OK. Will wait for later then. [11:52:01] But maybe you could use a ticket [12:31:32] 10Continuous-Integration-Config: [Regression] Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657891 (10JanZerebecki) That was me. I thought we had an assertion for that, but that was probably another type of publish jobs. [12:31:50] 10Continuous-Integration-Config: Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657892 (10JanZerebecki) [12:37:43] !log gallium:~$ sudo -u jenkins-slave rm -rf /srv/org/wikimedia/integration/cover/\$DOC_PROJECT/ [12:37:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [13:01:35] 10Gerrit-Migration, 10Gitblit-Deprecate, 3releng-201516-q1, 10Diffusion: [keyresult] Allow cloning of Phabricator hosted git repositories - https://phabricator.wikimedia.org/T128#1657933 (10mmodell) >>! In T128#1657344, @greg wrote: > > Is there a benefit to one over the other? SSH is much more efficient. [13:04:13] 10Gerrit-Migration, 10Gitblit-Deprecate, 3releng-201516-q1, 10Diffusion: [keyresult] Allow cloning of Phabricator hosted git repositories - https://phabricator.wikimedia.org/T128#1657936 (10mmodell) Also password authentication isn't as secure. Passwords have to be stored somewhere or entered repeatedly. [13:10:43] (03CR) 10Zfilipin: [C: 032] Login helper for fast API-based authentication [selenium] - 10https://gerrit.wikimedia.org/r/226950 (https://phabricator.wikimedia.org/T112908) (owner: 10Dduvall) [13:12:01] (03CR) 10Zfilipin: [C: 04-1] "Looks good, voting -1 for now since it is not finished." [selenium] - 10https://gerrit.wikimedia.org/r/239132 (https://phabricator.wikimedia.org/T89353) (owner: 10Dduvall) [13:12:03] (03Merged) 10jenkins-bot: Login helper for fast API-based authentication [selenium] - 10https://gerrit.wikimedia.org/r/226950 (https://phabricator.wikimedia.org/T112908) (owner: 10Dduvall) [13:18:26] 10Gerrit-Migration, 10Gitblit-Deprecate, 3releng-201516-q1, 10Diffusion: [keyresult] Allow cloning of Phabricator hosted git repositories - https://phabricator.wikimedia.org/T128#1657967 (10greg) /me nods, let's hope we can get this sorted soon. [13:21:03] (03PS1) 10JanZerebecki: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) [13:38:31] 10Beta-Cluster, 10Analytics-EventLogging, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 3Fundraising Sprint Tom Waits: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657984 (10Ottomata) Just deleted deployment-eventlogging02 [13:41:13] PROBLEM - Host deployment-eventlogging02 is DOWN: CRITICAL - Host Unreachable (10.68.16.52) [13:46:12] PROBLEM - Host mira is DOWN: CRITICAL - Host Unreachable (10.68.17.215) [13:46:31] 10Beta-Cluster, 10Analytics-EventLogging, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 3Fundraising Sprint Tom Waits: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657993 (10AndyRussG) Thanks so much! Closing... For MySQL querying of the bann... [13:46:44] 10Beta-Cluster, 10Analytics-EventLogging, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 3Fundraising Sprint Tom Waits: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657995 (10AndyRussG) 5Open>3Resolved [13:47:43] 10Continuous-Integration-Config, 5Patch-For-Review: Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1657998 (10JanZerebecki) I think we can fully replace the custom zuul injected DOC_PROJECT with builders accessing the existing ZUUL_* parameters. [13:56:57] (03PS2) 10JanZerebecki: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) [13:57:58] hashar: wanna take a quick peek at https://gerrit.wikimedia.org/r/#/c/239827/ before I deploy it? [13:58:21] jzerebecki: !!!! [14:06:00] (03CR) 10Hashar: [C: 04-1] "Yeah set -u is an excellent choice :-}" (035 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [14:06:06] jzerebecki: done, see my notes :-} [14:06:11] kudos on the set -u idea! [14:07:26] (03PS10) 10Hashar: Update WikidataPageBanner tests [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [14:08:25] (03PS11) 10Hashar: Update WikidataPageBanner tests [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [14:08:42] (03CR) 10Hashar: [C: 032] "There is a single phpcs error but will fix it." [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [14:15:21] (03CR) 10JanZerebecki: Make sure that destination for publishers is ok (035 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [14:16:46] (03Merged) 10jenkins-bot: Update WikidataPageBanner tests [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [14:17:47] (03PS1) 10Hashar: Throttle '{name}-phplint' [integration/config] - 10https://gerrit.wikimedia.org/r/239839 [14:20:12] (03CR) 10Hashar: [C: 032] Throttle '{name}-phplint' [integration/config] - 10https://gerrit.wikimedia.org/r/239839 (owner: 10Hashar) [14:23:06] (03PS2) 10Hashar: Throttle '{name}-phplint' [integration/config] - 10https://gerrit.wikimedia.org/r/239839 [14:23:12] (03CR) 10Hashar: [C: 032] Throttle '{name}-phplint' [integration/config] - 10https://gerrit.wikimedia.org/r/239839 (owner: 10Hashar) [14:23:35] (03CR) 10Hashar: "The sole PHP CS error is fixed by https://gerrit.wikimedia.org/r/#/c/239841/" [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [14:25:09] (03Merged) 10jenkins-bot: Throttle '{name}-phplint' [integration/config] - 10https://gerrit.wikimedia.org/r/239839 (owner: 10Hashar) [14:30:52] 10Continuous-Integration-Config, 7I18n, 7Jenkins: npm job should not fail on superfluous qqq entries - https://phabricator.wikimedia.org/T112854#1658106 (10hashar) @Mattflaschen mentioned on the Gerrit change: > > Patch Set 6: > It's deliberate. It avoids the problem of a non-existent message lingering in... [14:31:31] 10Continuous-Integration-Config, 7I18n, 7Jenkins: npm job should not fail on superfluous qqq entries - https://phabricator.wikimedia.org/T112854#1658113 (10hashar) 5Open>3Resolved a:3hashar Seems it is working as intended. If it is a concern / unwanted behavior, you will want to fill an issue on the b... [14:32:56] 10Deployment-Systems: Trebuchet blockers for MediaWiki (tracking) - https://phabricator.wikimedia.org/T45338#1658119 (10hashar) Just decline it? We are not going to use what was known as Trebuchet for mw deployment. [14:40:32] (03PS1) 10Hashar: Trigger jobs for puppet-compiler [integration/config] - 10https://gerrit.wikimedia.org/r/239845 [14:40:42] (03CR) 10Hashar: [C: 032] Trigger jobs for puppet-compiler [integration/config] - 10https://gerrit.wikimedia.org/r/239845 (owner: 10Hashar) [14:41:30] (03Merged) 10jenkins-bot: Trigger jobs for puppet-compiler [integration/config] - 10https://gerrit.wikimedia.org/r/239845 (owner: 10Hashar) [14:47:51] (03CR) 10Hashar: "Sorry for the set -e drama :-)" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [14:48:21] jzerebecki: sorry for set -u / set -e [14:48:37] still need the [ -z $ZUUL_UUID ] assertion though [14:55:52] (03CR) 10Paladox: "Thankyou." [integration/config] - 10https://gerrit.wikimedia.org/r/226913 (owner: 10Paladox) [15:00:51] pfff [15:01:00] I have been triaging my emails all day long :( [15:01:21] down to 132 Gerrit messages \O/ [15:02:26] (03PS2) 10Hashar: Add Jenkins test for Patroller [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:02:31] (03PS3) 10Hashar: Add Jenkins test for Patroller [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:03:00] (03PS3) 10JanZerebecki: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) [15:03:45] (03PS4) 10JanZerebecki: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) [15:11:19] (03PS4) 10Hashar: Add Jenkins test for Patroller [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:11:37] (03CR) 10Hashar: [C: 032] "Added composer since the repo has it." [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:12:38] (03Merged) 10jenkins-bot: Add Jenkins test for Patroller [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:15:58] (03CR) 10Paladox: "Thankyou." [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:20:25] (03CR) 10Paladox: "@Hashar should this be abandoned and replaced with the archive test. and the repo cleaned out." [integration/config] - 10https://gerrit.wikimedia.org/r/239189 (https://phabricator.wikimedia.org/T93727) (owner: 10Paladox) [15:27:43] (03CR) 10Hashar: "Fixes are on https://gerrit.wikimedia.org/r/#/c/239849/" [integration/config] - 10https://gerrit.wikimedia.org/r/237981 (owner: 10Paladox) [15:28:30] (03CR) 10Hashar: "https://gerrit.wikimedia.org/r/#/c/237489/ got merged :)" [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [15:36:04] (03CR) 10Hashar: [C: 031] "Sounds good :-} I am in a hurry and can deploy / review right now though, but feel free to do so!" [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [15:36:53] kids time! [15:38:25] 10Deployment-Systems, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 10Unplanned-Sprint-Work, and 2 others: is shown instead of the actual message on Special:ContentTranslation in ... - https://phabricator.wikimedia.org/T112964#1658252 [15:44:52] bye bye wikibugs [16:03:19] PROBLEM - Puppet failure on deployment-parsoidcache02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:03:20] PROBLEM - Puppet failure on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:03:20] PROBLEM - Puppet failure on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:07:57] I need some help with a failed build, it's something related to TimedMediaHandler and 'mw.EmbedPlayer' ( https://integration.wikimedia.org/ci/job/mwext-CirrusSearch-testextension-zend/983/consoleFull ) [16:08:15] the change is https://gerrit.wikimedia.org/r/#/c/238767/ [16:08:34] I don't understand why it's related to my change [16:13:42] PROBLEM - Puppet staleness on deployment-restbase01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [16:20:22] (03CR) 10JanZerebecki: "Thank you." [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [16:20:26] (03PS5) 10JanZerebecki: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) [16:23:23] (03PS1) 10Paladox: Move jshint test to check: in UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239869 [16:24:44] (03PS1) 10Paladox: Remove experimental support for php composer test in UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239870 [16:27:53] (03Abandoned) 10Paladox: Add jshint to npm in UniversalLanguageSelector extension [integration/config] - 10https://gerrit.wikimedia.org/r/228881 (owner: 10Paladox) [16:37:44] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1659794 (10Yurik) @hashar, tell you the truth, I don't know myself how new code is deployed to betacluster, especially entry points - its all pretty scary to me too ) [16:38:41] (03CR) 10JanZerebecki: [C: 032] "Deployed to Jenkins: (['doxygen-publish', 'mediawiki-core-code-coverage', 'mediawiki-core-doxygen-publish', 'mediawiki-core-jsduck-publish" [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [16:38:51] (03PS1) 10Paladox: Add composer-test to UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239873 [16:39:32] 10Continuous-Integration-Config, 5Patch-For-Review: Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1659808 (10JanZerebecki) https://integration.wikimedia.org/ci/job/phpunit-coverage-publish/44/console Does now fail before rsync as expected: DOC_... [16:40:18] (03PS1) 10Paladox: Remove phpcs test from UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239875 [16:40:22] (03Merged) 10jenkins-bot: Make sure that destination for publishers is ok [integration/config] - 10https://gerrit.wikimedia.org/r/239827 (https://phabricator.wikimedia.org/T113195) (owner: 10JanZerebecki) [16:40:52] (03PS2) 10Paladox: Remove phpcs test from UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239875 [16:41:40] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1659818 (10greg) >>! In T71757#1659794, @Yurik wrote: > @hashar, tell you the truth, I don't know myself how new code is deployed to betacluster, especially entry p... [16:41:47] (03PS2) 10Paladox: Add composer-test to UniversalLanguageSelector [integration/config] - 10https://gerrit.wikimedia.org/r/239873 [16:43:55] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1659827 (10greg) >>! In T71757#1659794, @Yurik wrote: > @hashar, tell you the truth, I don't know myself how new code is deployed to betacluster, especially entry p... [16:46:59] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1659844 (10Yurik) @greg, this is awesome list of links! (is it posted on wikitech somewhere?). When scap and config updates run, do they include the entry points... [16:47:12] 10Continuous-Integration-Config, 7Technical-Debt: move setting of DOC_* variables from zuul to inside the build scripts - https://phabricator.wikimedia.org/T113250#1659848 (10JanZerebecki) 3NEW [16:47:29] 10Continuous-Integration-Config, 5Patch-For-Review: Broken coverage report published to literal dir name "$DOC_PROJECT" - https://phabricator.wikimedia.org/T113195#1659856 (10JanZerebecki) 5Open>3Resolved a:3JanZerebecki [16:50:02] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1659875 (10greg) >>! In T71757#1659844, @Yurik wrote: > @greg, this is awesome list of links! (is it posted on wikitech somewhere?). When scap and config updates... [16:53:48] 10Continuous-Integration-Infrastructure, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: Let contint-admins force run puppet with /usr/local/sbin/puppet-run - https://phabricator.wikimedia.org/T110943#1659880 (10RobH) Update from meeting: This was approved by ops in the operations meeting. (Discuss... [16:57:51] 10Continuous-Integration-Infrastructure, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: Let contint-admins force run puppet with /usr/local/sbin/puppet-run - https://phabricator.wikimedia.org/T110943#1659899 (10Dzahn) @hashar brought it up in meeting and it has been approved. @Andrew might follow-u... [17:32:59] (03Abandoned) 10Chad: Simplify logging in ssh module [tools/scap] - 10https://gerrit.wikimedia.org/r/238959 (owner: 10Chad) [17:53:11] 6RelEng-Admin, 15User-greg: Propose changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1660169 (10greg) [18:21:50] !log deployed up-to-date scap in deployment-prep [18:21:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [18:26:13] 10Continuous-Integration-Infrastructure, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: Let contint-admins force run puppet with /usr/local/sbin/puppet-run - https://phabricator.wikimedia.org/T110943#1660294 (10Dzahn) 5stalled>3Resolved applied on gallium [18:27:21] People of Jenkins! I just had a patch rejected due to a pep8 failure in a file that is not in my patch :( https://gerrit.wikimedia.org/r/#/c/239183/ [18:27:26] That’s not supposed to happen, is it? [18:28:08] andrewbogott: was the change that introduced that pep8 failure force-merged? [18:28:28] someone can break all subsequent builds that way [18:28:37] greg-g: I don’t know, maybe. I’ll investigate [18:28:41] kk [18:31:44] greg-g: Looks like that file has broken pep8 since 2011. So… something else ‘interesting’ is going on [18:31:52] sounds fun [18:31:58] I'm going to go get some lunch :P [18:32:55] 10Beta-Cluster, 10MediaWiki-File-management, 6Multimedia: Low-quality images are not rendered in beta - https://phabricator.wikimedia.org/T71757#1660335 (10Tgr) Thumbnail entry points are not deployed, they are just there. See the blocked task. [18:34:48] 10Continuous-Integration-Infrastructure, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: Let contint-admins force run puppet with /usr/local/sbin/puppet-run - https://phabricator.wikimedia.org/T110943#1660349 (10hashar) Works on labnodepool1001.eqiad.wmnet as well. Thank you! [18:44:03] (03CR) 10JanZerebecki: [C: 031] Migrate ops/debs/adminbot to tox/flake8 [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [18:44:10] (03CR) 10JanZerebecki: [C: 031] Remove pep8/pyflakes for mw/tools/scap [integration/config] - 10https://gerrit.wikimedia.org/r/237617 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [18:44:35] Project beta-scap-eqiad build #71019: FAILURE in 8.5 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/71019/ [18:45:39] ^ me messing things up [18:48:03] (03CR) 10JanZerebecki: [C: 031 V: 04-1] "Waiting for dependency." [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [18:55:51] Yippee, build fixed! [18:55:51] Project beta-scap-eqiad build #71020: FIXED in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/71020/ [19:09:11] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling: Nodepool images need Gerrit mirror for git-clone performance - https://phabricator.wikimedia.org/T87294#1660543 (10hashar) [19:10:14] (03CR) 10Thcipriani: [C: 032] Support atomic promotion and rollback [tools/scap] - 10https://gerrit.wikimedia.org/r/238839 (https://phabricator.wikimedia.org/T109514) (owner: 10Dduvall) [19:10:49] (03PS1) 10Hashar: nodepool: rename dib element wikimedia{-puppet,} [integration/config] - 10https://gerrit.wikimedia.org/r/239907 [19:10:51] (03PS1) 10Hashar: nodepool: rename populate repo script [integration/config] - 10https://gerrit.wikimedia.org/r/239908 [19:10:53] (03PS1) 10Hashar: nodepool: cleanup git mirror cache [integration/config] - 10https://gerrit.wikimedia.org/r/239909 (https://phabricator.wikimedia.org/T87294) [19:12:18] twentyafterfour: https://gerrit.wikimedia.org/r/#/c/239112/ is basically rolled into its parent already right? [19:15:28] (03CR) 10Hashar: [C: 032] nodepool: rename dib element wikimedia{-puppet,} [integration/config] - 10https://gerrit.wikimedia.org/r/239907 (owner: 10Hashar) [19:15:35] (03CR) 10Hashar: [C: 032] nodepool: rename populate repo script [integration/config] - 10https://gerrit.wikimedia.org/r/239908 (owner: 10Hashar) [19:15:41] (03CR) 10Hashar: [C: 032] nodepool: cleanup git mirror cache [integration/config] - 10https://gerrit.wikimedia.org/r/239909 (https://phabricator.wikimedia.org/T87294) (owner: 10Hashar) [19:17:00] Yep. [19:17:12] (03CR) 10Thcipriani: [C: 04-1] "Causes error on regular scap when cherry-picked to beta: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/71019/console" [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [19:17:37] (03Abandoned) 10Chad: Convert tasks.* to use context logger [tools/scap] - 10https://gerrit.wikimedia.org/r/239112 (owner: 10Chad) [19:19:06] (03Merged) 10jenkins-bot: Support atomic promotion and rollback [tools/scap] - 10https://gerrit.wikimedia.org/r/238839 (https://phabricator.wikimedia.org/T109514) (owner: 10Dduvall) [19:19:11] (03Merged) 10jenkins-bot: nodepool: rename dib element wikimedia{-puppet,} [integration/config] - 10https://gerrit.wikimedia.org/r/239907 (owner: 10Hashar) [19:19:12] (03Merged) 10jenkins-bot: nodepool: rename populate repo script [integration/config] - 10https://gerrit.wikimedia.org/r/239908 (owner: 10Hashar) [19:19:14] (03Merged) 10jenkins-bot: nodepool: cleanup git mirror cache [integration/config] - 10https://gerrit.wikimedia.org/r/239909 (https://phabricator.wikimedia.org/T87294) (owner: 10Hashar) [19:19:34] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 5Patch-For-Review: Nodepool images need Gerrit mirror for git-clone performance - https://phabricator.wikimedia.org/T87294#1660583 (10hashar) a:3hashar [19:22:18] (03CR) 10Chad: "We should probably stop passing loggers to it then. I'll write up a patch and cherry pick this on top." [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [19:22:33] thcipriani: I think ^ [19:24:40] 10Continuous-Integration-Infrastructure: Remove PhantomJS from the CI infrastructure - https://phabricator.wikimedia.org/T113279#1660602 (10hashar) 3NEW [19:24:52] ostriches: I think the problem is that when you pass in a logger without specifying it as a keyword arg, but rather as a positional arg. [19:25:41] the decorator should be agnostic to how the logger is passed, or, maybe, it shouldn't allow a logger argument. [19:25:55] I poked twentyafterfour about it, too. [19:26:06] Yeah, I'm thinking it's best to just stop passing them about [19:26:51] Lemme amend. [19:27:49] (I mean avoiding passing them was the whole point, right? :) [19:27:51] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-ZeroBanner, 6Zero: Stop using PhantomJS for ZeroBanner browser test - https://phabricator.wikimedia.org/T113280#1660610 (10hashar) 3NEW [19:30:32] !log lets get rid of PhantomJS https://phabricator.wikimedia.org/T113279 :D [19:30:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:30:38] ostriches: right, would be ideal to just use the new mechanism. using a decorator for this is a bit tricky. [19:31:10] Project browsertests-ZeroBanner-en.m.wikipedia.org-linux-chrome build #1: FAILURE in 20 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroBanner-en.m.wikipedia.org-linux-chrome/1/ [19:32:03] we'd either have access to an argument in the body of the function that wasn't specified as a function argument, or, we have to make sure that we're overwriting the correct argument in either kwargs or args depending on how the logger was passed. [19:34:21] (03CR) 10Thcipriani: [C: 032] Support batch size configuration per stage [tools/scap] - 10https://gerrit.wikimedia.org/r/239016 (https://phabricator.wikimedia.org/T112841) (owner: 10Dduvall) [19:34:36] (03Merged) 10jenkins-bot: Support batch size configuration per stage [tools/scap] - 10https://gerrit.wikimedia.org/r/239016 (https://phabricator.wikimedia.org/T112841) (owner: 10Dduvall) [19:38:44] (03PS1) 10Hashar: Switch ZeroBanner browsertests to Chrome [integration/config] - 10https://gerrit.wikimedia.org/r/239913 (https://phabricator.wikimedia.org/T113280) [19:39:00] (03CR) 10Hashar: [C: 04-1] "Waiting for https://gerrit.wikimedia.org/r/#/c/239912/" [integration/config] - 10https://gerrit.wikimedia.org/r/239913 (https://phabricator.wikimedia.org/T113280) (owner: 10Hashar) [19:40:56] (03PS1) 10Hashar: Remove PhantomJS support from browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/239914 (https://phabricator.wikimedia.org/T113279) [19:45:22] thcipriani: Going through and naming everything logger=logger seems silly as I got halfway through the patch [19:48:35] ostriches: yeah, fragile I think, too. The utils.inside_git_dir is the same kind of idea for a decorator, looking at the location argument. [19:49:24] or the logging context could just not a decorator. [19:54:41] 10Deployment-Systems, 3Scap3, 5Patch-For-Review: Scap3 should support rollback - https://phabricator.wikimedia.org/T109514#1660699 (10Nemo_bis) Nice! This has proved to be useful in translatewiki.net too (see http://laxstrom.name/blag/2015/01/02/oregano-deployment-tool/ ). [19:56:47] 10Continuous-Integration-Infrastructure, 5Patch-For-Review: Fix tox installation on new Precise slaves - https://phabricator.wikimedia.org/T110506#1660703 (10hashar) 5Open>3Resolved The puppet patch was already on the CI puppetmaster. It landed in puppet.git. [19:56:49] 10Continuous-Integration-Infrastructure, 6Labs, 10Labs-Infrastructure: integration-slave-trusty-1014 and integration-slave-trusty-1017 instances can't boot anymore, ended up corrupted. Need rebuild - https://phabricator.wikimedia.org/T110052#1660705 (10hashar) [20:00:31] (03PS1) 10Hashar: Adjust .gitreview for the scap3 branch [tools/scap] (scap3) - 10https://gerrit.wikimedia.org/r/239977 [20:00:51] (03CR) 10jenkins-bot: [V: 04-1] Adjust .gitreview for the scap3 branch [tools/scap] (scap3) - 10https://gerrit.wikimedia.org/r/239977 (owner: 10Hashar) [20:01:07] holy hell [20:01:31] marxarelli: thcipriani: ostriches: is it known that tests fails on scap3 branch ? ;D [20:02:12] scap3 branch is mostly useless lmao [20:02:24] I don't think anything on it hasn't landed in master. [20:03:34] ah [20:04:18] Eh, or not. Landed in different forms. [20:04:34] I made a few patches to space [20:04:43] scap3 [20:04:53] I though it would land in master eventually [20:09:51] (03CR) 10Zfilipin: "This should be fine, if the commit it depends on does not break everything. :)" [integration/config] - 10https://gerrit.wikimedia.org/r/239913 (https://phabricator.wikimedia.org/T113280) (owner: 10Hashar) [20:10:45] Ouch, they've diverged pretty bad. [20:16:45] ostriches: yeah and there is a bunch of very experimental / pet project stuff in it apparently [20:17:26] (03CR) 10Chad: [C: 032 V: 032] Adjust .gitreview for the scap3 branch [tools/scap] (scap3) - 10https://gerrit.wikimedia.org/r/239977 (owner: 10Hashar) [20:17:58] I am wondering how patches landed in [20:18:05] maybe mukunda pushed them directly [20:31:51] (03PS2) 10Hashar: Migrate ops/debs/adminbot to tox/flake8 [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) [20:32:12] (03PS2) 10Hashar: Remove pep8/pyflakes for mw/tools/scap [integration/config] - 10https://gerrit.wikimedia.org/r/237617 (https://phabricator.wikimedia.org/T55354) [20:33:45] (03CR) 10Hashar: [C: 032] Migrate ops/debs/adminbot to tox/flake8 [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:33:50] (03CR) 10Hashar: [C: 032] Remove pep8/pyflakes for mw/tools/scap [integration/config] - 10https://gerrit.wikimedia.org/r/237617 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:35:43] (03Merged) 10jenkins-bot: Migrate ops/debs/adminbot to tox/flake8 [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:35:45] (03Merged) 10jenkins-bot: Remove pep8/pyflakes for mw/tools/scap [integration/config] - 10https://gerrit.wikimedia.org/r/237617 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:36:36] (03CR) 10Hashar: "Jobs deleted" [integration/config] - 10https://gerrit.wikimedia.org/r/237617 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:37:06] (03CR) 10Hashar: "Jobs deleted" [integration/config] - 10https://gerrit.wikimedia.org/r/237492 (https://phabricator.wikimedia.org/T55354) (owner: 10Hashar) [20:37:11] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1660861 (10greg) [20:38:01] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1660863 (10greg) Right now, since I've waited so long, I'm just going to do everything updates since Sept 1st and resolved. https://phabricator.wikimedia.org/maniphest/query/HoyG... [20:46:40] spam incoming! [20:46:52] 10Browser-Tests, 7WorkType-NewFunctionality: Tutorial to setup a Raita dev environment - https://phabricator.wikimedia.org/T111493#1660896 (10greg) [20:46:54] 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-NewFunctionality: Write a diskimage-builder element to run puppet - https://phabricator.wikimedia.org/T110735#1660897 (10greg) [20:46:56] 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-NewFunctionality: Wrap diskimage-builder for easy image creations - https://phabricator.wikimedia.org/T110733#1660898 (10greg) [20:46:58] 6Release-Engineering-Team, 7WorkType-NewFunctionality: Create slides about CI browser tests for Metrics meeting - https://phabricator.wikimedia.org/T110385#1660899 (10greg) [20:47:00] 10Deployment-Systems, 3Scap3, 5Patch-For-Review, 7WorkType-NewFunctionality: Scap3 should break up remote deploy tasks - https://phabricator.wikimedia.org/T109861#1660901 (10greg) [20:47:02] 10Continuous-Integration-Config, 6Release-Engineering-Team, 5Patch-For-Review, 3Wikipedia-Android-App, 7WorkType-NewFunctionality: JJB support for Android Emulator plugin - https://phabricator.wikimedia.org/T110307#1660900 (10greg) [20:47:04] 10Continuous-Integration-Config, 6Release-Engineering-Team, 5Patch-For-Review, 3Wikipedia-Android-App, 7WorkType-NewFunctionality: Create jenkins slave instance dedicated to Android runs - https://phabricator.wikimedia.org/T107336#1660902 (10greg) [20:47:12] annnnd killed it [20:47:23] 10Beta-Cluster, 6Scrum-of-Scrums, 6Security-Team, 6operations, and 4 others: Setup a dedicated mediawiki host in Beta Cluster that we can use for security scanning - https://phabricator.wikimedia.org/T72181#1660914 (10greg) [20:49:04] 6Release-Engineering-Team, 10MediaWiki-Unit-tests, 5Patch-For-Review, 7WorkType-Maintenance: mediawiki: Release tarballs do not contain XCF images for test suite - https://phabricator.wikimedia.org/T103683#1660916 (10demon) [20:50:00] 10Deployment-Systems, 6Release-Engineering-Team, 10MediaWiki-Unit-tests, 5MW-1.25-release, and 2 others: Tests directory subtly missing from core/extension releases - https://phabricator.wikimedia.org/T91426#1660919 (10demon) [20:50:02] 6Release-Engineering-Team, 10MediaWiki-Maintenance-scripts, 10MediaWiki-Redirects, 5Patch-For-Review, 7WorkType-Maintenance: namespaceDupes not handling deleted namespace redirects as desired - https://phabricator.wikimedia.org/T91401#1660920 (10demon) [20:51:01] 5Continuous-Integration-Scaling, 7WorkType-NewFunctionality: Evaluate angry-caching-proxy as a package managers cache - https://phabricator.wikimedia.org/T112561#1660923 (10greg) [20:51:05] I'm beginnin gto think that worktype stuff really could be a custom field [20:51:08] a lot of teams are into it [20:51:18] that's somewhat heresy between andre and I tho [20:52:37] ostriches: I'm going to double tag that one real quick, it's not worth it to me to go back a screen, re-add 67 tasks, just to momentarily not mark that one "maint" [20:52:50] chasemp: it should be, yes [20:52:56] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-Maintenance: colordiff: command not found - https://phabricator.wikimedia.org/T112821#1660936 (10greg) [20:52:57] 10Continuous-Integration-Config, 5Patch-For-Review, 7WorkType-Maintenance: Setup CI for mediawiki/extensions/ImportArticles - https://phabricator.wikimedia.org/T112249#1660938 (10greg) [20:53:00] 10Continuous-Integration-Config, 7I18n, 7Jenkins, 7WorkType-Maintenance: npm job should not fail on superfluous qqq entries - https://phabricator.wikimedia.org/T112854#1660935 (10greg) [20:53:02] 10Beta-Cluster, 10Continuous-Integration-Infrastructure, 5Patch-For-Review, 7WorkType-Maintenance: deployment-logstash2 puppet certificate - https://phabricator.wikimedia.org/T112537#1660937 (10greg) [20:53:04] 10Continuous-Integration-Infrastructure, 10Gather, 5Patch-For-Review, 5WMF-deploy-2015-09-15_(1.26wmf23), 7WorkType-Maintenance: Ruby syntax check and RuboCop for Gather - https://phabricator.wikimedia.org/T112097#1660942 (10greg) [20:53:04] another batch of 68 being tagged... [20:53:05] 10Browser-Tests, 10Gather, 5Patch-For-Review, 5WMF-deploy-2015-09-15_(1.26wmf23), and 2 others: Fix easy problems reported by RuboCop in Gather - https://phabricator.wikimedia.org/T112099#1660941 (10greg) [20:54:01] greg-g: https://phabricator.wikimedia.org/maniphest/query/3FCKG5coQ7s4/#R is also a nice query. It excludes things we've already tagged. [20:54:36] nice [20:54:41] (don't tag those two) [20:56:12] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1661017 (10greg) Some assumptions I'm making as I go along (so far only tagging tasks that are "New"): * If the work is novel work towards fulfilling a quarterly goal (which are m... [20:57:45] ostriches: how'd you type the not-in part? [20:57:56] Start typing "Not in" before the project name [20:58:00] And it'll auto-complete you [20:58:36] coolio [20:58:40] Er. [20:58:44] Type the project name. [20:58:51] And it'll give you a "Not in" option in the suggestions. [20:58:56] It's weird. But powerfu [21:00:09] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1661022 (10greg) [21:01:26] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1547416 (10greg) [21:02:20] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1661032 (10greg) As of right now, since Sept 1st: * 21 New * 67 Maint aka: ~24% new work [21:03:55] greg-g: I'm not sure I believe those stats one bit. Looking at the tasks just tagged, I see the biggest flaw. [21:04:15] Untracked work. [21:04:19] ostriches: totally [21:04:59] I'd wager, if we could as the universe/god/allah/whomever, our team probably spends 20% of our time in untracked work [21:05:18] but without omniscience, I don't know how to measure it :) [21:05:41] (maybe more than 20%, I don't know) [21:05:51] also, task != time [21:05:58] which, is another critique of this method [21:06:06] (NEW) RelEng-Admin, Epic, Q3-Goal: Get Greg some omniscience [21:06:09] tasks as units are unequal [21:06:12] :) :) [21:13:10] marxarelli: you will want to polish up the puppet role for raita. Guiseppe commented on it https://gerrit.wikimedia.org/r/208024 :-} [21:15:04] hashar: it's in my long queue of gerrit patches :) [21:15:14] trying to do unto others at the moment [21:15:24] * hashar tags marxarelli with Worktype-Maintenance [21:15:44] :) [21:17:48] 6RelEng-Admin, 15User-greg: Create working definition of Core vs Maint vs New for RelEng - https://phabricator.wikimedia.org/T108635#1661085 (10greg) 5Open>3Invalid a:3greg I don't know if the "core v maint v new" distinction is going to survive, I just did "new vs maint" for {T109375}. Closing this for... [21:20:24] 6RelEng-Admin, 15User-greg: Tag some portion of RelEng team tasks with "New" or "Maint" - https://phabricator.wikimedia.org/T109375#1661100 (10greg) 5Open>3Resolved a:3greg Done for now, we'll see what the follow up is. [21:34:38] ping marxarelli this was the patch i'm trying to get merged https://gerrit.wikimedia.org/r/#/c/239158/ [21:34:47] if you have time [21:34:54] need to show product owner some stuff [21:36:03] jdlrobson: ah, i would if i could, but i don't have +2 on that repo [21:36:32] marxarelli: damn. hopefully legoktm has some bandwidth to look at it today [21:36:40] i never know who to ping about config changes [21:41:59] jdlrobson: deployers should have +2 to that repo [22:11:26] * Krenair grumbles [22:11:37] It takes more than a +2 to properly process those commits [22:18:15] that too [22:22:56] 3releng-201516-q2: [keyresult] Release MW 1.26 - https://phabricator.wikimedia.org/T110486#1661386 (10greg) [22:23:07] 3releng-201516-q2: [keyresult] Migrate all Service team owned services and MW deploys to scap3 - https://phabricator.wikimedia.org/T109926#1661389 (10greg) [22:24:00] 3releng-201516-q2: [keyresult] CI cluster responds to spike in queued builds by starting and registering additional jenkins slaves - https://phabricator.wikimedia.org/T111106#1661394 (10greg) [22:32:37] PROBLEM - Puppet failure on integration-slave-trusty-1023 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [22:33:51] 10Deployment-Systems, 3Scap3, 5Patch-For-Review: Allow batch_size to be configured for each deploy stage - https://phabricator.wikimedia.org/T112841#1661428 (10mobrovac) >>! In T112841#1655432, @greg wrote: > Does this implement that request, Yes. There are some services, like #Parsoid, which have large clu... [22:34:58] the scap3 branch was intended to be for integrating our work, thus bypassing gerrit, that was kinda the whole point [22:35:52] 10Deployment-Systems, 3Scap3, 5Patch-For-Review: Allow batch_size to be configured for each deploy stage - https://phabricator.wikimedia.org/T112841#1661436 (10greg) Sweet, thanks for the confirmation, @mobrovac. [22:45:17] 10Gerrit-Migration, 10Gitblit-Deprecate, 6Release-Engineering-Team, 10Diffusion: Import all gerrit.wikimedia.org repositories with Diffusion - https://phabricator.wikimedia.org/T616#1661452 (10Spage) >>>! In T108902#1567500, @demon wrote: >> Any repo created after I mass-created them awhile back is likely... [23:05:27] 10Gerrit-Migration, 10Gitblit-Deprecate, 6Release-Engineering-Team, 10Diffusion: Import all gerrit.wikimedia.org repositories with Diffusion - https://phabricator.wikimedia.org/T616#1661488 (10demon) >>! In T616#1661452, @Spage wrote: >>>>! In T108902#1567500, @demon wrote: >>> Any repo created after I mas... [23:07:39] RECOVERY - Puppet failure on integration-slave-trusty-1023 is OK: OK: Less than 1.00% above the threshold [0.0] [23:45:22] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<11.11%)