[00:15:27] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:17:08] Project selenium-Flow » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #183: 04FAILURE in 1 min 7 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/183/ [00:29:17] RECOVERY - Puppet run on deployment-changeprop is OK: OK: Less than 1.00% above the threshold [0.0] [00:30:47] RECOVERY - Puppet run on deployment-sca03 is OK: OK: Less than 1.00% above the threshold [0.0] [00:50:23] RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [01:59:25] PROBLEM - Puppet run on deployment-conftool is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [02:23:11] PROBLEM - Puppet run on integration-slave-trusty-1012 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:03:12] RECOVERY - Puppet run on integration-slave-trusty-1012 is OK: OK: Less than 1.00% above the threshold [0.0] [03:16:23] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [03:51:25] RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [04:42:23] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [04:54:14] PROBLEM - Puppet run on integration-slave-trusty-1012 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [05:22:24] RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [05:34:12] RECOVERY - Puppet run on integration-slave-trusty-1012 is OK: OK: Less than 1.00% above the threshold [0.0] [06:29:58] Yippee, build fixed! [06:29:59] Project selenium-Wikibase » chrome,test,Linux,contintLabsSlave && UbuntuTrusty build #151: 09FIXED in 1 hr 49 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/151/ [06:47:10] Project selenium-Wikibase » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #151: 04FAILURE in 2 hr 7 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/151/ [07:12:33] PROBLEM - Puppet run on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:12:59] PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [07:22:38] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2724169 (10ArielGlenn) Regarding the gc patch, can we get some logging around how often full gcs are happening first... [07:43:24] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [07:52:31] RECOVERY - Puppet run on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [07:53:01] RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0] [08:34:56] PROBLEM - DPKG on scandium is CRITICAL: DPKG CRITICAL dpkg reports broken packages [08:39:37] RECOVERY - DPKG on scandium is OK: All packages OK [08:45:06] PROBLEM - Host gallium is DOWN: PING CRITICAL - Packet loss = 100% [08:45:50] RECOVERY - Host gallium is UP: PING OK - Packet loss = 0%, RTA = 0.43 ms [08:53:26] RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [08:57:07] Project beta-code-update-eqiad build #127139: 04FAILURE in 4 min 5 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/127139/ [09:04:56] Yippee, build fixed! [09:04:57] Project beta-code-update-eqiad build #127140: 09FIXED in 1 min 55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/127140/ [09:14:41] !log rebasing integration puppet master [09:14:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:43:32] 10Continuous-Integration-Infrastructure: debian-glue-non-voting jobs seem to be failing consistently and cause is not clear - https://phabricator.wikimedia.org/T148664#2729415 (10hashar) One of the issue we have is that piuparts runs from a snapshot Debian install which is never updated and does not get jessie-b... [09:43:50] 10Continuous-Integration-Infrastructure: debian-glue-non-voting jobs seem to be failing consistently and cause is not clear - https://phabricator.wikimedia.org/T148664#2737488 (10hashar) p:05Triage>03Normal [09:45:56] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2737489 (10hashar) [09:51:34] PROBLEM - Puppet run on deployment-apertium01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [09:54:32] PROBLEM - Puppet run on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [10:04:30] 10Continuous-Integration-Infrastructure: debian-glue-non-voting jobs seem to be failing consistently and cause is not clear - https://phabricator.wikimedia.org/T148664#2737502 (10Paladox) @hashar or we could upload it to the wikimedia mirror? [10:07:51] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2724601 (10hashar) My editor uses both and they are quite useful to avoid mistakes and preventing corner cases issues. The main challenge though is n... [10:10:53] 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling: Reduce number of CI permanent slaves - https://phabricator.wikimedia.org/T148183#2737514 (10hashar) [10:21:32] (03PS1) 10Hashar: Remove composer jobs from permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/317479 [10:23:02] (03CR) 10Hashar: [C: 032] Remove composer jobs from permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/317479 (owner: 10Hashar) [10:24:00] (03Merged) 10jenkins-bot: Remove composer jobs from permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/317479 (owner: 10Hashar) [10:25:14] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [10:29:30] RECOVERY - Puppet run on integration-slave-trusty-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [10:35:36] 05Continuous-Integration-Scaling, 10releng-201516-q3: [keyresult] Migrate majority of CI jobs to Nodepool (part 2) - https://phabricator.wikimedia.org/T119138#2737526 (10hashar) [10:35:38] 10Continuous-Integration-Config, 05Continuous-Integration-Scaling: Track what tests we have left to convert to nodepool - https://phabricator.wikimedia.org/T136249#2737528 (10hashar) [10:36:22] hashar hi, im wondering could you help set up postgres on ci please? [10:36:47] It's too prevent regressions like https://phabricator.wikimedia.org/T147599 [10:42:31] paladox: hello, postgre support is nowhere near a priority! [10:43:15] (03PS1) 10Hashar: Migrate visualeditor-coverage to Nodepool instance [integration/config] - 10https://gerrit.wikimedia.org/r/317481 (https://phabricator.wikimedia.org/T119138) [10:43:17] Oh nope, but would we be able to make it a non voting test even if it is broken? [10:46:52] (03CR) 10Hashar: [C: 032] "Verified and that even fixed the job that got broke due to node 0.10 -> 4.x transition." [integration/config] - 10https://gerrit.wikimedia.org/r/317481 (https://phabricator.wikimedia.org/T119138) (owner: 10Hashar) [10:47:54] (03Merged) 10jenkins-bot: Migrate visualeditor-coverage to Nodepool instance [integration/config] - 10https://gerrit.wikimedia.org/r/317481 (https://phabricator.wikimedia.org/T119138) (owner: 10Hashar) [10:51:26] (03PS1) 10Hashar: Move npm coverage jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/317482 (https://phabricator.wikimedia.org/T119138) [11:00:14] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [11:05:56] (03CR) 10Hashar: [C: 032] "Validated by triggering the coverage jobs on the last three merged changes:" [integration/config] - 10https://gerrit.wikimedia.org/r/317482 (https://phabricator.wikimedia.org/T119138) (owner: 10Hashar) [11:07:28] (03Merged) 10jenkins-bot: Move npm coverage jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/317482 (https://phabricator.wikimedia.org/T119138) (owner: 10Hashar) [11:34:15] Hey, Can you check this patch and/ore merge it? https://gerrit.wikimedia.org/r/#/c/316787/ [11:34:25] We need it [12:00:16] Amir1: hello :) [12:00:28] hey hashar [12:00:38] Amir1: sorry for last week, we were attending an off site and were quite busy [12:00:53] (03PS2) 10Hashar: Make jenkins working on mediawiki/extensions/WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/316787 (owner: 10Ladsgroup) [12:00:56] (03CR) 10Hashar: [C: 032] Make jenkins working on mediawiki/extensions/WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/316787 (owner: 10Ladsgroup) [12:00:59] Yeah, I heard from Tyler. No worries :) I hope you had fun [12:01:24] Thanks [12:02:04] hashar: Also is this okay? https://gerrit.wikimedia.org/r/#/c/316216/ [12:02:09] (03Merged) 10jenkins-bot: Make jenkins working on mediawiki/extensions/WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/316787 (owner: 10Ladsgroup) [12:02:21] I think I need to rebase it but other than that. Is it enough? [12:02:48] not sure :] [12:03:05] will review / merge / test it [12:03:47] Yes, Thanks [12:14:25] (03PS2) 10Hashar: Let Wikibase publish its docs [integration/config] - 10https://gerrit.wikimedia.org/r/316216 (owner: 10Ladsgroup) [12:14:31] (03CR) 10Hashar: [C: 032] Let Wikibase publish its docs [integration/config] - 10https://gerrit.wikimedia.org/r/316216 (owner: 10Ladsgroup) [12:15:33] (03Merged) 10jenkins-bot: Let Wikibase publish its docs [integration/config] - 10https://gerrit.wikimedia.org/r/316216 (owner: 10Ladsgroup) [12:16:57] Thanks! [12:17:06] fails though https://integration.wikimedia.org/ci/job/mwext-doxygen-publish/214/console :D [12:17:22] the doxygen job needs a Doxyfile apparently [12:19:57] there are examples in Kartographer / MobileFrontend and Thanks [12:20:26] * hashar does a copy paste [12:21:08] hashar hi, could you review https://gerrit.wikimedia.org/r/#/c/316908/ please? [12:27:55] (03CR) 10Hashar: "That requires a Doxygen file: https://gerrit.wikimedia.org/r/317489 then we will also want to update the doc.wikimedia.org/ page to poin" [integration/config] - 10https://gerrit.wikimedia.org/r/316216 (owner: 10Ladsgroup) [12:28:45] Amir1: https://gerrit.wikimedia.org/r/#/c/317489/ would add the Doxyfile [12:29:05] Thanks hashar [12:29:11] I'm waiting on jenkins [12:29:15] and then I merge it [12:29:18] Amir1: looks like that generates some kind of doc:] [12:30:09] (03PS3) 10Hashar: [PageForms] Adding jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/316908 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [12:30:15] paladox: yes doing [12:30:34] (03CR) 10Hashar: [C: 032] [PageForms] Adding jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/316908 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [12:30:46] hashar thankyou [12:31:00] It will most likly fail, but is ok since it will help with fixing it [12:31:08] SemanticForms is being renamed to PageForms [12:31:34] (03Merged) 10jenkins-bot: [PageForms] Adding jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/316908 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [12:32:18] paladox: done :) [12:32:27] Thankyou :) [12:33:14] Amir1: I am having a nap. Will look at the Wikibase doxygen job once the change land, then add the link to doc.wm.o later on :D [12:33:37] hashar: the link on doc.wm.o needs a patch? [12:33:49] or you need to login and add manually? [12:34:24] Amir1: https://doc.wikimedia.org/ page is a manually maintained list [12:34:37] that is integration/docroot.git file is org/wikimedia/doc/default.html [12:34:42] there are previous example :] [12:35:05] ssomething like:
  • Wikibase
    • [12:35:08] :D [12:35:09] Okay [12:35:19] I will make it while you rest [12:36:18] it is pretty much trivial :] [12:36:26] I will be back later! [12:45:43] (03PS1) 10Ladsgroup: Add link to Wikibase docs to doc.wm.o [integration/docroot] - 10https://gerrit.wikimedia.org/r/317497 [13:13:59] PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [13:48:03] (03CR) 10Thiemo Mättig (WMDE): [C: 031] Add link to Wikibase docs to doc.wm.o [integration/docroot] - 10https://gerrit.wikimedia.org/r/317497 (owner: 10Ladsgroup) [13:54:00] RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0] [14:18:42] (03CR) 10Hashar: [C: 032] "Congrats :)" [integration/docroot] - 10https://gerrit.wikimedia.org/r/317497 (owner: 10Ladsgroup) [14:19:09] (03Merged) 10jenkins-bot: Add link to Wikibase docs to doc.wm.o [integration/docroot] - 10https://gerrit.wikimedia.org/r/317497 (owner: 10Ladsgroup) [14:42:10] 06Release-Engineering-Team, 10MediaWiki-General-or-Unknown, 06Operations, 10Traffic, and 5 others: Make sure we're not relying on HTTP_PROXY headers - https://phabricator.wikimedia.org/T140658#2738137 (10demon) p:05High>03Low >>! In T140658#2684595, @BBlack wrote: > Is there more to do here on the MW-C... [14:46:35] PROBLEM - Puppet run on zuul-dev-jessie is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [14:47:03] Amir1: everything good now : [14:47:05] ) [14:47:28] hashar: Thank you! I will probably add js docs today or tomorrow [14:49:59] (03PS1) 10Paladox: Add PageForms as a wmf extension [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) [14:50:45] 10Gerrit, 06Repository-Admins, 13Patch-For-Review: Rename the Semantic Forms extension to "Page Forms" - https://phabricator.wikimedia.org/T147582#2738188 (10Paladox) @Yaron_Koren PageForms now has jenkins tests. Ive also added PageForms to the wmf branching list here https://gerrit.wikimedia.org/r/317516 [14:51:50] ostriches im wonder could you review https://gerrit.wikimedia.org/r/#/c/317516/1 please? [14:53:13] PROBLEM - Host deployment-pdf02 is DOWN: CRITICAL - Host Unreachable (10.68.16.129) [14:54:29] PROBLEM - Host deployment-conftool is DOWN: CRITICAL - Host Unreachable (10.68.20.30) [14:55:07] (03CR) 10Chad: "We'll actually want to have them both, briefly, so just add the new one for now." [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [14:55:35] (03CR) 10Paladox: "Yep, smooth transition :)" [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [14:59:04] 06Release-Engineering-Team, 06Operations, 07HHVM, 13Patch-For-Review, 06Services (doing): Migrate deployment servers (tin/mira) to jessie - https://phabricator.wikimedia.org/T144578#2738249 (10hashar) [14:59:07] 06Release-Engineering-Team, 06Operations, 07Beta-Cluster-reproducible, 13Patch-For-Review: mwscript on jessie mediawiki fails - https://phabricator.wikimedia.org/T146286#2738245 (10hashar) 05Open>03Resolved a:03hashar https://gerrit.wikimedia.org/r/#/c/315260/ adds mediawiki::packages::php5 on the ro... [15:04:57] (03CR) 10Hashar: [C: 031] "Looks like a good transition plan :]" [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [15:11:42] (03PS1) 10Ladsgroup: Publish jsduck for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/317520 [15:14:29] hashar: I'm working to pass jenkins on WikibaseLexeme. I made this patch https://gerrit.wikimedia.org/r/#/c/317490/ which fixes some stuff but still it doesn't install wikibase/data-model from composer.json and the phpunit fails [15:14:49] Amir1: cause the jobs are not using composer install [15:14:51] maybe I'm missing something obvious [15:15:01] the deps are provided from mediawiki/vendor iirc [15:15:05] which is thenused for deployment [15:15:35] or gotta inject some extensions as dependencies [15:16:20] Your test needs updating to the extension unit tests composer job [15:16:26] na [15:17:05] Amir1 you made a mistake in composer.json [15:17:24] let me check another ext [15:17:51] aohhohouazeaze [15:18:07] paladox: looks like you are right :D Some other extensions requiring wikibase/data-model do use composer [15:18:14] Yep :) [15:18:41] paladox: okay, what is the mistake [15:18:57] (03PS1) 10Hashar: [WikibaseLexeme] use composer to ship dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/317525 [15:18:58] Amir1 i left the comment here https://gerrit.wikimedia.org/r/#/c/317490/12/composer.json [15:19:06] (03PS2) 10Hashar: [WikibaseLexeme] use composer to ship dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/317525 [15:19:12] thanks [15:19:17] Your welcome :) [15:19:25] https://gerrit.wikimedia.org/r/#/c/317525/2/zuul/layout.yaml :D [15:19:36] paladox: does ^^ looks right to you? [15:19:49] hashar yes :) [15:19:55] (03CR) 10Paladox: [C: 031] [WikibaseLexeme] use composer to ship dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/317525 (owner: 10Hashar) [15:20:03] (03CR) 10Hashar: [C: 032] "Per Paladox review via irc :]" [integration/config] - 10https://gerrit.wikimedia.org/r/317525 (owner: 10Hashar) [15:20:16] looks like the test against mediawiki/vendor is done at a later stage [15:20:26] eg when extensions + deps are composed in the Wikidata extension [15:20:46] Yep [15:20:59] (03Merged) 10jenkins-bot: [WikibaseLexeme] use composer to ship dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/317525 (owner: 10Hashar) [15:21:10] btw we have this: https://gerrit.wikimedia.org/r/#/c/317520/ [15:21:48] hashar ^^ [15:24:59] the composer still fails [15:25:24] Amir1 the test hasent been deployed yet [15:25:41] oh, Do you have an ETA? [15:26:28] I doint have one, but hashar may :) [15:26:52] its deployed now [15:26:55] Amir1 ^^ [15:27:06] thanks [15:27:09] Amir1: I have changed WikibaseLexme to use composer and did a recheck on https://gerrit.wikimedia.org/r/#/c/317490/13 [15:27:16] sorry I was having a coffee [15:27:32] both you and paladox did :D [15:27:39] Give jenkins a break guys [15:27:42] Yep my mistake [15:28:10] Amir1 it passes :) [15:28:13] hashar ^^ :) [15:28:18] YESSSS [15:28:23] after 13 patch sets [15:28:34] The Irony [15:28:45] Yep :) [15:29:24] (03PS5) 10Jean-Frédéric: Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 [15:30:03] (03CR) 10Jean-Frédéric: "Rebased this :)" [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:31:15] (03PS2) 10Hashar: Publish jsduck for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/317520 (owner: 10Ladsgroup) [15:32:37] hashar it seems tests are slow on nodepool again https://integration.wikimedia.org/zuul/ [15:32:44] (03CR) 10Hashar: [C: 032] "Note that the repository will need a jsduck.json file that output the generation documentations to /docs/ (that is where the mwext-jsduck" [integration/config] - 10https://gerrit.wikimedia.org/r/317520 (owner: 10Ladsgroup) [15:33:43] (03Merged) 10jenkins-bot: Publish jsduck for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/317520 (owner: 10Ladsgroup) [15:34:03] Amir1: all done :) [15:34:16] Yess [15:34:17] thanks [15:37:21] (03CR) 10Ladsgroup: "Already there: https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/jsduck.json" [integration/config] - 10https://gerrit.wikimedia.org/r/317520 (owner: 10Ladsgroup) [15:39:49] PROBLEM - Puppet run on deployment-phab02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:40:45] :) [15:44:35] (03PS6) 10Hashar: Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:46:48] (03PS7) 10Hashar: Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:47:32] (03CR) 10Hashar: [C: 032] "I have hijacked the change. Namely:" [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:49:22] (03CR) 10jenkins-bot: [V: 04-1] Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:50:01] (03CR) 10Hashar: [C: 032] Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [15:51:04] PROBLEM - Puppet run on deployment-phab01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:52:06] 03Scap3, 06Services (watching), 15User-mobrovac: Scap3 fails to restart the service on deploy - https://phabricator.wikimedia.org/T148407#2738475 (10thcipriani) 05Open>03Resolved [15:52:38] (03Merged) 10jenkins-bot: Publish code coverage post-merge in labs/tools/heritage [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [16:00:28] (03CR) 10Hashar: "I have manually triggered the job on the latest merged change https://gerrit.wikimedia.org/r/#/c/314172/ Coverage magically appeared on h" [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [16:02:17] (03CR) 10Jean-Frédéric: "> I have manually triggered the job on the latest merged change" [integration/config] - 10https://gerrit.wikimedia.org/r/314171 (owner: 10Jean-Frédéric) [16:03:58] ostriches hi, are we going with 1.29 or going to do one more wmf release for 1.28? [16:04:37] No 1.29 this week. [16:04:40] I'll be sending an e-mail today [16:04:41] Ok thanks :) [16:13:32] Are releng people back (or at least partially back) from the offsite? [16:16:38] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T145220#2623702 (10hashar) Note: mw ext SemanticPageForms extension is going to be renamed at some point https://gerrit.wikimedia.org/r/#/c/317516/ should only impact wikitech [16:16:42] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T146998#2677385 (10hashar) Note: mw ext SemanticPageForms extension is going to be renamed at some point https://gerrit.wikimedia.org/r/#/c/317516/ should only impact wikitech [16:17:47] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T147517#2695117 (10hashar) Note: mw ext SemanticPageForms extension is going to be renamed at some point https://gerrit.wikimedia.org/r/#/c/317516/ should only impact wikitech [16:19:50] !log upgrading deployment-puppetmaster to puppet 3.8.5 packages [16:19:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [16:22:59] PROBLEM - Puppet run on deployment-eventlogging03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [16:24:33] PROBLEM - Puppet run on deployment-mira is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [16:25:37] PROBLEM - Puppet run on deployment-kafka05 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [16:25:54] 10Continuous-Integration-Config, 06Release-Engineering-Team: Prepare CI for MediaWiki REL1_29 - https://phabricator.wikimedia.org/T148987#2738655 (10hashar) [16:26:15] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 06Release-Engineering-Team: Prepare CI for MediaWiki REL1_29 - https://phabricator.wikimedia.org/T148987#2738671 (10hashar) [16:26:17] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [16:26:45] PROBLEM - Puppet run on deployment-mx is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [16:26:49] PROBLEM - Puppet run on deployment-aqs01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [16:26:51] PROBLEM - Puppet run on deployment-mediawiki04 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [16:32:19] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 06Release-Engineering-Team: Prepare CI for MediaWiki REL1_29 - https://phabricator.wikimedia.org/T148987#2738655 (10Paladox) @Hashar REL1_28 is being cut and yes it is being cut this week :). [16:34:32] RECOVERY - Puppet run on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [16:38:00] RECOVERY - Puppet run on deployment-eventlogging03 is OK: OK: Less than 1.00% above the threshold [0.0] [16:46:51] RECOVERY - Puppet run on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0] [16:46:53] PROBLEM - Puppet run on deployment-logstash2 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [16:59:02] 03Scap3, 07Documentation: Write "how to build scap plugins" developer documentation - https://phabricator.wikimedia.org/T147546#2738816 (10mmodell) 05Open>03Resolved https://doc.wikimedia.org/mw-tools-scap/dev/plugins.html [17:00:26] 06Release-Engineering-Team: Remove .gitreview from MediaWiki and Extensions - https://phabricator.wikimedia.org/T146293#2738822 (10demon) >>! In T146293#2660356, @mmodell wrote: > git-review version 1.25.0 is currently available as a .deb for Ubuntu 16.04 but debian jessie has only [[ https://packages.debian.org... [17:00:36] RECOVERY - Puppet run on deployment-kafka05 is OK: OK: Less than 1.00% above the threshold [0.0] [17:01:14] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [17:01:48] RECOVERY - Puppet run on deployment-aqs01 is OK: OK: Less than 1.00% above the threshold [0.0] [17:05:23] (03CR) 10Chad: [C: 032] Add PageForms as a wmf extension [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [17:06:12] (03Merged) 10jenkins-bot: Add PageForms as a wmf extension [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [17:06:25] (03CR) 10Paladox: "Thanks." [tools/release] - 10https://gerrit.wikimedia.org/r/317516 (https://phabricator.wikimedia.org/T147582) (owner: 10Paladox) [17:34:12] hello releng people. can someone fix the permissions on deployment-tin.deployment-prep.eqiad.wmflabs in /srv/deployment/parsoid/deploy. git pull is failing, "error: insufficient permission for adding an object to repository database .git/objects" [17:35:13] you should be able to use sudo to fix them [17:35:28] unless the problem is you don't know how [17:36:00] i didn't think i had sudo ... but let me see [17:36:28] if you can ssh into deployment-prep instances you have sudo on them [17:37:13] ok, thanks [17:37:47] if the problem is you don't know, that's okay, I may be able to help [17:38:55] arlolra, maybe you just need to `sudo -u trebuchet git <...>` to operate on that repo? [17:40:43] maybe ... but that wasn't necessary a few weeks ago [17:43:20] 03Scap3: Scap3 canonical target list - https://phabricator.wikimedia.org/T148992#2738970 (10thcipriani) [17:43:32] 06Release-Engineering-Team, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Add Tyler Cipriani to releasers-mediawiki - https://phabricator.wikimedia.org/T148681#2738982 (10Dzahn) a:03Dzahn [17:46:44] Krenair: ok, fixed. two .git/objects were missing group write permission and i added `sharedRepository = group` to .git/config [17:46:51] thanks [17:47:41] 03Scap3: Scap3 canonical target list - https://phabricator.wikimedia.org/T148992#2738984 (10thcipriani) p:05Triage>03Normal In the deployment tooling meeting [[ https://www.mediawiki.org/wiki/Deployment_tooling/Cabal/2016-10-24 | today ]] we discussed having a global list of pooled servers per service (alrea... [17:49:04] 10Beta-Cluster-Infrastructure: Import rights on wmflabs wiki for Baha - https://phabricator.wikimedia.org/T148993#2738987 (10bmansurov) [17:55:37] Hi jenkins is really slow ago https://integration.wikimedia.org/zuul/ [17:57:21] ack, noticed that too [18:01:40] there was a spike a little bit ago: https://grafana.wikimedia.org/dashboard/db/releng-kpis?from=now-12h&to=now [18:01:45] (bottom graph) [18:01:54] https://grafana.wikimedia.org/dashboard/db/releng-kpis?panelId=5&fullscreen&from=now-12h&to=now [18:02:52] paladox: you said it's a quota issue? [18:02:53] 10Beta-Cluster-Infrastructure: Import rights on wmflabs wiki for Baha - https://phabricator.wikimedia.org/T148993#2739047 (10Krenair) 05Open>03Resolved a:03Krenair (del/undel) 18:02, 24 October 2016 Krenair (talk | contribs | block) changed group membership for Baha from (none) to importer (request) [18:03:27] Yes [18:03:27] has it been raised before? [18:03:27] and is now maxed out? [18:03:34] Yes, but we had to go down due to labs having resources constrait's i think [18:03:44] how is right now a quota issue? [18:03:57] there are no nodepool instances running according to Jenkins: https://integration.wikimedia.org/ci/ [18:04:09] there is an "quota issue" but it is not the blind "reason for all issues" [18:04:50] and WMF Labs Opsen know of "the quota issue" and are working with antione actively on it. It's not a simple fix. (and again, is not shown to be the issue at hand) [18:05:04] https://integration.wikimedia.org/zuul/ [18:05:32] greg-g i carn't see any tests running now [18:05:49] paladox: that what I just said: "there are no nodepool instances running according to Jenkins: https://integration.wikimedia.org/ci/" [18:06:10] Oh sorry [18:06:25] one, to get some confusion out of the way pre-emptively: the android jobs do not run on nodepool, they are a permanent slave only [18:07:15] Yep [18:09:34] nodepool is working: this change was just +2's and went straight to a nodepool instance: https://gerrit.wikimedia.org/r/#/c/316494/ [18:09:45] zero(effetively) wait [18:10:00] I'm not sure why the mw/core and other extensions test jobs are not going to nodepool quickly [18:10:32] also, please describe what "slow" means to you paladox. what exact symptom were/are you seeing? [18:11:01] mw-config jobs in -operations are going quickly [18:11:04] greg-g only a couple of tests are running, and the page is freezing my browser [18:11:23] well, that second bit (freezing your browser) is your browser's fault [18:11:32] example from my side, i have rebased 316843 at 10:44 and did not get a vote from jenkins-bot yet [18:11:40] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2739102 (10Paladox) We could do this -Xloggc:/var/lib/gerrit2/review_site/logs/`date +%F_%H-%M-%S`-gc.log -XX:+Prin... [18:11:46] yeah, for some reason a few queues are not processing [18:12:12] I'm not going to recommend kicking anything yet until after swat is done (it's going fine so far wrt to Jenkins/zuul/nodepool) [18:12:29] ok [18:15:31] (03CR) 10BearND: "@Niedzielski, where would the effect of this show up?" [integration/config] - 10https://gerrit.wikimedia.org/r/317068 (owner: 10Niedzielski) [18:18:36] things are running in nodepool again, I'm going to say it was the overload of Android patches [18:20:07] I forgot what hashar told me to look at on https://grafana.wikimedia.org/dashboard/db/nodepool [18:20:16] Im thinking instances creations [18:20:41] no [18:20:44] it wasn't [18:21:26] oh [18:21:34] there were the baseline 4 or 5 (I forget exactly how many we have just waiting even when no jobs are running) just sitting there doing nothing [18:21:37] thank you, confired i got the votes [18:21:46] ok [18:37:01] greg-g: thcipriani: a user has been created on "bromine" so you can upload mw releases now [18:38:43] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2739193 (10demon) Close. I was thinking more like... -Xloggc:/var/log/gerrit/jvm_gc.%p.log -XX:+PrintGCDetails... [18:38:54] 06Release-Engineering-Team, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Add Tyler Cipriani to releasers-mediawiki - https://phabricator.wikimedia.org/T148681#2739195 (10Dzahn) on bromine.eqiad.wmnet Notice: /Stage[main]/Admin/Admin::Hashuser[thcipriani]/Admin::User[thcipriani]/User[thcipriani... [18:39:13] 06Release-Engineering-Team, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Add Tyler Cipriani to releasers-mediawiki - https://phabricator.wikimedia.org/T148681#2739196 (10Dzahn) 05Open>03Resolved [18:39:29] 06Release-Engineering-Team, 06Operations, 10Ops-Access-Requests: Add Tyler Cipriani to releasers-mediawiki - https://phabricator.wikimedia.org/T148681#2729869 (10Dzahn) [18:39:29] (03CR) 10Niedzielski: "@bearND, this change would show colors in the Jenkins build console instead of ANSI escape sequences." [integration/config] - 10https://gerrit.wikimedia.org/r/317068 (owner: 10Niedzielski) [18:40:02] mutante: sweet, thanks :) [18:42:42] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2739204 (10Paladox) Oh yes please :), would you like me to do the change or are you? [18:43:57] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2739218 (10ArielGlenn) I'd like to get this going during a normal usage period with babysitting for the first little... [18:45:14] 10Gerrit, 06Release-Engineering-Team, 06Operations, 13Patch-For-Review: Investigate why gerrit slowed down on 17/10/2016 / 18/10/2016 / 21/10/2016 - https://phabricator.wikimedia.org/T148478#2739221 (10Paladox) Normal usage should be around now, highest peak time. [19:03:05] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2724601 (10fgiunchedi) >>! In T148494#2737503, @hashar wrote: > Finally we will need an entry point in puppet.git to easily run the command (maybe reu... [19:08:25] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2724601 (10Dzahn) Looks like we have about 206 scripts with "bin/bash" in ops/puppet. About 56 are called .sh or .sh.erb. Would we want to rename the... [19:09:13] niedzielski: are you around ? :] [19:09:39] hashar: o/ :] [19:10:00] niedzielski: seeing all your patches, I guess you should probably just get CR+2 on integration/config [19:10:20] since you update the Jenkins jobs already [19:11:20] hashar: well, i think some of the android stuff is pretty isolated. i definitely wouldn't feel comfortable with puppet at this point but the jenkins job builder configs seem to make sense [19:11:32] oh [19:11:38] I have no access on puppet.git either :D [19:11:44] was really thinking about JJB config [19:11:58] * niedzielski is a young pup when it comes to puppet [19:12:00] lets do that [19:12:24] ok, that'd be cool. are you fine with me just +2ing my own config patches then? [19:12:36] sold :D [19:12:42] well [19:12:43] lol [19:13:01] given you already self deploy the job, I have no reason to prevent self merging own patches [19:13:24] specially when I do self merge about everything :] [19:13:24] see it more as a journal of changes [19:13:39] the more or less gentleman agreement is that we self merge when we now of the impact / there is a limited scope [19:13:57] for big changes we make sure to test it before CR+2 [19:14:05] hashar: as long as i'm not breaking any processes for you folks, that works for me [19:14:08] and sometime just push to prod to see what happens :] [19:14:32] things to watch is the output of the jenkins job that does the diff of xml config with previous version [19:14:59] and if you had to deploy a zuul/layout.yaml change, that currently requires shell access to gallium (which host zuul) [19:15:09] but I guess 99% of your patches are solely for android [19:15:18] niedzielski: you got +2 now :) [19:16:27] hashar: well thanks! i'll try to practice safe merging. i hear abstinence is the only truly safe merge but isn't very fun :] [19:19:17] and posted the announce on the QA list [19:19:45] niedzielski: I guess if you double check that your jjb change only impact a short amount of jobs [19:19:48] and you get the job deployed [19:19:51] and it works just fine [19:19:56] then do CR+2 as needed :] [19:20:39] your changes are all fine anyway, so do not worry [19:20:58] about puppet I havent looked at the Xdummy thing at all :((( [19:21:20] hashar: thanks! and the primary mechanism to verify no extra jobs leaked in is to diff the xml config? [19:22:05] yes ! [19:22:37] locally I use a makefile https://gerrit.wikimedia.org/r/#/c/313998/2/makefile [19:22:57] so I can just "make" to generate the current config, the output is saved to "output-prev" dir [19:23:01] then fetch a patch / hack [19:23:03] run "make" [19:23:26] and it generate the new conf then show a diff with the saved reference (in "output-prev") [19:23:42] which ends up being essentially the same as the jjb-diff job [19:24:13] hashar: that sounds pretty neat [19:25:18] hashar: i was thinking another mechanism might be to use the git cache but i suppose that would require a change to .gitignore [19:25:33] git cache ? [19:26:31] hashar: oh, i just mean run the build, git add the output, checkout master and run the build again. [19:26:41] then git diff --cached shows the delta [19:26:54] ohhhh [19:27:07] yeah that would be similar [19:27:11] er, i guess just git diff in that flow [19:27:23] though I am not sure I am willing to mess with the git index by randomly git adding stuff [19:27:40] yeah [19:29:50] hashar: maybe git is more risky than it's worth. we also have some screenshot tests and do something like your makefile. we have a reference directory that is committed to git but the test results are stored in an unversioned, ignored directory. when we run a diff script, the diffs go in a third (also unversioned) directory. https://phabricator.wikimedia.org/diffusion/APAW/browse/master/scripts/diff-screenshots [19:31:58] hashar: any recommendations on how best to move forward with the puppet xdummy patch? i'll re-review my changes on the next rebase but is there another puppet master i should bother about it? i'd like to get these dependencies updated in puppet so the CI system doesn't disappear in the future if i'm not around [19:33:36] sorry was out [19:33:43] niedzielski: catching up [19:34:24] makefiles everywhere :] [19:36:40] niedzielski: looks like your git screenshot thing is all fine [19:36:45] for xdummy, I guess I have to get into it [19:36:54] and switch us out of xvfb [19:38:02] hashar: ok (sorry for the hassle). maybe i can ping you on the patch on my next rebase? i can be responsible for making a WIP patch to do a swap from xvfb as a subsequent change if it all works well and that's wanted [19:38:47] the challenge is really how to do the change and verify all the jobs that rely on xvfb [19:38:51] but maybe it is just a non issue [19:39:03] the android job already uses it isn't it ? [19:40:32] hashar: no, it's using an Xorg instance i started manually on the android server [19:41:28] ahhh [19:41:33] er, android is using the new Xdummy instance and never used the xvfb instance except during initial investigation [19:45:04] niedzielski: yeah so it looks all fine to me [19:45:32] niedzielski: might want to figure out how to get Xorg to spawn when machines start though [19:46:48] hashar: ok, thanks :] [19:47:10] Trusty instances have run-level 2 , so I guess upstart would start it [19:47:48] jessie have run-level 5, no idea how it relates to systemd WantedBy=multi-user.target [19:48:02] so I guess seek someone from operations to land the patch and you are all set! [19:50:56] hashar: sounds good, thanks! [19:52:11] any runlevel between 2 and 5 .. _should_ be when multi-user.target is running [19:52:31] systemd doesnt have a concept of runlevels like that, but since 2-5 mean "multiuser modes" [19:52:47] that should be the same for this purpose [19:53:28] neat [19:53:43] one "just" have to figure out how to get xorg to spawn then :] [19:56:14] oh, i have not read all the context but there is a "graphical.target" too [19:56:52] that would be "higher" than multi-user but include that [20:00:56] mutante: that might what we would need (namely get Xorg to run on instance boot) [20:05:36] systemctl set-default graphical.target [20:06:55] that should do it on jessie it looks, feel free to add me to that stuff [20:06:59] bbl [20:15:39] !sal [20:15:39] https://tools.wmflabs.org/sal/releng [20:18:44] mutante: added you to the xdummy change and did a summary https://gerrit.wikimedia.org/r/#/c/264303/ [20:32:32] 03Scap3, 10Parsoid: Canary didn't rollback? - https://phabricator.wikimedia.org/T149008#2739558 (10Arlolra) [20:40:34] 03Scap3, 10Parsoid: Did not rollback? - https://phabricator.wikimedia.org/T149012#2739623 (10Arlolra) [21:01:54] 03Scap3, 10Parsoid: Canary didn't rollback? - https://phabricator.wikimedia.org/T149008#2739728 (10thcipriani) p:05Triage>03Low This is currently expected behavior. `Continue? no` is just a stop since nothing is "wrong". We could implement a rollback option here, likely. [21:02:17] 03Scap3, 10Parsoid: Canary doesn't rollback if you don't continue - https://phabricator.wikimedia.org/T149008#2739745 (10thcipriani) [21:02:55] 10Gerrit: Update gerrit to 2.13.1 - https://phabricator.wikimedia.org/T146350#2739748 (10Paladox) [21:03:03] 10Gerrit: Update gerrit to 2.13.2 - https://phabricator.wikimedia.org/T146350#2658303 (10Paladox) [21:03:38] 10Gerrit: Update gerrit to 2.12.5 - https://phabricator.wikimedia.org/T143089#2739750 (10Paladox) [21:08:13] 03Scap3, 10Parsoid: Did not rollback? - https://phabricator.wikimedia.org/T149012#2739782 (10thcipriani) p:05Triage>03Normal This one seems different than {T145460} but for the same reasons. As part of the `promote` stage we remove the `.in-progress` symlink. Without an `.in-progress` symlink, scap doesn't... [21:08:48] 03Scap3, 10Parsoid: Scap rollback fails after promote completes - https://phabricator.wikimedia.org/T149012#2739787 (10thcipriani) [21:09:17] 03Scap3, 10Parsoid: Canary doesn't rollback if you don't continue - https://phabricator.wikimedia.org/T149008#2739788 (10Arlolra) I see. I guess the process the current canary is catering for are automated checks that would fail and induce a rollback. We tend deploy the canary and then manually monitor the h... [21:10:37] 03Scap3, 10Parsoid: Scap rollback fails after promote completes - https://phabricator.wikimedia.org/T149012#2739623 (10thcipriani) Adding @dduvall if case he has any thoughts about rollback behavior. [21:12:18] 03Scap3, 10Parsoid: Canary doesn't rollback if you don't continue - https://phabricator.wikimedia.org/T149008#2739831 (10thcipriani) >>! In T149008#2739788, @Arlolra wrote: > We tend deploy the canary and then manually monitor the health of the cluster before continuing. At which point, a rollback would be qu... [21:12:34] 03Scap3, 10Parsoid: Scap rollback fails after promote completes - https://phabricator.wikimedia.org/T149012#2739846 (10Arlolra) Fair enough. Thanks @thcipriani [21:19:19] 10Beta-Cluster-Infrastructure, 10WikimediaPageViewInfo: Deploy WikimediaPageViewInfo extension to beta cluster - https://phabricator.wikimedia.org/T129602#2739924 (10greg) >>! In T129602#2733581, @MZMcBride wrote: >>>! In T129602#2733358, @Tgr wrote: >> It would defeat the point of beta (even if that point is... [21:41:14] Yippee, build fixed! [21:41:15] Project selenium-RelatedArticles » chrome,beta-desktop,Linux,contintLabsSlave && UbuntuTrusty build #187: 09FIXED in 43 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-desktop,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/187/ [22:00:55] (03PS1) 10Chad: Remove weird legacy extension config list thingie [tools/release] - 10https://gerrit.wikimedia.org/r/317661 [22:08:20] (03CR) 10Chad: [C: 032] Remove weird legacy extension config list thingie [tools/release] - 10https://gerrit.wikimedia.org/r/317661 (owner: 10Chad) [22:09:08] (03Merged) 10jenkins-bot: Remove weird legacy extension config list thingie [tools/release] - 10https://gerrit.wikimedia.org/r/317661 (owner: 10Chad) [22:12:49] qa-morebots leaving the channel is me. I'm testing stashbot as a replacement [22:17:24] (03CR) 10BearND: [C: 031] "It would be nice to add that to the commit message." [integration/config] - 10https://gerrit.wikimedia.org/r/317068 (owner: 10Niedzielski) [22:18:05] 03Scap3, 10Parsoid: Scap rollback fails after promote completes - https://phabricator.wikimedia.org/T149012#2740375 (10dduvall) >>! In T149012#2739787, @thcipriani wrote: > Adding @dduvall if case he has any thoughts about rollback behavior. Oh state... There are currently a few final operations tacked on to... [22:24:04] 05Gitblit-Deprecate, 06Release-Engineering-Team: Clones from git.wikimedia.org are not redirected - https://phabricator.wikimedia.org/T139206#2422729 (10demon) To be honest I think we should allow the clones to break. [22:45:09] 06Release-Engineering-Team: Remove .gitreview from MediaWiki and Extensions - https://phabricator.wikimedia.org/T146293#2740431 (10demon) Also, I'll note this doesn't affect puppet, or any other repos other than MW + core + extensions + vendor. Other repos are more than welcome to continue using defaultbranch. T... [22:51:04] (03PS3) 10Niedzielski: Android: enable ANSI console coloring & timestamps [integration/config] - 10https://gerrit.wikimedia.org/r/317068 [22:51:06] (03PS5) 10Niedzielski: Android: upgrade emulator to API 24 [integration/config] - 10https://gerrit.wikimedia.org/r/314788 (https://phabricator.wikimedia.org/T133183) [22:51:23] (03CR) 10Niedzielski: "Done. apps-android-wikipedia-test already has these settings enabled. This is for the similarly named apps-android-wikipedia-periodic-test" [integration/config] - 10https://gerrit.wikimedia.org/r/317068 (owner: 10Niedzielski) [23:04:31] (03CR) 10BearND: "Thanks. Looks better. :)" [integration/config] - 10https://gerrit.wikimedia.org/r/317068 (owner: 10Niedzielski) [23:13:14] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T146998#2740543 (10greg) 05Open>03Resolved [23:13:38] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T149059#2740552 (10greg) [23:13:53] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T147517#2695117 (10greg) [23:14:24] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T149059#2740552 (10greg) FTR: @thcipriani will do the branch cut and testwikis on Tuesday @mmodell will do Wed/Thurs deploys [23:48:19] 03Scap3, 10Parsoid: Scap rollback fails after promote completes - https://phabricator.wikimedia.org/T149012#2740642 (10thcipriani) >>! In T149012#2740375, @dduvall wrote: > One possible refactoring I can think of would be to change this behavior by decoupling those state-keeping and cleanup operations into a f...