[01:17:43] 03Scap3, 10Parsoid: Canary doesn't rollback if you don't continue - https://phabricator.wikimedia.org/T149008#2897788 (10dduvall) 05Open>03Resolved [01:17:46] 03Scap3, 10Parsoid: Rollback failed when target is down - https://phabricator.wikimedia.org/T145460#2897792 (10dduvall) 05Open>03Resolved [01:17:50] 03Scap3, 10Parsoid, 06Services, 15User-mobrovac: Allow failures for a percentage of targets - https://phabricator.wikimedia.org/T145512#2897790 (10dduvall) 05Open>03Resolved [01:26:13] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2897804 (10Paladox) oh. I did that on a test install and could not reproduce. but creating th... [02:25:10] 06Release-Engineering-Team, 10Wikimedia-Developer-Summit, 06Developer-Relations (Oct-Dec-2016), 07Documentation: Developer Summit 2017: Work with TPG and RelEng on solution to event documenting - https://phabricator.wikimedia.org/T132400#2897874 (10Rfarrand) Working on this next week... @srishakatux it wo... [02:49:57] 06Release-Engineering-Team, 10Wikimedia-Developer-Summit, 06Developer-Relations (Oct-Dec-2016), 07Documentation: Developer Summit 2017: Work with TPG and RelEng on solution to event documenting - https://phabricator.wikimedia.org/T132400#2897933 (10srishakatux) @Rfarrand planning to help with you on this o... [06:33:58] Yippee, build fixed! [06:33:59] Project selenium-Wikibase » chrome,test,Linux,contintLabsSlave && UbuntuTrusty build #216: 09FIXED in 1 hr 53 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/216/ [06:49:03] 10Continuous-Integration-Config, 10MediaWiki-Unit-tests, 05MW-1.29-release-notes, 13Patch-For-Review, 05WMF-deploy-2017-01-03_(1.29.0-wmf.7): Karma qunit proxy fails setting Host: header - https://phabricator.wikimedia.org/T153757#2898162 (10hashar) 05Open>03Resolved Solved, Verified by @Krinkle \O/ [06:49:54] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2898164 (10Osnard) I was not aware that this could cause problems. But why did the issues occu... [07:02:38] 10Gerrit, 10MediaWiki-Vagrant, 13Patch-For-Review: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#2898183 (10hashar) On T152801#2895705 I mentioned all missing objects are tip of wmf/* branches. `mediawiki/vagrant.git` issue looks like related to jg... [07:03:08] 10Gerrit, 06Operations, 07Beta-Cluster-reproducible, 13Patch-For-Review: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#2898185 (10hashar) [07:03:10] 10Gerrit, 10MediaWiki-Vagrant, 13Patch-For-Review: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#2898184 (10hashar) [07:04:54] 10Gerrit, 06Operations, 07Beta-Cluster-reproducible, 13Patch-For-Review: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#2824332 (10hashar) T152801 is the same issue for `mediawiki/vagrant.git` which has bunch of deleted `wmf/*` branches still referencing some... [07:06:06] 10Gerrit, 06Operations, 07Beta-Cluster-reproducible, 13Patch-For-Review: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#2898191 (10hashar) [07:06:07] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2898190 (10hashar) [07:09:57] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2898193 (10hashar) p:05Unbreak!>03High Clearing priority. Unbreak now! would be the whole... [07:12:03] 10Continuous-Integration-Infrastructure, 10Gerrit, 13Patch-For-Review, 07Zuul: jenkins-bot not able to post postmerge comment - https://phabricator.wikimedia.org/T153737#2898197 (10hashar) p:05Unbreak!>03High Not unbreak now worthy. It is just post merge reports not happening which really is not the en... [07:15:39] (03Abandoned) 10Hashar: Jenkins do not report verified 0 on postmerge changes [integration/config] - 10https://gerrit.wikimedia.org/r/328534 (https://phabricator.wikimedia.org/T153737) (owner: 10Paladox) [07:22:50] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [07:33:50] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [08:47:33] (03CR) 10Hashar: [C: 031] "The job has concurrent: false, so even if it takes longer than hour there will never be more than one instance build at any time. Feel fr" [integration/config] - 10https://gerrit.wikimedia.org/r/328159 (https://phabricator.wikimedia.org/T151197) (owner: 10Phedenskog) [08:49:30] (03CR) 10Hashar: [C: 031] "I usually just drop them from the CI config but - archived will work as well :} Pending confirmation that PrefStats is actually legacy" [integration/config] - 10https://gerrit.wikimedia.org/r/328477 (https://phabricator.wikimedia.org/T134441) (owner: 10Aklapper) [08:54:28] 06Release-Engineering-Team, 10Wikimedia-Developer-Summit, 06Developer-Relations (Oct-Dec-2016), 07Documentation: Developer Summit 2017: Work with TPG and RelEng on solution to event documenting - https://phabricator.wikimedia.org/T132400#2898283 (10Qgil) I hope we have those cards for each role (notetaker... [10:54:08] 10Gerrit, 06Operations, 10Ops-Access-Requests: Root for Mukunda for Gerrit machine(s) - https://phabricator.wikimedia.org/T152236#2842574 (10mark) This is approved,. [11:01:55] 10Continuous-Integration-Infrastructure, 10Gerrit, 13Patch-For-Review, 07Zuul: jenkins-bot not able to post postmerge comment - https://phabricator.wikimedia.org/T153737#2898530 (10Paladox) @hashar hi this https://gerrit.wikimedia.org/r/328534 should fix it [11:03:42] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2898532 (10Paladox) The problem with jgit should be fixed in gerrit 2.12 just now the reminent... [11:07:30] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2898534 (10Paladox) >>! In T153079#2898164, @Osnard wrote: > I was not aware that this could c... [11:15:42] (03PS2) 10Hashar: Ensure postmerge reports to Gerrit with no votes [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) [11:17:43] (03CR) 10Hashar: "Made the gerrit success/failure to pass an empty dictionary. That is what we have done on experimental pipeline." [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) (owner: 10Hashar) [11:19:08] (03CR) 10Paladox: [C: 031] Ensure postmerge reports to Gerrit with no votes [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) (owner: 10Hashar) [11:23:42] 10Continuous-Integration-Infrastructure, 10Gerrit, 13Patch-For-Review, 07Zuul: jenkins-bot not able to post postmerge comment - https://phabricator.wikimedia.org/T153737#2898569 (10hashar) The experimental pipeline has: ``` success: gerrit: {} ``` IIRC the review command required a vote to be passed wh... [11:27:22] 10Gerrit, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Root for Mukunda for Gerrit machine(s) - https://phabricator.wikimedia.org/T152236#2842574 (10ArielGlenn) Please verify that you have the rights you need, and I will then close this ticket. [11:39:20] (03CR) 10Legoktm: [C: 04-1] "This has no effect, [[mw:CC/JJB]] is not currently translatable." [integration/config] - 10https://gerrit.wikimedia.org/r/328437 (owner: 10Zppix) [11:55:05] (03PS1) 10Legoktm: Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 [12:14:22] (03CR) 10Legoktm: [C: 032] Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 (owner: 10Legoktm) [12:15:48] (03CR) 10jerkins-bot: [V: 04-1] Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 (owner: 10Legoktm) [12:16:43] (03PS2) 10Legoktm: Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 [12:17:05] (03CR) 10Legoktm: [C: 032] Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 (owner: 10Legoktm) [12:18:33] (03Merged) 10jenkins-bot: Add php70 MediaWiki core phpunit jobs to experimental [integration/config] - 10https://gerrit.wikimedia.org/r/328886 (owner: 10Legoktm) [12:18:47] !log deploying https://gerrit.wikimedia.org/r/328886 [12:18:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:54:43] 10Gerrit, 06Release-Engineering-Team, 07Upstream: Gerrit java.lang.ArrayIndexOutOfBoundsException when querying changes - https://phabricator.wikimedia.org/T153642#2898987 (10Paladox) Patch for upstream stable-2.13 has been merged and should be released in gerrit 2.13.5 [14:38:30] (03PS3) 10Hashar: Ensure postmerge reports to Gerrit with no votes [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) [14:44:55] 10Gerrit: Code merged for core without verified score shown - https://phabricator.wikimedia.org/T45485#476066 (10Paladox) This should now be fixed. Haven't seen this problem in a while now. Especially as we upgraded to gerrit 2.12 and then to 2.13. If it does continue to happen then we should file a bug report u... [14:45:35] (03CR) 10Hashar: [C: 032] Ensure postmerge reports to Gerrit with no votes [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) (owner: 10Hashar) [14:46:07] 10Gerrit, 07Upstream: Gerrit should allow a published patch to be set to "draft" again. - https://phabricator.wikimedia.org/T63124#2899035 (10Paladox) This is now supported in gerrit 2.14 I think but only admin's can convert open patches back to draft. [14:46:32] (03Merged) 10jenkins-bot: Ensure postmerge reports to Gerrit with no votes [integration/config] - 10https://gerrit.wikimedia.org/r/328535 (https://phabricator.wikimedia.org/T153737) (owner: 10Hashar) [14:48:00] (03CR) 10jenkins-bot: doc: Add RemexHtml [integration/docroot] - 10https://gerrit.wikimedia.org/r/328337 (owner: 10Legoktm) [14:50:33] 10Continuous-Integration-Infrastructure, 10Gerrit, 13Patch-For-Review, 07Zuul: jenkins-bot not able to post postmerge comment - https://phabricator.wikimedia.org/T153737#2899040 (10hashar) 05Open>03Resolved a:03hashar I have picked the last merged change of integration/docroot : https://gerrit.wikime... [14:52:44] (03CR) 10Hashar: [C: 04-1] "I don't see the point really. If it was to be translated, I guess a top infobox providing a list of translations would be sufficient." [integration/config] - 10https://gerrit.wikimedia.org/r/328437 (owner: 10Zppix) [14:59:53] 10Gerrit, 07Regression, 07Upstream: Gerrit doesn't display committed images - https://phabricator.wikimedia.org/T153085#2899050 (10Paladox) Actually it was reported for polygerrit, so I filled the report here https://bugs.chromium.org/p/gerrit/issues/detail?id=5197 for gwt and gerrit 2.13+ [15:15:15] 10Gerrit, 10MediaWiki-Vagrant, 13Patch-For-Review: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#2899065 (10bd808) >>! In T152801#2898183, @hashar wrote: > On T152801#2895705 I mentioned all missing objects are tip of wmf/* branches. `mediawiki/vag... [15:17:12] 06Release-Engineering-Team, 07Security-General: Download of composer cweiske/php-sqllint requires to disable https security - https://phabricator.wikimedia.org/T153842#2899066 (10hashar) Got a reply from the author Christian Weiske: > > However on packagist.org it is registered with the git protocol which > >... [15:23:53] 06Release-Engineering-Team, 13Patch-For-Review, 07Security-General: Download of composer cweiske/php-sqllint requires to disable https security - https://phabricator.wikimedia.org/T153842#2899071 (10hashar) 05Open>03Resolved a:03hashar Aced! [15:25:10] 06Release-Engineering-Team, 13Patch-For-Review, 07Security-General: Download of composer cweiske/php-sqllint requires to disable https security - https://phabricator.wikimedia.org/T153842#2899077 (10JeanFred) *high-five* [15:29:40] 06Release-Engineering-Team, 13Patch-For-Review, 07Security-General: Download of composer cweiske/php-sqllint requires to disable https security - https://phabricator.wikimedia.org/T153842#2899079 (10hashar) antoine-approve [15:39:49] PROBLEM - Puppet run on deployment-phab02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:46:09] 10Continuous-Integration-Infrastructure, 06Collaboration-Team-Triage, 10Thanks: Merge failed due to unrelated test failure in Thanks - https://phabricator.wikimedia.org/T154029#2899095 (10Paladox) [15:47:03] 10Continuous-Integration-Infrastructure, 06Collaboration-Team-Triage, 10Thanks: Merge failed due to unrelated test failure in Thanks - https://phabricator.wikimedia.org/T154029#2898999 (10Paladox) Fails with 12:50:07 .......................PHP Fatal error: Call to a member function getId() on a non-object... [16:09:42] (03PS1) 10Zfilipin: mediawiki-core-selenium-jessie treats all XML files as JUnit reports [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) [16:11:07] (03PS2) 10Zfilipin: mediawiki-core-selenium-jessie treats all XML files as JUnit reports [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) [16:11:20] (03CR) 10Zfilipin: "#2 is a rebase" [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) (owner: 10Zfilipin) [16:15:56] PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [16:18:04] (03CR) 10Hashar: [C: 031] "Good to go :}" [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) (owner: 10Zfilipin) [16:36:26] (03CR) 10Zfilipin: [C: 032] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) (owner: 10Zfilipin) [16:38:24] (03Merged) 10jenkins-bot: mediawiki-core-selenium-jessie treats all XML files as JUnit reports [integration/config] - 10https://gerrit.wikimedia.org/r/328918 (https://phabricator.wikimedia.org/T151442) (owner: 10Zfilipin) [16:50:54] RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0] [16:51:29] 10Continuous-Integration-Infrastructure, 06Collaboration-Team-Triage, 10Thanks: Merge failed due to unrelated test failure in Thanks - https://phabricator.wikimedia.org/T154029#2899167 (10hashar) 05Open>03Resolved p:05Unbreak!>03High a:03hashar The last change to Thanks passed properly at 13:49UTC... [17:04:16] 10Continuous-Integration-Infrastructure, 06Collaboration-Team-Triage, 10Thanks: Merge failed due to unrelated test failure in Thanks - https://phabricator.wikimedia.org/T154029#2899211 (10hashar) And I have looked in the log history of HHVM and Zend build, there is no other occurrence of that fatal error. So... [17:14:26] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2899242 (10hashar) @Osnard we have to identify the issue in Gerrit/Git which is impacting a fe... [17:41:55] PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [17:43:27] 10Gerrit, 10BlueSpice, 07Upstream: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2899290 (10Paladox) I'm wondering would checking out the commit you have open for review on br... [18:21:54] RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0] [18:27:37] 10Gerrit, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Root for Mukunda for Gerrit machine(s) - https://phabricator.wikimedia.org/T152236#2842574 (10Dzahn) I confirmed the access on cobalt. ``` [cobalt:~] $ id twentyafterfour uid=4967(twentyafterfour) gid=500(wikidev) groups=500(wikidev),703(... [18:27:52] 10Gerrit, 06Operations, 10Ops-Access-Requests, 13Patch-For-Review: Root for Mukunda for Gerrit machine(s) - https://phabricator.wikimedia.org/T152236#2899343 (10Dzahn) 05Open>03Resolved a:03Dzahn [18:37:19] 10Gerrit, 06Operations, 10Ops-Access-Requests: Root for Mukunda for Gerrit machine(s) - https://phabricator.wikimedia.org/T152236#2899355 (10Dzahn) [18:53:30] fyi: https://about.gitlab.com/2016/12/22/gitlab-8-15-released/ [19:09:15] 10Continuous-Integration-Config, 05Goal, 07I18n: Configure banana checker for i18n files to run on all MediaWiki extensions and skins - https://phabricator.wikimedia.org/T94547#2899453 (10Umherirrender) [23:02:25] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:02:55] 10Gerrit, 10MediaWiki-Vagrant, 13Patch-For-Review: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#2899920 (10Tgr) Note that in both cases the error is on the server side - apparently something breaks so bad that it can't even handle the connection pr...