[00:06:29] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885544 (10Paladox) It could be related to this bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=846648 and https://bugs.debian.org/cgi-...
[00:12:47] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885545 (10Paladox) Lets downgrade to chrome 53 as it seems it is broken in chrome 54 too https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=8...
[02:19:51] <grrrit-wm>	 (03PS2) 10Legoktm: Add prometheus-related repositories [integration/config] - 10https://gerrit.wikimedia.org/r/327692 (owner: 10Filippo Giunchedi) 
[02:20:42] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Add prometheus-related repositories (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/327692 (owner: 10Filippo Giunchedi) 
[02:21:38] <grrrit-wm>	 (03Merged) 10jenkins-bot: Add prometheus-related repositories [integration/config] - 10https://gerrit.wikimedia.org/r/327692 (owner: 10Filippo Giunchedi) 
[02:25:18] <legoktm>	 !log deploying https://gerrit.wikimedia.org/r/327692
[02:25:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[02:50:50] <TimStarling>	 I think it's time to move RemexHtml to proper version control
[02:51:34] <TimStarling>	 I started it off under tstarling/ in github since it wasn't clear how far I was going to take it, or whether it would end up integrated with something else
[03:15:06] <legoktm>	 TimStarling: mediawiki/libs/Remex ?
[03:16:19] <TimStarling>	 something like that I guess
[03:16:35] <TimStarling>	 how does the github bridge work? I see the existing composer libraries pull from github
[03:17:46] <TimStarling>	 if I want to use remex from MW core, I'm going to need it to be working in packagist
[03:18:03] <legoktm>	 You need to manually create a empty repository on github with the name repo.replace('/','-') and then Gerrit will automatically mirror the repo after a commit is merged. 
[03:18:33] <legoktm>	 We point packagist at github because then it has code to download zips from github, which it caches instead of cloning using git (and doesn't get cached)
[03:19:28] <legoktm>	 And once the Gerrit -> Github replication runs at least once, you can rename the github repo to something else (drop mediawiki-libs-) and Gerrit will follow the redirect properly
[03:19:42] <legoktm>	 (so like mediawiki/core is just mediawiki on github)
[03:21:38] <TimStarling>	 I'm only a "member" of wikimedia on github, I can't create repositories
[03:22:41] <legoktm>	 >  Made tstarling an owner. 
[03:22:56] <TimStarling>	 thanks
[03:22:56] <legoktm>	 You should also enable 2fa for github
[03:35:50] <TimStarling>	 done
[04:18:30] <wmf-insecte>	 Yippee, build fixed!
[04:18:31] <wmf-insecte>	 Project selenium-MultimediaViewer » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #238: 09FIXED in 22 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/238/
[04:34:15] <TimStarling>	 force push is not allowed anymore, even with configuration?
[04:34:30] <TimStarling>	 there's about 30 existing commits
[04:35:29] <TimStarling>	 I guess I'll just submit them
[04:54:47] <TimStarling>	 ah, but then they would all have to have change-id footers, which means the repository can't have the same state as the existing one on github
[04:56:40] <TimStarling>	 never mind, I'm just forgetting how to use gerrit apparently
[05:03:43] <Zppix>	 TimStarling: dont screw up gerrit only cause you use github now. :P
[05:03:56] <Zppix>	 Anyway need a hand?
[05:04:51] <TimStarling>	 I'll let you know if anything goes wrong, thanks
[05:05:59] <TimStarling>	 there it is: https://github.com/wikimedia/mediawiki-libs-RemexHtml
[05:06:09] <wikibugs>	 03Scap3, 15User-mobrovac: Scap deploy failed to sync git-fat artifacts - https://phabricator.wikimedia.org/T147856#2885665 (10thcipriani) I managed to make the scap-vagrant box reproduce this (although I don't have steps to reproduce, just tried deploys until a large binary failed to hydrate).  The files are m...
[05:07:08] <TimStarling>	 I wonder if tags are mirrored
[05:09:46] <legoktm>	 TimStarling: yes, tags are mirrored
[06:02:32] <wikibugs_>	 10Gerrit: GitHub repository descriptions should point directly to the Gerrit projects - https://phabricator.wikimedia.org/T153614#2885704 (10scfc)
[07:24:21] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885148 (10Nikerabbit) Also affects Translate btw. The "script error" is another thing that hints at a Cross-Origin issue, which is also what...
[07:25:26] <wikibugs>	 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-ContentTranslation, 10Wikidata, 07Jenkins, 07WorkType-Maintenance: Wikibase Qunit test fails with CX + ULS - https://phabricator.wikimedia.org/T120907#2885780 (10Nikerabbit) Is this still happening? Is there trace with debug=true?
[07:51:05] <grrrit-wm>	 (03PS9) 10Legoktm: [CentralAuth] Add composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/288819 (owner: 10Paladox) 
[07:51:07] <grrrit-wm>	 (03PS3) 10Legoktm: [EducationProgram] Add composer-test and move jshint to check: [integration/config] - 10https://gerrit.wikimedia.org/r/276065 (owner: 10Paladox) 
[07:51:09] <grrrit-wm>	 (03PS3) 10Legoktm: Make mwext-PoolCounter-build-jessie more like the Debian package [integration/config] - 10https://gerrit.wikimedia.org/r/325144 (https://phabricator.wikimedia.org/T152338) 
[07:51:11] <grrrit-wm>	 (03PS1) 10Legoktm: Configure jobs for mediawiki/libs/RemexHtml [integration/config] - 10https://gerrit.wikimedia.org/r/328136 
[07:54:49] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Make mwext-PoolCounter-build-jessie more like the Debian package [integration/config] - 10https://gerrit.wikimedia.org/r/325144 (https://phabricator.wikimedia.org/T152338) (owner: 10Legoktm) 
[07:54:55] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Configure jobs for mediawiki/libs/RemexHtml [integration/config] - 10https://gerrit.wikimedia.org/r/328136 (owner: 10Legoktm) 
[07:55:00] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] [EducationProgram] Add composer-test and move jshint to check: [integration/config] - 10https://gerrit.wikimedia.org/r/276065 (owner: 10Paladox) 
[07:55:07] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] [CentralAuth] Add composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/288819 (owner: 10Paladox) 
[07:56:03] <grrrit-wm>	 (03Merged) 10jenkins-bot: [CentralAuth] Add composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/288819 (owner: 10Paladox) 
[07:56:05] <grrrit-wm>	 (03Merged) 10jenkins-bot: [EducationProgram] Add composer-test and move jshint to check: [integration/config] - 10https://gerrit.wikimedia.org/r/276065 (owner: 10Paladox) 
[07:56:44] <grrrit-wm>	 (03Merged) 10jenkins-bot: Make mwext-PoolCounter-build-jessie more like the Debian package [integration/config] - 10https://gerrit.wikimedia.org/r/325144 (https://phabricator.wikimedia.org/T152338) (owner: 10Legoktm) 
[07:56:53] <grrrit-wm>	 (03Merged) 10jenkins-bot: Configure jobs for mediawiki/libs/RemexHtml [integration/config] - 10https://gerrit.wikimedia.org/r/328136 (owner: 10Legoktm) 
[08:00:00] <legoktm>	 !log deploying https://gerrit.wikimedia.org/r/288819 https://gerrit.wikimedia.org/r/276065 https://gerrit.wikimedia.org/r/328136
[08:00:04] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[08:06:50] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885869 (10Paladox) @Nikerabbit hi, this looks like a different problem to that as it does  16:56:07 Chrome 55.0.2883 (Linux 0.0.0) ERROR 16:...
[08:14:08] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885879 (10Nikerabbit) The error is different, but symptom **might** be the same. Please read https://sentry.io/answers/javascript-script-err...
[08:15:22] <wikibugs_>	 10Gerrit, 10GitHub-Mirrors: GitHub repository descriptions should point directly to the Gerrit projects - https://phabricator.wikimedia.org/T153614#2885881 (10Paladox)
[08:16:39] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885897 (10Paladox) Oh thanks, I guess something big changed between chrome 53 and 54 to cause the cross origin error to spread.
[08:18:45] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2885921 (10Paladox) Should we set Access-Control-Allow-Origin: * in apache?
[09:13:41] <shinken-wm>	 PROBLEM - Puppet run on deployment-redis02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[09:14:00] <hashar>	 o/
[09:15:29] <shinken-wm>	 PROBLEM - Puppet run on deployment-logstash2 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[09:17:03] <shinken-wm>	 PROBLEM - Puppet run on deployment-apertium02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[09:17:55] <shinken-wm>	 PROBLEM - Puppet run on deployment-fluorine02 is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0]
[09:18:21] <shinken-wm>	 PROBLEM - Puppet run on deployment-kafka04 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:18:25] <shinken-wm>	 PROBLEM - Puppet run on deployment-mcs01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[09:18:28] <shinken-wm>	 PROBLEM - Puppet run on deployment-zookeeper01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:18:48] <shinken-wm>	 PROBLEM - Puppet run on deployment-sca03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[09:20:12] <shinken-wm>	 PROBLEM - Puppet run on deployment-salt02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[09:20:32] <shinken-wm>	 PROBLEM - Puppet run on deployment-pdfrender02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[09:20:34] <shinken-wm>	 PROBLEM - Puppet run on deployment-elastic06 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[09:21:08] <shinken-wm>	 PROBLEM - Puppet run on deployment-jobrunner02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[09:22:00] <shinken-wm>	 PROBLEM - Puppet run on deployment-mira is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[09:22:04] <shinken-wm>	 PROBLEM - Puppet run on deployment-copper is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[09:22:24] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-docker-1000 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[09:22:56] <shinken-wm>	 PROBLEM - Puppet run on deployment-phab01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]
[09:23:15] <shinken-wm>	 PROBLEM - Puppet run on deployment-zotero01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:23:57] <shinken-wm>	 PROBLEM - Puppet run on deployment-prometheus01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[09:24:23] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-jessie-android is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:24:23] <shinken-wm>	 PROBLEM - Puppet run on deployment-changeprop is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:27:28] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]
[09:32:23] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-trusty-1006 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:33:09] <shinken-wm>	 PROBLEM - Puppet run on integration-publisher is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:33:55] <grrrit-wm>	 (03PS1) 10Legoktm: Revert "Configure jobs for mediawiki/libs/RemexHtml" [integration/config] - 10https://gerrit.wikimedia.org/r/328145 
[09:33:59] <grrrit-wm>	 (03CR) 10Legoktm: [C: 032] Revert "Configure jobs for mediawiki/libs/RemexHtml" [integration/config] - 10https://gerrit.wikimedia.org/r/328145 (owner: 10Legoktm) 
[09:34:55] <grrrit-wm>	 (03Merged) 10jenkins-bot: Revert "Configure jobs for mediawiki/libs/RemexHtml" [integration/config] - 10https://gerrit.wikimedia.org/r/328145 (owner: 10Legoktm) 
[09:35:06] <legoktm>	 !log deploying https://gerrit.wikimedia.org/r/328145
[09:35:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:35:40] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-trusty-1011 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:35:56] <shinken-wm>	 PROBLEM - Puppet run on integration-saltmaster is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[09:39:43] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-precise-1011 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:41:04] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[09:42:06] <grrrit-wm>	 (03PS1) 10Hashar: Add new MediaWiki extensions [integration/config] - 10https://gerrit.wikimedia.org/r/328146 
[09:42:52] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:42:56] <shinken-wm>	 PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:42:58] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-precise-1012 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[09:45:51] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-jessie-1002 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[09:46:05] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[09:47:53] <hashar>	 puppet spam should be fixed
[09:47:58] <hashar>	 just had to restart the puppet master
[09:48:06] <hashar>	 ( systemctl restart apache2 )
[09:48:31] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] Add new MediaWiki extensions [integration/config] - 10https://gerrit.wikimedia.org/r/328146 (owner: 10Hashar) 
[09:49:19] <grrrit-wm>	 (03Merged) 10jenkins-bot: Add new MediaWiki extensions [integration/config] - 10https://gerrit.wikimedia.org/r/328146 (owner: 10Hashar) 
[09:53:22] <shinken-wm>	 RECOVERY - Puppet run on deployment-kafka04 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:53:42] <shinken-wm>	 RECOVERY - Puppet run on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:53:46] <wikibugs_>	 10Continuous-Integration-Config, 10MediaWiki-extensions-PoolCounter, 13Patch-For-Review: PoolCounter daemon CI tests should use dependencies from debian, not bundler - https://phabricator.wikimedia.org/T152338#2886089 (10Legoktm) 05Open>03Resolved a:03Legoktm
[09:55:10] <shinken-wm>	 RECOVERY - Puppet run on deployment-salt02 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:55:28] <shinken-wm>	 RECOVERY - Puppet run on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:55:30] <shinken-wm>	 RECOVERY - Puppet run on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:55:35] <shinken-wm>	 RECOVERY - Puppet run on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:56:09] <shinken-wm>	 RECOVERY - Puppet run on deployment-jobrunner02 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:57:51] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:57:53] <shinken-wm>	 RECOVERY - Puppet run on deployment-fluorine02 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:57:57] <shinken-wm>	 RECOVERY - Puppet run on deployment-phab01 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:58:13] <shinken-wm>	 RECOVERY - Puppet run on deployment-zotero01 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:58:23] <shinken-wm>	 RECOVERY - Puppet run on deployment-mcs01 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:58:29] <shinken-wm>	 RECOVERY - Puppet run on deployment-zookeeper01 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:58:45] <shinken-wm>	 RECOVERY - Puppet run on deployment-sca03 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:58:57] <shinken-wm>	 RECOVERY - Puppet run on deployment-prometheus01 is OK: OK: Less than 1.00% above the threshold [0.0]
[09:59:23] <shinken-wm>	 RECOVERY - Puppet run on deployment-changeprop is OK: OK: Less than 1.00% above the threshold [0.0]
[10:02:00] <shinken-wm>	 RECOVERY - Puppet run on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0]
[10:02:04] <shinken-wm>	 RECOVERY - Puppet run on deployment-apertium02 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:02:04] <shinken-wm>	 RECOVERY - Puppet run on deployment-copper is OK: OK: Less than 1.00% above the threshold [0.0]
[10:02:26] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-docker-1000 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:02:26] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-trusty-1004 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:04:24] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-jessie-android is OK: OK: Less than 1.00% above the threshold [0.0]
[10:07:21] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-trusty-1006 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:08:09] <shinken-wm>	 RECOVERY - Puppet run on integration-publisher is OK: OK: Less than 1.00% above the threshold [0.0]
[10:11:44] <grrrit-wm>	 (03PS1) 10Hashar: [WikimediaMessages] add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/328149 (https://phabricator.wikimedia.org/T150332) 
[10:11:59] <wikibugs_>	 10Continuous-Integration-Config, 07I18n, 13Patch-For-Review, 05WMF-deploy-2017-01-03_(1.29.0-wmf.7): Wikimedia deployed extensions lack npm entry point / banana checker - https://phabricator.wikimedia.org/T150332#2886146 (10hashar)
[10:13:00] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] [WikimediaMessages] add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/328149 (https://phabricator.wikimedia.org/T150332) (owner: 10Hashar) 
[10:13:46] <grrrit-wm>	 (03Merged) 10jenkins-bot: [WikimediaMessages] add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/328149 (https://phabricator.wikimedia.org/T150332) (owner: 10Hashar) 
[10:15:40] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-trusty-1011 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:15:56] <shinken-wm>	 RECOVERY - Puppet run on integration-saltmaster is OK: OK: Less than 1.00% above the threshold [0.0]
[10:16:48] <wikibugs_>	 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development, 13Patch-For-Review: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2886184 (10ArielGlenn) >>! In T148494#2880204, @Volans wrote: > @ArielGlenn it's surely depends on the specific cases, but I thi...
[10:19:43] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-precise-1011 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:21:03] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:21:04] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:21:26] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2886204 (10Paladox) Should we make the qunit test non voting for now?  Is this a bug in chrome or does something need to be updated in mw to...
[10:22:54] <shinken-wm>	 RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0]
[10:22:58] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-precise-1012 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:25:49] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:55:59] <wikibugs_>	 10Continuous-Integration-Config, 13Patch-For-Review, 07Upstream, 07Zuul: free some repositories from their unintended chain to mediawiki/core - https://phabricator.wikimedia.org/T107529#2886305 (10hashar)
[10:56:01] <wikibugs>	 10Continuous-Integration-Config, 10OOjs-UI, 13Patch-For-Review: Move OOUI out of the MediaWiki gate-and-submit queue - https://phabricator.wikimedia.org/T134946#2886306 (10hashar)
[10:58:03] <wikibugs_>	 10Continuous-Integration-Config, 10OOjs-UI, 13Patch-For-Review: Move OOUI out of the MediaWiki gate-and-submit queue - https://phabricator.wikimedia.org/T134946#2283244 (10hashar) `oojs/ui` is decoupled but `oojs/core` is still in the shared queue due to the job `npm-node-4`.  There is a more generic task T1...
[11:00:36] <grrrit-wm>	 (03PS1) 10Hashar: [oojs/core] decouple from mediawiki pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/328156 (https://phabricator.wikimedia.org/T134946) 
[11:03:28] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] [oojs/core] decouple from mediawiki pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/328156 (https://phabricator.wikimedia.org/T134946) (owner: 10Hashar) 
[11:03:57] <wikibugs>	 10Continuous-Integration-Config, 13Patch-For-Review, 07Upstream, 07Zuul: free some repositories from their unintended chain to mediawiki/core - https://phabricator.wikimedia.org/T107529#2886328 (10hashar)
[11:04:00] <wikibugs_>	 10Continuous-Integration-Config, 10OOjs-UI, 13Patch-For-Review: Move OOUI out of the MediaWiki gate-and-submit queue - https://phabricator.wikimedia.org/T134946#2886325 (10hashar) 05Open>03Resolved a:03hashar So it is indeed fixed. The generalization will be handled via T107529
[11:05:00] <grrrit-wm>	 (03Merged) 10jenkins-bot: [oojs/core] decouple from mediawiki pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/328156 (https://phabricator.wikimedia.org/T134946) (owner: 10Hashar) 
[11:07:02] <grrrit-wm>	 (03PS10) 10Hashar: Test mediawiki gate queue only has mw projects [integration/config] - 10https://gerrit.wikimedia.org/r/241660 (https://phabricator.wikimedia.org/T107529) 
[11:23:11] <wikibugs_>	 10Gerrit, 06Release-Engineering-Team: Gerrit java.lang.ArrayIndexOutOfBoundsException when querying changes - https://phabricator.wikimedia.org/T153642#2886349 (10hashar)
[11:29:37] <wikibugs_>	 10Gerrit, 06Release-Engineering-Team: Gerrit lacks a 'change' cache - https://phabricator.wikimedia.org/T153645#2886392 (10hashar)
[12:01:47] <wmf-insecte>	 Yippee, build fixed!
[12:01:48] <wmf-insecte>	 Project selenium-RelatedArticles » chrome,beta-mobile,Linux,contintLabsSlave && UbuntuTrusty build #247: 09FIXED in 46 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-mobile,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/247/
[12:05:20] <grrrit-wm>	 (03PS1) 10Phedenskog: Run WebPageTest jobs every hour [integration/config] - 10https://gerrit.wikimedia.org/r/328159 (https://phabricator.wikimedia.org/T151197) 
[12:26:51] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2886834 (10Osnard) This seems to occur only on repo `mediawiki/extensions/BlueSpiceExtensions`, but there on all changes to branch `REL1_27` [1]. Th...
[12:35:28] <wikibugs_>	 06Release-Engineering-Team, 10MediaWiki-JobRunner, 07HHVM, 07Wikimedia-log-errors: Jobs issue due to HHVM json_decode() error - https://phabricator.wikimedia.org/T153676#2886866 (10hashar)
[12:42:09] <wikibugs>	 06Release-Engineering-Team, 10MediaWiki-JobRunner, 07HHVM, 07Wikimedia-log-errors: Jobs issue due to HHVM json_decode() error - https://phabricator.wikimedia.org/T153676#2886885 (10hashar) I think the trick is the rpc runJobs.php is supposed to returns a json status of the jobs that run.  But some jobs do...
[12:53:49] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2886887 (10Paladox) @Osnard hi can you merge anything in BlueSpiceExtensions?  If not then I think this task priority should be unbreak.
[12:58:14] <wikibugs_>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2886888 (10Osnard) Not in `REL1_27`. In `master` it works. Unfortunately we deliver only from our LTS branches `REL1_23` and `REL1_27`not from `mast...
[13:02:04] <wikibugs_>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2886892 (10Paladox) I guess this would make this a higher priority task for that branch.  @demon I wonder if we could create a REL1_271 branch then...
[13:06:41] <zeljkof>	 hashar: can you take a look at this again? https://gerrit.wikimedia.org/r/#/c/327211/
[13:06:52] <zeljkof>	 I think I have done everything that you have suggested
[13:07:29] <wikibugs_>	 06Release-Engineering-Team, 10MediaWiki-JobRunner, 07HHVM, 07Wikimedia-log-errors: Jobs issue due to HHVM json_decode() error - https://phabricator.wikimedia.org/T153676#2886894 (10hashar) And I guess the root cause is operations/mediawiki-config rpc/RunJobs.php. It runs the job and emits a json_encode of...
[13:07:59] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2886896 (10Mglaser) p:05High>03Unbreak! Following this discussion, I took the liberty of raising the priority.
[13:09:09] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] Fix NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) (owner: 10Zfilipin) 
[13:09:15] <hashar>	 zeljkof: yeah good to me :
[13:09:16] <hashar>	 }
[13:09:24] <zeljkof>	 hashar: thanks! :)
[13:11:06] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fix NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) (owner: 10Zfilipin) 
[13:13:59] <wikibugs_>	 10Browser-Tests-Infrastructure, 13Patch-For-Review, 07Ruby, 15User-zeljkofilipin: NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords - https://phabricator.wikimedia.org/T153202#2886904 (10zeljkofilipin) 05Open>03Resolved
[13:43:56] <shinken-wm>	 PROBLEM - Puppet run on repository is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[13:58:16] <wikibugs_>	 06Release-Engineering-Team, 10MediaWiki-JobRunner, 07HHVM, 07Wikimedia-log-errors: Jobs issue due to HHVM json_decode() error - https://phabricator.wikimedia.org/T153676#2886852 (10Joe) please note this doesn't really pose any serious issue to the jobs being processed.
[14:05:07] <paladox>	 hashar hi, im wondering could you look into https://phabricator.wikimedia.org/T153597 please?
[14:14:24] <wikibugs>	 10Continuous-Integration-Infrastructure, 07Jenkins, 07Upstream, 07WorkType-NewFunctionality: Jenkins trilead-ssh2 doesn't support our MAC/KEX algorithms - https://phabricator.wikimedia.org/T103351#2886980 (10Paladox) I've created this https://github.com/jenkinsci/ssh-slaves-plugin/issues/39 upstream.  Also...
[14:23:54] <shinken-wm>	 RECOVERY - Puppet run on repository is OK: OK: Less than 1.00% above the threshold [0.0]
[14:32:24] <wikibugs>	 10Continuous-Integration-Infrastructure, 07Jenkins, 07Upstream, 07WorkType-NewFunctionality: Jenkins trilead-ssh2 doesn't support our MAC/KEX algorithms - https://phabricator.wikimedia.org/T103351#2887036 (10Paladox) This https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310849&version=123...
[14:52:45] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2887048 (10Aklapper)
[15:39:49] <shinken-wm>	 PROBLEM - Puppet run on deployment-phab02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[15:43:03] <andrewbogott>	 ostriches: regarding my ops-list email about python package upgrades… can we run a scap test with the new packages in deployment-prep?  Or does scap have its own testing env?
[15:51:47] <wmf-insecte>	 Project selenium-MobileFrontend » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #265: 04FAILURE in 29 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/265/
[16:04:53] <hashar>	 andrewbogott: we have a scap/vagrant test environment
[16:05:16] <hashar>	 andrewbogott: at least scap only depends on requests :D
[16:05:35] <andrewbogott>	 hashar: where is that?
[16:05:45] <hashar>	 in differential I believe
[16:06:06] <andrewbogott>	 oh, so not actually running anywhere?
[16:06:06] <hashar>	 andrewbogott: https://phabricator.wikimedia.org/diffusion/GSV/
[16:06:22] <hashar>	 then
[16:07:35] <hashar>	 maybe the package will not be installed unless they apt::pin the package to whatever component ( jessie-wikimedia/backports ?) 
[16:08:22] <wikibugs_>	 10Continuous-Integration-Infrastructure, 07Jenkins, 07Upstream, 07WorkType-NewFunctionality: Jenkins trilead-ssh2 doesn't support our MAC/KEX algorithms - https://phabricator.wikimedia.org/T103351#2887177 (10hashar) Feel free to raise it on the upstream bug tracker: https://issues.jenkins-ci.org/browse/JEN...
[16:08:33] <andrewbogott>	 ah, yeah, I'd have to be able to dpkg —install after the fact
[16:08:34] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: Reenable ssh MAC/KEX hardening on beta cluster and integration labs project - https://phabricator.wikimedia.org/T100518#2887179 (10hashar)
[16:08:36] <wikibugs_>	 10Continuous-Integration-Infrastructure, 06Operations, 13Patch-For-Review, 07WorkType-Maintenance: Jenkins master / client ssh connection fails due to missing ssh algorithm - https://phabricator.wikimedia.org/T100509#2887180 (10hashar)
[16:08:38] <wikibugs>	 10Continuous-Integration-Infrastructure, 07Jenkins, 07Upstream, 07WorkType-NewFunctionality: Jenkins trilead-ssh2 doesn't support our MAC/KEX algorithms - https://phabricator.wikimedia.org/T103351#2887178 (10hashar) 05Open>03stalled
[16:11:11] <wikibugs_>	 10Continuous-Integration-Config, 07I18n, 13Patch-For-Review, 05WMF-deploy-2017-01-03_(1.29.0-wmf.7): Wikimedia deployed extensions lack npm entry point / banana checker - https://phabricator.wikimedia.org/T150332#2887182 (10hashar)
[16:12:25] <wikibugs>	 10Continuous-Integration-Config, 07I18n, 13Patch-For-Review, 05WMF-deploy-2017-01-03_(1.29.0-wmf.7): Wikimedia deployed extensions lack npm entry point / banana checker - https://phabricator.wikimedia.org/T150332#2782348 (10hashar)
[16:13:21] <wikibugs_>	 10Continuous-Integration-Config, 06Release-Engineering-Team, 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream: Improve tests for KafkaSSE - https://phabricator.wikimedia.org/T150436#2887184 (10mforns) a:05mforns>03JAllemandou
[16:48:57] <ostriches>	 andrewbogott: we could use beta
[16:49:15] <ostriches>	 we run master branch of scap there
[16:50:20] <andrewbogott>	 ostriches: would you be available to walk me through that later today?  (you have a meeting in 10 minutes iirc)
[16:51:19] <ostriches>	 I've got a few meetings but after that ya
[16:51:26] <andrewbogott>	 thanks!
[16:51:56] <Zppix>	 andrewbogott: shouldnt we be on .7 even though no train since holiday ?
[16:52:30] <andrewbogott>	 Zppix: I don't think I'm the right person to ask
[16:52:48] <Zppix>	 Ok ill try again in ops
[16:52:53] <Zppix>	 Thanks!
[16:54:31] <wikibugs_>	 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development, 13Patch-For-Review: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2887285 (10Volans) @ArielGlenn remember that if the file will still be a `.py.erb` it will be skipped by our tox checker as of n...
[16:55:39] <Zppix>	 On behalf of my family to yours, thanks WMF staff and volunteers for all you do, have a safe and merry christmas.
[16:57:20] <wikibugs_>	 03Scap3, 15User-mobrovac: Scap deploy failed to sync git-fat artifacts - https://phabricator.wikimedia.org/T147856#2705818 (10Eevans) >>! In T147856#2885665, @thcipriani wrote: [ ... ] > I'm not sure what the "right" fix is for this inside scap is. Certainly an ugly bug.  Nice analysis!
[17:03:49] <ostriches>	 Zppix: No, we're on wmf.6. We missed a week back during Thanksgiving.
[17:03:56] <ostriches>	 Maybe that's where you're off-by-one?
[17:04:18] <Zppix>	 ostriches: i got my answer already from ops
[17:04:24] <Zppix>	 Thanks though!
[17:15:55] <bd808>	 Zppix: you can get all the important links about deployments and roadmaps from https://tools.wmflabs.org/versions/
[17:16:08] <bd808>	 and see what's running where on the prod cluster
[17:20:10] <Zppix>	 bd808: i knew that already but thanks!
[17:37:38] <grrrit-wm>	 (03PS1) 10Hashar: zuul: fix appending to SKIN_DEPENDENCIES [integration/config] - 10https://gerrit.wikimedia.org/r/328198 
[17:39:27] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] zuul: fix appending to SKIN_DEPENDENCIES [integration/config] - 10https://gerrit.wikimedia.org/r/328198 (owner: 10Hashar) 
[17:41:42] <grrrit-wm>	 (03PS2) 10Hashar: zuul: fix appending to SKIN_DEPENDENCIES [integration/config] - 10https://gerrit.wikimedia.org/r/328198 
[17:43:52] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] zuul: fix appending to SKIN_DEPENDENCIES [integration/config] - 10https://gerrit.wikimedia.org/r/328198 (owner: 10Hashar) 
[17:45:17] <grrrit-wm>	 (03Merged) 10jenkins-bot: zuul: fix appending to SKIN_DEPENDENCIES [integration/config] - 10https://gerrit.wikimedia.org/r/328198 (owner: 10Hashar) 
[18:12:05] <wikibugs_>	 10Gerrit, 10GitHub-Mirrors: GitHub repository descriptions should point directly to the Gerrit projects - https://phabricator.wikimedia.org/T153614#2885704 (10greg) Where is this text managed?
[18:26:20] <wikibugs_>	 10Gerrit, 10GitHub-Mirrors: GitHub repository descriptions should point directly to the Gerrit projects - https://phabricator.wikimedia.org/T153614#2885704 (10demon) >>! In T153614#2887678, @greg wrote: > Where is this text managed?  It isn't. A long time ago when I had the plugin that auto-created Github repo...
[19:37:20] <shinken-wm>	 PROBLEM - Puppet run on deployment-tmh01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[19:44:54] <SMalyshev>	 is something wrong with qunit tests? I'm getting failures on patches where everything worked before, e.g. https://integration.wikimedia.org/ci/job/mediawiki-extensions-qunit-jessie/19472/console
[19:45:31] <Reedy>	 SMalyshev: It's been reported yeah
[19:45:35] <Reedy>	 I think there's a bug for it
[19:45:45] <Reedy>	 Something to do with a Chrom(e|ium) change
[19:55:27] <andrewbogott>	 ostriches: are you post-meeting (and post-lunch)?
[19:59:17] <paladox>	 SMalyshev: Hi please see https://phabricator.wikimedia.org/T153597
[20:00:49] <SMalyshev>	 paladox: thank you! looks like it breaks all merges for CirrusSearch :( will wait for resolution then
[20:01:01] <paladox>	 Ok, your welcome
[20:01:35] <paladox>	 SMalyshev most likly we may have to go back to chromium 53 since the problem exists in chrome 53 but did not affect as much as it does now.
[20:02:01] <paladox>	 See https://phabricator.wikimedia.org/T153038
[20:03:10] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2888066 (10Paladox) Another user reported this problem on irc  <SMalyshev> is something wrong with qunit tests? I'm getting failures on patch...
[20:04:33] <wikibugs>	 10Continuous-Integration-Config, 10MediaWiki-extensions-ContentTranslation, 05Language-Engineering October-December 2016, 03Language-Q2-2016-17 Sprint 5, and 2 others: mwext-qunit-jessie test fails on unrelated change - https://phabricator.wikimedia.org/T153038#2888071 (10Paladox) Since the upgrade to chro...
[20:16:42] <wikibugs>	 03Scap3, 15User-mobrovac: Scap deploy failed to sync git-fat artifacts - https://phabricator.wikimedia.org/T147856#2888122 (10fgiunchedi) To add to last weekend's event about twcs repo, at the time I forced `git fat pull` on restbase1* nodes after finding some were not updated:   ``` root@neodymium:~# salt 're...
[20:24:52] <ostriches>	 andrewbogott: Finishing up lunch right now :)
[20:36:23] <andrewbogott>	 ostriches: so, I want to test scap and eventlogging.  I'm guessing that there's probably something we can do that both scaps and logs things from the scap?
[20:36:46] <ostriches>	 eventlogging uses scap3. Right thcipriani?
[20:37:12] <thcipriani>	 I think so... /me doublechecks
[20:37:14] <ostriches>	 Er, that won't really test eventlogging :\
[20:37:20] <ostriches>	 Unless it has checks
[20:37:41] <wikibugs_>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597#2888176 (10Paladox) Oh wow stretch is using an old version of chromium https://packages.debian.org/source/stretch/chrom...
[20:38:01] <andrewbogott>	 Yeah — I don't actually know an example of what tests eventlogging except I'm guessing… everything
[20:38:23] <thcipriani>	 ah, no it doesn't, it's blocked on: https://phabricator.wikimedia.org/T131977
[20:43:58] <andrewbogott>	 ostriches: I have no idea what I'm doing :)  I presume that scap has a client and a server, so I should do a 'scap pull' on an appserver in beta, then upgrade pacakages on server and that appserver, then 'scap pull' again and declare victory if it doesn't throw any errors...
[20:44:17] <andrewbogott>	 Does that sound roughly correct?  Or is the 'server' in this case just gerrit.wikimedia.org?
[20:44:30] <ostriches>	 Something like that. Or just upgrade the packages on both, then wait 15-20m for the auto-scap-run to go
[20:44:31] <ostriches>	 :)
[20:44:49] <ostriches>	 (and no, not gerrit.wm.o, deployment-tin and deployment-mediawiki*)
[20:45:22] <andrewbogott>	 ok
[20:46:18] <andrewbogott>	 ostriches: and an appserver would like deployment-mediawiki01?
[20:46:27] <ostriches>	 Yep
[20:47:01] <ottomata>	 hiya, how do I tell if an extension chnage that has been merged is live on beta?
[20:47:39] <andrewbogott>	 ottomata: https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:Version ?
[20:48:01] <ottomata>	 AMAZING
[20:48:16] <ottomata>	 thanks andrewbogott
[20:48:34] <ostriches>	 That, plus anything in master will be live on beta Real Soon™
[20:48:36] <ostriches>	 :)
[20:52:25] <andrewbogott>	 ostriches: ok, I upgraded and everything looks the same to me.  Let me know if something starts to misbehave.
[20:52:36] <andrewbogott>	 ostriches: Any thoughts about how I can test eventlogging?
[21:07:54] <ostriches>	 Not a clue, I know nothing about it :(
[21:10:55] <andrewbogott>	 ostriches: ok!
[21:11:07] * andrewbogott waits quietly for an expert to appear
[21:11:29] <greg-g>	 andrewbogott: probably won't find an expert on eventlogging in here :)
[21:11:44] <andrewbogott>	 greg-g: it runs in beta though, doesn't it?
[21:12:39] <greg-g>	 andrewbogott: sure, but we don't know how every single peice of software that runs there works. We aren't operations. There's only 6 of us
[21:12:56] <andrewbogott>	 that's fine, I'm just making sure
[21:13:00] <greg-g>	 :)
[21:13:14] <ottomata>	 andrewbogott:  can I help?
[21:13:41] <andrewbogott>	 ottomata: maybe?  I just want to try upgrading some pacackage dependencies, restart eventlogging, and verify that it still works
[21:13:56] <andrewbogott>	 For which I need to know 1) where it runs 2) how to restart it 3) how to tell if it's working properly
[21:16:02] <Zppix>	 greg-g: im interested in helping with releng.
[21:16:13] <ottomata>	 andrewbogott: it runs on deployment-eventlogging03
[21:16:57] <ottomata>	 https://wikitech.wikimedia.org/wiki/Analytics/EventLogging/TestingOnBetaCluster
[21:16:58] <ottomata>	 could help
[21:17:11] <andrewbogott>	 ottomata: cool, thank you
[21:17:36] <ottomata>	 you can tail -f /srv/log/eventlogging/all-events.log
[21:17:39] <ottomata>	 and then make an edit
[21:17:42] <ottomata>	 you should see stuff there
[21:17:46] <ottomata>	 to restart eventlogging
[21:18:03] <ottomata>	  sudo  eventloggingctl stop &&  sleep 5 && sudo  eventloggingctl start
[21:18:06] <ottomata>	 (I don't trust restart)
[21:18:37] <andrewbogott>	 ottomata: yep, I see logs appearing, I'll upgrade the packages now
[21:18:59] <ottomata>	 cool
[21:22:14] <andrewbogott>	 ottomata: ok, done.  Looks the same to me...
[21:22:30] <ottomata>	 great
[21:29:09] <greg-g>	 Zppix: That's a big topic :) What are you interested in helping with, specifically?
[21:29:46] <Zppix>	 CI
[21:30:00] <paladox>	 Zppix ci is also big
[21:30:08] <Zppix>	 Like jenkins
[21:30:25] <greg-g>	 Zppix: anything in particular you're thinking about?
[21:31:42] <Zppix>	  Job management
[21:34:08] <greg-g>	 Zppix: ok, so, I know you've been following along somewhat in here generally, so feel free to propose patches to the CI config repos, we'll review as time allows. But, we have a new hire coming on board in the new year, so most of our "teaching other people our systems" time will be spent on them, just FYI.
[21:34:48] <Zppix>	 Ok, no worries i love fresh bait :D
[21:35:48] <grrrit-wm>	 (03PS1) 10Paladox: Temporarily make mediawiki-extensions-qunit-jessie non voting [integration/config] - 10https://gerrit.wikimedia.org/r/328238 (https://phabricator.wikimedia.org/T153597) 
[21:37:30] <paladox>	 ostriches thcipriani twentyafterfour Hi, could you reviewer / merge ^^ please?
[21:37:47] <paladox>	 Currently qunit tests are breaking, see T153597 please
[21:37:48] <stashbot>	 T153597: mediawiki-extensions-qunit-jessie tests are failing - https://phabricator.wikimedia.org/T153597
[21:41:00] <paladox>	 legoktm ^^
[21:45:34] <ostriches>	 No.
[21:45:50] <ostriches>	 I'm busy
[21:45:54] <paladox>	 ok
[21:50:14] <wikibugs>	 06Release-Engineering-Team, 06Operations, 10Phabricator, 13Patch-For-Review: Setup test domain for phab2001 - https://phabricator.wikimedia.org/T152132#2888316 (10Zppix) 05Open>03stalled >>! In T152132#2873547, @Dzahn wrote: >  traffic team asked to wait a couple days because they were in the middle of...
[22:15:37] <wikibugs_>	 10Gerrit, 10MediaWiki-Vagrant: "index-pack failed" when installing new MediaWiki-Vagrant box - https://phabricator.wikimedia.org/T152801#2888413 (10demon) >>! In T152801#2863456, @bd808 wrote: > Another report at https://www.mediawiki.org/wiki/Talk:MediaWiki-Vagrant#Cloning_MediaWiki_fails_during_first_setup >...
[22:33:08] <wikibugs>	 10Gerrit, 06Release-Engineering-Team: Gerrit lacks a 'change' cache - https://phabricator.wikimedia.org/T153645#2886392 (10demon) The point is moving towards a multi-master with Redis as our shared cache store. Cf T152802, which should probably block this.
[22:35:20] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2888445 (10demon) >>! In T153079#2886892, @Paladox wrote: > I guess this would make this a higher priority t...
[22:36:23] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2888454 (10Paladox) oh.
[22:40:18] <wikibugs>	 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" for BlueSpiceExtensions' REL1_27 branch - https://phabricator.wikimedia.org/T153079#2888473 (10demon) A manual rebase didn't work. Ideally what I want is to find where that missing object is (...
[22:40:48] <wikibugs_>	 10Gerrit, 06Labs, 10Labs-Infrastructure: Gerrit reviewer bot dosent seem to be working - https://phabricator.wikimedia.org/T153719#2888474 (10Paladox)
[22:41:32] <ostriches>	 INTERESTING
[22:41:33] <ostriches>	 https://github.com/wikimedia/mediawiki-extensions-BlueSpiceExtensions/commit/a177cdabc7e83c30b71786f19e42055db873cfda
[22:43:00] <wikibugs>	 10Gerrit, 06Labs, 10Labs-Infrastructure: Gerrit reviewer bot dosent seem to be working - https://phabricator.wikimedia.org/T153719#2888490 (10Paladox)
[22:44:07] <paladox>	 Oh that commit is missing when searching for it
[22:44:07] <paladox>	 https://gerrit.wikimedia.org/r/#/q/a177cdabc7e83c30b71786f19e42055db873cfda
[22:46:31] <ostriches>	 Yeah I found it in github
[22:46:35] <ostriches>	 But it's missing on disk
[22:46:49] <greg-g>	 milimetric: yo, my first thought is "I wonder if that happens if #analytics-kanban is a milestone of #analytics" (cf: https://www.mediawiki.org/wiki/Phabricator/Project_management#Parent_Projects.2C_Subprojects_and_Milestones )
[22:47:00] <paladox>	 oh
[22:47:06] <ostriches>	 It's also just...missing...
[22:47:13] <ostriches>	 I think I see what happens.
[22:47:18] <paladox>	 oh
[22:47:39] <ostriches>	 So, branch FOO is created. JoeDeveloper comes along and clones the repo.
[22:47:46] <paladox>	 yep
[22:47:46] <ostriches>	 Branch FOO is deleted and recreated.
[22:48:06] <ostriches>	 JoeDeveloper has the old sha1 in his history, which is needed to reconstruct the dependency chain
[22:48:14] <paladox>	 yep
[22:48:15] <ostriches>	 Pushes a change to Gerrit involving that sha1s history
[22:48:15] <paladox>	 oh
[22:48:18] <ostriches>	 Gerrit goes BOOM
[22:48:35] <paladox>	 Is that considered a bug or just the Developer needs to reclone?
[22:48:57] <ostriches>	 Actually I'm not sure about my theory now
[22:49:01] <paladox>	 ok
[22:49:07] <ostriches>	 Cuz I rebased the change manually and that didn't fix it :)
[22:49:17] <paladox>	 oh
[22:49:30] <paladox>	 It seems the problem happened after updating gerrit
[22:49:41] <ostriches>	 Moral of the story is probably "stop using git and go back to subversion"
[22:49:43] <ostriches>	 Git sucks
[22:49:55] <paladox>	 LOL, but i find git mutch better then svn
[22:50:50] <marktraceur>	 ostriches: +1, git's the worst, after svn
[22:51:02] <ostriches>	 You have that backwards
[22:51:05] <ostriches>	 svn is so much better
[22:51:07] <ostriches>	 I miss it!
[22:51:20] <Reedy>	 ostriches: I still want VSS
[22:51:24] <marktraceur>	 ostriches: And of course svn is the worst after cvs
[22:51:28] <ostriches>	 Reedy: Those were the days.
[22:51:33] <marktraceur>	 Oh, Reedy found the one to beat cvs.
[22:51:36] <milimetric>	 greg-g: interesting
[22:51:44] <ostriches>	 paladox: I wonder if pushing the sha1 to some random temporary branch would help us for now
[22:51:55] <mutante>	 it log | grep cvs
[22:51:55] <mutante>	     Tool Labs: install cvs on dev_environ
[22:51:57] <paladox>	 Oh yep
[22:51:59] <milimetric>	 greg-g: is it possible to set analytics-kanban to be a milestone of analytics and try?
[22:52:04] <milimetric>	 I can do the trying and report back
[22:52:08] <marktraceur>	 Reedy: Wait a minute, don't we have a VCS system in MediaWiki? Dogfood much?
[22:52:32] <Reedy>	 marktraceur: HACKATHON PROJECT
[22:52:35] <milimetric>	 greg-g: and sorry about assuming there was a task (this was very low priority for us before that thread)
[22:52:37] <ostriches>	 Like push SOmething like `git push <url> <sha1>:refs/heads/temporary-bogus-missing-sha1s`
[22:52:39] <paladox>	 ostriches try the temp branch>
[22:52:52] <marktraceur>	 Reedy: Great, RIP gerrit, long live Extension:VCS
[22:53:06] <paladox>	 ostriches what about mirroring it from github
[22:53:16] <paladox>	 from github to gerrit
[22:53:26] <paladox>	 as github store all, the changes should not be lost.
[22:54:13] <ostriches>	 We'd still need somewhere to put the sha1
[22:54:24] <ostriches>	 I'm basically mirroring it back now, just putting it in the right branch
[22:54:32] <ostriches>	 s/right/random temporary branch/
[22:54:47] <wikibugs_>	 10Gerrit, 06Labs, 10Labs-Infrastructure: Gerrit reviewer bot dosent seem to be working - https://phabricator.wikimedia.org/T153719#2888520 (10greg) @valhallasw Kinda offtopic, but where should gerrit-reviewer-bot tasks be filed? This current list of projects doesn't make sense to me. I don't see one in Phab...
[22:54:47] <paladox>	 Oh
[22:54:48] <paladox>	 :)
[22:55:54] <greg-g>	 milimetric: yeah, we can convert. It takes a manual run of a script on the phab host, but doable. Wanna file a task describing the issue and then comment with this as a potential fix (I'm only grasping for straws, I/someone should/could test on the test phab instance)
[22:56:21] <ostriches>	 gerrit2@cobalt /srv/gerrit/git/mediawiki/extensions/BlueSpiceExtensions.git (BARE:master)$ git fetch https://github.com/wikimedia/mediawiki-extensions-BlueSpiceExtensions a177cdabc7e83c30b71786f19e42055db873cfda:refs/heads/REL1_27-backup-from-github-dont-delete
[22:56:21] <ostriches>	 error: no such remote ref a177cdabc7e83c30b71786f19e42055db873cfda
[22:56:29] <ostriches>	 So, github lies, it doesn't actually have that sha1!
[22:56:32] <ostriches>	 Just pretending :p
[22:56:42] <ostriches>	 Fan-fucking-tastic.
[22:56:48] <TabbyCat>	 lol
[22:56:56] <paladox>	 LOLOLOL
[22:57:09] <paladox>	 ostriches, i see now, you theory looks correct to me
[22:57:09] <milimetric>	 greg-g: do I have rights to do that on the test instance?  Happy to
[22:57:10] <paladox>	 https://phabricator.wikimedia.org/diffusion/EBSE/history/REL1_27/?offset=700
[22:57:28] <greg-g>	 milimetric: yeah, I bet!
[22:57:29] <paladox>	 It seems they just recreated the branch straight of master
[22:57:41] <milimetric>	 ok, doing so then making the issue, thx
[22:57:49] <greg-g>	 milimetric: https://phab-01.wmflabs.org/
[22:57:50] <paladox>	 https://phabricator.wikimedia.org/rEBSE9f5048e323e8f2813106194a3530b49f3b835b08
[22:57:54] <paladox>	 ostriches ^^
[22:57:57] <paladox>	 This is a bug
[22:58:01] <paladox>	 in gerrit.
[22:58:10] <greg-g>	 paladox: is https://phab-01.wmflabs.org/ still the best phab labs host to test things on?
[22:58:15] <paladox>	 Yep
[22:58:19] <ostriches>	 well I don't care about the bug at all :p
[22:58:24] <ostriches>	 Since I'm never turning jgit's gc back on
[22:58:29] <ostriches>	 I WANT TO KNOW HOW TO UNDO THE MADNESS
[22:58:33] <greg-g>	 paladox: thanks
[22:58:43] <greg-g>	 milimetric: go collaboration! :)
[22:58:50] <paladox>	 Nope i doint mean trun gc back on, i mean bug as in gerrit's branche system is broken
[22:58:53] <paladox>	 greg-g your welcome :)
[22:58:59] <paladox>	 trun turn
[22:59:02] <ostriches>	 Well, it's both :)
[22:59:08] <milimetric>	 paladox: what's the user/password on that labs instance?
[22:59:09] <ostriches>	 It's gc + branch deletion
[22:59:10] <ostriches>	 :)
[22:59:17] <milimetric>	 (for phab, not for the instance, sorry)
[22:59:29] <paladox>	 milimetric you can create an account
[22:59:37] <milimetric>	 k
[23:00:13] <paladox>	 ostriches but strange that it let's you delete the branch but recreating it with the same name brakes
[23:00:13] <paladox>	 it
[23:00:17] <greg-g>	 milimetric: random pass plz, test instance on labs :)
[23:00:20] <paladox>	 brakes = breaks
[23:00:48] <ostriches>	 I can kinda see why.
[23:00:51] <ostriches>	 I mean it *almost* makes sense
[23:00:52] <ostriches>	 Sorta
[23:00:56] <paladox>	 oh
[23:00:56] <ostriches>	 In bizarro world
[23:01:00] <paladox>	 lol
[23:01:09] * paladox reports the bug
[23:01:26] <ostriches>	 Don't bother
[23:01:34] <ostriches>	 No fix they'll have will help us unbreak our repos
[23:01:42] <ostriches>	 Plus it'll never happen again as long as I don't do gc.
[23:02:03] <Reedy>	 ostriches: First rule of garbage collection; don't call the garbage collector
[23:02:03] <paladox>	 oh
[23:02:05] <paladox>	 I see
[23:02:22] <paladox>	 lol, ostriches tell upstream your gc is broken, disable it. lol
[23:02:25] <shinken-wm>	 PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[23:02:34] <ostriches>	 Reedy: True story....
[23:03:02] <ostriches>	 I'm going to go for a walk, gerrit made me angry
[23:03:05] <paladox>	 ok
[23:03:06] <paladox>	 lol
[23:03:32] <wikibugs_>	 10Gerrit, 06Labs, 10Labs-Infrastructure: Gerrit reviewer bot dosent seem to be working - https://phabricator.wikimedia.org/T153719#2888548 (10Legoktm) Issues should be filed at <https://github.com/valhallasw/gerrit-reviewer-bot>.
[23:03:40] <paladox>	 why not use this java to php converter http://javatophp.com/
[23:03:43] <paladox>	 ostriches ^^
[23:05:00] <Zppix>	 Unreliable
[23:05:18] <Reedy>	 paladox: go get it gerritphp running in labs and come back to us
[23:05:54] <paladox>	 LOL
[23:05:59] <paladox>	 i tryed that
[23:06:03] <paladox>	 it was missing a clas
[23:06:05] <paladox>	 class
[23:06:20] <mutante>	 bug about filing bugs in the same place :p
[23:09:29] <milimetric>	 greg-g: it says I don't have rights to create "global" herald rules, and it seems like I need that to create an "add project tag" rule.  In the "personal" herald rules I don't see the option
[23:09:51] <paladox>	 Oh
[23:10:09] <paladox>	 how can we allow users to set global herald rules?
[23:10:35] <milimetric>	 paladox: it looks like I'd need to be an admin
[23:10:36] <paladox>	 ah
[23:10:36] <mutante>	 make them admins?
[23:10:37] <paladox>	 found it
[23:10:41] <paladox>	 fixed it now
[23:10:43] <milimetric>	 thx!
[23:10:47] <paladox>	 Your welcome
[23:10:47] <paladox>	 :)
[23:11:57] <paladox>	 Reedy i now get internal error
[23:11:58] <paladox>	 lol
[23:12:03] <paladox>	 http://javatophp.com/demo.html
[23:12:20] <shinken-wm>	 RECOVERY - Puppet run on deployment-tmh01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:13:13] <paladox>	 http://javatophp.com/demoupload.php
[23:14:45] <paladox>	 ostriches looks like php can execute java, http://php.net/manual/fa/java.examples-basic.php yes that is in fa since en lang not available
[23:14:55] <paladox>	 I am just looking at the example, i carnt read fa
[23:14:55] <mutante>	 are we also converting mediawiki to java with PtoJ?
[23:15:08] <Zppix>	 Php can excute java in <script> tags
[23:15:11] <paladox>	 lol no why would we want to convert mediawiki to java
[23:15:57] <Zppix>	 Mediawiki being converted to java is the same as me winning the lottery not gonna happen. :P
[23:16:26] <milimetric>	 greg-g: the report: apparently if I mimic the rule, the bug still happens even with kanban being a milestone of analytics: https://phab-01.wmflabs.org/T126
[23:16:55] <milimetric>	 greg-g: however! the rules are more flexible than we were led to believe!  So I just changed the rule and everything's fine: https://phab-01.wmflabs.org/H3
[23:17:00] <milimetric>	 https://phab-01.wmflabs.org/T127
[23:17:48] <ostriches>	 I'd rather convert MW to Java than convert Gerrit to PHP :p
[23:17:48] <paladox>	 LOL
[23:17:51] <milimetric>	 I honestly would love to just be allowed to make changes in phab.  I promise not to do anything evil, but the mystical shroud around it has prevented us from doing a lot of things we want to do
[23:17:57] <paladox>	 Why?
[23:18:07] <paladox>	 Java is not easy, whereas php is.
[23:18:16] <ostriches>	 Java is super easy though
[23:18:23] <ostriches>	 PHP is just dumb
[23:18:36] <paladox>	 Oh, /me does not understand any java.
[23:18:42] <Zppix>	 I am better at php :/
[23:18:51] <mutante>	 paladox: it would resolve T31169
[23:18:51] <stashbot>	 T31169: PHP Sucks - https://phabricator.wikimedia.org/T31169
[23:19:04] <ostriches>	 Zppix: I'm better at PHP too, but it's still a dumb language :p
[23:19:08] <paladox>	 lolololol
[23:20:10] <paladox>	 But with java you have to keep restarting
[23:20:11] <Zppix>	 ostriches: not only that but my beginnings as a dev was php and html (i self taught myself html at age 6)
[23:20:16] <paladox>	 for minor changes or big changes
[23:20:29] <paladox>	 whereas with php you can have it contantly on.
[23:20:42] <Zppix>	 paladox: that depends
[23:20:53] <milimetric>	 yeah, you can hot-load classes if you want
[23:21:33] <paladox>	 Oh
[23:21:36] <milimetric>	 obviously C# is the best, but I'm not sending this message 'cause I'll get crucified
[23:21:43] <milimetric>	 oh $#!%
[23:21:44] <Reedy>	 paladox: We have to keep restarting hhvm
[23:21:49] <Reedy>	 milimetric: VS is awesome too
[23:21:56] <paladox>	 LOL, and oh, but what about php?
[23:22:00] <Reedy>	 fsck ASP .NET though
[23:22:02] <paladox>	 hhvm is a different thing to php
[23:22:06] <Reedy>	 paladox: Not really
[23:22:09] <paladox>	 it is based on php
[23:22:13] <Reedy>	 no it's not
[23:22:18] <paladox>	 but it has it's own set's on bugs
[23:22:19] <milimetric>	 asp.net is not a thing... asp.net does not look like anything at all to me
[23:22:20] <Reedy>	 it's a different runtime engine for PHP
[23:22:29] <Reedy>	 hack is based on PHP
[23:22:33] <paladox>	 yep
[23:22:38] <milimetric>	 Bernard, what does this look like?  AAAAH, asp.net!! segfault
[23:22:39] <paladox>	 asp.net  is microsoft?
[23:22:47] <paladox>	 lol
[23:23:03] <milimetric>	 yeah, it's like the world's most comprehensive crippling of a perfectly fine technology (HTTP / HTML)
[23:23:19] <paladox>	 lol
[23:23:54] <paladox>	 gerrit's ui is horrible but has very good dev tools, phabricators ui is very user freindly but lacks the dev tools
[23:24:01] <ostriches>	 milimetric: "How can we make HTML worse?!"
[23:24:05] <greg-g>	 milimetric: yay!
[23:24:06] <paladox>	 lol
[23:24:08] <ostriches>	 ^ Exactly how that conversation started
[23:24:57] <Zppix>	 paladox: dont hurt gerrits feelings its ui just needs some botox
[23:25:08] <ostriches>	 Gerrit has no feelings
[23:25:08] <milimetric>	 it literally is, they did one of these: http://www.snopes.com/computer/program/stroustrup.asp
[23:25:10] <ostriches>	 Or soul
[23:25:23] <milimetric>	 aw that's not true, it does have a soul, big Austrian soul called Christian :)
[23:25:24] <ostriches>	 Reedy: Remember the first experiments with hphpc? :P
[23:25:31] <Reedy>	 lols
[23:25:41] <ostriches>	 Recompiling MW took like 2 hours
[23:25:45] <mutante>	 that was also Reedy's bug
[23:25:50] <mutante>	 i wonder if he is ok with that being closed 
[23:25:51] <Zppix>	 Ive heard about that ostriches did you guys crash servers with that
[23:25:52] <paladox>	 LOL
[23:26:07] <milimetric>	 greg-g: can I get permits to change herald rules in prod or should I file a task?
[23:26:07] <ostriches>	 Zppix: It never went live, we couldn't get it all working in our dev environments
[23:26:13] <paladox>	 gerrit though is getting a refresh with polygerrit
[23:26:16] <ostriches>	 So we shelved it until they rewrote the whole thing as a VM
[23:26:32] <paladox>	 gerrit-new.wmflabs.org/?polygerrit=1
[23:26:34] <Zppix>	 ostriches: so everything i write in code basically , k.
[23:26:36] <paladox>	 http://gerrit-new.wmflabs.org/?polygerrit=1
[23:26:41] <paladox>	 http://gerrit-new.wmflabs.org/?polygerrit=0
[23:26:53] <mutante>	 paladox: how about a nice Greasmonkey script that makes Gerrit the prettiest of them all
[23:27:02] <paladox>	 LOL
[23:27:06] <Zppix>	 I want thy polygerrit
[23:27:06] <paladox>	 yeh
[23:28:05] <greg-g>	 milimetric: did you file the one about the kanban issue? just file that and I can either give you the rights or jfdi
[23:29:20] <Zppix>	 greg-g: im trying to find any decent easy ci bugs, to figure out  jenkins code got any ?
[23:29:44] <paladox>	 try https://phabricator.wikimedia.org/project/view/1208/
[23:30:55] <mutante>	 "add a check for whitespace before leading <?php"  (it's rarely actually that easy though)
[23:31:06] <mutante>	 T92531
[23:31:07] <stashbot>	 T92531: add a check for whitespace before leading <?php - https://phabricator.wikimedia.org/T92531
[23:31:10] <paladox>	 lol
[23:31:28] <Reedy>	 alll of the whitespaces
[23:31:43] * paladox goes and watch ncis lost agelas.
[23:31:53] <mutante>	 see the "good luck with that" part in the comments... hmmm
[23:35:45] <grrrit-wm>	 (03CR) 10Legoktm: [C: 04-2] "We're not going to do this." [integration/config] - 10https://gerrit.wikimedia.org/r/328238 (https://phabricator.wikimedia.org/T153597) (owner: 10Paladox) 
[23:40:17] <wikibugs>	 06Release-Engineering-Team, 10Phabricator: Herald rule thought to conflict - https://phabricator.wikimedia.org/T153723#2888645 (10Milimetric)
[23:40:34] <milimetric>	 in case I didn't tag it right, greg-g: https://phabricator.wikimedia.org/T153723
[23:40:39] <milimetric>	 thanks yall
[23:40:39] <milimetric>	 nite
[23:40:50] <greg-g>	 milimetric: adios
[23:41:22] <wikibugs_>	 06Release-Engineering-Team, 10Phabricator: Herald rule thought to conflict - https://phabricator.wikimedia.org/T153723#2888660 (10greg)
[23:50:20] <grrrit-wm>	 (03CR) 10Paladox: [] "@Legoktm oh what should we do?" [integration/config] - 10https://gerrit.wikimedia.org/r/328238 (https://phabricator.wikimedia.org/T153597) (owner: 10Paladox)