[00:04:46] <grrrit-wm>	 (03PS1) 10Paladox: Replace global-teardown with castor-save in test mwext-VisualEditor-publish [integration/config] - 10https://gerrit.wikimedia.org/r/322375 
[00:04:52] <paladox>	 Krinkle ^^
[00:05:21] <grrrit-wm>	 (03CR) 10Paladox: "See an example on this being removed here https://phabricator.wikimedia.org/rCICFf2d1c618b21e4dfacaa0233b1cdf22d16c22a6d4" [integration/config] - 10https://gerrit.wikimedia.org/r/322375 (owner: 10Paladox)
[00:06:38] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] Apply nodepool_params to mwext-VisualEditor-publish job [integration/config] - 10https://gerrit.wikimedia.org/r/322370 (owner: 10Krinkle)
[00:08:31] <grrrit-wm>	 (03Merged) 10jenkins-bot: Apply nodepool_params to mwext-VisualEditor-publish job [integration/config] - 10https://gerrit.wikimedia.org/r/322370 (owner: 10Krinkle)
[00:10:09] <Krinkle>	 !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/322370 
[00:10:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[00:22:07] <grrrit-wm>	 (03PS2) 10Krinkle: Replace global-teardown with castor-save in test mwext-VisualEditor-publish [integration/config] - 10https://gerrit.wikimedia.org/r/322375 (owner: 10Paladox)
[00:23:33] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] "Compiled and deployed updated mwext-VisualEditor-publish job." [integration/config] - 10https://gerrit.wikimedia.org/r/322375 (owner: 10Paladox)
[00:25:23] <grrrit-wm>	 (03Merged) 10jenkins-bot: Replace global-teardown with castor-save in test mwext-VisualEditor-publish [integration/config] - 10https://gerrit.wikimedia.org/r/322375 (owner: 10Paladox)
[01:52:00] <paladox>	 thanks
[03:35:19] <wikibugs>	 10Deployment-Systems, 06Release-Engineering-Team (Long-Lived-Branches), 10releng-201617-q1, 07Epic: Merge to deployed branches instead of cutting a new deployment branch every week. - https://phabricator.wikimedia.org/T89945#2807767 (10mmodell) >>! In T89945#2113537, @Krinkle wrote: >>>! In T89945#1554422,...
[06:46:03] <shinken-wm>	 PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[07:21:04] <shinken-wm>	 RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0]
[08:33:35] <wikibugs_>	 10Continuous-Integration-Config, 10MediaWiki-extensions-Other, 13Patch-For-Review: [LightweightRDFa] The module 'ext.wikiEditor' required by 'ext.LightweightRDFa.button' must exist - https://phabricator.wikimedia.org/T93727#2807871 (10Raymond) >>! In T93727#2806421, @hashar wrote: > I have archived it.  Does...
[09:51:33] <shinken-wm>	 PROBLEM - Puppet run on deployment-apertium01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[14:53:10] <shinken-wm>	 PROBLEM - Host deployment-pdf02 is DOWN: CRITICAL - Host Unreachable (10.68.16.129)
[14:54:29] <shinken-wm>	 PROBLEM - Host deployment-conftool is DOWN: CRITICAL - Host Unreachable (10.68.20.30)
[15:29:49] <shinken-wm>	 PROBLEM - Puppet run on deployment-cache-text04 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]
[15:39:48] <shinken-wm>	 PROBLEM - Puppet run on deployment-phab02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[15:43:13] <wmf-insecte>	 Project selenium-MobileFrontend » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #233: 04FAILURE in 21 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/233/
[15:51:03] <shinken-wm>	 PROBLEM - Puppet run on deployment-phab01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[15:54:47] <shinken-wm>	 RECOVERY - Puppet run on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0]
[18:37:24] <shinken-wm>	 PROBLEM - Host deployment-puppetmaster is DOWN: CRITICAL - Host Unreachable (10.68.16.63)
[19:05:54] <wikibugs_>	 03Scap3 (Scap3-Adoption-Phase1), 10releng-201516-q4: [keyresult] Migrate remaining trebuchet deployed services - https://phabricator.wikimedia.org/T129290#2808322 (10ori)
[19:05:56] <wikibugs>	 10Deployment-Systems, 03Scap3 (Scap3-Adoption-Phase1), 10scap: remove fluoride from the deployments directory - https://phabricator.wikimedia.org/T129145#2808321 (10ori) 05Open>03Resolved
[19:27:59] <shinken-wm>	 PROBLEM - Puppet run on logstash is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[19:37:23] <hashar>	 i give up with wmflabs
[19:37:27] <hashar>	 cant spawn an instance properly :(
[19:37:31] <hashar>	 good week-end
[19:37:44] <shinken-wm>	 PROBLEM - Host logstash is DOWN: CRITICAL - Host Unreachable (10.68.23.175)
[19:53:28] <wikibugs_>	 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible, 07Puppet: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2736876 (10hashar)
[19:54:14] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible, 07Puppet: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2736876 (10hashar) @mobrovac here is the task related to the instance...
[19:54:26] <wikibugs_>	 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible, 07Puppet: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2808351 (10hashar)
[20:01:23] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible, 07Puppet: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2808357 (10hashar) Note that once one has deleted `/var/lib/puppet/ss...
[20:06:32] <shinken-wm>	 RECOVERY - Host logstash is UP: PING OK - Packet loss = 0%, RTA = 0.97 ms
[20:13:29] <shinken-wm>	 PROBLEM - Puppet run on deployment-logstash2 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[20:14:40] <wikibugs_>	 10Continuous-Integration-Config, 10MediaWiki-extensions-Other, 13Patch-For-Review: [LightweightRDFa] The module 'ext.wikiEditor' required by 'ext.LightweightRDFa.button' must exist - https://phabricator.wikimedia.org/T93727#2808373 (10hashar) @Raymond I guess yes.  It is no more maintained and probably not u...
[20:22:39] <wikibugs_>	 10Continuous-Integration-Config, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 13Patch-For-Review: Qunit tests for MultimediaViewer fail with the currently submitted code - https://phabricator.wikimedia.org/T150575#2808375 (10hashar) a:03Tgr Looks like @tgr aced it entirely. Root cause...
[20:23:28] <shinken-wm>	 RECOVERY - Puppet run on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0]
[21:01:30] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Discovery, 06Discovery-Search, 10Elasticsearch: ElasticSearch 2.3.5 + plugins 2.3.5 raise jar hell - https://phabricator.wikimedia.org/T151128#2808438 (10hashar)
[21:18:16] <wikibugs_>	 03Scap3, 10Parsoid, 06Services, 15User-mobrovac: Env vars being overwritten - https://phabricator.wikimedia.org/T150897#2808471 (10thcipriani) 05Open>03Resolved
[21:26:47] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Discovery, 06Discovery-Search, 10Elasticsearch: ElasticSearch 2.3.5 + plugins 2.3.5 raise jar hell - https://phabricator.wikimedia.org/T151128#2808473 (10hashar) Hijacking `/usr/share/elasticsearch/bin/elasticsearch` to use strace with:      exec /usr/bin/strace -...
[21:47:02] <wikibugs_>	 10Continuous-Integration-Infrastructure, 06Discovery, 06Discovery-Search, 10Elasticsearch: ElasticSearch 2.3.5 + plugins 2.3.5 raise jar hell - https://phabricator.wikimedia.org/T151128#2808474 (10hashar) 05Open>03Resolved a:03hashar And obviously the .jar files are NOT jar files: ``` tail /srv/deplo...
[21:47:44] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Discovery, 06Discovery-Search, 10Elasticsearch: ElasticSearch 2.3.5 + plugins 2.3.5 raise jar hell - https://phabricator.wikimedia.org/T151128#2808477 (10hashar)
[22:54:03] <bd808>	 what do I type in a gerrit comment to make zuul run tests for a patch that has an untrusted uploader? Is it "check" or "test" or ??
[22:54:25] <paladox>	 it's check
[22:54:28] <paladox>	 i think
[22:54:39] <paladox>	 bd808 is this for Zppix patch?
[22:54:49] <bd808>	 yes
[22:55:26] <paladox>	 Ok, bd808 you can do recheck i think
[22:55:33] <paladox>	 most of the tests should run on nodepool
[22:56:32] <paladox>	 bd808 it should be safe as tox-jessie is on nodepool
[22:56:56] <bd808>	 yeah that seems to have done it
[22:57:12] <paladox>	 :)
[23:12:14] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Wikimedia-Logstash, 07Jenkins: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#2808556 (10hashar) I have created a very rough proof of concept in labs. Instance logstash.integration.eaiad.wmflabs. I have reused the role...