[01:17:03] <shinken-wm>	 PROBLEM - Host deployment-parsoidcache02 is DOWN: CRITICAL - Host Unreachable (10.68.16.145)
[01:32:48] <grrrit-wm>	 (03PS1) 10Paladox: [ConventionExtension] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253287 
[01:33:20] <grrrit-wm>	 (03PS2) 10Paladox: [ConventionExtension] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253287 
[01:33:47] <grrrit-wm>	 (03PS3) 10Paladox: [ConventionExtension] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253287 
[01:34:06] <grrrit-wm>	 (03CR) 10Paladox: [C: 04-1] "Requires source change be merged first." [integration/config] - 10https://gerrit.wikimedia.org/r/253287 (owner: 10Paladox)
[02:35:10] <bmansurov>	 hello, is this the channel for being online during swat deploys?
[03:17:23] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #781: 04FAILURE in 27 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/781/
[05:55:20] <rasel_>	 like/rasel/help
[05:56:09] <shinken-wm>	 PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:06:00] <shinken-wm>	 RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 38944 bytes in 1.151 second response time
[06:12:08] <shinken-wm>	 PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:16:57] <shinken-wm>	 RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 38948 bytes in 0.761 second response time
[06:23:07] <shinken-wm>	 PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:37:24] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK
[08:08:53] <wikibugs>	 10Differential, 5Gerrit-Migration, 6Phabricator: Have to login to view differential stuff - https://phabricator.wikimedia.org/T118666#1807106 (10demon) It already is?
[08:51:45] <hashar>	 good morning
[09:06:40] <grrrit-wm>	 (03CR) 10Thiemo Mättig (WMDE): "Thank you very much for fixing the unit tests. :-) I would have been lost with that." [selenium] - 10https://gerrit.wikimedia.org/r/252936 (owner: 10Thiemo Mättig (WMDE))
[09:48:45] <grrrit-wm>	 (03PS4) 10Hashar: [Git2Pages] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/249481 (owner: 10Paladox)
[09:51:06] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] [Git2Pages] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/249481 (owner: 10Paladox)
[09:52:01] <grrrit-wm>	 (03Merged) 10jenkins-bot: [Git2Pages] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/249481 (owner: 10Paladox)
[09:54:47] <shinken-wm>	 PROBLEM - Puppet staleness on integration-dev is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0]
[10:00:01] <grrrit-wm>	 (03Abandoned) 10Hashar: Experiment YAML aliases and template inheritance [integration/config] - 10https://gerrit.wikimedia.org/r/243164 (https://phabricator.wikimedia.org/T107529) (owner: 10Hashar)
[10:06:34] <grrrit-wm>	 (03CR) 10Hashar: [C: 04-1] "Pending https://gerrit.wikimedia.org/r/#/c/243391/" [integration/config] - 10https://gerrit.wikimedia.org/r/243394 (owner: 10Paladox)
[10:07:10] <grrrit-wm>	 (03PS2) 10Hashar: Configure thumbor/result-storage [integration/config] - 10https://gerrit.wikimedia.org/r/252638 (owner: 10Gilles)
[10:07:17] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] Configure thumbor/result-storage [integration/config] - 10https://gerrit.wikimedia.org/r/252638 (owner: 10Gilles)
[10:08:11] <grrrit-wm>	 (03Merged) 10jenkins-bot: Configure thumbor/result-storage [integration/config] - 10https://gerrit.wikimedia.org/r/252638 (owner: 10Gilles)
[10:09:49] <wikibugs>	 10Gitblit-Deprecate, 10Diffusion: redirect gerrit repo paths to diffusion callsigns - https://phabricator.wikimedia.org/T110607#1807292 (10mmodell)
[10:11:52] <wikibugs>	 10Deployment-Systems, 3Scap3, 7Documentation, 5Patch-For-Review: Add documentation of the new scap3 features to the scap docs - https://phabricator.wikimedia.org/T112554#1807296 (10mmodell)
[10:11:53] <wikibugs>	 10Deployment-Systems, 3Scap3: Document Scap3 post-stage checks - https://phabricator.wikimedia.org/T116636#1807295 (10mmodell) 5Open>3Resolved
[10:16:33] <grrrit-wm>	 (03CR) 10Hashar: [C: 04-1] "Pending https://gerrit.wikimedia.org/r/#/c/243208/" [integration/config] - 10https://gerrit.wikimedia.org/r/243209 (https://phabricator.wikimedia.org/T90943) (owner: 10Paladox)
[10:21:49] <wikibugs>	 10Deployment-Systems, 3Scap3, 5Patch-For-Review, 7Puppet: Refactor `mediawiki::scap` to make sure Scap dependencies are not dependent on mediawiki - https://phabricator.wikimedia.org/T116606#1807309 (10Joe) @thcipriani the patch is mostly ok as is, sorry for not working on this last week, but it has been s...
[10:27:16] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team: Mira failed to sync - https://phabricator.wikimedia.org/T118555#1807313 (10jcrespo) For reference, I will ignore new and future errors like this until further notice:  ``` jynus@tin:/srv/mediawiki-staging$ sync-file wmf-config/db-eqiad.php "Repool db1015, depool...
[11:06:59] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: check experimental is not showing jobs running - https://phabricator.wikimedia.org/T118082#1807397 (10hashar) 5Open>3Invalid a:3hashar I could not reproduce. The jobs in experimental shows up fine in my experience.
[11:28:00] <wikibugs>	 10Continuous-Integration-Config, 10ArticlePlaceholder, 10Wikidata, 5Patch-For-Review, 3Wikidata-Sprint-2015-11-03: [Task] add CI to extension ArticlePlaceholder - https://phabricator.wikimedia.org/T113049#1807424 (10daniel)
[11:31:05] <wikibugs>	 10Continuous-Integration-Config, 10ArticlePlaceholder, 10Wikidata, 5Patch-For-Review, 3Wikidata-Sprint-2015-11-03: [Task] add CI to extension ArticlePlaceholder - https://phabricator.wikimedia.org/T113049#1807438 (10daniel) Pending review (probably by @Lucie): https://gerrit.wikimedia.org/r/#/c/252458/ "...
[12:23:02] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UrlShortener, 10Wikimedia-Extension-setup, 5Patch-For-Review: Set up UrlShortener extension on the beta cluster - https://phabricator.wikimedia.org/T116444#1807518 (10hashar) 5Open>3Resolved a:3hashar Gave it a try using the side bar link "Get sho...
[12:23:33] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UrlShortener, 10Wikimedia-Extension-setup, 5Patch-For-Review: Set up UrlShortener extension on the beta cluster - https://phabricator.wikimedia.org/T116444#1807524 (10hashar) a:5hashar>3Legoktm
[12:43:08] <wikibugs>	 10Differential, 5Gerrit-Migration, 6Phabricator: Have to login to view differential stuff - https://phabricator.wikimedia.org/T118666#1807531 (10Luke081515) 5Open>3Resolved a:3Luke081515 Hm, I mean, last time there was the setting: "Can Use Application: All Users"
[13:00:16] <wmf-insecte>	 Yippee, build fixed!
[13:00:16] <wmf-insecte>	 Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #850: 09FIXED in 28 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/850/
[13:03:07] <grrrit-wm>	 (03PS13) 10Hashar: [Maintenance] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/225222 (owner: 10Paladox)
[13:10:15] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] [Maintenance] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/225222 (owner: 10Paladox)
[13:11:26] <grrrit-wm>	 (03Merged) 10jenkins-bot: [Maintenance] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/225222 (owner: 10Paladox)
[13:40:15] <shinken-wm>	 PROBLEM - Host integration-labsvagrant is DOWN: CRITICAL - Host Unreachable (10.68.16.4)
[14:10:42] <hashar>	 !log deleted integration-labsvagrant
[14:38:06] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: [Task] Move Wikidata brosertests into Wikibase repository - https://phabricator.wikimedia.org/T118727#1807678 (10Tobi_WMDE_SW) 3NEW
[14:38:19] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: [Task] Move Wikidata browsertests into Wikibase repository - https://phabricator.wikimedia.org/T118727#1807678 (10Tobi_WMDE_SW)
[15:15:01] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text04 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]
[15:16:11] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1016 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[15:16:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-mobile04 is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0]
[15:23:34] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload04 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]
[15:26:16] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave-trusty-1016 is OK: OK: Less than 1.00% above the threshold [0.0]
[15:40:24] <wikibugs>	 10Deployment-Systems, 3Scap3: Scap3 repo-cache should clean unused revs - https://phabricator.wikimedia.org/T118305#1807919 (10thcipriani) a:3thcipriani
[15:41:17] <wikibugs>	 10Deployment-Systems, 3Scap3: Scap3 repo-cache should clean unused revs - https://phabricator.wikimedia.org/T118305#1796903 (10thcipriani) p:5Triage>3Normal
[15:42:42] <wikibugs>	 10Deployment-Systems, 3Scap3, 5Patch-For-Review, 7Puppet: Refactor `mediawiki::scap` to make sure Scap dependencies are not dependent on mediawiki - https://phabricator.wikimedia.org/T116606#1807931 (10thcipriani) 5Open>3Resolved
[15:46:01] <wikibugs>	 10Deployment-Systems, 3Scap3: End user tutorial docs for Scap - https://phabricator.wikimedia.org/T118738#1807938 (10thcipriani) 3NEW
[15:50:16] <grrrit-wm>	 (03PS2) 10Zfilipin: Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) 
[15:50:28] <grrrit-wm>	 (03CR) 10Zfilipin: Run Ruby jobs using Rake (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[15:50:46] <grrrit-wm>	 (03PS3) 10Zfilipin: Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) 
[15:51:17] <grrrit-wm>	 (03CR) 10Zfilipin: "Patch set 2 implements changes requested in comments of patch set 1. Patch set 3 is just a rebase." [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[15:52:41] <zeljkof>	 hashar: I think this is ready https://gerrit.wikimedia.org/r/#/c/252690/
[15:52:55] <hashar>	 will be for tomorrow I am taking a break
[15:53:32] <hashar>	 zeljkof: though you get it merged, we would need to backport the patch to the bunch of branches we support
[15:54:20] <zeljkof>	 hashar: but the only repos touched are ruby gems and oojs/ui
[15:54:29] <hashar>	 ah
[15:54:31] <zeljkof>	 mw/core will be in a different patch
[15:54:31] <shinken-wm>	 RECOVERY - Host integration-labsvagrant is UP: PING OK - Packet loss = 0%, RTA = 0.87 ms
[15:54:38] <zeljkof>	 working on it now
[15:54:50] <hashar>	 confused because you marked it as depending on https://gerrit.wikimedia.org/r/#/q/I83d16759597a90ee9a082eb4029fedca7af10a30,n,z
[15:54:55] <hashar>	 which is a change-id used on mediawiki/core :D
[15:54:58] <zeljkof>	 so I need to port Rakefile to supported branches for mw/core?
[15:55:01] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0]
[15:55:38] <zeljkof>	 hashar: I have used your trick of adding the same gerrit change id to all related patches in different repos :D
[15:56:03] <zeljkof>	 but looks like gerrit by default links to just one of them, does not open a page with all of them, if there is more than one
[15:56:13] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-mobile04 is OK: OK: Less than 1.00% above the threshold [0.0]
[15:56:14] <hashar>	 now, Zuul will refuses to merge the CI patch until the mediawiki/core patch is merged
[15:56:14] <hashar>	 :D
[15:56:28] <hashar>	 just drop the depends-on
[15:58:22] <zeljkof>	 hashar: :D
[15:58:26] <zeljkof>	 shoot myself in the foot
[15:58:28] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [0.0]
[15:58:34] <zeljkof>	 ok, will do
[15:59:57] <zeljkof>	 hashar: well, zull commit not longer depends on commit in mediawiki/core ;) https://gerrit.wikimedia.org/r/#/c/252682/1..2//COMMIT_MSG,cmhttps://gerrit.wikimedia.org/r/#/c/252682/1..2//COMMIT_MSG,cm
[16:01:30] <hashar>	 go for it I guess :}
[16:01:51] <zeljkof>	 https://gerrit.wikimedia.org/r/#/c/252682/1..2//COMMIT_MSG,cm
[16:01:59] <zeljkof>	 looks like I have pasted the link twice
[16:02:03] <zeljkof>	 this is the correct one
[16:07:31] <hashar>	 oh
[16:07:35] <wikibugs>	 10Beta-Cluster-Infrastructure, 7Blocked-on-RelEng, 6operations, 7HHVM, 5Patch-For-Review: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1808045 (10Joe) Terbium is now done; I'll look at reimaging tin next.
[16:07:50] <hashar>	 zeljkof: you can't change the change-id in the commit message after patchset #1
[16:07:54] <hashar>	 just drop the depends-on from the integration/config change
[16:08:01] <hashar>	 that will let it go
[16:08:15] <zeljkof>	 hashar: and I thought I was so smart :(
[16:08:17] <zeljkof>	 will do
[16:09:09] <zeljkof>	 hashar: but wait
[16:09:28] <zeljkof>	 when at https://gerrit.wikimedia.org/r/#/c/252690/
[16:09:38] <zeljkof>	 I click Depends-On: I83d16759597a90ee9a082eb4029fedca7af10a30
[16:09:45] <zeljkof>	 I land at https://gerrit.wikimedia.org/r/#/q/I83d16759597a90ee9a082eb4029fedca7af10a30,n,z
[16:09:58] <zeljkof>	 and only two commits (both in ruby gems) are listed there
[16:10:18] <zeljkof>	 but one is abandoned :(
[16:10:25] <zeljkof>	 ok, dropping the depends-on
[16:10:26] <zeljkof>	 what a mess
[16:10:39] <grrrit-wm>	 (03PS4) 10Zfilipin: Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) 
[16:10:46] <zeljkof>	 the definition of shooting myself in the foot
[16:11:58] <hashar>	 :-}
[16:12:01] <hashar>	 I am off
[16:12:07] <hashar>	 need some fresh air before the weekly checkin
[16:16:35] <grrrit-wm>	 (03PS1) 10Zfilipin: Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/253343 (https://phabricator.wikimedia.org/T114860) 
[16:17:25] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/253343 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[16:18:02] <grrrit-wm>	 (03CR) 10Zfilipin: Run Ruby jobs using Rake (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/252690 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[16:19:51] <grrrit-wm>	 (03PS2) 10Zfilipin: Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/253343 (https://phabricator.wikimedia.org/T114860) 
[16:20:40] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Run Ruby jobs using Rake [integration/config] - 10https://gerrit.wikimedia.org/r/253343 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[16:37:15] <grrrit-wm>	 (03CR) 10Zfilipin: "hashar: the commit you have linked to is ready to be merged, as far as I can see." [selenium] - 10https://gerrit.wikimedia.org/r/252693 (https://phabricator.wikimedia.org/T117993) (owner: 10Zfilipin)
[16:38:28] <grrrit-wm>	 (03CR) 10Zfilipin: "Not sure what is the problem here :(" [integration/config] - 10https://gerrit.wikimedia.org/r/253343 (https://phabricator.wikimedia.org/T114860) (owner: 10Zfilipin)
[16:38:36] <wikibugs>	 10Deployment-Systems, 3Scap3, 7Documentation: Document Scap3 config-deploy - https://phabricator.wikimedia.org/T116634#1808136 (10thcipriani) Addition of tests to ensure inheritance happens correctly would be Nice To Have.
[16:42:14] <wikibugs>	 10Deployment-Systems, 3Scap3: Document Scap3's `--limit` flag - https://phabricator.wikimedia.org/T118745#1808147 (10thcipriani) 3NEW
[16:55:27] <wikibugs>	 10Deployment-Systems, 3Scap3, 7Documentation: End user tutorial docs for Scap - https://phabricator.wikimedia.org/T118738#1808209 (10Aklapper)
[17:12:30] <shinken-wm>	 PROBLEM - Host integration-labsvagrant is DOWN: PING CRITICAL - Packet loss = 100%
[17:17:22] <hashar>	 jzerebecki: I moved the Tuesday CI checkin from tomorrow to thursday same date
[17:17:24] <hashar>	 same time
[17:23:17] <wikibugs>	 10Differential, 5Gerrit-Migration, 7Documentation: Create example workflows for differential showing old way and new way side by side - https://phabricator.wikimedia.org/T117058#1808279 (10mmodell)
[17:23:18] <wikibugs>	 10Differential, 5Gerrit-Migration: Document git-review -> arc mapping. - https://phabricator.wikimedia.org/T112967#1808278 (10mmodell)
[17:25:12] <wikibugs>	 5Continuous-Integration-Scaling, 6operations, 5Patch-For-Review: install/deploy scandium as zuul merger (ci) server - https://phabricator.wikimedia.org/T95046#1808301 (10hashar) Scheduled for Tuesday 17th November  15:00–16:00 UTC #  07:00–08:00 PST  16:00–17:00 UTC+1
[17:38:59] <wikibugs>	 5Gerrit-Migration, 10Analytics-Tech-community-metrics: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#1808364 (10Aklapper) 3NEW
[17:45:03] <wikibugs>	 10Deployment-Systems, 3Scap3, 7Documentation: Document Scap3's `--limit` flag - https://phabricator.wikimedia.org/T118745#1808400 (10Aklapper)
[18:11:23] <wikibugs>	 3Scap3: create a scap3 command to bootstrap a new deployment repo - https://phabricator.wikimedia.org/T118760#1808552 (10mmodell) 3NEW a:3mmodell
[18:20:23] <wikibugs>	 10Deployment-Systems, 3Scap3: enforcing deployment from `/srv/deployment` is wrong - https://phabricator.wikimedia.org/T116207#1808595 (10mmodell) a:5mmodell>3thcipriani
[18:21:29] <shinken-wm>	 RECOVERY - Host integration-labsvagrant is UP: PING OK - Packet loss = 0%, RTA = 1.13 ms
[18:29:22] <grrrit-wm>	 (03PS1) 10Ottomata: Set up CI for eventlogging (python) repo [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) 
[18:30:11] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Set up CI for eventlogging (python) repo [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) (owner: 10Ottomata)
[18:45:25] <ottomata>	 heya
[18:45:30] <ottomata>	 who can tell me about scap2?
[18:45:59] <ottomata>	 or um sacp3?
[18:46:00] <ottomata>	 scap3?
[18:47:00] <thcipriani>	 ottomata: myself, twentyafterfour marxarelli ostriches are all pretty in-the-know on that topic. what's up?
[18:47:33] <ottomata>	 so, we're revamping some eventlogging python server stuff for the EventBus project.  currently EventLogging server is deployed via tin + upstart
[18:47:44] <ottomata>	 want to make it work with jessie and_whateverdeploymentsystemisbest_
[18:47:50] <ostriches>	 ottomata: via tin, so with trebuchet?
[18:47:57] <ottomata>	 ja
[18:47:59] <ottomata>	 sorry
[18:48:00] <ottomata>	 git-deploy
[18:48:08] <ottomata>	 (aka trebuchet), 
[18:49:01] <ostriches>	 Hmm, that sounds reasonable.
[18:49:16] <ottomata>	 just found https://doc.wikimedia.org/mw-tools-scap/index.html
[18:49:25] <ottomata>	 hmm, -php
[18:49:26] <ottomata>	 ?
[18:49:29] <ottomata>	 is it PHP specific?
[18:49:33] <ostriches>	 Ignore that bit :p
[18:49:44] <ottomata>	 haha
[18:49:45] <ottomata>	 "It used to mean, “Sync Common All PHP.” Now, it doesn’t make sense.
[18:49:45] <ottomata>	 "
[18:49:45] <ostriches>	 "It used to mean, “Sync Common All PHP.” Now, it doesn’t make sense."
[18:49:46] <ottomata>	 nice.
[18:49:47] <ottomata>	 hahha
[18:49:50] <greg-g>	 :)
[18:49:54] <ottomata>	 reading..
[18:50:02] <greg-g>	 names are hard to change, yo
[18:50:10] <ottomata>	 sorry, would ahve read this before poking, but was searching for scap2 for a while and not finding what I expected
[18:50:13] <ottomata>	 didn't know this was 3!
[18:50:25] <greg-g>	 2.0 is so last decade
[18:50:27] <ostriches>	 scap 1 (or just scap) was the bash-based original version
[18:50:42] <ostriches>	 scap 2 is the python-based `scap` we currently use in prod for MW deploys
[18:51:00] <ostriches>	 scap 3 uses the same codebase as 2, but is gaining support for non-MW service deploys.
[18:53:47] <twentyafterfour>	 so yeah, ottomata, the scap3 way is similar to `git-deploy` but without the dependency on Salt.  The docs are lacking a good introductory section though
[18:54:14] <ottomata>	 ja reading, kinda get it, would be nice to have a 'getting started' section
[18:54:24] <ottomata>	 for deploying a simple repo with a running service
[18:54:58] <ottomata>	 cool, hm, intersting, nrpe cheks
[18:55:00] <ottomata>	 kinda cool
[18:55:34] <twentyafterfour>	 ottomata: We can probably help you get the initial configuration set up. It sounds like your service would probably be a good fit for this.
[18:55:48] <ottomata>	 yeah, there might actually be many daemons
[18:55:57] <ottomata>	 but, the http service we are working on is only one
[18:56:05] <ottomata>	 we will port analytics eventlogging to this as well
[18:56:08] <ottomata>	 but don't need to think about it yet
[18:56:26] <ottomata>	 i'm not actually sure if it is better to use deployment or a .deb for this yet...
[18:56:44] <ottomata>	 where do systemd unit files live in the scap3 way?
[18:56:46] <ottomata>	 in puppet?
[18:57:15] <twentyafterfour>	 generally anything owned by root would still need to be in puppet
[18:57:22] <ottomata>	 hmm, ok, got it.
[18:57:24] <ottomata>	 ok i think that's fine
[18:57:26] <twentyafterfour>	 but scap3 can run commands via ssh + sudo
[18:57:44] <twentyafterfour>	 so if you have sudoers rules on the target machine you can trigger them to restart services
[18:57:49] <ottomata>	 k
[18:58:15] <ottomata>	 can we reload and/or sighup?  there will be config changes for which sighup is enough
[18:58:23] <ottomata>	 guess so if we can run whatever command :)
[18:58:30] <twentyafterfour>	 yep
[18:58:35] <ottomata>	 k
[18:58:47] <twentyafterfour>	 I think we can even do that conditionally based on whether it's just a config change or whether the code changed
[18:59:14] <ottomata>	 cool
[18:59:30] <ottomata>	 ok, cool, then I think i will work on some of the puppet stuff then (systemd mostly)
[18:59:36] <ottomata>	 cause that needs to be done anyway
[18:59:59] <ottomata>	 just planning some work this week...if you guys find it in your hearts to write a little getting started documentation, I would be much obliged :)
[19:00:13] <twentyafterfour>	 ottomata: that's on our TODO list
[19:00:26] <thcipriani>	 https://phabricator.wikimedia.org/T118738
[19:00:28] <twentyafterfour>	 along with creating a tool to bootstrap the deployment configurations
[19:00:40] <twentyafterfour>	 https://phabricator.wikimedia.org/T118760
[19:00:50] <ottomata>	 oh cool :)
[19:00:57] <ottomata>	 this is not bad though https://doc.wikimedia.org/mw-tools-scap/scap3/repo_config.html 
[19:16:45] <wikibugs>	 10Deployment-Systems, 6operations: install/deploy mira as codfw deployment server - https://phabricator.wikimedia.org/T95436#1808803 (10mmodell)
[19:16:46] <wikibugs>	 10Deployment-Systems, 3Scap3, 5Patch-For-Review: [scap] Add support for syncing /srv/mediawiki-staging including fully working git data to warm spare deploy server - https://phabricator.wikimedia.org/T104826#1808800 (10mmodell) 5Resolved>3Open This is still not done.   There is a patch against [[ https:/...
[19:16:54] <wikibugs>	 10Deployment-Systems, 3Scap3, 5Patch-For-Review: [scap] Add support for syncing /srv/mediawiki-staging including fully working git data to warm spare deploy server - https://phabricator.wikimedia.org/T104826#1808807 (10mmodell)
[19:27:42] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 3Scap3: scap creating directories owned by root on mira - https://phabricator.wikimedia.org/T118691#1808825 (10Reedy)
[19:28:45] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 5Patch-For-Review, 7WorkType-Maintenance: beta-scap-eqiad mira / deployment-bastion permissions problem - https://phabricator.wikimedia.org/T117016#1808838 (10mmodell)
[19:28:47] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 3Scap3: scap creating directories owned by root on mira - https://phabricator.wikimedia.org/T118691#1808837 (10mmodell)
[19:29:17] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team: Move the train deployment from Thursday to Wednesday for some Wikipedia sites - https://phabricator.wikimedia.org/T115002#1808843 (10greg) So, status summary.  * Catalan looks positive, right? * Hewbrew has only supports (and nothing new for a while): [[ https:/...
[19:29:33] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 3Scap3: scap creating directories owned by root on mira - https://phabricator.wikimedia.org/T118691#1806687 (10mmodell) D48 addresses this I think
[19:31:09] <grrrit-wm>	 (03PS1) 10Paladox: [Gadgets] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253369 
[19:31:47] <grrrit-wm>	 (03PS2) 10Paladox: [Gadgets] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253369 
[19:32:03] <grrrit-wm>	 (03CR) 10Paladox: [C: 04-1] "Needs source change merged first." [integration/config] - 10https://gerrit.wikimedia.org/r/253369 (owner: 10Paladox)
[19:32:11] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team: Move the train deployment from Thursday to Wednesday for some Wikipedia sites - https://phabricator.wikimedia.org/T115002#1808847 (10Amire80) Yes, both Catalan and Hebrew are positive.  As far as I am concerned let's do it.
[19:33:22] <shinken-wm>	 PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<12.50%)
[19:38:20] <grrrit-wm>	 (03CR) 10Nuria: [C: 031] Set up CI for eventlogging (python) repo [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) (owner: 10Ottomata)
[19:43:55] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 5Patch-For-Review: Move the train deployment from Thursday to Wednesday for some Wikipedia sites - https://phabricator.wikimedia.org/T115002#1808899 (10Legoktm) Per {T118212} I'd think we want to slow down the train rather than speed it up...
[19:46:23] <ostriches>	 ottomata, twentyafterfour: We should file a task for moving EventLogging
[19:46:45] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 5Patch-For-Review: Move the train deployment from Thursday to Wednesday for some Wikipedia sites - https://phabricator.wikimedia.org/T115002#1808907 (10greg) >>! In T115002#1808899, @Legoktm wrote: > Per {T118212} I'd think we want to slow down the train rathe...
[19:47:16] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10MobileFrontend: MobileFrontend is failing mwext-mw-selenium test - https://phabricator.wikimedia.org/T118771#1808909 (10Paladox) 3NEW
[19:47:29] <ostriches>	 Filing now
[19:47:35] <ottomata>	 ok, hm
[19:47:49] <ottomata>	 hm, ostriches, we may want to deploy a new eventlogging service with this
[19:48:00] <ottomata>	 before we move the analytisc eventlogging stuff from git-deploy to scap3
[19:48:08] <ottomata>	 because, that will require us to upgrade the server to jessie
[19:48:14] <ostriches>	 Fair enough
[19:48:15] <ottomata>	 but, yes, eventually i think we will want to do that
[19:50:09] <wikibugs>	 10Deployment-Systems, 6Release-Engineering-Team, 3Scap3, 10Analytics-EventLogging: Move EventLogging service to scap3 - https://phabricator.wikimedia.org/T118772#1808930 (10demon) 3NEW
[19:51:29] <grrrit-wm>	 (03PS1) 10Paladox: [WikiEditor] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253376 
[19:52:34] <grrrit-wm>	 (03PS2) 10Paladox: [WikiEditor] Update Jenkins tests [integration/config] - 10https://gerrit.wikimedia.org/r/253376 
[19:58:42] <grrrit-wm>	 (03CR) 10Paladox: [C: 04-1] "Requires source patch merged first." [integration/config] - 10https://gerrit.wikimedia.org/r/253376 (owner: 10Paladox)
[19:58:45] <shinken-wm>	 PROBLEM - Host integration-labsvagrant is DOWN: CRITICAL - Host Unreachable (10.68.16.4)
[20:02:43] <wikibugs>	 10Beta-Cluster-Infrastructure, 5Patch-For-Review: deployment-bastion puppet runs failing due to add_ip6_mapped.pp template parsing error - https://phabricator.wikimedia.org/T118422#1809025 (10thcipriani) 5Open>3Resolved a:3thcipriani
[20:19:47] <wikibugs>	 10Beta-Cluster-Infrastructure, 5Patch-For-Review: deployment-bastion puppet runs failing due to add_ip6_mapped.pp template parsing error - https://phabricator.wikimedia.org/T118422#1809042 (10Dzahn) sorry, i did not expect this to break in labs., should have tested.   it would still be interesting why we can't...
[20:22:22] <shinken-wm>	 RECOVERY - Host integration-labsvagrant is UP: PING OK - Packet loss = 0%, RTA = 0.92 ms
[20:22:46] <shinken-wm>	 PROBLEM - Puppet staleness on deployment-restbase01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0]
[20:39:12] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: [Task] Move Wikidata browsertests into Wikibase repository - https://phabricator.wikimedia.org/T118727#1809061 (10JanZerebecki)
[20:39:35] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: [Task] Move Wikidata browsertests into Wikibase repository - https://phabricator.wikimedia.org/T118727#1807684 (10JanZerebecki)
[20:39:37] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: create a Wikibase browser test job running against a fresh MediaWiki installation - https://phabricator.wikimedia.org/T118284#1809062 (10JanZerebecki)
[20:39:47] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: [Task] Move Wikidata browsertests into Wikibase repository - https://phabricator.wikimedia.org/T118727#1807684 (10JanZerebecki)
[20:39:48] <wikibugs>	 7Browser-Tests, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2015-11-03: create a Wikibase browser test job running against a fresh MediaWiki installation - https://phabricator.wikimedia.org/T118284#1796346 (10JanZerebecki)
[20:55:13] <ottomata>	 ostriches:  this is the repo we'll want to use scap3 for sooner rather than later
[20:55:14] <ottomata>	 https://gerrit.wikimedia.org/r/#/admin/projects/eventlogging
[20:55:19] <hashar>	 andrewbogott: thank you for the zuul-merger deployment tomorrow: -}
[20:55:30] <ottomata>	 currently a fork of the code in mediawiki/extensions/EventLogging server/
[20:55:35] <andrewbogott>	 hashar: we can do it another time if tomorrow is too busy...
[20:55:56] <hashar>	 andrewbogott: I had a conflict with the weekly CI checkin but moved that one to later
[20:56:01] <andrewbogott>	 ok
[20:56:02] <hashar>	 rest of week is busy anyway
[20:56:07] <hashar>	 so tomorrow is perfect
[20:57:54] <hashar>	 ottomata: you know https://gerrit.wikimedia.org/r/#/c/253359/1/zuul/layout.yaml looks wrong :-}
[20:58:16] <ottomata>	 doh!
[20:58:17] <hashar>	 ottomata: you edited the wrong extension ( mediawiki/extensions/ExpandTemplates )  instead of EventLogging :-D
[20:58:22] <ottomata>	 UHHHH
[20:58:37] <ottomata>	 OOokkk not sure how that happened, fixing...
[20:58:49] <hashar>	 ottomata: and you can just use the template:  - name: tox-jessie
[20:58:52] <hashar>	 that runs tox
[20:59:09] <hashar>	 which would executes serially all envs listed in tox.ini  envlist =  
[20:59:09] <ottomata>	 ok
[20:59:15] <hashar>	 which one can see by running:  tox -l
[20:59:34] <hashar>	 only drawback, is that each env is tested one after the other, so that slow things down a bit
[21:02:58] <grrrit-wm>	 (03PS2) 10Ottomata: Set up CI for eventlogging (python) repo [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) 
[21:08:34] <hashar>	 almost
[21:08:46] <hashar>	 ottomata: almost, though there is no .json files in the repo https://gerrit.wikimedia.org/r/#/c/253359/2/zuul/layout.yaml :D
[21:08:48] <hashar>	 check:
[21:08:52] <hashar>	   - jsonlint
[21:10:34] <grrrit-wm>	 (03CR) 10Hashar: [C: 04-1] "almost!" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) (owner: 10Ottomata)
[21:10:56] <hashar>	 anyway sleep time
[21:27:02] <ottomata>	 hmmmm, k, yeah, there might be...but ja
[21:29:53] <grrrit-wm>	 (03PS3) 10Ottomata: Set up CI for eventlogging (python) repo [integration/config] - 10https://gerrit.wikimedia.org/r/253359 (https://phabricator.wikimedia.org/T118761) 
[21:33:25] <shinken-wm>	 PROBLEM - Host integration-labsvagrant is DOWN: CRITICAL - Host Unreachable (10.68.16.4)
[22:21:30] <shinken-wm>	 RECOVERY - Host integration-labsvagrant is UP: PING OK - Packet loss = 0%, RTA = 2.24 ms
[22:22:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[22:47:17] <shinken-wm>	 PROBLEM - Host integration-labsvagrant is DOWN: CRITICAL - Host Unreachable (10.68.16.4)
[22:57:20] <shinken-wm>	 RECOVERY - Host integration-labsvagrant is UP: PING OK - Packet loss = 0%, RTA = 0.70 ms
[22:57:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]
[23:43:13] <wikibugs>	 5Continuous-Integration-Scaling, 6operations: Upload new Zuul packages on apt.wikimedia.org for Precise / Trusty / Jessie - https://phabricator.wikimedia.org/T118340#1809485 (10Andrew) I'd be a bit happier if the source tree that these packages are built from is checked into gerrit, with clearly labeled branch...