[00:03:04] PROBLEM - Puppet run on deployment-ms-be01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [00:35:19] PROBLEM - Puppet run on deployment-eventlogging04 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [00:38:05] RECOVERY - Puppet run on deployment-ms-be01 is OK: OK: Less than 1.00% above the threshold [0.0] [01:07:01] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618053 (10Jdforrester-WMF) [01:09:51] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618056 (10Jdforrester-WMF) Maybe someone is doing something... [01:12:10] 10Beta-Cluster-Infrastructure, 10Parsoid, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618063 (10Jdforrester-WMF) I think the Special... [01:15:19] RECOVERY - Puppet run on deployment-eventlogging04 is OK: OK: Less than 1.00% above the threshold [0.0] [01:33:55] PROBLEM - Puppet run on deployment-db03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [02:10:07] 10Beta-Cluster-Infrastructure, 10Parsoid, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618147 (10ssastry) Try refreshing the content... [02:12:42] PROBLEM - Puppet run on deployment-mathoid is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [02:13:55] RECOVERY - Puppet run on deployment-db03 is OK: OK: Less than 1.00% above the threshold [0.0] [02:25:25] 10Beta-Cluster-Infrastructure, 10Parsoid, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2613500 (10AlexMonk-WMF) So, the cause will be... [02:30:32] what magic does scap3 do on the target hosts? Had to cd `pwd` to get the new changes to show up in git... [02:37:54] PROBLEM - Puppet run on deployment-redis01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [02:43:06] Krenair: that probably means it deleted and recreated the directory [02:44:38] yep, look: [02:44:49] /tmp/quux ] rmdir /tmp/quux [02:44:51] /tmp/quux ] mkdir /tmp/quux [02:44:52] /tmp/quux ] touch /tmp/quux/a [02:45:00] ls gives me a still empty directory [02:45:35] When the inode changes, you need to go to the new inode, not remain on the old. [02:46:21] some OS (Windows) will forbid to delete a directory someone is on to avoid that [02:47:26] so you can fail maintenance script only with in idle console :/ [02:49:44] does this make the deployment effectively atomic? [02:50:13] as in the program won't be able to accidentally read some files from the new version and some from the old version while it's downloading changes? [02:52:43] RECOVERY - Puppet run on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [02:59:49] 10Beta-Cluster-Infrastructure, 10Parsoid, 10VisualEditor, 10VisualEditor-MediaWiki, 10VisualEditor-MediaWiki-Media: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618197 (10ssastry) >>! In T144884#2618153, @Al... [03:12:55] RECOVERY - Puppet run on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0] [03:40:48] 10Beta-Cluster-Infrastructure, 10Parsoid, 10VisualEditor, 10VisualEditor-MediaWiki, and 2 others: On beta cluster, images are replaced with failed (404) links to Special:FilePath when you load VE - https://phabricator.wikimedia.org/T144884#2618215 (10Ryasmeen) 05Open>03Resolved [03:48:00] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T142855#2618238 (10Ryasmeen) [03:48:19] (03CR) 10Krinkle: rake: tweak files filter (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/308150 (https://phabricator.wikimedia.org/T144325) (owner: 10Hashar) [04:53:43] PROBLEM - Puppet run on deployment-mx is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [05:33:46] RECOVERY - Puppet run on deployment-mx is OK: OK: Less than 1.00% above the threshold [0.0] [06:17:54] 06Release-Engineering-Team, 10DBA, 10MediaWiki-Maintenance-scripts, 06Operations, and 2 others: Add section for long-running tasks on the Deployment page (specially for database maintenance) - https://phabricator.wikimedia.org/T144661#2618330 (10jcrespo) > Also, re the inclusion criteria part: if we list t... [06:34:02] PROBLEM - Puppet run on deployment-elastic07 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:14:02] RECOVERY - Puppet run on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:15] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:04:14] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [08:21:56] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 10MediaWiki-extensions-Examples, 07Documentation, and 3 others: Improve documentation around running/writing (with lots of examples) browser tests - https://phabricator.wikimedia.org/T108108#2618427 (10zeljkofilipin) Argh. Selenium page was redire... [09:06:32] 10Continuous-Integration-Infrastructure (phase-out-gallium), 13Patch-For-Review: Firewall rules for labs support host to communicate with contint1001.wikimedia.org (new gallium) - https://phabricator.wikimedia.org/T137323#2618545 (10hashar) [09:09:25] 10Continuous-Integration-Infrastructure (phase-out-gallium), 13Patch-For-Review: Firewall rules for labs support host to communicate with contint1001.wikimedia.org (new gallium) - https://phabricator.wikimedia.org/T137323#2618551 (10hashar) **Status** I have verified labnodepool1001 -- contint1001 flows. They... [09:11:16] 10Continuous-Integration-Infrastructure (phase-out-gallium), 06Operations, 10Traffic: Move gallium to an internal host? - https://phabricator.wikimedia.org/T133150#2618555 (10hashar) 05stalled>03declined From T140257#2595926 and follow up response from ops, we are keeping the status quo of using a public... [09:39:46] 10Continuous-Integration-Infrastructure (phase-out-gallium), 06Operations: Upgrade Zuul on scandium.eqiad.wmnet (Jessie zuul-merger) - https://phabricator.wikimedia.org/T145057#2618640 (10hashar) [09:40:29] 10Continuous-Integration-Infrastructure, 06Operations, 07Zuul: Upgrade Zuul on scandium.eqiad.wmnet (Jessie zuul-merger) - https://phabricator.wikimedia.org/T140894#2479965 (10hashar) I have bumped the version to `zuul_2.5.0-8-gcbc7f62-wmf2jessie1` and the new package `.changes` file properly list the orig... [10:02:05] !log Delete mwext-VisualEditor-sync-gerrit job, already got removed by ostriches in 139d17c8f1c4bcf2bb761e13a6501e4d85684066 . The issue in Gerrit (T51846) has been fixed. Poke T86659 , one less job on slaves. [10:02:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:02:34] 10Continuous-Integration-Infrastructure: Migrate all jobs to labs slaves - https://phabricator.wikimedia.org/T86659#2618710 (10hashar) [10:03:29] !log Delete Jenkins job https://integration.wikimedia.org/ci/job/mwext-VisualEditor-sync-gerrit/ that has been left behind. It is no more needed. T51846 T86659 [10:03:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:06:00] (03PS1) 10Hashar: Remove VE script to sync the Gerrit repo [integration/jenkins] - 10https://gerrit.wikimedia.org/r/309272 (https://phabricator.wikimedia.org/T51846) [10:13:38] 10Continuous-Integration-Infrastructure, 13Patch-For-Review, 07Upstream: mediawiki/extensions.git does not update some extensions - https://phabricator.wikimedia.org/T51846#2618725 (10hashar) This is confirmed to have been fixed when we have upgraded to Gerrit 2.12. The workaround script and Jenkins job hav... [10:13:38] 10Continuous-Integration-Infrastructure: Migrate all jobs to labs slaves - https://phabricator.wikimedia.org/T86659#973822 (10Paladox) This is almost finished we just need to do jobs which have no node: stanza in JJB configuration file. Important now. [10:25:14] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [10:27:44] hashar you know this task erblint-HEAD, could we just use https://github.com/katgironpe/rails-erb-lint please? [10:27:53] Or how do we install erblint on jessie? [10:32:35] (03PS1) 10Paladox: Migrate erblint-HEAD from Precise to DebianJessie [integration/config] - 10https://gerrit.wikimedia.org/r/309278 [10:35:01] PROBLEM - Puppet run on deployment-elastic07 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [11:00:15] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [11:15:01] RECOVERY - Puppet run on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0] [11:28:13] 10Continuous-Integration-Infrastructure (phase-out-gallium), 13Patch-For-Review: Firewall rules for labs support host to communicate with contint1001.wikimedia.org (new gallium) - https://phabricator.wikimedia.org/T137323#2618806 (10hashar) [11:30:59] 10Continuous-Integration-Infrastructure (phase-out-gallium), 13Patch-For-Review: Firewall rules for labs support host to communicate with contint1001.wikimedia.org (new gallium) - https://phabricator.wikimedia.org/T137323#2618810 (10hashar) [12:11:04] (03PS1) 10Mobrovac: RESTBaseUpdateJobs: Do not branch the extension any more [tools/release] - 10https://gerrit.wikimedia.org/r/309294 (https://phabricator.wikimedia.org/T144843) [13:00:46] (03PS5) 10Paladox: Add rm -fR "$WORKSPACE/modules/*/bin" to jenkins job operations-puppet-doc [integration/config] - 10https://gerrit.wikimedia.org/r/307654 (https://phabricator.wikimedia.org/T143233) [13:00:53] hashar ^^ ? [13:03:31] paladox, is that urgent? [13:03:56] andre__ according to https://phabricator.wikimedia.org/T143233 yes, since it is set at high priority [13:04:06] and that patch has been sitting there for 5+ days. [13:04:29] Ah, I see. Thanks for explaining. [13:05:14] Your welcome :) [13:10:47] (03CR) 10Mobrovac: [C: 032] "I5c45d676da01332a2f43cd51caca0d8a164eb8c2 has been merged and deployed, so this can take effect now." [tools/release] - 10https://gerrit.wikimedia.org/r/309294 (https://phabricator.wikimedia.org/T144843) (owner: 10Mobrovac) [13:11:15] (03Merged) 10jenkins-bot: RESTBaseUpdateJobs: Do not branch the extension any more [tools/release] - 10https://gerrit.wikimedia.org/r/309294 (https://phabricator.wikimedia.org/T144843) (owner: 10Mobrovac) [13:47:27] Project selenium-VisualEditor » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #139: 04FAILURE in 3 min 26 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/139/ [13:50:00] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619197 (10hashar) Puppet has a blacklist of files pattern to not pass to rdoc: ``` name=lib/puppet/util/rdoc.rb, lang=... [14:00:12] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet, 07Upstream: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619213 (10hashar) [14:06:41] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet, 07Upstream: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619216 (10hashar) I have manually hacked the Jenkins job via the web UI to add `RDOCOPT='--exclude=/modu... [14:13:43] PROBLEM - Puppet run on deployment-mathoid is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [14:28:29] PROBLEM - Puppet staleness on deployment-salt02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [14:42:30] PROBLEM - Puppet run on deployment-cache-text04 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [14:53:43] RECOVERY - Puppet run on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [14:56:15] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [14:56:27] (03PS1) 10Paladox: Add env params['RDOCOPT'] = '--exclude=/modules/[^/]*/bin/.*$' [integration/config] - 10https://gerrit.wikimedia.org/r/309327 (https://phabricator.wikimedia.org/T143233) [15:00:38] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet, 07Upstream: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619387 (10hashar) Looks like previously it took several minutes to build anyway:* | #25603 | 8 min 7 se... [15:19:47] 03Scap3 (Scap3-Adoption-Phase1), 10releng-201516-q2, 10releng-201516-q3, 10releng-201516-q4, 10scap: [keyresult] Migrate all Service team owned services and MW to scap - https://phabricator.wikimedia.org/T109926#2619433 (10thcipriani) [15:19:51] 03Scap3 (Scap3-Adoption-Phase1), 10scap, 10Graphoid: Deploy Graphoid with scap3 - https://phabricator.wikimedia.org/T120102#2619430 (10thcipriani) 05Open>03Resolved a:03thcipriani Closing—Graphoid was migrated to scap3 on 2016-06-21. [15:21:07] (03PS1) 10Hashar: Puppet doc now ignore /bin files [integration/config] - 10https://gerrit.wikimedia.org/r/309332 (https://phabricator.wikimedia.org/T143233) [15:21:30] (03CR) 10Hashar: "Job updated / working." [integration/config] - 10https://gerrit.wikimedia.org/r/309332 (https://phabricator.wikimedia.org/T143233) (owner: 10Hashar) [15:21:40] (03CR) 10Paladox: [C: 031] Puppet doc now ignore /bin files [integration/config] - 10https://gerrit.wikimedia.org/r/309332 (https://phabricator.wikimedia.org/T143233) (owner: 10Hashar) [15:22:26] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet, 07Upstream: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619451 (10hashar) 05Open>03Resolved a:03hashar I have updated the job in JJB and refreshed it http... [15:22:28] RECOVERY - Puppet run on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [15:22:47] paladox: got the puppet doc fixed [15:22:55] in a better way than mass deleting the /bin dirs :] [15:22:59] hasharAway thanks [15:23:07] But when i click on the files [15:23:12] it says they doint exist [15:23:15] hasharAway [15:23:26] looks like something is generated and more or less represent puppet docuemntation on https://doc.wikimedia.org/puppet/ [15:23:32] so assuming it is fixed [15:23:37] IE [15:23:37] https://doc.wikimedia.org/mnt/jenkins-workspace/workspace/operations-puppet-doc/src/modules/admin/README.html [15:23:47] That shows [15:23:52] bah [15:23:53] The requested URL /mnt/jenkins-workspace/workspace/operations-puppet-doc/src/modules/admin/README.html was not found on this server. [15:23:58] And it happends for the rest [15:23:59] too [15:24:09] can you reopen the task ? [15:24:13] might look at it later this evening [15:24:16] gotta leave for now [15:24:16] Ok [15:24:18] thanks [15:25:26] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review, 07Puppet, 07Upstream: post build failures for operations/puppet on operations-puppet-doc - https://phabricator.wikimedia.org/T143233#2619470 (10Paladox) 05Resolved>03Open Re opening per @hashar it seems that when you click on for exam... [15:27:22] (03CR) 10Jforrester: [C: 032] Add test to verify SpaceBeforeClassBraceSniff handles extends [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/308700 (https://phabricator.wikimedia.org/T142343) (owner: 10Legoktm) [15:30:50] (03Merged) 10jenkins-bot: Add test to verify SpaceBeforeClassBraceSniff handles extends [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/308700 (https://phabricator.wikimedia.org/T142343) (owner: 10Legoktm) [15:31:14] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [16:35:11] PROBLEM - Puppet run on integration-aptly01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [16:40:28] bd808: Where does the beta kibana live again? I can never remember the url [16:41:14] http://logstash-beta.wmflabs.org/ [16:41:29] That -beta [16:42:33] so tricky! [16:43:09] Krenair hi, for https://gerrit.wikimedia.org/r/#/c/308152/ the reason why it breaks for labs is because we need to update the [16:43:09] https://wikitech.wikimedia.org/w/index.php?title=Special:NovaInstance&action=configure&instanceid=3bf01df8-2be7-4ca2-b08f-9187eb8ed6d7&project=phabricator®ion=eqiad [16:43:21] Global groups -> building: [16:43:25] to use the new class [16:45:10] RECOVERY - Puppet run on integration-aptly01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:46:13] paladox, you don't think I know why it breaks for labs? [16:46:55] Krenair oh sorry wrong person [16:47:14] sorry [16:50:50] !log deleted integration-aptly01 [16:50:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [16:55:14] PROBLEM - Host integration-aptly01 is DOWN: CRITICAL - Host Unreachable (10.68.22.227) [17:04:53] PROBLEM - Puppet run on deployment-redis01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [17:08:07] !log deleted integration-jessie-lego-test01 [17:08:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [17:11:44] (03PS1) 10Chad: Use scap as source of wikiversions rather than multiversion [tools/release] - 10https://gerrit.wikimedia.org/r/309355 [17:12:55] PROBLEM - Host integration-jessie-lego-test01 is DOWN: CRITICAL - Host Unreachable (10.68.16.187) [17:13:37] PROBLEM - Puppet run on deployment-elastic08 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [17:14:42] PROBLEM - Puppet run on deployment-mathoid is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [17:18:42] 10Continuous-Integration-Infrastructure (phase-out-gallium), 06Operations: Upgrade Zuul on scandium.eqiad.wmnet (Jessie zuul-merger) - https://phabricator.wikimedia.org/T145057#2618640 (10elukey) @hashar let's sync and make this happen, maybe on Monday? [17:28:44] hihyaa [17:28:51] where should I ask if I have differential/diffusion/arcanist questions? [17:29:39] #wikimedia-devtools [17:29:51] danke [17:38:54] So what exactly gives someone the ability to move tasks between spaces, or create them into a private space? (I need to give it to the new buyer) [17:39:07] right now her user just has the normal create task, no space (advaced) options [17:39:11] advanced even. [17:40:32] robh: within an existing space? [17:40:42] editing the members of the underlying ACL project [17:40:47] by adding that person as member [17:41:16] I'm not quite following, sorry =[ [17:41:34] robh: are we talking about an existing space? [17:41:39] or is that a request for some new space? [17:41:44] existing space [17:42:01] So I have s4 for all our vendor quotes. Pam is our new buyer (pswaby user) and needs to have the create task (advanced) option [17:42:09] so she can create tasks and sub tasks directly into the private space. [17:42:28] robh: Should that person be able to see **all** the tasks that are in S4? [17:42:31] yep [17:42:34] and they can [17:42:42] They can view them all, since they are a member of #acl*procurement-review [17:42:46] but they cannot create task (advanced) [17:42:50] so they cannot make new s4 tasks. [17:43:06] they can even comment and edit existing tasks in the space [17:44:11] robh: https://phabricator.wikimedia.org/spaces/edit/4/ shows that "Visible To" (and "Editable By") are set to admins, you, and members of acl*operations-team and acl*procurement-review [17:44:28] yes, but edit the space isnt the same as moving a task in and out of a space, afaik [17:44:34] ah. true that [17:44:56] RECOVERY - Puppet run on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0] [17:44:58] this is an issue where i have to grant her user some kind of advanced right to move tasks between spaces and/or create a task into a private space [17:45:06] which i know i have, but not sure if its due to admin or what. [17:45:48] Let me check the Edit controls of the "Task Advanced" form at https://phabricator.wikimedia.org/maniphest/task/edit/form/3/ [17:46:19] Uhm. "Can View" does not include #acl*procurement-review [17:46:31] but stuff like #acl*operations-team [17:46:43] that's probably the reason why you and many others can see that but not that person [17:46:50] ahhh, yep [17:47:00] let me be bold and add #acl*procurement-review here [17:47:07] please do, saves me logging in via command line admin [17:47:13] cuz my user cannot edit that form perms =] [17:47:36] andre__: i think you solved the issue, i'll drop her a hangout ping when you finish the edit and have her try it out [17:47:46] i forget those forms have their own perms. [17:48:09] robh, done [17:48:12] thanks! [17:48:15] yw [17:48:20] I admit it also still confuses me :-/ [17:53:37] RECOVERY - Puppet run on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0] [17:54:41] RECOVERY - Puppet run on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [17:56:35] andre__ hi, could you copy over https://phab-01.wmflabs.org/transactions/editengine/phame.post/view/5/ to https://phabricator.wikimedia.org/transactions/editengine/phame.post/ please? and set visability to public please? [17:56:37] It will fix https://phabricator.wikimedia.org/T144338 [17:56:52] and in my testing that form is needed for posting comments [17:57:20] paladox: why do you ping me on IRC about this? :) [17:58:07] andre__ Because it seemed to be better asked on irc then on a task [17:58:16] But if you want it on a task i can do that [18:00:26] paladox: when things are not urgent, I'd say it's *never* better to ask on irc than on a task. [18:00:40] Ok, i will write on a task then [18:00:52] paladox: On IRC your ping requires me to have attention at the very same moment. In tasks I can provide attention when *I* have time. [18:01:09] Oh sorry [18:01:21] paladox: This also applies for everybody else on IRC. [18:01:31] Ok [18:01:54] paladox: it's asynchron communication. Send me a letter and I answer when I have time. Call me via phone and I must have time in that moment you expect me to have time. [18:02:13] *asynchronous [18:02:35] Ok [18:02:37] sorry [18:02:45] no problem :) [18:04:22] (03CR) 10Thcipriani: [C: 032] Use scap as source of wikiversions rather than multiversion [tools/release] - 10https://gerrit.wikimedia.org/r/309355 (owner: 10Chad) [18:05:58] (03Merged) 10jenkins-bot: Use scap as source of wikiversions rather than multiversion [tools/release] - 10https://gerrit.wikimedia.org/r/309355 (owner: 10Chad) [18:29:57] 10Continuous-Integration-Infrastructure (phase-out-gallium), 06Operations: Upgrade Zuul on scandium.eqiad.wmnet (Jessie zuul-merger) - https://phabricator.wikimedia.org/T145057#2620190 (10hashar) Sounds good thanks! [18:30:26] twentyafterfour: so I just landed https://phabricator.wikimedia.org/D333 I have in my notes that I should: arc patch --no-branch D333; git push; arc close-revision D333 [18:30:58] but I'm getting, "You can not close this revision because you do not own it. To close a revision, you must be its owner" [18:31:26] Ah yep there is a config for ^^ [18:32:16] We doint allow users to configure other users patches, not sure why but there is no way to configure on a per user basis [18:32:27] so we either allow it or disable it for everyone [18:32:59] hrm, that's no good :\ [18:36:59] ah, autoclosed because I committed. [18:37:03] ok. [18:54:35] Krenair: thank you for the puppet fix on aptly01 :) [19:00:06] thcipriani: good news contint1001 is moved to production lan with a public IP : D [19:00:15] I got a bunch of random patches for puppet [19:00:42] hashar: I saw that/see your bunch of patches :) [19:00:52] scheduled for puppet swat [19:01:06] though I was not there. So unsurprisingly the serie will get merged later [19:01:13] late being very late on thursday PST [19:01:28] (or friday morning in Europe but tssk tssk dont tell anyone) [19:01:54] 06Release-Engineering-Team, 15User-greg: Create P&T offsite slides (due 9/12) - https://phabricator.wikimedia.org/T144511#2620428 (10greg) Good example from FR-Tech: https://docs.google.com/presentation/d/1ZPDkdTacgyH_Cd96BUHIEdJEO8KF4fHLwzozwNc6NAo/edit#slide=id.g11764204a4_0_5 [19:02:56] ostriches: are you phasing out mwMultiversion almost entirely ? [19:02:57] hashar: _joe_ said he would merge tomorrow in -operations, IIRC [19:03:05] thcipriani: yeah noticed that [19:03:18] hashar: Starting by removing all the cruft we don't need. [19:03:21] he is more careful than I am [19:03:35] neat [19:03:38] greg-g: LOL, I've never seen those slides until now [19:04:05] awight: they were just created this week, so no surprise there! [19:04:22] awight: for that manager "onsite/offsite" next week [19:04:48] hashar: Right now, we've got too many entry points and they're all a pile of includes. [19:04:49] harrr [19:05:14] Want to clean up the unused entry points/consolidate them, and reduce the # of requires and dependency chain. [19:05:28] ostriches: yeah it is a bit aged :( [19:08:33] ostriches: and Gerrit now properly sync the VisualEditor extension to mw/ext.git \o/ [19:08:39] (03CR) 10Hashar: [C: 032] Remove VE script to sync the Gerrit repo [integration/jenkins] - 10https://gerrit.wikimedia.org/r/309272 (https://phabricator.wikimedia.org/T51846) (owner: 10Hashar) [19:09:50] I know, it's been doing it :) [19:10:00] (03Merged) 10jenkins-bot: Remove VE script to sync the Gerrit repo [integration/jenkins] - 10https://gerrit.wikimedia.org/r/309272 (https://phabricator.wikimedia.org/T51846) (owner: 10Hashar) [19:11:39] yeah and I noticed one of your comment stating you could not find the related shell scripts [19:11:41] so here they are gone [19:12:56] !log integration: salt -v '*' cmd.run 'cd /srv/deployment/integration/slave-scripts; git pull' | https://gerrit.wikimedia.org/r/309272 [19:13:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:16:50] (03PS3) 10Hashar: Get rid of manual PHP_BIN handling [integration/jenkins] - 10https://gerrit.wikimedia.org/r/308931 (owner: 10Legoktm) [19:18:15] (03CR) 10Hashar: [C: 032] Get rid of manual PHP_BIN handling [integration/jenkins] - 10https://gerrit.wikimedia.org/r/308931 (owner: 10Legoktm) [19:18:28] cause really lets be bold [19:18:44] (03Merged) 10jenkins-bot: Get rid of manual PHP_BIN handling [integration/jenkins] - 10https://gerrit.wikimedia.org/r/308931 (owner: 10Legoktm) [19:18:47] :D [19:19:00] !log integration: salt -v '*' cmd.run 'cd /srv/deployment/integration/slave-scripts; git pull' | https://gerrit.wikimedia.org/r/308931 [19:19:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:19:07] legoktm: kudos on that patch [19:19:30] legoktm: also while you are there, it seems integration-jessie-lego-test01 and integration-aptly01 are down. Have they been deleted? [19:19:41] hashar yes [19:19:42] yes, I logged that in SAL [19:19:43] both are unreacheable from salt. If they are gone I can just drop the keys [19:19:56] ah yeah I forgot to RTFS [19:20:28] how do I delete stuff from salt? I didn't know I was supposed to do that [19:20:39] oh I never figured that out [19:20:46] so I just delete the instance and create a new one with same role [19:20:59] then it autosign all the keys again giving us a fresh / uptodate list [19:21:18] no really: sudo salt-key -d whatever.tenant.eqiad.wmflabs [19:21:35] to get list of hosts: salt-key [19:25:44] ok :D [19:26:15] !log repooled integration-slave-jessie-1005 now that php7 testing is done [19:26:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:26:30] :) [19:27:23] PROBLEM - Puppet run on deployment-conftool is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [19:27:51] (03CR) 10Legoktm: [C: 04-1] "We don't need php70lint yet, so I'd rather not add it." [integration/config] - 10https://gerrit.wikimedia.org/r/309048 (https://phabricator.wikimedia.org/T144961) (owner: 10Paladox) [19:28:30] (03PS4) 10Paladox: Add support for php7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/309048 (https://phabricator.wikimedia.org/T144961) [19:28:48] ostriches i carnt seem to clone https://gerrit.wikimedia.org/r/#/c/309048/ please? [19:28:59] When i click download it shows only the download files not the clone links [19:29:22] Oh hang [19:29:23] on [19:29:26] theres a bug [19:29:37] worksforme [19:29:38] It wont work on Microsoft Edge but does on Internet Explorer [19:32:56] (03PS5) 10Paladox: Add support for php7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/309048 (https://phabricator.wikimedia.org/T144961) [19:33:02] legoktm ^^ done [19:40:36] :) [19:46:10] !log Nodepool looping over some tasks since 17:45 ( https://grafana.wikimedia.org/dashboard/db/nodepool?panelId=21&fullscreen ) [19:46:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:47:30] !log nodepool cant delete: ci-jessie-wikimedia-369422 [ delete | 2.24 hours . Stuck in task_state=deleting :( [19:47:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:49:47] !log Nodepool, deleting instances that Nodepool lost track of (from nodepool alien-list) [19:49:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:51:47] til that Grafana dont let you invert a serie (eg: 1 / x ) :( [19:58:02] lol gerrit sets display: none on the box that decides weather too choose ssh or http [20:00:54] !log nova delete ci-jessie-wikimedia-369422 (was stuck in deleting state) [20:00:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [20:07:23] RECOVERY - Puppet run on deployment-conftool is OK: OK: Less than 1.00% above the threshold [0.0] [20:13:28] 10Gerrit: Microsoft Edge in gerrit dosent display the git commands in the download box - https://phabricator.wikimedia.org/T145130#2620802 (10Paladox) [20:15:02] 10Gerrit, 13Patch-For-Review: Microsoft Edge in gerrit dosent display the git commands in the download box - https://phabricator.wikimedia.org/T145130#2620818 (10Paladox) p:05Triage>03High This affects users who use Microsoft Edge so setting as high priority. [20:23:55] ostriches, im getting this error in my develeper tools [20:23:56] DENIED - The requested resource requires user authentication. [20:24:02] GET - https://gerrit.wikimedia.org/r/login/%23%2Fq%2Fstatus%3Aopen [20:24:07] HTTP401: [20:24:22] On Microsoft Edge when visiting the login page [20:24:41] I don't know anything about edge or its devtools [20:24:57] Oh [20:27:15] PROBLEM - Puppet run on deployment-ores-redis is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [20:43:48] Project selenium-Echo » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #142: 04FAILURE in 2 min 47 sec: https://integration.wikimedia.org/ci/job/selenium-Echo/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/142/ [20:43:50] Project selenium-Echo » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #142: 04FAILURE in 2 min 50 sec: https://integration.wikimedia.org/ci/job/selenium-Echo/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/142/ [21:00:59] remote: Processing changes: closed: 9, refs: 1, done [21:01:02] lovely gerrit [21:02:08] ? [21:02:13] RECOVERY - Puppet run on deployment-ores-redis is OK: OK: Less than 1.00% above the threshold [0.0] [21:16:41] 10Continuous-Integration-Infrastructure, 07Nodepool: Investigate use of Nodepool ListFloatingIPsTask - https://phabricator.wikimedia.org/T143943#2620992 (10hashar) a:03hashar The code path summary is: ``` name=nodepool/provider_manager.py, lang=python def cleanupServer() if self.hasExtension('os-floating... [21:19:35] Hi all. Is someone on top of the php53lint syntax error? [21:20:11] ejegg: Thx for asking... [21:20:31] For example, https://integration.wikimedia.org/ci/job/php53lint/5299/console [21:20:42] /tmp/hudson3459139067776911937.sh: line 3: -v: command not found [21:20:59] hashar ^^ [21:21:11] strange someone may have hacked the test since it does -v [21:23:41] 10Continuous-Integration-Infrastructure: Give @mobrovac access to CI instances - https://phabricator.wikimedia.org/T129880#2621006 (10mobrovac) Yup, will surely come useful soon enough. [21:23:55] 10Continuous-Integration-Infrastructure, 15User-mobrovac: Give @mobrovac access to CI instances - https://phabricator.wikimedia.org/T129880#2621007 (10mobrovac) [21:24:33] paladox: ahhazraze [21:24:42] ejegg: awight yeah the php script seems bugged (poke legoktm) [21:24:51] LOL, [21:25:03] guessing it was https://phabricator.wikimedia.org/rCIJE0faca12019bf1bc93a50f2f486040a590884e7e4 [21:25:34] $PHP_BIN -v [21:25:38] awight: ejegg will fix it [21:25:51] thanks! [21:25:53] it invokes an env variable which is no more set [21:26:00] we have phased out $PHP_BIN [21:26:07] used to switch between different flavor [21:26:10] and it is not set bah [21:26:52] :P [21:27:19] is fr-tech the last team using php53lint? [21:27:32] Yep [21:27:35] probably yeah [21:27:40] * awight blasts the stars [21:27:44] and all mediawiki branches before REL1_27 [21:28:02] thankfully we're off the pre-1.27 ones [21:28:11] Yep [21:28:30] oh hey, if anybody's keeping those around on our account feel free to clean 'em up! [21:29:00] looks like a simple fix anyway [21:29:43] We need to use them for mw 1.23 [21:29:52] so we will be using them until next year [21:30:00] then we can get rid of php53 :) [21:30:23] we SHOULD have the rest of the payments cluster on php55 by then [21:30:35] (03PS1) 10Hashar: Inject PHP_BIN=php5 for php53 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/309413 [21:30:41] err, the fundraising cluster rather [21:31:27] Oh [21:31:32] (03CR) 10Hashar: [C: 032] "This is terrible but I am tired :(" [integration/config] - 10https://gerrit.wikimedia.org/r/309413 (owner: 10Hashar) [21:31:33] (03CR) 10Ejegg: [C: 031] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/309413 (owner: 10Hashar) [21:31:42] ejegg: awight thank you for reporting here :) [21:31:46] thanks hashar ! [21:32:11] gotta deploy the change [21:32:16] then PHP_BIN will be set [21:32:34] (03Merged) 10jenkins-bot: Inject PHP_BIN=php5 for php53 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/309413 (owner: 10Hashar) [21:33:16] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/309413 " Inject PHP_BIN=php5 for php53 jobs" [21:33:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:33:49] paladox: thanks for the ping :) [21:34:00] Your welcome :) [21:34:05] thanks for fixing it hashar [21:34:06] :) [21:34:41] ejegg: awight: while you are around, I noticed a few FR related patches or bugs [21:34:49] I have only seen them today :( [21:35:01] the symfony polyfill is kind of a nice issue [21:35:02] Yep [21:35:04] hashar: no worries! [21:35:11] surely something can be hacked up :] [21:35:20] maybe using composer-test [21:35:23] hehe yeah polyfill looks fun. I think paladox has the right approach there, to use a more sophisticated lint wrapper from within composer test [21:35:26] that [21:35:37] to run parallel lint with a proper --exclude --do-no-lint-that-path [21:35:45] ah [21:35:50] Yep [21:35:55] if there is already a patch floating around that is even better ! [21:36:02] Yep, plus [21:36:07] i fixed a few things [21:36:12] that were failing [21:36:19] PROBLEM - Puppet run on deployment-eventlogging04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [21:36:20] the good thing is that one of the composer-job automatically install the require-devs dependencies [21:36:29] so you dont need to add them to the vendor repo (iirc) [21:37:03] oh right! I got started reviewing paladox's stuff yesterday. i should look at the rest [21:37:04] awight: ejegg php53lint should be good [21:37:15] thanks again hashar ! [21:37:18] will try to shine in tomorrow [21:37:24] :) [21:37:40] but yeah usually Paladox is quite successful at quickly figuring out solutions :D [21:37:47] Thanks :) [21:39:00] ejegg awight https://gerrit.wikimedia.org/r/#/c/307528/ :) [21:39:13] hashar ^^ :) [21:45:37] nodepool pre-depends on python2.7 --> python2.7 is not installed. [21:45:38] well [21:45:55] paladox yep, merged it! Thanks again [21:46:04] ejegg thanks [21:46:19] hashar im going to submit a patch to add composer to ^^ [21:47:11] ejegg|afk it seems nothing merge? [21:47:16] merge = merged [21:47:43] awight could you +2 https://gerrit.wikimedia.org/r/#/c/307528/ please? [21:47:53] Ah [21:47:59] now i see it meged [21:48:01] oh you even have a yaml linter! [21:48:05] gerrit ui was lieing to me [21:48:14] Who has a yaml linter? [21:48:44] in DonationInterface [21:49:29] Ah [21:49:32] thanks [21:50:07] hashar: I was trying, but haven't finished it yet... Quite a horrific thing. [21:51:31] (03PS1) 10Paladox: [DonationInterface] Adding composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/309456 [21:51:34] hashar ^^ [21:51:44] (03PS2) 10Paladox: [DonationInterface] Adding composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/309456 [21:54:41] hashar also there is https://gerrit.wikimedia.org/r/#/c/307543/ :) [21:54:47] (03PS4) 10Paladox: [DonationInterface] Switch jenkins tests to extension-unittests-composer-non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/307543 [22:03:14] paladox: can you stop adding CR+1 on everything ? :) [22:03:53] Ok sorry [22:07:41] sometime it does not make sense [22:07:54] an example is https://gerrit.wikimedia.org/r/#/c/309459/ [22:08:00] with the build still failing [22:08:11] when the intent of this patch is to eventually fix the failling test! [22:08:23] Ok [22:11:19] RECOVERY - Puppet run on deployment-eventlogging04 is OK: OK: Less than 1.00% above the threshold [0.0] [22:15:46] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621158 (10Paladox) [22:16:53] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621160 (10Ottomata) mocha or maybe better, npm test [22:18:25] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621167 (10Paladox) Yep, we can do npm test, but just to let you know jenkins is currently broken in diff... [22:26:48] 10Continuous-Integration-Infrastructure, 07Nodepool, 13Patch-For-Review: Investigate use of Nodepool ListFloatingIPsTask - https://phabricator.wikimedia.org/T143943#2621212 (10hashar) We will want to upgrade Nodepool to 0.1.1-wmf5 Packages build and published at: https://people.wikimedia.org/~hashar/debs/n... [22:31:30] 10Continuous-Integration-Infrastructure, 06Labs, 06Operations, 07Nodepool: Upgrade Nodepool to 0.1.1-wmf5 to reduce requests made to OpenStack API - https://phabricator.wikimedia.org/T145142#2621216 (10hashar) [22:31:49] 10Continuous-Integration-Infrastructure, 07Nodepool: 2016-08-10 CI incident follow-ups - https://phabricator.wikimedia.org/T142952#2621236 (10hashar) [22:31:51] 10Continuous-Integration-Infrastructure, 07Nodepool, 13Patch-For-Review: Investigate use of Nodepool ListFloatingIPsTask - https://phabricator.wikimedia.org/T143943#2584195 (10hashar) 05Open>03stalled Patch ready. Pending upgrade in production via T145142 [22:32:00] yeah well [22:32:05] .deb packaging is long :( [22:32:22] LOL [22:32:31] a good one hour to add a: "False and " [22:32:32] :=] [22:32:46] anyway sleep sleep sleep [22:32:52] LOL and bye [22:32:53] :) [22:33:00] * paladox will watch tv in a min [22:40:19] (03CR) 10Paladox: [C: 031] Phabricator/harbormaster job templates [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [22:40:24] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621287 (10mmodell) it should now be set up to run `npm test` [22:41:04] (03PS12) 10Paladox: Phabricator/harbormaster job templates [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [22:41:40] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621295 (10Paladox) Thanks @mmodell [22:42:28] (03PS13) 10Paladox: Phabricator/harbormaster job templates [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [22:42:39] (03CR) 10Paladox: "Rebased again." [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [22:45:21] (03PS14) 1020after4: Phabricator/harbormaster job templates [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) [22:45:46] (03CR) 10Paladox: "Changed node to contintLabsSlave && DebianJessie" [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [22:45:56] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621308 (10mmodell) @ottomata: running `npm test` locally on a checkout of this repo throws a bunch of er... [22:51:10] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621117 (10mobrovac) How about using Travis for now since the repo is on GitHub? [22:52:41] greg-g, legoktm, are you able to ssh to deployment-fluorine? I'm getting 'administratively prohibited'. [22:53:07] nope.... [22:54:44] I forget, does running out of disk space cause that? [22:54:52] I feel like that's one of the ssh root causes. [22:55:37] No, apparently then it just becomes unresponsive, unless it depends. [22:55:47] legoktm i fixed https://gerrit.wikimedia.org/r/309048 [22:55:48] :) [22:55:57] and removed the part you wanted to be removed [22:55:57] :) [22:56:55] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible: deployment-fluorine ssh 'administratively prohibited' - https://phabricator.wikimedia.org/T145147#2621385 (10Mattflaschen-WMF) [22:57:32] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible: deployment-fluorine ssh 'administratively prohibited' - https://phabricator.wikimedia.org/T145147#2621397 (10Mattflaschen-WMF) [22:57:45] 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible: deployment-fluorine ssh 'administratively prohibited' - https://phabricator.wikimedia.org/T145147#2621385 (10Krenair) open failed because that host does not exist anymore [23:00:42] 10Beta-Cluster-Infrastructure: deployment-fluorine ssh 'administratively prohibited' - https://phabricator.wikimedia.org/T145147#2621419 (10greg) [23:01:14] (03CR) 10Paladox: [C: 031] "Please merge :)" [integration/config] - 10https://gerrit.wikimedia.org/r/295396 (https://phabricator.wikimedia.org/T130950) (owner: 1020after4) [23:01:26] (03CR) 10Paladox: [C: 031] install contint::arcanist module on contint slave snapshots [integration/config] - 10https://gerrit.wikimedia.org/r/295976 (owner: 1020after4) [23:05:37] 10Beta-Cluster-Infrastructure: deployment-fluorine ssh 'administratively prohibited' - https://phabricator.wikimedia.org/T145147#2621431 (10Krenair) 05Open>03Resolved a:03Krenair I've updated the Overview page to show the new hostname [23:29:40] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621501 (10mmodell) @mobrovac I think that is the status quo for now. This task, however, is about differ... [23:49:28] 10Continuous-Integration-Infrastructure, 10Analytics, 10Differential, 10EventBus, 10Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2621577 (10Ottomata) Yeah, npm test 'works' in travis now because `.travis.yml` sets up the KAFKA_HOME bi... [23:54:43] PROBLEM - Puppet run on deployment-mx is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]