[00:18:32] RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [00:19:30] RECOVERY - Puppet errors on deployment-memc06 is OK: OK: Less than 1.00% above the threshold [0.0] [04:43:39] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<44.44%) [04:47:32] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [10.0] [05:22:30] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [10.0] [06:12:29] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:41:48] 10Gerrit: Setup mirror for gerrit to clone from GitHub for repo design/style-guide - https://phabricator.wikimedia.org/T189370#4041713 (10demon) If development is going to happen on Github I would call Gerrit a mirror yes. I don't see the problem [07:08:39] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:18:35] 10RelEng-Archive-FY201718-Q1, 10Wikimedia-Developer-Summit, 10User-greg: Facilitate Wikidev'17 main topic "How to manage our technical debt" - https://phabricator.wikimedia.org/T147937#4041734 (10Joe) [07:33:04] 10Gerrit, 10Release-Engineering-Team (Next), 10DBA, 10Operations, 10Patch-For-Review: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532#3628916 (10Joe) Is anyone working on this issue? @Dzahn @jcrespo if neither of you is wo... [07:34:45] 10Gerrit, 10Release-Engineering-Team (Next), 10DBA, 10Operations, 10Patch-For-Review: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532#4041753 (10Marostegui) The proxies for codfw have been budget but not yet ordered, so rig... [08:50:08] PROBLEM - Puppet errors on integration-slave-jessie-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:50:58] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:56:27] 10Phabricator (Upstream), 10Upstream: When navigating from a ticket to a tag, highlight the ticket in the workboard - https://phabricator.wikimedia.org/T189207#4041858 (10Pginer-WMF) 05Open>03Invalid >>! In T189207#4035340, @Aklapper wrote: > If you are after moving one specific ticket on workboard, the re... [09:14:05] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [09:18:36] PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [09:55:22] hi zeljkof hashar . Any idea what's broken at https://gerrit.wikimedia.org/r/#/c/309724/ ? I can't merge it, and the testing error it doesn't seem to be related, but I might be wrong. [09:55:40] aharoni: looking... [09:58:15] aharoni: looks like jenkins is complaining about something in a npm package :| [09:58:26] zeljkof: should I report? [09:58:32] does it happen elsewhere? [09:58:57] aharoni: did not notice it elsewhere, if a `recheck` does not help, you should report it [09:59:23] zeljkof: already tried recheck. I'll report. [09:59:38] aharoni: looks like the job works fine for other patches: https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/ [10:01:39] aharoni: and looks like dependencies did not change for months https://phabricator.wikimedia.org/source/mediawiki/history/master/package.json [10:01:52] zeljkof: https://phabricator.wikimedia.org/T189451 [10:02:38] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042171 (10zeljkofilipin) [10:14:07] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042228 (10zeljkofilipin) Looks like the problem is in sntp npm package, a dependency of nodemw (mediawiki API) package: ``` 00:01:... [10:21:09] aharoni: usually I would do a diff of the console output between a SUCCESS and FAILURE builds [10:21:19] eg https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/16422/console vs https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/16424/console [10:24:20] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042271 (10zeljkofilipin) Note to self (thanks to @hashar): compare output of a failed job with a job that did not help. [10:24:36] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042276 (10zeljkofilipin) a:03zeljkofilipin [10:27:12] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042327 (10hashar) On https://gerrit.wikimedia.org/r/#/c/309724/ a good build had: nodemw@0.11.0 -> request@2.83.0 -> hawk@6.0... [10:27:26] zeljkof: https://phabricator.wikimedia.org/T189451#4042327 ;] [10:27:34] so yeah what you said [10:27:41] sntp@3 does not support nodejs 6 [10:27:59] and nodemw is the one to blame because it is too liberal in the request version it uses (more or less) [10:28:14] so MAYBE we can add pin request@2.83.* [10:28:25] hashar: ok, that is fixed by https://gerrit.wikimedia.org/r/#/c/417848/ ;) [10:28:33] WIP Replace nodemw with mwbot [10:28:50] well yeah eventually long terme [10:28:51] term [10:28:55] or we could just merge mwbot change [10:29:21] it's ready, it's WIP because I guess I forgot to remove it :D [10:29:48] oh, no, I was just waiting for feedback, that's why I left WIP [10:30:11] but as far as I care, removing WIP and merging it is fine [10:30:19] let me try pinning to 2.83 [10:31:47] 10Release-Engineering-Team (Kanban), 10Jenkins, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042345 (10zeljkofilipin) p:05Normal>03Unbreak! @hashar uh oh, in that case this is UBN! [10:31:51] "request": "<2.84.0", ;D [10:31:59] trying that locally [10:32:12] hashar: yes, that is what I plan to do [10:32:55] zeljkof: so yeah locally that works if I do <2.84.0 [10:33:37] guess you can try, send patch to gerrit and if it passes CI we can +2 it :] [10:33:58] hashar: hm, but sntp is the one that fails, so if we lock it to 2.1.0 it has the same effect, right? [10:34:33] well request requires hawk@7.0.7 which requires sntp@3.0.1 [10:34:44] so I guess if you pin to sntp to 2.x [10:34:48] hawk will probably not work [10:34:58] testing ;) [10:35:27] I think we should pin up in the tree instead of something deep inside it [10:36:04] zeljkof: and upstream bug is https://github.com/request/request/issues/2891 [10:36:41] 10Release-Engineering-Team (Kanban), 10Jenkins, 10Patch-For-Review, 10Upstream, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042362 (10hashar) [10:37:06] 10Release-Engineering-Team (Kanban), 10Jenkins, 10Patch-For-Review, 10Upstream, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042157 (10hashar) The upstream bug for `request` is https://github.com/request/request/issues/... [10:37:32] zeljkof: hey it is transient [10:37:42] zeljkof: the 'request' author is acutally fixing it as I write this [10:37:54] he commented on the github issue a minute or so ago [10:37:57] hashar: just reading the thread :) [10:38:07] hashar: should we just wait ;) [10:38:46] zeljkof: he is just going to release a new version [10:39:28] zeljkof: so yeah just wait (tm) [10:39:34] 10Release-Engineering-Team (Kanban), 10Jenkins, 10Patch-For-Review, 10Upstream, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042369 (10hashar) So yeah request@2.84.0 breaks on Node 6 and apparently upstream is going to... [10:40:16] https://github.com/request/request/commit/21ef363b91c17763d6c79a639a197bf72135b97a :D [10:40:59] that should give us 2.85.1 [11:05:33] 10Release-Engineering-Team (Kanban), 10Jenkins, 10Patch-For-Review, 10Upstream, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042420 (10zeljkofilipin) 05Open>03Resolved Resolved upstream! The commit is now ready to b... [12:06:42] 10Gerrit, 10Release-Engineering-Team (Next): Upgrade gerrit from 2.14.6 to 2.14.7 - https://phabricator.wikimedia.org/T186135#4042713 (10Paladox) 2.14.7 was released today [12:17:06] 10Release-Engineering-Team (Kanban), 10Jenkins, 10Patch-For-Review, 10Upstream, 10User-zeljkofilipin: Jenkins fails to merge a patch with "Unexpected token function" - https://phabricator.wikimedia.org/T189451#4042736 (10hashar) I also rebuild [[https://integration.wikimedia.org/ci/job/wikimedia-portals-... [12:24:32] !log Jenkins: uninstalled the "cvs" plugin [12:24:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:27:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Someday), 10Jenkins: Install the BlueOcean plugin on jenkins - https://phabricator.wikimedia.org/T155840#4042781 (10hashar) It has a LOT of dependencies: Design Language Blue Ocean Core JS GitHub API GitHub GitHub Branch Source... [12:28:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Someday), 10Jenkins: Install the BlueOcean plugin on jenkins - https://phabricator.wikimedia.org/T155840#4042785 (10hashar) It is now available from https://integration.wikimedia.org/ci/blue/ There are surely a few things we want to tweak... [12:29:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Jenkins: Install the BlueOcean plugin on jenkins - https://phabricator.wikimedia.org/T155840#4042787 (10hashar) [12:33:05] !log Jenkins: installed Blue Ocean plugin. Eg: https://integration.wikimedia.org/ci/blue/ | T155840 [12:33:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:33:07] T155840: Install the BlueOcean plugin on jenkins - https://phabricator.wikimedia.org/T155840 [14:22:03] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10cloud-services-team, 10Privacy, 10Vuln-Infoleak: logstash-beta: stop exposing IP addresses to the public - https://phabricator.wikimedia.org/T189489#4043190 (10MarcoAurelio) [14:27:48] * paladox wonders why empty tool repos were created on phab see https://phabricator.wikimedia.org/diffusion/?after=2293 [14:30:00] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10cloud-services-team, 10Privacy, 10Vuln-Infoleak: logstash-beta: stop exposing IP addresses to the public - https://phabricator.wikimedia.org/T189489#4043247 (10Huji) p:05Triage>03Unbreak! This is an overt violation of Privacy Policy as well. Marking... [14:30:00] twentyafterfour hi, im wondering would you be able to fix this repo https://phabricator.wikimedia.org/diffusion/EBSP/manage/basics/ please? When clicking storage it says "Unable to load cluster service." but also it is stuck on importing. (i didnt create the repo but found it stuck on importing) [14:34:28] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10cloud-services-team, 10Privacy, 10Vuln-Infoleak: logstash-beta: stop exposing IP addresses to the public - https://phabricator.wikimedia.org/T189489#4043190 (10Bawolff) There's a related [secret] bug at T161051 [14:35:23] 10Beta-Cluster-Infrastructure, 10Wikidata: Domain 'sdwiki' is not recognized. - https://phabricator.wikimedia.org/T189493#4043275 (10Reedy) [14:36:12] https://phabricator.wikimedia.org/diffusion/EBSP/manage/storage/ <-- never saw that error before [14:37:11] twentyafterfour: ^ [14:40:20] I think there's a command to remove it from cluster mode [14:40:45] ./bin/repository clusterize --remove-service [14:40:51] https://secure.phabricator.com/book/phabricator/article/cluster_repositories/ [14:41:24] 10Beta-Cluster-Infrastructure, 10Wikidata: Domain 'sdwiki' is not recognized. - https://phabricator.wikimedia.org/T189493#4043275 (10MarcoAurelio) Yup. Spotted that as well while looking for somewhere else. Do we know who issued that? Puppet? [14:44:33] 10Beta-Cluster-Infrastructure, 10Wikidata: Domain 'sdwiki' is not recognized. - https://phabricator.wikimedia.org/T189493#4043320 (10Reedy) I'm guessing the dispatchChanges is possibly using the wrong wiki list or similar for beta [14:56:37] 10Release-Engineering-Team, 10Epic, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review, 10User-zeljkofilipin: Q3 Selenium framework improvements - https://phabricator.wikimedia.org/T182421#4043373 (10zeljkofilipin) [14:56:39] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Find a few people interested in reviewing Selenium patches - https://phabricator.wikimedia.org/T188744#4043371 (10zeljkofilipin) 05Open>03Resolved Thanks to everybody that already helped out. Resolving so it does not stay open forever, but you can... [15:20:38] PROBLEM - Free space - all mounts on integration-slave-docker-1005 is CRITICAL: CRITICAL: integration.integration-slave-docker-1005.diskspace.root.byte_percentfree (<50.00%) [15:27:45] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:33:43] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:54:10] (03CR) 10Robert Vogel: [C: 031] Added dependency "ExtJSBase" to "BlueSpiceFoundation" [integration/config] - 10https://gerrit.wikimedia.org/r/389412 (owner: 10Robert Vogel) [16:10:44] what's the easy way to get wfdebuglog output to go somewhere on deployment-prep images, anyone know offhand? [16:11:06] I want to switch it on for just a bit on my snapshot instance for testing, without spamming whatever logstash and etc [16:26:12] apergos: it is in logstash [16:26:30] yeah, I wanna not do that for a bit [16:26:45] there is this variable I found MW_DEBUG_LOCAL, anyone use that? [16:26:50] then filter by host / debug level [16:26:50] seems to be an environ var [16:26:54] no clue [16:27:13] I'll try it later and see [16:42:18] 10Scap, 10Operations, 10Packaging, 10Patch-For-Review: Install git-lfs client (at least on scap targets & masters) - https://phabricator.wikimedia.org/T180628#4043780 (10akosiaris) The scap targets that would benefit from this (namely `ores*` boxes) now have git-lfs installed. @mmodell do we also need this... [16:46:25] so the answer is, prefix your php command (cli right) with MW_DEBUG_LOCAL=something and it writes to /tmp/wiki.log this is very nice [17:06:22] 10Gerrit, 10Release-Engineering-Team (Next), 10DBA, 10Operations, 10Patch-For-Review: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532#4043863 (10Dzahn) Let's not set it to role::spare::system please. That would mean activel... [18:02:50] 10Scap, 10Operations, 10Packaging, 10Patch-For-Review: Install git-lfs client (at least on scap targets & masters) - https://phabricator.wikimedia.org/T180628#4044173 (10mmodell) I think it's needed on masters at least for deployers to issue git-lfs commands if not for scap itself. [18:16:32] paladox: Building gerrit 2.14.7 (includes the its-base fix) [18:17:22] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/gerrit/gerrit/+log/wmf/stable-2.14 [18:39:23] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:40:47] PROBLEM - Free space - all mounts on deployment-mediawiki05 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki05.diskspace.root.byte_percentfree (<11.11%) [18:50:47] RECOVERY - Free space - all mounts on deployment-mediawiki05 is OK: OK: All targets OK [18:57:32] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [19:13:00] no_justification: thank you :) [19:13:55] no_justification: I will try it out in a bit (currently having dinner :)) [19:38:17] i will test i tnow [19:46:27] no_justification somehow it is throwing "Exception in thread "main" java.io.IOException: Permission denied "(not realted to your change) [19:46:40] I pressed n on plugins as they would have been updated when i scapped [19:46:57] but some how i think it's trying to do something in the plugins folder [19:47:16] ah [19:47:20] found the plugin [19:47:21] com.googlesource.gerrit.plugins.replication.Init.run [19:47:35] (03PS1) 10Hashar: parsoidsvc-hhvm-parsertests to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/418984 [19:47:55] i will do the upgrade as root then, and then chrown it back to gerrit2 [19:49:45] no_justification upgraded worked, https://gerrit.git.wmflabs.org/r/#/q/status:open (will +1 your change :)) [19:52:33] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [10.0] [19:54:28] !log Cherry picked https://gerrit.wikimedia.org/r/#/c/418986/ to deployment-puppetmaster02 [19:54:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:55:55] !log Forced puppet run on deployment-logstash2. Failed due to bad logstash::eventlogging hiera data [19:55:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:09:04] !log Removed role::logstash::eventlogging from deployment-logstash2 because the hiera config is failing (undefined method `[]' for nil:NilClass at /etc/puppet/modules/role/manifests/logstash/eventlogging.pp:11) [20:09:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:09:53] !log Forced puppet run on deployment-logstash2 for {{gerrit|418986}} [20:09:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:11:28] how log has puppet been busted on that host?! :/ [20:13:32] !log deployed to BC: [mobileapps/deploy@c764714]: Update mobileapps to 5c90db7 [20:13:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:15:24] bd808: Who knows [20:15:36] paladox: Yeah, I usually have to run init as root then re-chown it to gerrit2 [20:15:40] Always been mildly annoying [20:15:47] yeh [20:22:22] no_justification i think i can keep gerrit-test3 @ same version as prod whilst testing 2.15 :) (as i found there's the gerrit project seperate to git) and i can have a floating ip in there [20:23:03] Were there any diffs on *.config files post-upgrade puppet run? [20:23:47] no_justification let me check the logs as i didnt run the puppet command after upgrading [20:23:56] the puppet logs [20:24:24] no_justification yep there was [20:24:25] - javaOptions = -Xmx2g -Xms2g -Dlog4j.configuration=file:///var/lib/gerrit2/review_site/etc/log4j.xml [20:24:25] + javaOptions = -Xmx2g -Xms2g -Dlog4j.configuration=file:///var/lib/gerrit2/review_site/etc/log4j.xml [20:24:40] and [20:24:41] - cookiePath = / [20:24:41] + cookiePath = "/" [20:24:41] registerUrl = https://wikitech.wikimedia.org/w/index.php?title=Special:CreateAccount&returnto=Gerrit/NewUse [20:24:48] Ok, so cookiepath we should fix. [20:24:49] no_justification we need to remove the qoutes ^^ [20:24:51] That's easy [20:24:53] heh yeh [20:25:01] should i let you do that ? :) [20:25:04] The other one is the usual trailing whitespace one [20:25:06] Go for it :) [20:25:08] * no_justification is busy [20:25:10] ok :) [20:25:11] thanks [20:27:49] no_justification https://gerrit.wikimedia.org/r/#/c/418990/ [20:28:19] RECOVERY - Puppet errors on deployment-logstash2 is OK: OK: Less than 1.00% above the threshold [0.0] [20:28:34] Yippee, build fixed! [20:28:34] Project selenium-Wikibase-chrome ยป chrome,beta,Linux,DebianJessie && contintLabsSlave build #143: 09FIXED in 41 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/143/ [20:39:00] PROBLEM - Puppet errors on deployment-mediawiki07 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:48:00] 10Release-Engineering-Team (Next), 10Scap, 10Wikimedia-Incident: Scap sync-file: allow to sync multiple files in different directories - https://phabricator.wikimedia.org/T186067#4044711 (10thcipriani) 05Open>03declined Syncing multiple files in different directories can be done via `scap sync`, so there... [20:55:17] PROBLEM - Puppet errors on deployment-maps01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:30:32] 10Phabricator, 10Release-Engineering-Team (Someday), 10Operations, 10Patch-For-Review: Add support for stretch in the phabricator puppet class - https://phabricator.wikimedia.org/T187127#4044767 (10Paladox) Needs package libsodium23 to also be imported into php 7.2 in thirdparty/component php7.2-cli : De... [21:41:25] PROBLEM - Free space - all mounts on integration-slave-jessie-1001 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1001.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1001.diskspace._srv.byte_percentfree (<40.00%) [22:00:15] hi releng! [22:00:30] do you know who's responsible for the ExtensionDistributor? [22:00:46] It's giving 404s for all the download links [22:00:48] e.g. [22:00:50] https://www.mediawiki.org/wiki/Special:ExtensionDistributor?extdistname=CentralNotice&extdistversion=master [22:04:03] legoktm ^^ [22:06:24] it works for me [22:07:13] It dosen't work for me [22:07:21] ah works now [22:09:50] paladox: same here, I guess there's a delay creating the tarballs [22:09:57] yep [22:10:18] cool, working on all the branches now [22:14:49] 10Gerrit: Support OAuth for login onto gerrit.wikimedia.org - https://phabricator.wikimedia.org/T147864#4044871 (10Tgr) [22:17:19] 10Phabricator, 10Release-Engineering-Team (Someday), 10Operations, 10Patch-For-Review: Add support for stretch in the phabricator puppet class - https://phabricator.wikimedia.org/T187127#4044877 (10Paladox) We will also need to import php-apcu and php-mailparse from there too. [22:38:00] paladox: I updated webhooks plugin to use 21e71dc9abb42f5ac80c51d2df2a374c3414b9d2 of bazlets, but still fails? [22:38:11] no_justification /me has a look [22:38:34] using the stable-2.14 branch? [22:38:47] Yep [22:39:32] ok [22:39:36] * paladox clones it :) [22:40:17] no_justification strange it works for me [22:40:28] Something's cached somewhere :\ [22:40:43] no_justification yeh, run bazel clean && bazel build [22:40:56] I ran bazel clean and `bazel build` [22:41:01] With no tho [22:41:19] The failure is on running ./tools/eclipse/project.py [22:41:34] oh [22:41:40] no_justification what failure does it say? [22:41:44] is it python3? [22:42:15] I though so...pastebinning [22:42:26] https://www.irccloud.com/pastebin/Lqo1jArT/ [22:43:47] yep that's python3 [22:44:17] I have no idea how mine went back to python2 by default though [22:44:55] no_justification update to 3dbcd76602dd93fddb9deae4cacd1699164a19e9 [22:47:57] no_justification that one includes the full ranges of python3 fixes [22:48:01] 10Gerrit, 10WikimediaUI Style Guide: Setup mirror for gerrit to clone from GitHub for repo design/style-guide - https://phabricator.wikimedia.org/T189370#4044959 (10Volker_E) [22:48:04] including the ones in gerrit core [22:48:12] 10Gerrit, 10WikimediaUI Style Guide: Setup mirror for gerrit to clone from GitHub for repo design/style-guide - https://phabricator.wikimedia.org/T189370#4044961 (10demon) 05stalled>03Resolved a:03demon Imported history from Github to Gerrit. Spoke with @Volker_E about the deployment process: # Review h... [22:49:56] no_justification aha, 21e71dc9abb42f5ac80c51d2df2a374c3414b9d2 does not include python3 fixes [22:50:00] but 3dbcd76602dd93fddb9deae4cacd1699164a19e9 does :) [22:50:03] Derp [22:50:09] https://github.com/GerritCodeReview/bazlets/commit/21e71dc9abb42f5ac80c51d2df2a374c3414b9d2 [22:54:52] no_justification i need to update it upstream too [22:55:00] it's using an old version heh (in webhooks) [22:57:24] no_justification https://gerrit-review.googlesource.com/c/plugins/webhooks/+/165237 [22:59:16] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10cloud-services-team, 10Privacy, 10Vuln-Infoleak: logstash-beta: stop exposing IP addresses to the public - https://phabricator.wikimedia.org/T189489#4045012 (10Tgr) [23:00:32] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10cloud-services-team, 10Privacy, 10Vuln-Infoleak: logstash-beta: stop exposing IP addresses to the public - https://phabricator.wikimedia.org/T189489#4043190 (10Tgr) As discussed elsewhere, this was not a violation of the Labs ToU, nevertheless not a grea... [23:00:42] * paladox has just went around updating plugins to 3dbcd76602dd93fddb9deae4cacd1699164a19e9 :) [23:03:21] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikimedia-Logstash, 10Patch-For-Review, 10User-bd808: Make logstash in beta public - https://phabricator.wikimedia.org/T76784#819741 (10Tgr) This was reverted following discussion in {T161051}. [23:05:04] 10Scap, 10Packaging, 10Patch-For-Review: SCAP: Upload debian package version 3.7.7-1 - https://phabricator.wikimedia.org/T189306#4045039 (10mmodell) [23:08:15] no_justification do we want to install https://gerrit.googlesource.com/plugins/metrics-reporter-jmx/+/refs/heads/master ? [23:08:27] Idk yet. Low priority [23:09:08] ok [23:09:23] i requested a 2.14 branch at https://groups.google.com/forum/#!topic/repo-discuss/XfipArhZnaI [23:11:21] Same thing with go-import, btw [23:11:38] And reviewers [23:11:44] (bazlets outta date) [23:12:05] https://phabricator.wikimedia.org/T176532#4041751 [23:12:11] https://gerrit.wikimedia.org/r/419080 [23:12:59] ok [23:13:01] thanks [23:13:14] no_justification ^^ (mutante comment) :) [23:13:15] https://gerrit.wikimedia.org/r/419080 [23:13:24] ah you saw [23:13:54] the process check is easy, the systemd check needs change in base.. additoinally [23:14:48] 10Beta-Cluster-Infrastructure, 10Privacy: Flush private data on Beta Cluster - https://phabricator.wikimedia.org/T189541#4045055 (10Tgr) [23:17:23] no_justification i think i've done all the ones we have deployed @ wmf now :) [23:17:28] https://gerrit-review.googlesource.com/q/owner:thomasmulhall410%2540yahoo.com [23:19:04] no_justification also i finally got https://gerrit-review.googlesource.com/c/gerrit/+/160034 merged :) [23:21:56] 10Beta-Cluster-Infrastructure, 10Privacy: Flush private data on Beta Cluster - https://phabricator.wikimedia.org/T189541#4045074 (10Tgr) [23:22:37] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#4045082 (10Tgr) [23:22:39] 10Beta-Cluster-Infrastructure, 10Privacy: Flush private data on Beta Cluster - https://phabricator.wikimedia.org/T189541#4045055 (10Tgr) [23:24:46] 10Beta-Cluster-Infrastructure, 10Patch-For-Review, 10Privacy, 10User-MarcoAurelio: Disable the collection of private information on abusefilter log for Beta Cluster wikis - https://phabricator.wikimedia.org/T188862#4022014 (10Tgr) It is unrealistic to assume that only people with an NDA get access to priva... [23:27:40] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#831102 (10Tgr) I don't think making the privacy policy link to the labs terms of use is helpful. The kind of confusion it creates can be evidenced in... [23:28:24] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#4045122 (10Krenair) >>! In T77858#4045113, @Tgr wrote: > I don't think making the privacy policy link to the labs terms of use is helpful. The kind o... [23:30:57] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [23:37:05] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#4045141 (10Tgr) Anyway it creates confusion :-) The Labs ToU is between tool developers and the WMF, not users and the WMF. It does not really make s... [23:39:07] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#4045143 (10Krenair) >>! In T77858#4045141, @Tgr wrote: > Anyway it creates confusion :-) The Labs ToU is between tool developers and the WMF, not use... [23:43:14] no_justification: any reason for me to not merge https://gerrit.wikimedia.org/r/#/c/358896/ ? [23:43:32] Not that I can think of [23:43:53] ok. Let me know if it immediately and horribly breaks the deployer workflow [23:45:23] i wonder what the issue of contint2001 is .. looking [23:46:14] none... [23:46:23] 10Beta-Cluster-Infrastructure: Make Privacy Policy/ToS on Beta Cluster link to the labs version (not production version) - https://phabricator.wikimedia.org/T77858#4045162 (10Tgr) >>! In T77858#4045143, @Krenair wrote: > I don't think it creates as much confusion as pointing at the prod privacy policy. It might... [23:51:03] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [23:51:13] mutante i think it was puppetdb [23:52:27] 10Gerrit, 10Release-Engineering-Team (Next), 10DBA, 10Operations, 10Patch-For-Review: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532#4045167 (10Dzahn) p:05Low>03Normal We still want this just as before. We were just as... [23:55:55] no_justification all the 2.14 changes where merged that i uploaded :) [23:56:03] (so it should make building easier for you now :)) [23:56:11] now on to part 2, doing 2.15 [23:56:12] lol [23:56:52] after merging the branches