[00:14:46] 10Deployment-Systems, 06Operations: dologmsg doesn't work on terbium - https://phabricator.wikimedia.org/T141619#2505074 (10Dzahn) @Anomie see my changes above, which i merged now. how about now? [00:16:57] Project selenium-Flow » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #99: 04FAILURE in 56 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/99/ [00:17:06] Project selenium-Flow » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #99: 04FAILURE in 1 min 5 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/99/ [00:24:17] why isn't grrrit-wm talking in here? [00:24:48] Oh, not sure [00:25:14] > error: prefix=weber.freenode.net, server=weber.freenode.net, command=err_cannotsendtochan, rawCommand=404, commandType=error, args=[grrrit-wm, #wikimedia-releng, Cannot send to channel] [00:25:16] legoktm ^^ [00:31:44] 10Deployment-Systems, 03Scap3: More atomic directory operations - https://phabricator.wikimedia.org/T141913#2517632 (10Dereckson) Yes, if it's already implemented in scap3, this task is indeed a duplicate. [00:42:04] not sure why it says it cannot send to channel [00:46:54] 10Deployment-Systems, 06Operations: dologmsg doesn't work on terbium - https://phabricator.wikimedia.org/T141619#2517673 (10Dzahn) i see there is still an issue .. ferm rules not on neon yet.. hmm.. [00:52:26] Yippee, build fixed! [00:52:27] Project selenium-Wikidata » firefox,test,Linux,contintLabsSlave && UbuntuTrusty build #74: 09FIXED in 2 hr 40 min: https://integration.wikimedia.org/ci/job/selenium-Wikidata/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/74/ [01:31:39] 10Deployment-Systems, 06Operations: dologmsg doesn't work on terbium - https://phabricator.wikimedia.org/T141619#2517783 (10Dzahn) the reason why it doesn't work yet even after merge is currently T141957 an unrelated issue that prevents the new ferm rules from being applied [02:22:58] 10Deployment-Systems, 06Operations: dologmsg doesn't work on terbium - https://phabricator.wikimedia.org/T141619#2517803 (10Dzahn) That issue has been fixed.. now we have these iptables rules ACCEPT tcp -- terbium.eqiad.wmnet anywhere tcp dpt:9200 ACCEPT tcp -- wasat.codfw.wmnet an... [02:26:53] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, 07Wikimedia-Incident: Config beta ORES extension to use the beta ORES service - https://phabricator.wikimedia.org/T141825#2513412 (10Legoktm) Well, I merged it. But I don't think we're still at the right... [03:25:47] 10Deployment-Systems, 06Operations: dologmsg doesn't work on terbium - https://phabricator.wikimedia.org/T141619#2517862 (10Dzahn) 05Open>03Resolved a:03Dzahn @anomie finally works now :) 20:29 < logmsgbot> testing from terbium T141619 20:29 < stashbot> T141619: dologmsg doesn't work on terbium - https:... [03:56:16] Project selenium-MultimediaViewer » chrome,beta,OS X 10.9,contintLabsSlave && UbuntuTrusty build #93: 04FAILURE in 15 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=contintLabsSlave%20&&%20UbuntuTrusty/93/ [03:56:19] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,contintLabsSlave && UbuntuTrusty build #93: 04FAILURE in 19 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=contintLabsSlave%20&&%20UbuntuTrusty/93/ [03:56:21] Project selenium-MultimediaViewer » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #93: 04FAILURE in 20 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/93/ [05:22:51] !log Jenkins job beta-mediawiki-config-update-eqiad has been stuck and unrun for 6 hours [05:22:58] greg-g: ^ [05:23:19] Shown on https://integration.wikimedia.org/zuul/ and https://integration.wikimedia.org/ci/ [05:47:05] Looks like Jenkins connection to deployment-tin.eqiad.wmflabs is deadlocked again [07:34:34] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518118 (10Nemo_bis) [08:39:29] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518118 (10Danny_B) What is the desired target? [08:41:15] 05Gitblit-Deprecate, 10Diffusion, 13Patch-For-Review: Update all references to git.wikimedia.org and replace them with the Phabricator equivalent - https://phabricator.wikimedia.org/T137353#2518196 (10Aklapper) [08:41:18] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518195 (10Aklapper) [08:41:35] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518118 (10Aklapper) Expected target should be https://phabricator.wikimedia.org/diffusion/GTWN/browse/irc-relay IMO [08:43:58] 10Deployment-Systems, 03Scap3: Update Debian Package for Scap3 - https://phabricator.wikimedia.org/T127762#2518201 (10fgiunchedi) >>! In T127762#2516893, @thcipriani wrote: > I fixed and bumped the minor version—new version on Beta seems to have fixed the issue. > > Could I get you to build/upload: `debian/3.... [08:46:47] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2518205 (10jayvdb) 05Open>03stalled Thankfully this appears to be on hold until the broken redirects are fixed. [08:46:50] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2518207 (10jayvdb) 05Open>03stalled Thankfully this appears to be on hold until the broken redirects are fixed. [08:46:53] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2518209 (10jayvdb) Thankfully this appears to be on hold until the broken redirects are fixed. [08:48:11] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2518213 (10jayvdb) Interesting; I only submitted one comment. I see it appearing three times. [08:50:58] 05Gitblit-Deprecate, 06Release-Engineering-Team, 06Operations: Clones from git.wikimedia.org are not redirected - https://phabricator.wikimedia.org/T139206#2518214 (10Paladox) This will require us to generate another list to go on due to the difference in the names. [08:52:05] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518215 (10Paladox) We should just remove HEAD from URL, ie strip it when it goes to the URL. [08:53:08] 05Gitblit-Deprecate, 10Diffusion: Redirect git.wikimedia.org HEAD URLs to Diffusion - https://phabricator.wikimedia.org/T141965#2518221 (10Paladox) [08:53:11] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2518220 (10Paladox) [09:25:24] !log deploying https://gerrit.wikimedia.org/r/302665 [09:26:41] legoktm it seems the log bot broke [09:27:24] It seems that no bot is working in here but are in the other channels [09:29:39] i get this error [09:29:40] error: prefix=barjavel.freenode.net, server=barjavel.freenode.net, command=err_cannotsendtochan, rawCommand=404, commandType=error, args=[grrrit-wm, #wikimedia-releng, Cannot send to channel] [09:30:28] hm [09:31:11] I think it may be the same error in logs bot [09:32:48] no idea [09:33:13] oh [09:34:22] legoktm i think i know why [09:34:33] valhallasw`cloud suggested this channel may be muted [09:34:35] and yes it was [09:34:38] [09:50:13] [ChanServ] greg-g quieted *!*@208.80.155.255 [09:34:43] it was muted yesturday for the labs upgrade [09:34:48] did he ever unquiet? [09:34:53] Nope [09:34:57] but grrrit-wm should have a cloak anyways [09:35:18] It dosent seem [09:35:19] the bot [09:35:21] has a cloak [09:35:22] ~lolrrit@208.80.155.255 [09:35:25] legoktm: I think the authentication stuff was lost in the move to k8s [09:35:39] ah [09:35:46] it always was a bit hacky (cherry-pick some commit) [09:35:54] well that explains everything [09:36:01] yep [09:39:20] I am running http://meta.wikimedia.org/wiki/WM-Bot version wikimedia bot v. 2.8.0.0 [libirc v. 1.0.3] my source code is licensed under GPL and located at https://github.com/benapetr/wikimedia-bot I will be very happy if you fix my bugs or implement new features [09:39:20] @help [09:42:20] valhallasw`cloud is there a way we can give the bot a cloak please? [09:42:35] paladox: the docs describe how to cherry-pick the auth info [09:42:43] Ok [09:42:54] This one https://wikitech.wikimedia.org/wiki/Grrrit-wm [09:43:17] valhallasw`cloud ^^ [09:44:18] paladox: https://wikitech.wikimedia.org/w/index.php?title=Grrrit-wm&oldid=134594 [09:44:24] Thanks [09:50:14] valhallasw`cloud i carnt find anything for cherry-pick the auth info [09:50:15] there [09:50:16] ? [09:50:26] local-lolrrit-wm@tools-login:~$ git cherry-pick auth [09:50:35] ah sorry [09:50:37] I found it [09:53:28] Yay it works [09:53:34] legoktm ^^ [09:53:42] woohoo [09:53:51] * legoktm hands paladox a shiny barnstar [09:53:52] valhallasw`cloud should i submit a patch that merges it into master [09:53:56] :) [09:53:58] no [09:54:01] Ok [09:54:01] of course not [09:54:05] ok [09:54:06] because then the password becomes public [09:54:09] oh [09:54:11] yeh sorry [09:54:14] forgot about that [09:54:26] thank you for fixing the cloak :-) [09:54:32] Your welcome [09:54:44] valhallasw`cloud is there a way i can symblink the file [09:54:55] so that we doint undue it [09:55:05] when we pull in master [09:55:13] paladox: iirc the problem is that the auth config is in the same file as other configuration [09:55:20] that should really be split out [09:55:25] Oh ah yep [09:57:03] valhallasw`cloud we could do password: in a patch but leave it empty then, get it to require a file that has the password [09:57:11] As not to expose [09:57:13] the password [09:57:23] and to prevent it from loosing the cloak [09:58:42] It seems it still dosent work [09:59:19] error: prefix=verne.freenode.net, server=verne.freenode.net, command=err_cannotsendtochan, rawCommand=404, commandType=error, args=[grrrit-wm, #wikimedia-releng, Cannot send to channel] [09:59:54] valhallasw`cloud ^^ [10:00:03] hey, anyone would be able to help me to decrypt a browser test failure? https://integration.wikimedia.org/ci/job/selenium-CirrusSearch/105/ [10:00:14] I'm not sure to understand what's wrong here [10:00:18] zeljkof ^^ [10:00:21] paladox: my guess still stands [10:00:30] Oh [10:00:35] paladox: and possibly ip +q's also apply to cloaked users? [10:00:43] Oh [10:00:45] yeh maybe [10:02:34] paladox: sorry, on vacation [10:02:37] Oh ok [10:39:40] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, 07Wikimedia-Incident: Config beta ORES extension to use the beta ORES service - https://phabricator.wikimedia.org/T141825#2518502 (10Ladsgroup) >>! In T141825#2517811, @Legoktm wrote: > Well, I merged it.... [10:59:09] 10Continuous-Integration-Infrastructure (phase-out-gallium), 06Operations, 10hardware-requests: Allocate contint1001 to releng and allocate to a vlan - https://phabricator.wikimedia.org/T140257#2518566 (10mark) a:05mark>03None The allocation of contint1001 is fine. What's the current status on the discu... [11:19:06] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer browser tests - https://phabricator.wikimedia.org/T129483#2107040 (10phuedx) I'm seeing this same error on the MobileFrontend b... [11:19:36] 07Browser-Tests, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity, 07Regression, 07Unplanned-Sprint-Work: [Regression] Fix browser tests for language switching on the beta cluster - https://phabricator.wikimedia.org/T141647#2518627 (10phuedx) All scenarios are failing with the error descr... [13:17:20] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, 07Wikimedia-Incident: [Spike] Should we make a model for ores in beta? - https://phabricator.wikimedia.org/T141980#2518777 (10Ladsgroup) [13:39:45] ostriches sorry about https://gerrit.wikimedia.org/r/#/c/302601/ but i would have created a seperate patch but im testing https://phabricator.wikimedia.org/T141329 [13:40:38] Maybe use your test install for that instead of polluting production with needless edits. [13:41:13] Oh, but i doint have the bot on the test install [13:42:05] In that case, do like I originally said and make a specific testing change. [13:42:10] *sigh* [13:42:16] ostriches could you take the quiet of the bot [13:42:18] please [13:42:24] * ostriches goes back to bed [13:42:28] [09:50:13] [ChanServ] greg-g quieted *!*@208.80.155.255 [13:42:35] ok [15:06:24] ostriches does https://gerrit-review.googlesource.com/#/c/80830/1 that look like it will fix it for us? [15:09:39] i hope that does fix our problem [15:09:48] they submitted that for https://bugs.chromium.org/p/gerrit/issues/detail?id=4324&can=2&start=0&num=100&q=&colspec=ID%20Type%20Stars%20Milestone%20Status%20Priority%20Owner%20Summary&groupby=&sort= [15:19:32] blerg. code updates for beta have been stuck a while :(( [15:21:42] thcipriani hi could you unquiet the grrrit-wm bot please [15:21:48] [09:50:13] [ChanServ] greg-g quieted *!*@208.80.155.255 [15:22:03] That was quiet yesturday due to the labs upgrade [15:27:15] Thanks [15:27:46] (03CR) 10Paladox: "recheck" [integration/zuul] (debian/trusty-wikimedia) - 10https://gerrit.wikimedia.org/r/301805 (owner: 10Paladox) [15:57:26] Yay /me built a gerrit.war for the first time [15:57:57] PROBLEM - Puppet run on integration-slave-trusty-1013 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:58:40] 10Deployment-Systems, 06Discovery, 07Regression: mwgrep getting HTTP 500 from elastic - https://phabricator.wikimedia.org/T141996#2519347 (10Krinkle) [16:16:49] ostriches tested and it works [16:16:53] with that patch [16:17:05] havent tested with irc but stream-events seems to show the correct user [16:17:08] and also the owner [16:22:58] ostriches hi im wondering could you when you build the war, could you go with gerrit 2.12.4 which hasent been released yet, or go with gerrit 2.12.3 with this https://gerrit-review.googlesource.com/#/c/80830/ backported please [16:22:59] ? [16:23:11] it deffintly fixes https://phabricator.wikimedia.org/T141329 [16:24:33] what does "build the war" and "could you go" mean? [16:25:59] It means that when he builds the war, meaning ie building gerrit per https://gerrit-review.googlesource.com/Documentation/dev-buck.html [16:26:43] could he backport https://gerrit-review.googlesource.com/#/c/80830/ before he builds it please? it's a small fix and dosent break anything, it fixes the wrong author thing in gerrit when using inline editing [16:27:12] https://gerrit-review.googlesource.com/Documentation/dev-buck.html#_gerrit_development_war_file [16:28:57] paladox: if it fixes https://phabricator.wikimedia.org/T141329 why didn't you just add a comment to that task proposing to backport? [16:29:09] Oh sorry [16:29:13] i should have done that [16:29:17] sorry [16:29:41] andre__ should i asked it here https://phabricator.wikimedia.org/T141245 or https://phabricator.wikimedia.org/T141329 here [16:29:44] please? [16:30:15] as Chad's manager, I would like him to be able to work on the things that he needs to work on. I'd like to ask you, paladox, to refrain from pinging him (or other WMF Release Engineering people) as often as you do. Thanks for your understanding. [16:30:37] Ok sorry [16:30:53] no worries, I know you are really tryign to help, and I appreciate that [16:30:56] really I do [16:31:06] ok [16:31:07] it's just the frequency of pings is a bit high :) [16:31:12] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, 07Wikimedia-Incident: [Spike] Should we make a model for ores in beta? - https://phabricator.wikimedia.org/T141980#2519511 (10Halfak) Right now, the testwiki model is a model for all intents and purposes.... [16:31:18] Yep, sorry about the high pings [16:31:27] paladox: Pick one of them? :P [16:31:48] Ok, i will do it in the task about the upgrade, since it is better in there. [16:32:10] paladox: Pinging people on IRC requires the people to be around **at the very same time**. It's synchronous communication. Like a phone call. [16:32:20] Oh, sorry. [16:32:48] paladox: Replying on Gerrit or on a Phab task (and summarizing your thoughts in one comment, instead of three comments within 5 minutes) allows others to get to it when they have time. It's asynchronous communication. :) [16:32:56] Yep [16:33:01] And it's all about finding a good balance. :) [16:34:03] Yep [16:34:08] paladox: In the worst case, people will start ignoring your pings if they are too much. Or your comments on Phab or rebases on Gerrit if they are too much. And that is probably nothing you want. :) [16:34:18] Yep [16:34:19] and comparing to other users, you are... let's say... "very active". :P [16:34:27] Oh yep [16:34:48] so I also kindly ask you to consider summarizing thoughts, and pinging less. Thanks for your help! :) [16:35:05] Ok, yep i will do. [16:35:09] thx :) [16:35:15] Your welcome :) [16:36:27] cool [16:38:22] go team :) [16:55:16] (03CR) 10Niedzielski: "@Thcipriani, would it be possible to get a +2? :]" [integration/config] - 10https://gerrit.wikimedia.org/r/301370 (https://phabricator.wikimedia.org/T141440) (owner: 10Mholloway) [16:59:05] We are seeing lots of browser test failures. Does anyone know if anything has broken on the beta cluster or changed in CI ? [16:59:56] greg-g: ^ [17:00:12] do the browser tests go to http: ? [17:00:15] or https: ? [17:00:24] if they go to http:, can they handle the redirect? [17:00:39] also beta updates were stalled overnight, but should be up-to-date now. [17:01:29] jdlrobson: will need more info/investigation. nothing comes to mind about changes/breakages other than yesterday's labs outage/upgrade and the issue thcipriani mentioned just now [17:01:40] okay thanks greg-g i'll keep an eye on it [17:02:00] greg-g someone said this morning that they would like help unecrypting it. [17:02:13] hey, anyone would be able to help me to decrypt a browser test failure? https://integration.wikimedia.org/ci/job/selenium-CirrusSearch/105/ [17:02:13] I'm not sure to understand what's wrong here [17:02:35] paladox: "decrypt" means "to understand" in this contenxt [17:02:38] -n [17:02:41] Oh [17:03:11] greg-g: 52 failures on recent build - basically all of them - but tried again https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/104/ [17:03:38] You seem to be htting the error dcausse had [17:03:55] what error, exactly? [17:05:16] hi, it looks like James_F, I and the Collaboration team just collectively broke CI for all MediaWiki extensions. we're working to unbreak it [17:05:19] I carnt see any errors [17:05:22] except from [17:05:22] ha:AAABBB+LCAAAAAAAAP9djrFOwzAQhk0QPAAjYmGgU6gRAwvyUCAVEU6CkkBHZDenxMW1I/tctWNfgdeAhYfgcXgHInWi/3B3w6fv/q9fcuQdOe9C460Zz63xVsM4sw3ox00PTivznlsEsstBRMjakbM9/h96wz6/2Uf/E5EoJccaTIsdJ4fBaSQnfCFWgmphWlqhU6a9HXSne7r73d7/2w8HR7KlCyuph8GswvIys1JpmDprEExD78piViUlm3fOLiHOkod0Mkuf0rckf03LIs+SvGYSUMTPfFJPizJjXJmwjrWQoNlQAZVBLqSvtFjBxfXVaDSMFxkMhtoFjxv6B+bUEUc2AQAAselenium-MobileFrontend » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty [17:05:22] FAILURE [17:05:51] MatmaRex: how? Is there a bug open I can follow? [17:06:05] (03PS4) 10Thcipriani: Make apps-android-wikipedia-lint voting [integration/config] - 10https://gerrit.wikimedia.org/r/301370 (https://phabricator.wikimedia.org/T141440) (owner: 10Mholloway) [17:06:23] jdlrobson: it looks like they somehow managed to break the build on Echo's master [17:06:37] this caused https://gerrit.wikimedia.org/r/#/c/302736/ to fail the checks too [17:07:26] and because https://gerrit.wikimedia.org/r/302735 was merged, the OOjs UI version in composer.json is now not matching the one in mediawiki/vendor, and that breaks everything [17:08:07] (03CR) 10Thcipriani: [C: 032] Make apps-android-wikipedia-lint voting [integration/config] - 10https://gerrit.wikimedia.org/r/301370 (https://phabricator.wikimedia.org/T141440) (owner: 10Mholloway) [17:09:29] (03Merged) 10jenkins-bot: Make apps-android-wikipedia-lint voting [integration/config] - 10https://gerrit.wikimedia.org/r/301370 (https://phabricator.wikimedia.org/T141440) (owner: 10Mholloway) [17:09:45] MatmaRex: icanhaz phab ticket # ? [17:10:15] there isn't one [17:10:33] hmm. i just tried merging https://gerrit.wikimedia.org/r/#/c/302736/ again, and it went through this time. so the issue should be resolved [17:10:42] now i'm wondering why it failed before [17:10:47] jdlrobson: [17:11:17] please create so we can have this convo in a single place (my team has already spent lots of time investigating it). [17:11:44] I think ya'll are talking about different hings [17:12:03] MatmaRex: is talking about unit tests, jdlrobson is talking browser tests, no? [17:12:16] im talking abut browser tests yes. [17:12:24] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/301370 [17:13:18] that sounds like a different issue [17:14:32] It last past here https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/99/console [17:14:41] which makes me think it was the labs upgrade? [17:14:48] Since it was also around the same time [17:17:41] jdlrobson https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/104/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/consoleFull [17:19:06] and [17:19:06] https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/104/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/testReport/ [17:23:02] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10DBA, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2519772 (10dduvall) We've reached our vcpu and memory quota for deployment-prep so I'm una... [17:30:50] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10DBA, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2519803 (10greg) [17:37:35] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer browser tests - https://phabricator.wikimedia.org/T129483#2107040 (10Tgr) CentralAuth as well. [17:45:13] PROBLEM - Puppet run on deployment-db01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [18:09:01] 10Beta-Cluster-Infrastructure, 06Commons, 10MediaWiki-Authentication-and-authorization, 10MediaWiki-extensions-CentralAuth, 10MobileFrontend: Unable to log in on https://commons.m.wikimedia.beta.wmflabs.org/wiki/Special:UserLogin - https://phabricator.wikimedia.org/T142015#2519949 (10matmarex) [18:46:04] marxarelli, hey [18:46:06] re https://phabricator.wikimedia.org/T138778 [18:46:13] maybe one can be done at a time? [18:56:10] What did this plugin https://gerrit.wikimedia.org/r/#/c/192613/1/its-phabricator-from-bugzilla.jar,unified meant to do [18:56:11] ? [18:57:31] paladox: https://github.com/wikimedia/operations-gerrit-plugins [18:57:46] Oh thanks [18:59:26] this may or may not answer the question [18:59:47] but i think linking to bugzilla from gerrit [18:59:56] when people used Bug: [18:59:59] (03PS1) 10Dduvall: Support automatic redirects [ruby/api] - 10https://gerrit.wikimedia.org/r/302766 (https://phabricator.wikimedia.org/T1029483) [19:00:06] qchris made it [19:01:04] (03PS2) 10Dduvall: Support automatic redirects [ruby/api] - 10https://gerrit.wikimedia.org/r/302766 (https://phabricator.wikimedia.org/T129483) [19:01:35] Oh [19:01:36] ah [19:01:51] I was wondering since im tracing the gerritbot not adding projects problem [19:02:36] yea, also search phab for that, it's mentioned in some old bugs [19:03:01] Oh [19:03:24] (03CR) 10Thcipriani: [C: 032] Support automatic redirects [ruby/api] - 10https://gerrit.wikimedia.org/r/302766 (https://phabricator.wikimedia.org/T129483) (owner: 10Dduvall) [19:06:33] Krenair: hmm. how would we best do that? [19:06:46] set up the new instance as a replica of the existing master? [19:06:50] (03Merged) 10jenkins-bot: Support automatic redirects [ruby/api] - 10https://gerrit.wikimedia.org/r/302766 (https://phabricator.wikimedia.org/T129483) (owner: 10Dduvall) [19:12:28] Krenair: my thought was to provision both new databases from the same dump of the current master and do the mediawiki-config changes all at once, but i'm open to any other ideas really [19:12:33] marxarelli, I'm not completely sure, I was just wondering if there was a way around the quota issue :/ [19:13:16] i mean, we could. it would just make for more work :) [19:18:17] 1) dump current master 2) set up new instance as a slave of the current precise master 3) edit/deploy mediawiki-config to replace db2 4) terminate db2 instance 5) set up new instance to replace db1 6) dump current master again and restore on new db1 7) reconfigure new db2 to read from new db1 8) edit/deploy mediawiki-config to use new db1 9) terminate old db1 [19:18:19] James_F: https://gerrit.wikimedia.org/r/#/c/302638/ is still a draft? can you publish it? [19:18:33] lot-o-steps [19:18:34] (03Draft3) 10Legoktm: make-wmf-branch: Stop branching ImageMetrics (undeployed) [tools/release] - 10https://gerrit.wikimedia.org/r/302638 (owner: 10Jforrester) [19:18:38] Huh. [19:18:42] Krenair: ^ [19:18:57] (03CR) 10Legoktm: [C: 032] make-wmf-branch: Stop branching ImageMetrics (undeployed) [tools/release] - 10https://gerrit.wikimedia.org/r/302638 (owner: 10Jforrester) [19:19:33] provisioning both new instances at the same time i think would only require one dump of master and one change to mediawiki-config [19:19:39] marxarelli, yeah maybe easiest to just get the quota raised. long term though, I think we should aim to cut down on the resources used by deployment-prep [19:19:40] less overall downtime as well [19:20:00] (03Merged) 10jenkins-bot: make-wmf-branch: Stop branching ImageMetrics (undeployed) [tools/release] - 10https://gerrit.wikimedia.org/r/302638 (owner: 10Jforrester) [19:20:28] https://wikitech.wikimedia.org/wiki/Purge_2016#In_use_deployment-prep [19:21:57] marxarelli, is deployment-db01 yours? [19:22:12] Krenair: oh neat. i'll check in with releng peeps about those "delete" ones [19:22:17] it is, yeah [19:22:20] why not 03? [19:22:39] haha, i don't know honestly. because it's going to replace db1 [19:22:55] and i thought having just 03 and 04 in the end would be confusing [19:23:18] the 01 name is confusing but hopefully only in the interim [19:25:50] Krenair: heading out for a bit but i'll definitely checkin to see if we can terminate some of those instances. thanks for pointing that out [19:38:15] 07Browser-Tests, 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! - https://phabricator.wikimedia.org/T142033#2520344 (10Jdlrobson) [19:40:06] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 07Epic, 05MW-1.28-release-notes, and 3 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2520359 (10thcipriani) [19:40:09] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer browser tests - https://phabricator.wikimedia.org/T129483#2520358 (10thcipriani) 05Resolved>03Open [19:40:32] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer browser tests - https://phabricator.wikimedia.org/T129483#2107040 (10thcipriani) [19:40:34] 07Browser-Tests, 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! - https://phabricator.wikimedia.org/T142033#2520364 (10thcipriani) [19:43:00] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend and Wikibase browser tests - https://phabricator.wikimedia.org/T129483#2520371 (10Jdlrobson) [19:44:57] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend... - https://phabricator.wikimedia.org/T129483#2107040 [19:45:00] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend... - https://phabricator.wikimedia.org/T129483#2520374 [19:45:54] 07Browser-Tests, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity, 07Regression, 07Unplanned-Sprint-Work: [Regression] Fix browser tests for language switching on the beta cluster - https://phabricator.wikimedia.org/T141647#2520376 (10Jdlrobson) See T129483. This is not only a problem wit... [19:46:23] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend... - https://phabricator.wikimedia.org/T129483#2107040 [19:48:48] jdlrobson: marx...dan and I paired on this a little bit, discovered that the faraday gem doesn't support redirects, dan made a patch for the mediawiki_api gem, but hasn't gotten it updated yet afaik. In the interim, could you update: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/master/tests/browser/environments.yml#L31 to https? [19:48:51] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, Mo... - https://phabricator.wikimedia.org/T129483#2520384 [19:49:04] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, Mo... - https://phabricator.wikimedia.org/T129483#2107040 [19:55:27] I get error [19:55:28] [2016-08-03 01:32:28,589] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon [19:55:28] java.lang.NoClassDefFoundError: org/bouncycastle/openssl/PEMParser [19:55:47] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, Mo... - https://phabricator.wikimedia.org/T129483#2520398 [19:59:42] (03PS1) 10Jdlrobson: API should use https on the beta cluster [integration/config] - 10https://gerrit.wikimedia.org/r/302775 (https://phabricator.wikimedia.org/T129483) [20:00:25] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer, 06Reading-Web-Backlog, 03Reading-Web-Sprint-78-Terminal-Velocity: A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, Mo... - https://phabricator.wikimedia.org/T129483#2520418 [20:23:17] I now get [20:23:18] [2016-08-03 20:21:45,973] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon [20:23:18] com.google.inject.ProvisionException: Unable to provision, see the following errors: [20:23:18] 1) No index versions ready; run Reindex [20:25:16] RoanKattouw upstream have finally fixed https://phabricator.wikimedia.org/T141329 [20:25:34] yay [20:25:44] I tested the fix [20:25:57] and stream-events now shows us the correct author and the owner [20:26:15] But we wont be getting the fix until they merge it and roll out 2.12.4 [20:29:59] ostriches it seems that https://gerrit.wikimedia.org/r/#/c/302497/2/modules/gerrit/manifests/jetty.pp is not running automatically or dosent run at all [20:30:02] sorry for ping [20:30:13] I get error [20:30:13] [2016-08-03 20:21:45,906] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin singleusergroup, version v2.12.2 [20:30:14] [2016-08-03 20:21:45,973] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon [20:30:14] com.google.inject.ProvisionException: Unable to provision, see the following errors: [20:30:14] 1) No index versions ready; run Reindex [20:35:13] Then run reindex. [20:35:24] Ok [20:35:26] But isent [20:35:30] https://gerrit.wikimedia.org/r/#/c/302497/1/modules/gerrit/manifests/jetty.pp [20:35:33] that suppose to do it [20:35:34] ? [20:35:42] we just merged the change to run it [20:35:49] that's why i said it should have happened automatically now [20:35:54] yep [20:35:59] does the directory exist, paladox? [20:36:04] Yep [20:36:05] the one listed in the "creates" line [20:36:06] It exist [20:36:12] well then that's why it doesnt run [20:36:23] try deleting it and running puppet again [20:36:25] Oh, but it is gerrit package before running that [20:36:25] i guess [20:36:45] ? [20:37:27] The package shouldn't great that directory. [20:37:30] *create [20:37:35] But then wont it fail [20:37:39] because of the missing [20:37:42] plugin [20:38:25] you gotta stick to one issue at a time. does it still tell you to run re-index? [20:38:30] What plugin missing? [20:38:38] bouncycastle i think [20:38:51] Yep [20:38:58] the patch you uploaded in gerrit fixed it [20:39:04] Oh, the second piece of bouncycastle. [20:39:04] I have a patch for that :) [20:39:13] Yep [20:39:15] Found it [20:39:21] I downloaded the plugin manually [20:40:15] I thought the point of this exercise was not doing anything manually on that host? [20:40:46] paladox: does the plugin have anything to do with /var/lib/gerrit2/review_site/index ? no, right [20:40:55] that's the index directory [20:41:05] Yep [20:41:07] Plugin has nothing to do with that directory no. [20:41:09] No [20:41:20] so, does it still tell you to reindex? [20:41:28] then remove that directory and run puppet again [20:41:29] greg-g: That's the point. It's hard though, since gerrit doesn't like being set up automatically ;-) [20:41:46] phabricator froze internet explorer i carnt access it lol [20:41:48] greg-g: yes, what both of you said :p [20:41:54] trying hard :p [20:42:40] Were making progress though [20:42:46] paladox generally pastes manual steps on the ticket, right [20:42:56] Yep [20:43:15] progress confirmed :) [20:43:43] I did what was asked [20:43:46] and removed the files [20:43:49] and re did puppet [20:43:56] it seems to not have recreated them now [20:44:10] ie i have etc and static folder and thats it [20:44:13] we were expecting it to run the re-index command now [20:44:19] Info: /Stage[main]/Gerrit::Jetty/File[/var/lib/gerrit2/review_site/etc/secure.config]: Scheduling refresh of Service[gerrit] [20:44:20] Notice: /Stage[main]/Gerrit::Jetty/Exec[install_gerrit_jetty]/returns: Error: Unable to access jarfile gerrit.war [20:44:20] Error: /usr/bin/java -jar gerrit.war init -d review_site --batch --no-auto-start returned 1 instead of one of [0] [20:44:20] Error: /Stage[main]/Gerrit::Jetty/Exec[install_gerrit_jetty]/returns: change from notrun to 0 failed: /usr/bin/java -jar gerrit.war init -d review_site --batch --no-auto-start returned 1 instead of one of [0] [20:44:41] It seems to want to install gerrit jetty before actually installing gerrit first? [20:44:47] did you delete more than just the index dir? [20:44:54] Yeh [20:44:58] ugh, why [20:45:06] not sure [20:45:10] i wanted to make sure it worked [20:45:16] and it obvously didnt work [20:45:26] deleting unrelated things is normally not a way to make it work [20:45:46] you gotta get the .war file back and stuff [20:46:18] just let puppet redo all the things [20:46:22] from scratch [20:46:23] Thats why [20:46:28] i didnt remove the deb [20:47:11] start over, it shouldnt take long, go pass issues you had last time [20:47:13] Yep it installed the deb through puppet [20:47:13] now [20:47:16] Well removing other unrelated stuff probably has puppet all confused now. [20:47:26] But it seemes [20:47:32] to not do the reindex thing [20:47:36] or init the dir [20:47:46] so now i doint have bin/gerrit.war or bin/gerrit.sh [20:47:54] should i manually run the script again? [20:48:01] Because you deleted those when you shouldn't have.... [20:48:12] Yep, but shoulden they be recreated [20:48:16] when you run puppet [20:48:46] When you run the init step. [20:48:46] But if you deleted the gerrit.war too, it can't do that. [20:48:48] It should run [20:48:49] exec { 'install_gerrit_jetty' [20:48:50] paladox: delete instance, re-create instance, apply puppet role, run puppet again .. is the easiest way to find out and have a clean setup again [20:48:54] IOk [20:48:55] Ok [20:48:59] i will do that now [20:49:06] then you will see what the next issue is [20:49:17] Yes, listen to mutante. That's the best way to do it, when you're trying to get it working 100% with puppet. [20:49:44] Don't work around puppet. [20:50:03] Ok [20:53:45] Ok the instance is recreated [20:58:49] chad could you add install mysql to the step please? [21:02:26] No, I'm not adding mysql to the role. [21:03:29] Ok [21:07:49] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, and 2 others: [Spike] Should we make a model for ores in beta? - https://phabricator.wikimedia.org/T141980#2520650 (10Danny_B) [21:10:16] paladox: wait a moment, labs puppetmaster is being upgraded [21:10:29] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 10ORES, 06Revision-Scoring-As-A-Service, 07Spike: [Spike] Should we make a model for ores in beta? - https://phabricator.wikimedia.org/T141980#2520653 (10greg) [21:10:50] Ok [21:13:25] 05Gitblit-Deprecate, 06Project-Admins: Archive #Gitblit-Deprecate - https://phabricator.wikimedia.org/T138986#2520663 (10Danny_B) 05stalled>03Open No need to set stalled status to tasks which depend on subtasks which are open. [21:18:45] Project selenium-Wikidata » firefox,test,Linux,contintLabsSlave && UbuntuTrusty build #75: 04FAILURE in 2 hr 28 min: https://integration.wikimedia.org/ci/job/selenium-Wikidata/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/75/ [21:31:47] PROBLEM - Puppet run on deployment-aqs01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:24] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:38] PROBLEM - Puppet run on integration-slave-precise-1012 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:42] PROBLEM - Puppet run on integration-slave-precise-1011 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:59:37] Hi running just the reindex command seems to fail [21:59:51] because of the missing two git folders in review_site/git/ [22:04:07] it seems the init command needs to be run before the reindex command [22:15:08] yep [22:15:18] I have a patch here https://gerrit.wikimedia.org/r/#/c/302840/ [22:19:52] That's what it should do already from my patches, reindex requires init. [22:19:58] That patch reverses it to do the opposite. [22:19:59] Which is wrong [22:20:05] Oh [22:20:08] But init [22:20:09] is not run [22:20:31] it hasent created the AllProject.git folder in review_site/git/ [22:20:36] which fails on reindexing [22:20:43] because it is missing? [22:22:35] How do you know it's not run? Did it fail when it ran? Did you try looking into why it didn't run? [22:22:42] Yep [22:22:46] Well it failed [22:22:47] reindexing [22:22:57] Notice: /Stage[main]/Gerrit::Jetty/Exec[reindex_gerrit_jetty]/returns: [2016-08-03 21:36:28,472] [main] INFO com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /var/lib/gerrit2/review_site/cache [22:22:57] Notice: /Stage[main]/Gerrit::Jetty/Exec[reindex_gerrit_jetty]/returns: [2016-08-03 21:36:29,195] [main] INFO com.google.gerrit.server.project.ProjectCacheWarmer : Loading project cache [22:22:57] Notice: /Stage[main]/Gerrit::Jetty/Exec[reindex_gerrit_jetty]/returns: fatal: fetch failure on changes [22:22:57] Error: /usr/bin/java -jar gerrit.war reindex -d review_site --threads 4 returned 128 instead of one of [0] [22:22:59] Error: /Stage[main]/Gerrit::Jetty/Exec[reindex_gerrit_jetty]/returns: change from notrun to 0 failed: /usr/bin/java -jar gerrit.war reindex -d review_site --threads 4 returned 128 instead of one of [0] [22:23:04] That's not useful. [22:23:08] You should find out *why* it failed [22:23:17] It says something about fatal: fetch failure on changes [22:23:36] Something. What something? A full error message would be useful. [22:23:44] Let me check [22:23:57] Oh there [22:24:00] are no error_log [22:24:04] in logs dir in gerrit [22:24:11] Then maybe you should try running it by hand and see why :) [22:24:27] Ok [22:24:30] i did run it byt [22:24:31] cd /var/lib/gerrit2/; /usr/bin/java -jar gerrit.war reindex -d review_site --threads 4 [22:24:31] by [22:24:32] hand [22:24:41] And then that should yell at you if it breaks :) [22:25:14] oh [22:25:21] it says [22:25:22] root@gerrit-test3:/var/lib/gerrit2# /usr/bin/java -jar gerrit.war reindex -d review_site --threads 4 [22:25:22] [2016-08-03 22:25:09,788] [main] INFO com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /var/lib/gerrit2/review_site/cache [22:25:22] [2016-08-03 22:25:10,553] [main] INFO com.google.gerrit.server.project.ProjectCacheWarmer : Loading project cache [22:25:23] fatal: fetch failure on changes [22:25:25] root@gerrit-test3:/var/lib/gerrit2# [22:25:42] Well then it sounds like you never got init running properly either. Take a step back and figure that out. [22:25:46] Yep [22:25:59] I ran puppet [22:26:03] You should abandon that puppet change though. It does the *opposite* of what we want. [22:26:26] Yep [22:26:27] I did [22:26:55] Anyway, I gotta run. I'll look later. [22:27:02] Ok [22:42:43] 06Release-Engineering-Team, 15User-greg, 15User-zeljkofilipin: Add a European mid-day SWAT window - https://phabricator.wikimedia.org/T137970#2520968 (10greg) @Addshore I hear your morning SWAT activities are going well :) Can I add you to the list of names for the inaugural mid-day European SwAT members lis... [22:49:56] 06Release-Engineering-Team, 15User-greg, 15User-zeljkofilipin: Add a European mid-day SWAT window - https://phabricator.wikimedia.org/T137970#2521010 (10Addshore) @greg certainly! [22:50:08] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 03Fundraising Sprint Nitpicking, and 2 others: Continuous integration: DonationInterface needs composer variant - https://phabricator.wikimedia.org/T141309#2521016 (10DStrine) [23:16:58] (03PS1) 10Dduvall: Release minor version 0.7.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/302846 [23:28:06] (03CR) 10Thcipriani: [C: 032] Release minor version 0.7.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/302846 (owner: 10Dduvall) [23:29:18] (03Merged) 10jenkins-bot: Release minor version 0.7.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/302846 (owner: 10Dduvall) [23:31:40] Yay progress [23:31:53] Check out the new list https://phabricator.wikimedia.org/P3637 [23:31:56] im updating [23:33:38] (03PS1) 10Dduvall: Bump mediawiki_api dependency for redirect support [selenium] - 10https://gerrit.wikimedia.org/r/302849 [23:33:40] (03PS1) 10Dduvall: Release patch version 1.7.2 [selenium] - 10https://gerrit.wikimedia.org/r/302850 [23:35:17] 10Beta-Cluster-Infrastructure: deployment-sentry2 /var is full - https://phabricator.wikimedia.org/T132730#2208581 (10greg) >>! In T132730#2209331, @Tgr wrote: >>>! In T132730#2209158, @hashar wrote: >> Probably @tgr as part of {T106920} ? > > No, that was deployment-sentry01. Can sentry2 be deleted then? We'r... [23:35:36] (03CR) 10jenkins-bot: [V: 04-1] Bump mediawiki_api dependency for redirect support [selenium] - 10https://gerrit.wikimedia.org/r/302849 (owner: 10Dduvall) [23:35:40] (03CR) 10jenkins-bot: [V: 04-1] Release patch version 1.7.2 [selenium] - 10https://gerrit.wikimedia.org/r/302850 (owner: 10Dduvall) [23:38:04] paladox: are you using phab-beta.deployment-prep.eqiad.wmflabs ? [23:38:20] (03CR) 10Dduvall: "retest" [selenium] - 10https://gerrit.wikimedia.org/r/302849 (owner: 10Dduvall) [23:38:26] Nope [23:38:32] I wonder who is... [23:38:39] Oh [23:39:07] looks like mukunda: https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Phab-beta.deployment-prep.eqiad.wmflabs&action=history [23:39:21] just going through this list: https://wikitech.wikimedia.org/wiki/Purge_2016#In_use_deployment-prep [23:39:28] (03CR) 10Dduvall: "recheck" [selenium] - 10https://gerrit.wikimedia.org/r/302849 (owner: 10Dduvall) [23:39:47] Oh