[02:40:35] (03PS1) 10Legoktm: Add tox job for labs/tools/gblrenamemon [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150757 [02:41:57] (03PS1) 10Legoktm: Add tox job (flake8) for labs/tools/gblrenamemon [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150758 [06:36:39] (03PS1) 10Pastakhov: fix dependencies for mw/ext/PhpTagsWiki [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150768 [07:35:33] (03CR) 10Hashar: [C: 032] "Excellent!" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150758 (owner: 10Legoktm) [07:36:04] (03CR) 10Hashar: [C: 032] Add tox job for labs/tools/gblrenamemon [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150757 (owner: 10Legoktm) [07:36:20] (03Merged) 10jenkins-bot: Add tox job for labs/tools/gblrenamemon [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150757 (owner: 10Legoktm) [07:36:55] (03Merged) 10jenkins-bot: Add tox job (flake8) for labs/tools/gblrenamemon [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150758 (owner: 10Legoktm) [07:59:35] (03PS1) 10Zfilipin: Minor readme updates [selenium] - 10https://gerrit.wikimedia.org/r/150776 [08:20:09] (03PS2) 10Hashar: fix dependencies for mw/ext/PhpTagsWiki [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150768 (owner: 10Pastakhov) [08:21:42] (03CR) 10Hashar: [C: 032] "Job updated." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150768 (owner: 10Pastakhov) [08:23:11] (03Merged) 10jenkins-bot: fix dependencies for mw/ext/PhpTagsWiki [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150768 (owner: 10Pastakhov) [08:45:06] (03CR) 10Zfilipin: [C: 04-1] "Merge conflict + inline comments." (032 comments) [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (owner: 10Phuedx) [09:13:41] (03PS2) 10Phuedx: Enable browser tests for the GettingStarted extension [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 [09:15:02] (03CR) 10Phuedx: Enable browser tests for the GettingStarted extension (032 comments) [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (owner: 10Phuedx) [09:41:05] (03CR) 10Zfilipin: "Sam, why did you give this commit -2?" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (owner: 10Phuedx) [09:43:11] (03CR) 10Phuedx: "I wanted to make sure I79d9468e56bc7941c7c5bb27a77d1622bc7d3836 was merged before this so that we actually had a runnable browser test in " [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (owner: 10Phuedx) [09:43:43] zeljkof: i can remove it if that's not an issue ^ [09:45:43] (03PS3) 10Zfilipin: WIP Enable browser tests for the GettingStarted extension [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (owner: 10Phuedx) [09:46:04] phuedx: this ^ is how we usually mark WIP (work in progress) commits [09:46:19] feel free to remove -2 from the commit [09:46:49] -2 usually means you do not think the commit can be merged in the current state, until it is significantly changed [09:47:10] hoom [09:47:17] i'm happy to follow convention [09:47:40] phuedx: that is how I understand the convention(s), I could be wrong :) [09:47:44] not sure that it's quite what i'm trying to say though [09:48:02] phuedx: what _are_ you trying to say? [09:48:02] it's blocked, i don't think it requires further changes [09:48:20] it has a dependency on another change to another repo [09:48:37] phuedx: yes, that is why -2 does not apply, it is blocked by another commit [09:50:35] yeah, so "don't submit" ;) [09:54:22] phuedx: similar commit https://gerrit.wikimedia.org/r/#/c/144978/ [09:55:12] zeljkof: did you see my last message? [09:55:21] (03PS4) 10Zfilipin: WIP Enable browser tests for the GettingStarted extension [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150172 (https://bugzilla.wikimedia.org/52246) (owner: 10Phuedx) [09:55:32] phuedx: here? probably not, my client got disconnected [09:55:35] i read -2 "don't submit" as exactly that: don't submit this change (until…) [09:55:49] yeah -2 is meant to prevent a change from being merged [09:56:04] Gerrit will just refuse to merge it [09:56:23] phuedx: feel free to leave -2 for the commit [09:58:24] well now it's in the commit message it seems redundant [09:58:31] also, the big red x makes me feel sad [10:06:10] phuedx: yes, big red x does look sad :) [10:35:28] hashar: Hello ! beta labs are down :( [10:35:56] bah [10:36:04] gotta get traces :D [10:37:43] !log deployment-prep Taking gdb traces of hhvm on mediawiki01 and mediawiki02. Restarting hhvm [10:41:29] vikasyaligar: should be back now [10:42:29] hashar: thank you it's up ! [11:18:05] Krinkle: [11:18:19] Krinkle: in case you are around. I am working a bit on the Trusty jenkins slave [11:18:25] marking all jobs with UbuntuPrecise label [11:21:05] 3Wikimedia / 3Continuous integration: Jenkins: label slaves with their ubuntu version - 10https://bugzilla.wikimedia.org/68340#c2 (10Antoine "hashar" Musso) https://integration.wikimedia.org/ci/label/UbuntuPrecise/ lanthanum gallium integration-slave1001 integration-slave1002 integration-slave1003 https:... [11:21:53] (03PS1) 10Hashar: Tie all jobs to UbuntuPrecise [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150785 (https://bugzilla.wikimedia.org/68340) [11:24:05] hashar: k [11:24:19] Krinkle: basically created the label UbuntuPrecise [11:24:23] and updating all jobs to require it [11:24:31] we can then migrate the npm jobs to UbuntuTrusty [11:24:39] I have polled in integration-slave1004-trusty [11:51:47] well [11:51:49] jjb is too slow [11:52:01] will just rewrite the XML files :D [11:55:38] aharoni: coming to the hangout in 5 minutes? [11:55:45] yes [11:55:54] vikasyaligar: will you join us? [11:55:59] aharoni: great, see you there [11:56:24] aharoni: yup will be there in 10 minutes :) [12:05:47] (03CR) 10Hashar: [C: 032] "Deployed." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150785 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [12:05:47] zeljkof: aharoni did you send me the invite ? [12:06:09] 3Wikimedia / 3Continuous integration: Jenkins: label slaves with their ubuntu version - 10https://bugzilla.wikimedia.org/68340#c4 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX a:3Antoine "hashar" Musso Deployed. I did it manually though by using shell commands such as: sed -ibak -e 's%has... [12:10:49] vikasyaligar: I invited you [12:11:58] aharoni: I did not receive the invite :( [12:12:32] (03Merged) 10jenkins-bot: Tie all jobs to UbuntuPrecise [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150785 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [12:13:12] aharoni: can you send it to me again ? [12:16:41] vikasyaligar: both aharoni and me have invited you [12:16:45] did you check spam? [12:17:37] zeljkof: nothing in the spam [12:20:38] aharoni: https://www.mediawiki.org/wiki/Manual:Coding_conventions/Ruby [12:21:34] vikasyaligar: aharoni will set up a hangout from his personal account, we are not sure why wikimedia one is not inviting you [12:23:59] zeljkof: you can join this => https://plus.google.com/hangouts/_/lite/g5xieb6uzoqvrvygsj42wrd2f4a?pqs=1&authuser=0&hl=en [12:24:39] vikasyaligar: aharoni: You're not allowed to join this video call. :) [12:28:01] vikasyaligar, aharoni: can you sent the invite to me? [12:29:16] zeljkof: Can you start a hangout and invite all of us ? [12:29:25] vikasyaligar: sure [12:30:02] zeljkof: my hangout says that your chat client doesn't support group calls [12:30:06] vikasyaligar, aharoni: done [12:30:23] aharoni: wow, I do not have a chat client :) [12:30:49] zeljkof: Can you put the link here ? [12:35:06] vikasyaligar, aharoni: http://commons.wikimedia.beta.wmflabs.org/wiki/Special:ListFiles/LanguageScreenshotBot [12:35:17] http://commons.wikimedia.beta.wmflabs.org/wiki/File:VisualEditor_More_Settings-de.png [12:41:00] aharoni, vikasyaligar: https://docs.saucelabs.com/reference/test-configuration/#specifying-the-screen-resolution [12:46:34] vikasyaligar: http://rdoc.info/gems/watir-webdriver/Watir/Element#flash-instance_method [12:46:49] zeljkof: aharoni: https://commons.wikimedia.org/wiki/Commons:Bots/Requests/LanguageScreenshotBot [13:20:38] (03PS1) 10Vikassy: Language screenshot job should run on test2.wikipedia.org [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150803 [13:23:06] 3Wikimedia / 3Continuous integration: Jenkins: Set up integration slaves with Ubuntu Trusty - 10https://bugzilla.wikimedia.org/68256#c2 (10Antoine "hashar" Musso) I have pooled integration-slave1004-trusty.eqiad.wmflabs with Jenkins labels: UbuntuTrusty contintLabsSlave hasPhpUnit hasPhpcs All Jenkins jobs... [13:23:15] (03PS2) 10Vikassy: Language screenshot job should run on test2.wikipedia.org [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150803 [13:24:21] (03PS1) 10Hashar: Tie all jobs to UbuntuPrecise [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150804 (https://bugzilla.wikimedia.org/68340) [13:26:36] (03CR) 10Hashar: [C: 032] "Zeljkof, I have added a Jenkins slaves that is using Ubuntu Trusty. The browser tests can probably not run on it yet :/" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150804 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [13:28:10] (03Merged) 10jenkins-bot: Tie all jobs to UbuntuPrecise [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150804 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [13:28:23] zeljkof_: here it is https://gerrit.wikimedia.org/r/#/c/150803/ [13:28:23] (03PS3) 10Zfilipin: Language screenshot job should run on test2.wikipedia.org [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150803 (owner: 10Vikassy) [13:28:35] vikasyaligar: looking [13:29:58] vikasyaligar: does LanguageScreenshotBot have the same password at test2? [13:31:02] (03PS1) 10Hashar: Tie language screenshot to UbuntuPrecise [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150807 (https://bugzilla.wikimedia.org/68340) [13:32:02] zeljkof: yes :) [13:32:59] (03CR) 10Hashar: [C: 032] Tie language screenshot to UbuntuPrecise [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150807 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [13:33:10] zeljkof: vikasyaligar: I have added a new Jenkins slave [13:33:14] which uses Ubuntu Trusty [13:33:24] (03Merged) 10jenkins-bot: Tie language screenshot to UbuntuPrecise [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150807 (https://bugzilla.wikimedia.org/68340) (owner: 10Hashar) [13:33:28] to prevent job from running on it I created two labels UbuntuPrecise and UbuntuTrusty [13:33:35] and made all jobs tied to UbuntuPrecise [13:33:44] browsertests-VisualEditor-language-screenshot-linux-firefox-sauce might be buggy though :-/ [13:33:56] hashar: why? [13:34:09] (it might be buggy) [13:34:31] no idea :-D [13:34:38] I am suspecting something will be wrong [13:34:46] anyway https://gerrit.wikimedia.org/r/#/c/150803/ should be rebased now [13:35:37] (03PS4) 10Vikassy: Language screenshot job should run on test2.wikipedia.org [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/150803 [13:36:02] hashar: rebased ! thank you :) [13:36:22] 3Wikimedia / 3Continuous integration: Jenkins: label slaves with their ubuntu version - 10https://bugzilla.wikimedia.org/68340#c10 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX Should be fine now :) [13:53:02] vikasyaligar: deployed test2 commit, applied no scroll commit, running the job https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/87/console [13:53:22] hashar: this looks strange https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/87/console [13:54:07] hashar: but looks like the child job is running https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=he,label=contintLabsSlave%20&&%20UbuntuPrecise/87/console [13:54:14] vikasyaligar: ^ [13:54:40] :-( [13:56:06] zeljkof: yup ! looking at it :) [14:00:25] zeljkof: I can still see scrollbars here => https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=he,label=contintLabsSlave%20&&%20UbuntuPrecise/ws/log/VisualEditor_Toolbar_SpecialCharacters-he.png [14:00:49] vikasyaligar: can you reproduce it on your machine? [14:01:15] zeljkof: let me check [14:02:44] vikasyaligar: is this the job? [14:02:45] https://saucelabs.com/tests/0a959a75d85d4234a909cf97ce3daf68 [14:03:32] vikasyaligar: you might trying to zoom out :) [14:04:13] zeljkof: ok Let me try that :) [14:06:51] vikasyaligar: or try running the test with the same screen size as it is at sauce [14:07:06] the first step is reproducing the problem [14:07:18] let me know if you do not know how to run tests from your machine on sauce [14:07:21] zeljkof: I tried running it on my machine and I too get that scrollbar [14:07:30] vikasyaligar: ok, progress [14:07:40] try making the browser window bigger [14:07:53] try it manually, do you still get the scroll bar? [14:07:59] try zooming out [14:09:17] checking .. [14:09:39] I am out for the rest of the day [14:13:21] zeljkof: manually also I see the scrollbar ! But when I zoom out it goes off :) [14:13:42] vikasyaligar: victory! :) [14:13:53] try zooming out for that scenario [14:14:05] zeljkof: Let me try it out in terms of code :) [14:14:10] yup [14:18:04] ah ! I have to change control to command now for my machine :) [14:18:04] (03PS1) 10Hashar: Build PHP extension with HHVM [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150820 (https://bugzilla.wikimedia.org/63120) [14:18:55] zeljkof: wooohoo ! no scroll bar when zoom out [I had hardcoded the zoom out] :) [14:19:03] vikasyaligar: :) [14:20:49] (03PS2) 10Hashar: Build PHP extension with HHVM [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150820 (https://bugzilla.wikimedia.org/63120) [14:23:16] now need to make the code look good hmm ... [14:23:40] (03PS1) 10Hashar: Build PHP extension with HHVM (non voting) [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150822 (https://bugzilla.wikimedia.org/63120) [14:24:24] (03CR) 10Hashar: [C: 032] Build PHP extension with HHVM (non voting) [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150822 (https://bugzilla.wikimedia.org/63120) (owner: 10Hashar) [14:24:33] (03Merged) 10jenkins-bot: Build PHP extension with HHVM (non voting) [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/150822 (https://bugzilla.wikimedia.org/63120) (owner: 10Hashar) [14:25:17] (03CR) 10Hashar: "I have added the Zuul trigger with https://gerrit.wikimedia.org/r/#/c/150822/" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150820 (https://bugzilla.wikimedia.org/63120) (owner: 10Hashar) [14:28:35] (03PS3) 10Hashar: Build PHP extension with HHVM [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150820 (https://bugzilla.wikimedia.org/63120) [14:28:47] (03CR) 10Hashar: "PS3 change label:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150820 (https://bugzilla.wikimedia.org/63120) (owner: 10Hashar) [14:29:51] 3Wikimedia / 3Continuous integration: Set up automatic builds for extensions ported to HHVM - 10https://bugzilla.wikimedia.org/63120#c12 (10Antoine "hashar" Musso) A first build: https://integration.wikimedia.org/ci/job/php-FastStringSearch-hhvm-build/1/console 00:00:05.874 cc1plus: error: /root/hhvm/joe/hh... [14:35:36] vikasyaligar: does the scroll bar appear for a screenshot for all languages? [14:35:43] or just for one language? [14:37:00] zeljkof: some languages only [14:37:23] zeljkof: for that one, it was he and en, where we saw scrollbars [14:37:55] that => commons.wikimedia.beta.wmflabs.org/wiki/File:VisualEditor_Toolbar_SpecialCharacters-he.png [14:46:16] zeljkof: I am waiting for the job to complete, so that after seeing the uploads I can zoom in or zoom out correct scenario in https://gerrit.wikimedia.org/r/#/c/150833 [14:46:39] *corectly in scenarios [14:46:50] *correctly [14:47:22] will be back soon, out for dinner [14:50:48] beta labs died overnight it seemsd [14:51:53] chrismcmahon: yes, I think hashar restarted it a few hours ago [14:52:07] or in our morning, I remember seeing something in irc [16:58:19] (03CR) 10Cmcmahon: [C: 032] Minor readme updates [selenium] - 10https://gerrit.wikimedia.org/r/150776 (owner: 10Zfilipin) [16:58:21] (03Merged) 10jenkins-bot: Minor readme updates [selenium] - 10https://gerrit.wikimedia.org/r/150776 (owner: 10Zfilipin) [19:23:36] 3Wikimedia / 3Continuous integration: Set up automatic builds for extensions ported to HHVM - 10https://bugzilla.wikimedia.org/63120#c13 (10Antoine "hashar" Musso) 5PATC>3NEW Ori quoted a discussion from Giuseppe: > hphpize adds to the list of paths it configures the $PWD at the time of building > so wh... [19:35:37] 3Wikimedia / 3Continuous integration: Set up automatic builds for extensions ported to HHVM - 10https://bugzilla.wikimedia.org/63120 (10Antoine "hashar" Musso) [19:35:38] 3Wikimedia / 3Continuous integration: hphpize inject $PWD used at time of building to HHVM_INCLUDE_DIRS - 10https://bugzilla.wikimedia.org/68944 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3None Forked from bug 63120 Ori quoted a discussion from Giuseppe: > hphpize adds to the list of pat... [19:36:08] 3Wikimedia / 3Continuous integration: Set up automatic builds for extensions ported to HHVM - 10https://bugzilla.wikimedia.org/63120#c14 (10Antoine "hashar" Musso) Forked the issue to Bug 68944 - hphpize inject $PWD used at time of building to HHVM_INCLUDE_DIRS [19:39:23] (03PS1) 10Hashar: Workaround hphpize injecting some bad include path [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150895 (https://bugzilla.wikimedia.org/68944) [19:41:08] (03CR) 10Hashar: "deployed and fixed the issue https://integration.wikimedia.org/ci/job/php-FastStringSearch-hhvm-build/2/console :-]" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/150895 (https://bugzilla.wikimedia.org/68944) (owner: 10Hashar) [19:42:06] 3Wikimedia / 3Continuous integration: hphpize inject $PWD used at time of building to HHVM_INCLUDE_DIRS - 10https://bugzilla.wikimedia.org/68944#c2 (10Antoine "hashar" Musso) 5PATC>3NEW Using sed in Jenkins to get rid of the incorrect path has let Jenkins build the FastStringSearch extension! https://in... [19:50:37] 3Wikimedia / 3Continuous integration: Add pre-commit hook to check executable rights - 10https://bugzilla.wikimedia.org/68902#c1 (10christian) You can of course install your own pre-commit hook to do so, but git does not allow to automatically install hooks. But one could add such guards to Jenkins, if that... [20:19:21] 3Wikimedia / 3Continuous integration: Add pre-commit hook to check executable rights - 10https://bugzilla.wikimedia.org/68902 (10Greg Grossmeier) p:5Unprio>3Normal [20:57:28] bd808: should this slave be kicked? https://integration.wikimedia.org/ci/computer/puppet-compiler02.eqiad.wmflabs/ [20:57:43] god, I kinda want to hurt myself for saying that [20:57:56] greg-g: white guilt [20:58:14] I don't think I have access to that box [20:58:33] I have this nice "Launch slave agent" button when I'm logged in on that page [20:58:49] not sure if it works or what [20:59:02] there's a job waiting for that box to run [20:59:17] (sorry to divert your attention) [20:59:34] just, "puppet" and CI with no hashar means.... [20:59:53] oh hey, twentyafterfour maybe you'd be curious to just see what the decision is here ^ :) [21:00:48] greg-g: "ERROR: Server rejected the 1 private key(s) for jenkins-deploy" [21:01:10] oh.... right.... there was something about that in bugzilla or some such [21:01:16] * greg-g is helpful [21:01:29] My guess is that box has puppet disabled and it got bitten by the nscd package update that caused problems in beta yesterday [21:01:35] huh [21:02:47] There was a security update for a package that we apply custom config for. If puppet doesn't run after the package was upgraded then the host can't see the labs ldap server [21:03:00] hrmmm (all news to me) [21:03:12] We had 4 boxes in beta that were bittten by it yesterday [21:03:23] I fixed them (because I'm awesome) [21:03:33] indeed, this is true [21:04:38] If puppet is admin disabled on that box then the only way I know to fix it would be with salt commands from the labs salt master. [21:04:58] The root ssh keys may work without ldap as well [21:05:18] You should ping _joe_. It's his baby [21:07:42] he's asleep [21:07:48] or family-ing [21:08:48] That box doesn't run blocking tests does it? [21:09:11] this one https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/ [21:09:29] hasn't run for a while [21:14:09] Here's the wikitech page for the instance -- https://wikitech.wikimedia.org/wiki/Nova_Resource:I-00000387.eqiad.wmflabs [21:14:34] I'm not a member of the puppet3-diffs project so It won't let me see the console output [21:15:08] One of the labs folks (andrew, coren, yuvi) might be able to poke it [21:27:35] * greg-g is emailing hashar a summary of this and -labs convo, have no fear! ;) [22:25:00] Running phpunit is so much less painful when it can consume 6 GB of RAM in cache [22:36:29] bd808: is beta down? [22:36:43] bd808: kaldari was just complaining, I'm checking [22:36:50] deployment-mediawiki01 looks hung [22:37:11] let me ping Brett really quick [22:37:47] bd808: yeah, just not responding [22:39:05] chrismcmahon: the 02 server looks to be alive, but 01 is in the "all apache children busy" state [22:40:53] chrismcmahon: better now I think [22:48:35] (03PS1) 10Dduvall: Parsing of responses and additional query actions [ruby/api] - 10https://gerrit.wikimedia.org/r/150977 [22:53:14] chrismcmahon: Just so you're sort of in the loop... current thinking on the HHVM lockup issue is that the fastcgi bridge code in HHVM that we are talking to with apache has some bug. Brett is looking into it (by reading all the fcgi code in hhvm). [22:54:20] thanks bd808, sounds like not much fun [22:54:43] It seems to slowly "hang" individual apache children until apache hits it's maxchild limit. Then just 503s come out from the apache dispatcher loop after the max wait timeout. [22:55:44] FB doesn't use the fcgi connector internally so... new bugs for upstream! [23:01:10] 18:58 < kaldari> chrismcmahon: Something is definitely hosed on beta labs still. Half the time it won't load anything from bits. And every time I log in it wants to show me everything in German, even though my language is set to English in the prefs :P [23:01:14] from -operations [23:02:13] * bd808 looks around for a beta sys admin [23:04:29] greg-g, kaldari: It's working for me. Not always fast but I can load pages across many beta domains [23:05:13] ditto, I don't accidentally see german either :/ [23:06:13] wfm too [23:06:16] (03PS2) 10Dduvall: Parsing of responses and additional query actions [ruby/api] - 10https://gerrit.wikimedia.org/r/150977 [23:07:07] 3Wikimedia / 3Continuous integration: Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) - 10https://bugzilla.wikimedia.org/67216 (10Nemo) [23:10:38] 3Wikimedia / 3Continuous integration: Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) - 10https://bugzilla.wikimedia.org/67216 (10Nemo) [23:12:51] 3Wikimedia / 3Continuous integration: Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) - 10https://bugzilla.wikimedia.org/67216 (10Nemo) [23:19:23] 3Wikimedia / 3Continuous integration: Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) - 10https://bugzilla.wikimedia.org/67216 (10Nemo)