[00:00:51] (03PS1) 10Krinkle: (DRAFT) Hotfix: grunt-lib-phantomjs Date mockery protection [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125039 [00:02:41] (03PS1) 10Krinkle: Add bin/phantomjs symlink to avoid having to figure out the deep path [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125040 [00:03:14] (03PS2) 10Krinkle: Add bin/phantomjs symlink to deep node_modules path [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125040 [00:03:23] (03CR) 10Krinkle: [C: 032] Add bin/phantomjs symlink to deep node_modules path [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125040 (owner: 10Krinkle) [00:03:26] (03Merged) 10jenkins-bot: Add bin/phantomjs symlink to deep node_modules path [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125040 (owner: 10Krinkle) [00:58:28] (03PS1) 10Krinkle: Update tools/package.json to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125052 [01:01:30] (03PS2) 10Krinkle: Update tools/package.json to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125052 [01:01:36] (03CR) 10Krinkle: [C: 032] Update tools/package.json to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125052 (owner: 10Krinkle) [01:01:55] (03Merged) 10jenkins-bot: Update tools/package.json to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125052 (owner: 10Krinkle) [01:14:58] (03PS1) 10Ori.livneh: *.hhvm.beta.wmflabs.org -> *.wikipedia.beta-hhvm.wmflabs.org [operations/apache-config] (betacluster) - 10https://gerrit.wikimedia.org/r/125058 [01:15:36] (03CR) 10Ori.livneh: [C: 032] *.hhvm.beta.wmflabs.org -> *.wikipedia.beta-hhvm.wmflabs.org [operations/apache-config] (betacluster) - 10https://gerrit.wikimedia.org/r/125058 (owner: 10Ori.livneh) [01:15:38] (03Merged) 10jenkins-bot: *.hhvm.beta.wmflabs.org -> *.wikipedia.beta-hhvm.wmflabs.org [operations/apache-config] (betacluster) - 10https://gerrit.wikimedia.org/r/125058 (owner: 10Ori.livneh) [07:26:23] hello [07:30:40] (03PS1) 10Hashar: Mark ContributionReporting has obsolete [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125082 [07:30:49] (03CR) 10Hashar: [C: 032] Mark ContributionReporting has obsolete [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125082 (owner: 10Hashar) [07:35:42] (03PS1) 10Hashar: Jobs for mw/ext/TextExtracts [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125085 [07:35:54] (03CR) 10Hashar: [C: 032] Jobs for mw/ext/TextExtracts [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125085 (owner: 10Hashar) [07:36:46] (03PS2) 10Hashar: Mark ContributionReporting has obsolete [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125082 [07:36:55] (03CR) 10Hashar: [C: 032] Mark ContributionReporting has obsolete [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125082 (owner: 10Hashar) [07:37:19] (03Merged) 10jenkins-bot: Jobs for mw/ext/TextExtracts [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125085 (owner: 10Hashar) [07:37:53] (03PS1) 10Hashar: Trigger jobs for mw/ext/TextExtracts [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125087 [07:38:07] (03CR) 10Hashar: [C: 032] Trigger jobs for mw/ext/TextExtracts [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125087 (owner: 10Hashar) [07:38:38] (03Merged) 10jenkins-bot: Mark ContributionReporting has obsolete [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125082 (owner: 10Hashar) [07:38:44] (03Merged) 10jenkins-bot: Trigger jobs for mw/ext/TextExtracts [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125087 (owner: 10Hashar) [07:42:03] hi hashar [07:42:52] hashar: I filed https://bugzilla.wikimedia.org/show_bug.cgi?id=63745 yesterday [07:45:37] hold on busy taking traces :-D [07:47:29] hashar: :) [07:52:26] bahhh [07:55:44] i must have deployed that manually on production [07:55:50] the dir has proper ownerships [08:06:05] Invalid parameter user at /etc/puppet/modules/contint/manifests/maven_webproxy.pp:21 [08:06:11] I sometime hate puppet [08:25:00] hashar: heh [08:25:06] being worked on :-] [08:25:08] puppet is running [08:25:09] hashar: it broke after a puppet run I guess [08:25:10] hashar: ok :) [08:25:14] * yuvipanda is intermittently available [08:25:30] I will retrigger the job to see if it is fixed and close bug if so [08:26:31] yuvipanda: should be good rebuilding at https://integration.wikimedia.org/ci/job/apps-android-wikipedia-maven-checkstyle/53/console [08:26:43] hashar: woot! :) [08:26:44] quick afk brb [08:37:32] re [08:50:43] hashar: so if I upload a new patchset it should re trigger I guess? [08:56:52] yuvipanda: yeah or you can look at the job failing and click the [Rebuild] link in the left menu [08:57:03] hashar: yeah, I just did a trivial rebase and pushed them all up :) [08:57:04] that would show a form with all parameters filled, then just click [rebuild] [08:57:07] saves you a new patchset [08:57:14] or you can do a trivial rebase / commit summary edit [08:58:21] hashar: are these checks run parallelly or serially? [08:58:27] just looking for ways to speed it up :) [08:58:28] serially [08:58:31] going to adjust that [08:58:45] ah ok :D [08:59:16] and i found out that the maven template has a typo :)D [08:59:31] hashar: hah! [08:59:32] - tirggers: [08:59:32] + triggers: [08:59:32] - zuul [08:59:39] oh wow. and it still worked [08:59:52] I've the checkstyle errors down to under 200. [08:59:57] yeah because Zuul does not uses the parameters [08:59:59] should get it to zero soon [09:00:02] it is only used when triggering build manually [09:00:06] not sure why it worked though [09:01:53] (03PS1) 10Hashar: Typo for Zuul triggers in '{name}-maven-checkstyle' [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125106 [09:01:56] (03PS1) 10Hashar: Make '{name}-maven-checkstyle' concurrent [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125107 [09:02:10] yuvipanda: ^^^:-] [09:02:28] woo! [09:02:47] bah apps-android-wikipedia-tox-flake8 is blocked :( [09:02:50] https://integration.wikimedia.org/zuul/ [09:03:22] hashar: oh? is that why jenkins-bot isn't voting? [09:03:29] yeah [09:03:33] the jobs are not being triggered [09:03:48] that has happened from time to time since ~ 3 weeks [09:03:48] hmm [09:03:51] no clue what is happening [09:03:57] what's the fix? [09:04:12] need to diagnosse it [09:04:23] :( [09:31:24] (03CR) 10Addshore: [C: 031] ":D" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125106 (owner: 10Hashar) [09:31:36] (03CR) 10Addshore: [C: 031] ":D" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125107 (owner: 10Hashar) [09:50:02] I am off for lunch, will be back later [09:50:17] (03CR) 10Hashar: [C: 032] "Deployed" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125106 (owner: 10Hashar) [09:50:27] (03CR) 10Hashar: [C: 032] "Deployed" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125107 (owner: 10Hashar) [09:51:09] (03Merged) 10jenkins-bot: Typo for Zuul triggers in '{name}-maven-checkstyle' [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125106 (owner: 10Hashar) [09:51:12] (03Merged) 10jenkins-bot: Make '{name}-maven-checkstyle' concurrent [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/125107 (owner: 10Hashar) [11:04:36] Zeljkof:Hi! I had installed SulimeText ..was wondering is it an IDE or more of a text editor [11:04:46] Jagori: sorry, in a meeting with Tobi_WMDE [11:05:07] Zeljkof:Ok sorry [11:07:44] Jagori: it is pretty much a text editor [11:08:13] Yuvipanda:Thanks..yes:) [11:08:51] Yuvipanda:Since I had been using RubyMine for a while. are there any open source IDE too [11:09:15] Jagori: I usually just use Vim :D [11:09:22] :) Ohh.. [11:09:57] Yuvipanda:thanks..no issues..then I think Imay use Sublimetext only..notepad ++ is also a good editr [11:10:19] thanks :) [11:14:01] Jagori: :) [11:14:05] Jagori: good luck! [11:23:37] (03PS1) 10Zfilipin: If environment variable HEADLESS is set to true, run a local browser [selenium] - 10https://gerrit.wikimedia.org/r/125169 [11:48:53] zeljkof: just saw that if I turn a pageObject into a module I cannot do things like on(SpecialModifyEntityPage)... anymore. it would give me "undefined method `new' for SpecialModifyEntityPage:Module (NoMethodError)" [11:49:03] any idea? [11:49:13] Tobi_WMDE: hm [11:49:21] can you push it to gerrit? [11:49:24] so I could take a look [11:49:32] can do [11:49:34] one moment [11:50:41] (03PS1) 10Zfilipin: Fixed link to Wikibase Jenkins in readme file [selenium] - 10https://gerrit.wikimedia.org/r/125171 [11:54:39] zeljkof: https://gerrit.wikimedia.org/r/#/c/125172 [11:55:23] (03CR) 10Tobias Gritschacher: [C: 031] Fixed link to Wikibase Jenkins in readme file [selenium] - 10https://gerrit.wikimedia.org/r/125171 (owner: 10Zfilipin) [11:56:13] Tobi_WMDE: looking [11:56:23] zeljkof: any objections in giving me +2 rights in mediawiki/selenium repository [11:56:38] Tobi_WMDE: I thought you already have them :) [11:56:53] I am not the one that gives the permission [11:57:02] hashar should be able to do it [11:57:18] probably somebody else too, but not sure who [11:57:30] can you create a bug in bugzilla and assign it to hashar? [11:57:38] (for +2 on the gem) [11:58:48] can do [12:00:05] zeljkof: what's the right component for that bug? [12:00:11] :) [12:00:15] probably wikimedia-qa [12:00:25] ok [12:00:34] it is important that it is in bugzilla, hashar uses it as a todo list :) [12:00:41] he can change the component if needed [12:00:56] feel free to add me to cc, I will give my +1 on the bug [12:02:35] zeljkof: you were added automatically [12:02:58] Tobi_WMDE: yes, now I remember :) [12:06:42] Tobi_WMDE: I see the problem [12:06:50] in the commit [12:07:09] zeljkof: yes? [12:07:21] yes, when you want to do on(SpecialModifyEntityPage), that calls method #new on SpecialModifyEntityPage [12:07:34] and classes have that method defined by default, modules do not [12:07:56] zeljkof: so I just add it? [12:08:00] what should it do? [12:08:03] nothing? [12:08:21] I am not sure would adding #new method work, who knows what page-object gem expects it to do :) [12:08:34] is there a reason you have converted all classes to modules? [12:08:58] can you just copy/paste the code from the module to the class? and delete the module? [12:09:04] is the module included in several classes? [12:10:00] zeljkof: yes they will be included in several classes [12:10:36] the reason why I converted them is to no longer use inheritance but inclusion [12:10:48] if you need the module and the class with the same name [12:10:50] but I can live with having them as classes [12:11:05] you can create SpecialModifyEntityPage and SpecialModifyEntityModule [12:11:11] pull all the code into the module [12:11:18] zeljkof: yeah, was thinking of this [12:11:20] and include the module in all the classes that need it [12:11:38] not sure if this is nicer than just having classes [12:11:44] SpecialModifyEntityPage can be almost empty, except for including page object and the module [12:11:56] Tobi_WMDE: depends :) [12:12:13] if you want to ride with the pack, we usually use composition, not inheritance [12:12:25] but you are free to ride alone :) [12:13:23] hehe [12:13:40] I'll think over it. currently trying to get the headless thing working [12:16:45] (03CR) 10KartikMistry: [C: 032] "LGTM" [selenium] - 10https://gerrit.wikimedia.org/r/125171 (owner: 10Zfilipin) [12:16:47] (03Merged) 10jenkins-bot: Fixed link to Wikibase Jenkins in readme file [selenium] - 10https://gerrit.wikimedia.org/r/125171 (owner: 10Zfilipin) [12:38:19] zeljkof: after running bundle update (and updating to cucumber 1.3.14 I no longer can run specific tests by specifying the feature file - I'm getting "No such file or directory - features/special_set_label.feature. Please create a features/special_set_label.feature directory to get started." [12:38:39] do you know something about that? [12:38:45] no :) [12:38:56] did you read release notes? [12:39:02] maybe they have changed something [12:39:10] zeljkof: sorry [12:39:13] was my mistakle [12:39:17] arr [12:39:21] too confused today [12:39:32] was unrelated to the update [12:39:58] :) [12:54:07] hashar: where is node: hasAndroidSdk defined? [12:54:20] ohh [12:54:26] I am not sure Android SDK is working anymore [12:54:35] yeah, I think the builds have been broken for a while [12:54:40] http://integration.wikimedia.org/ci/label/hasAndroidSdk/ [12:54:40] is there any way I can help set one up on labs? [12:54:50] that is only for the job https://integration.wikimedia.org/ci/job/apps-android-commons-build/ [12:54:56] and only the gallium slave has it [12:55:17] gotta find a way to install the android SDK properly [12:56:20] hashar: hmm, would that have to be puppetized? [12:56:35] ideally [12:56:36] or packaged [12:56:57] last time I checked (several months ago) there was no Debian package though :( [12:57:11] hashar: yeah, there are some ubuntu PPAs [12:57:14] hashar: but a debian package I think is unlikely [12:57:18] hashar: a shell script is more doable [12:57:19] oh [12:57:20] https://packages.debian.org/source/sid/androidsdk-tools [12:57:43] waat [12:57:54] oh wow [12:57:56] and Ubuntu Trusty has http://packages.ubuntu.com/trusty/android [12:58:10] zeljkof: I managed to fix https://gerrit.wikimedia.org/r/#/c/125172/ [12:58:15] * hashar assigns bug to yuvipanda :-] [12:58:35] yuvipanda: you could try out the Ubuntu Trusty android package. Maybe it can be installed on Precise [12:58:38] hashar: it doesn't actually have support for sdkmanager :( [12:58:44] else we will have to backport it (that might be possibly) [12:58:46] * yuvipanda checks trusty [12:59:16] hashar: I can play around with it on a labs instance. Should I do that on mobile project or contint? [12:59:41] probably easier on mobile [12:59:48] Tobi_WMDE: looks good [12:59:56] try to figure out whether the package can be installed and if you can use what it provides to build your apps [13:00:13] + the commands involved (probably just need maven package or something and it would find the androidSDK provided by the package) [13:00:21] hashar: could you please take a look at https://bugzilla.wikimedia.org/show_bug.cgi?id=63761 [13:00:28] if that works fine (which I doubt) we can ask ops to publish the androidsdk package on apt.wikimedia.org [13:00:44] then get it installed on the labs Jenkins slaves and have the maven job to run there [13:01:34] zeljkof: oh [13:01:55] zeljkof: aren't you a project owner for mediawiki/selenium ? :-] [13:02:07] * hashar points at https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/selenium,access [13:02:38] hashar: ah, right. I'll create something on mobile and try it out [13:03:07] hashar: I wonder if I should just setup a cron on toollabs. Should be rather trivial to do there, since there's no packaging requirement [13:03:17] zeljkof: https://bugzilla.wikimedia.org/show_bug.cgi?id=63761#c2 [13:03:17] and I can have it do it once a day or so which is 'good enough' [13:03:37] zeljkof: just add tobias to the mediawiki-selenium group and he will be a owner and probably get +2 / merge rights as a result [13:04:06] yuvipanda: if the android package from trusty works we can get it installed on labs slaves [13:04:17] hmm [13:04:23] yuvipanda: then you can use jenkins which would provide you with IRC notifications/ Zuul triggering / dashboards / check style whatever :] [13:04:28] :D [13:04:32] at the expense of a bit of effort :] [13:04:44] but probably less than setting up a cron based continuous integration system hehe [13:05:16] hashar: created an instance under mobile to test it out. let's see how it goes :) [13:05:18] yuvipanda: can you bug fill it so we can track the progress ? [13:05:23] hashar: sure. moment [13:06:49] hashar: https://bugzilla.wikimedia.org/show_bug.cgi?id=63765 [13:08:42] hashar: brb, sunset time [13:10:48] * hashar today I learned indian people stop working during sunset [13:30:05] hashar: when I open the page you have linked to in the bug, "members" text field is disabled :( [13:30:11] I can not enter new people there [13:30:27] bah [13:30:28] link? [13:30:43] (cleared backlog and closed bug sorry) [13:30:52] hashar: https://bugzilla.wikimedia.org/show_bug.cgi?id=63761 [13:31:12] https://gerrit.wikimedia.org/r/#/admin/groups/600,members [13:31:12] hmm [13:31:18] maybe you dont have the right to edit that group [13:31:25] could be [13:31:29] belong to "Project and Group Creators" [13:31:31] sorry [13:32:30] added Tobias there [13:32:37] hashar: thanks [13:32:38] he should get +2 and basically any rights on that repository [13:32:41] sorry :-( [13:32:43] great [13:32:52] I am able to add new people or not? [13:32:52] I thought members of a group could add members hehe [13:32:56] na you can't [13:32:58] I see [13:33:05] ok, we do not have to do that often [13:33:07] need to be a member of "Project and Group Creators" [13:33:14] i.e. more or less a Gerrit bureaucrat or something [13:33:15] it only happened a couple of times [13:33:27] will be happy to address such requests :] [13:33:39] is there a reason kartik does not automatically have +2 on mediawiki/selenium repo? [13:33:41] either via bug if you CC me or by IRC ping hehe [13:33:48] since he is wmf [13:34:01] he must be using an email which is not @wikimedia.org [13:34:07] hence we have to whitelist him in zuul [13:34:12] I see [13:34:29] ^kartik\.mistry@gmail\.com$ [13:34:34] apparently he is whitelisted [13:34:51] hashar: he is a member of the group now [13:34:58] nevermind, just a thought [13:34:58] \O/ [13:35:05] our tooling is strange sometimes [13:35:07] you can close https://bugzilla.wikimedia.org/show_bug.cgi?id=63765 [13:35:13] hashar: will do [14:03:46] who is the right person to add to a mediawiki ruby api change? [14:04:14] zeljkof: http://wdjenkins.wmflabs.org/ci/job/wikibase-browsertests-smoke/14/console [14:04:33] thats waht I'm getting when trying to run a test in HEADLESS firefox on our jebnkins [14:04:38] manybubbles: me :) [14:04:46] k. cool. already done then:) [14:04:51] it seems the headless firefox cannot reach beta [14:05:02] manybubbles: Juliusz too [14:05:27] Tobi_WMDE: ruby-2.1.1 is not installed. [14:05:42] zeljkof: that's unrelated I guess [14:05:48] we're using 1.9.3 [14:06:01] zeljkof: do you know Juliusz's gerrit account name? [14:06:15] manybubbles: something jgonera I think [14:06:28] thanks! done [14:06:29] Tobi_WMDE: are you sure you have the correct gems installed/used? [14:06:54] zeljkof: I think so.. you see the list of gems in the console output [14:08:10] zeljkof: the request times out [14:08:32] if there were a problem with ruby or the gems it would not go so far [14:09:12] I think I remember something similar when I tried phantomjs back then [14:09:27] Tobi_WMDE: can you try with phantomjs? [14:09:37] just to make sure it is not related to headless gem [14:09:57] also, why don't you change ruby version in Gemfile to the one you use? [14:10:05] just to make sure that is not the problem [14:10:29] zeljkof: should do that [14:10:30] :) [14:10:37] I think you've put it there [14:13:11] Tobi_WMDE: I thought you will use the new and shiny version, not the old and rusty one :) [14:13:30] zeljkof: yeah.. guess you're right [14:13:32] :) [14:13:49] I'm using the old and rusty one on my machine that's why I installed it on jenkins too [14:14:17] Tobi_WMDE: I have tested it recently, ruby 2.0 and cucumber now work on windows [14:14:18] zeljkof: this is with phantomjs: http://wdjenkins.wmflabs.org/ci/job/wikibase-browsertests-smoke/15/console [14:14:20] try it [14:14:21] same thing [14:14:35] you can upgrade to the newest and greatest ruby now [14:14:35] zeljkof: you think it's related? [14:14:43] no, just random thought [14:16:58] zeljkof: and that's the same test when using sauce: http://wdjenkins.wmflabs.org/ci/job/wikibase-browsertests-smoke/16/console [14:17:01] it's passing [14:17:54] hashar: Tobi_WMDE is having problems with jenkins ^ [14:18:20] can their jenkins access the intertubes? [14:18:36] ha [14:19:03] Tobi_WMDE: maybe sauce works because their browsers are on the internet, and your jenkins is maybe behind a firewall or something [14:19:33] zeljkof: surprisingly our jenkins can access beta when we're doing api calls to wikidata.beta with ruby (to e.g. create items) [14:19:46] ruby-2.1.1 is not installed. [14:19:46] To install do: 'rvm install ruby-2.1.1' [14:19:53] zeljkof: it's just the browser that cannot access it [14:20:09] hashar: I think this is not the problem [14:20:14] what is the browser trying to reach ? [14:20:21] is that the beta cluster wikis? [14:20:50] hashar: http://wikidata.beta.wmflabs.org/wiki/Special:SetLabel [14:20:57] yes [14:21:04] ah [14:21:24] so here is the deal. Your labs instance wdjenkins.wmflabs.org query the DNS for wikidata.beta.wmflabs.org [14:21:31] that gives it a public IP address [14:21:41] but that public IP is NATed to an instance IP [14:22:02] and when you try to reach that public IP from your wdjenkins instanceIP, that is unsupported by the network stack [14:22:09] it basically has no clue what to do with it [14:22:27] hashar: but when ruby makes calls to http://wikidata.beta.wmflabs.org/w/api.php it works fine [14:22:39] the evil hack of doom is to add some iptables rules on the wdjenkins to rewrite the destination IP from a public IP to a private IP. [14:22:41] taht's what we're doing since a long time [14:22:52] hashar: we have that hack already in the iptables [14:22:52] ohh [14:23:04] another possibility is to use a web proxy [14:23:18] it's just that it seems the headless firefox and phantomjs cannot reach beta [14:23:21] but I am not sure whether we have any web proxy reachable from labs [14:23:29] :-/ [14:24:26] hashar: I'm trying something [14:25:03] on wdjenkins can you try: curl http://wikidata.beta.wmflabs.org/w/api.php [14:25:07] just to confirm [14:25:17] as for phantomjs maybe it is set to use a web proxy that has the same issue? :-( [14:25:18] hashar: curl works fine [14:25:20] I tried that [14:25:21] good [14:25:26] so the iptables works \O/ [14:25:30] yes [14:26:40] hashar: ok, I just tried in the console [14:27:00] opening google.com with phantomjs works [14:27:06] opening http://wikidata.beta.wmflabs.org/wiki/Special:SetLabel times out [14:28:08] would need to figure out what phantomjs is doing exactly [14:28:27] hashar: same with firefox [14:29:18] hashar: arn't you using phantomjs also on wmf jenkins? [14:29:38] to run the qunit tests but they are running against a local virtualhost [14:29:46] hashar: ok [14:29:54] and I did work a bit on having browser tests but they hit the local virtual host as well :-( [14:30:00] maybe tcpdump would help ? :D [14:30:47] hashar: hmm [14:31:45] or find out whether a http proxy is set with the snippet https://github.com/ariya/phantomjs/blob/master/examples/printenv.js [14:31:58] though unlikely [14:33:54] doesn't seem like [14:37:10] * zeljkof is out to lunch [14:39:50] https://www.irccloud.com/pastebin/NBsXpCDb [14:39:55] hashar: ^^ [14:40:06] that's what phantomjs is doing [14:41:26] so at least packets are being exchanges with the text cache which serves wikidata.beta.wmflabs.org [14:41:32] that is deployment-cache-text02.eqiad.wmflabs [14:41:47] maybe the logs on beta cluster got more informating, digging in [14:43:01] yeah I see them just fine [14:43:10] hashar: means? [14:43:12] X-FORWARDED-FOR: 10.68.16.79, 127.0.0.1 [14:43:35] maybe it has trouble to login? [14:43:47] hashar: login where? [14:43:56] it should not be necessary to login [14:45:09] it shoud just open http://wikidata.beta.wmflabs.org [14:45:16] no need to login for that [14:45:38] I got some GET /wiki/Special:SetLabel [14:45:46] ok [14:46:12] what happened to them then? [14:47:28] Tobi_WMDE: an example http://paste.openstack.org/show/75481/ [14:48:03] and there is one after doing Start request GET /wiki/Special:SetLabel?setlang=en [14:49:13] hashar: yeah, guess the one with the setlang is from the actual browsertest [14:49:30] while the other one is from my manual try with phantomjs in the console [14:49:55] but it seems ok. so why is the request then timing out? [14:50:10] and from the apache log [14:50:10] http://paste.openstack.org/show/75482/ [14:50:27] do you have access on the beta cluster ? [14:50:35] might be user for you to tail the log while the job is running :] [14:51:30] the apache log is probably not relevant.. that could also be from one of the jobs that run on aucelabs.. they're fine [14:52:03] I can just try again with phantomjs [14:52:04] !log deployment-prep Adding Tobias Gritschacher to the project so he can look at udp2log / apache logs whenever needed :-] [14:52:38] and then we should have a look at the logs [14:52:48] to be sure to look at the right time [14:53:11] hashar: ok, request is running [14:53:18] zeljkof: I am running a little behind right now, figuring out if a Mobile test failure is a bug or not... [14:53:47] can you look at the logs again? [14:54:36] hashar: request still pending.. think it times out after 1 minute or so [14:55:17] -:( [14:55:39] hm.. timeout [14:56:01] is there anything in the logs now regarding /wiki/Special:_SetLabel [14:56:04] is there anything in the logs now regarding /wiki/SpecialSetLabel [14:56:10] ? [14:56:22] within the last 2 minutes? [14:56:49] [10/Apr/2014:14:37:32 +0000] "GET /wiki/Special:SetLabel HTTP/1.1" 200 5181 xfp: - [14:56:50] [10/Apr/2014:14:44:26 +0000] "GET /wiki/Special:SetLabel HTTP/1.1" 200 5181 xfp: - [14:57:06] I gave you access to the project. You can probably login on deployment-bastion.eqiad.wmflabs now [14:57:23] the set label seems to be working on apache side [14:58:02] hashar: the entries are to old.. they should be around minute 50 or so [14:58:25] 14:44 is like over 190 minutes ago [14:58:31] I mean 10 [14:59:47] ah time is UTC [14:59:55] yeah yeah 10 [15:00:27] hashar: me trying was at 55 or something [15:00:40] and right now again [15:00:50] that is 15:00 UTC right? [15:01:34] yes [15:01:37] we are on GMT+2 [15:01:43] but the instances are using UTC [15:02:04] hashar: what is 208.80.155.137 [15:02:06] ? [15:02:24] * hashar looks at https://wikitech.wikimedia.org/wiki/Special:NovaAddress [15:02:31] ah that is deployment-cache-bits01 [15:02:39] 10.68.16.12 [15:02:44] it serves bits.beta.wmflabs.org [15:02:49] so maybe an iptables rule is missing ? [15:03:07] hashar: that might be the case [15:03:15] I'll check that [15:03:26] straight from puppet : http://paste.openstack.org/show/75483/ [15:03:31] zeljkof: almost ready... [15:03:38] chrismcmahon: :) [15:03:41] I am in the hangout [15:04:19] Tobi_WMDE: I got a puppet class role::beta::natfix to apply the iptables hack. But that uses ferm which DROP by default. [15:04:25] so that can get you unwanted side effect [15:04:31] ssh from the bastion is allowed though [15:04:48] hashar: I'll add it manually and to our puppet [15:04:53] for now [15:07:23] yeah it is probably safer [15:07:31] the instance IP are probably not going to change anytime soon [15:08:15] Tobi_WMDE: http://paste.openstack.org/show/75485/ should be appropriate [15:09:08] (03CR) 10KartikMistry: "Do we need to update README.md for this change?" [selenium] - 10https://gerrit.wikimedia.org/r/125169 (owner: 10Zfilipin) [15:13:04] hashar: works :D [15:13:11] great!!!!!!!!!!! [15:13:25] you would never believe how much time it took me to figure out that solution in the first place :-] [15:13:44] if only cucumber could tell which URL it is timing out on [15:13:52] I hehe [15:13:58] Tobi_WMDE: have you tried connecting to deployment-bastion.eqiad.wmflabs ? [15:13:59] hashar: I can imagine [15:14:04] logs are in /data/project/logs [15:14:07] that might be handy [15:14:13] hashar: yes, I have access [15:14:16] thx! [15:14:17] \O/ [15:14:47] all of the logs [15:15:54] most of them are produced via wfDebugLog() and udp2log [15:16:57] hashar: we were missing all rules except deployment-cache-text02 [15:17:15] but that was enough to make api calls so I did not notice earlier [15:17:33] you can look at using the role::beta::natfix class [15:17:47] but you would have to allow ports using ferm::rule {} [15:17:56] there is a few examples floating in operations/puppet.git [15:18:32] hashar: now that I know, i will just poke the class when problems occur and check if something changed [15:18:45] and keep it in our own puppet for the moment [15:18:59] an example was allowing Parsoid port 8080 on beta [15:19:14] parsoid is setup on beta cluster using the role::parsoid::beta [15:19:18] so I just added a ferm rule https://gerrit.wikimedia.org/r/#/c/101210/ [15:20:04] and there is a ferm rule to allow ssh access from theJenkins server at the bottom of https://gerrit.wikimedia.org/r/#/c/99656/8/manifests/role/parsoid.pp,unified [15:20:08] that is quite "easy" [15:20:15] akosiaris from ops can help review them if unsure [15:24:29] (03PS1) 10Hashar: dashboard: remove pmtpa beta cluster jobs [integration/docroot] - 10https://gerrit.wikimedia.org/r/125198 [15:24:37] (03CR) 10Hashar: [C: 032] dashboard: remove pmtpa beta cluster jobs [integration/docroot] - 10https://gerrit.wikimedia.org/r/125198 (owner: 10Hashar) [15:24:40] (03Merged) 10jenkins-bot: dashboard: remove pmtpa beta cluster jobs [integration/docroot] - 10https://gerrit.wikimedia.org/r/125198 (owner: 10Hashar) [15:46:35] (03PS2) 10Zfilipin: If environment variable HEADLESS is set to true, run a local browser [selenium] - 10https://gerrit.wikimedia.org/r/125169 [15:49:34] (03CR) 10Zfilipin: "Kartik: yes, we should, will do it in the next patch set" [selenium] - 10https://gerrit.wikimedia.org/r/125169 (owner: 10Zfilipin) [15:51:19] (03CR) 10KartikMistry: [C: 032] "LGTM." [selenium] - 10https://gerrit.wikimedia.org/r/125169 (owner: 10Zfilipin) [16:02:05] (03Merged) 10jenkins-bot: If environment variable HEADLESS is set to true, run a local browser [selenium] - 10https://gerrit.wikimedia.org/r/125169 (owner: 10Zfilipin) [16:43:39] zeljkof_: are you around? do you know what would cause this URL? https://saucelabs.com/jobs/0591beeda67946588235d78ec9110160 (with "$channel" where "beta" should be) [16:44:56] chrismcmahon: uh oh [16:44:58] strange [16:45:05] not sure what caused that [16:45:12] zeljkof: actually, it's doing it live now too [16:45:23] zeljkof: it's not just Sauce/the tests [16:45:29] * chrismcmahon figures this out [16:45:37] can you create a bug in bugzilla, I can take a look tomorrow if you do not find the problem [16:45:59] zeljkof: don't worry, it's not the test, it's beta [17:02:51] (03CR) 10Chad: [C: 032] Cirrus now needs the Cite extension for testing [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/124011 (owner: 10Manybubbles) [17:02:59] (03PS4) 10Chad: Cirrus now needs the Cite extension for testing [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/124011 (owner: 10Manybubbles) [17:03:06] (03CR) 10Chad: [C: 032] Cirrus now needs the Cite extension for testing [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/124011 (owner: 10Manybubbles) [17:04:00] (03Merged) 10jenkins-bot: Cirrus now needs the Cite extension for testing [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/124011 (owner: 10Manybubbles) [17:04:56] chrismcmahon: I think I'm going to blame Ori for the $channel stuff [17:05:15] chrismcmahon: See https://gerrit.wikimedia.org/r/#/c/125062/ [17:05:23] bd808: wfm [17:36:43] bd808: I commented on both changes and pinged Ori over in -operations. I'd like to get beta back asap. [17:40:15] chrismcmahon: We can revert and see if that fixes things. I know you know how to use the revert button :) [17:42:39] I don't know where in the code path that those strings get expanded. The use of single quotes in the arrays means that they aren't expanded inline. Something does that later in the request cycle. [17:43:30] bd808: not sure a revert would even help that much, it'd take longer to get that merged than it would to find Ori [17:43:55] It's nearing the time of day when Ori should show up [18:21:40] hashar: I updated on the bug about contint [18:22:48] yuvipanda: I received an email notification :-] [18:23:00] got to catch up with a bunch of other issues, will look at it tomorrow hopefully [18:23:14] hashar: ok! :) [19:00:49] (03PS1) 10Krinkle: Update grunt-cli and jshint to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125257 [19:03:00] (03CR) 10Krinkle: [C: 032] "MediaWiki core (and other projects using grunt/npm) were already using jshint 2.4.x via grunt-contrib-jshint. This makes the raw -jslint j" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125257 (owner: 10Krinkle) [19:03:05] (03Merged) 10jenkins-bot: Update grunt-cli and jshint to newer upstream versions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125257 (owner: 10Krinkle) [19:22:16] (03CR) 10Krinkle: [C: 04-1] (DRAFT) Hotfix: grunt-lib-phantomjs Date mockery protection [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125039 (owner: 10Krinkle) [20:40:54] chrismcmahon, hashar: I want to make some more changes to beta's code deployment path this afternoon. Would turning off the beta-code-update-eqiad job in jenkins for a couple of hours be incredibly disruptive? If so I can plan my steps more carefully and get a designated window to make changes in. If not I'd love to poke around with it this afternoon. [20:41:10] bd808: I think I'm OK with that [20:41:23] note that Parsoid is deployed differently [20:41:44] might want to poke the features team that you are going to pause the update since people rely on it somehow :] [20:42:08] The only reason I want to shut the job down is that I think mw-update-l10n will break while I'm testing. [20:42:13] bd808: I think I am as caught up as I'm going to get, as long as you turn it back on before EOD so the overnight runs make sense [20:42:47] bd808: or turn it back on sometime before the middle of the night Pacific time [20:42:49] best way would be to go to https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/configure then check [ ] Disable Build (no new builds will be executed until the project is reenabled) [20:43:07] hashar: Yup. That was my plan [20:43:20] I will be sleeping anyway so nobody is going to complain to me :] [20:43:35] just announce it on engineering list I guess [20:43:35] chrismcmahon: I will make sure it's running again before I stop for the day [20:43:50] bd808: awesome, thanks [20:43:52] since various teams are using beta [20:44:28] bd808: pro tip: add a reminder to your agenda so you get pinged and make sure you dont forget about it :D [20:44:43] Those admonishments are making me lean towards planning, announcing and doing the work another day... [20:44:44] gotta process all my pending emails tomorrow, will surely have a look at your scap mails [20:44:55] na just be bold [20:45:01] but says so :] [20:45:08] {{WP:BOLD}} [20:45:30] you can just shout something like "code on beta cluster is no more update this afternoon, I am working on largely improving the way it is being done." [20:45:42] then everyone will happily reply and rejoice with you :] [20:45:49] :) [20:45:59] I took a 101 reverse psychology [20:46:13] or to make folks happy when you are going to annoy them [20:47:06] Cool. I've been meaning to tell the world my evil plans anyway. I'll use this as the best excuse to write that email. [20:47:33] keep it short :] [20:47:45] definitely mention it is an evil plan that boost your karma by an order of magnitude! [20:48:37] 2nd pro tip (but you know about it) if it crash completely write a lengthy post mortem this way folks get impressed and know the situation is handled :D [20:48:59] I'm getting too much practice at those :/ [20:49:23] we are human :] [20:50:05] a word of caution: dont get trapped in having to redesign mw-update-l10n on your own [20:50:12] or you might burn out [20:54:21] off to sleep [21:14:31] chrismcmahon: I just disabled https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/ to start my experiments. [21:34:10] go bd808 go ! [22:15:17] (03CR) 10Brian Wolff: "Recently all the patches submitted by (non-trusted) users have been failing the jslint tests on files not touched by the patch. Maybe this" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125257 (owner: 10Krinkle) [22:18:59] (03CR) 10Krinkle: "No, this is what fixes that by upgrading the jshint used by non-trusted users (node-jshint) to the same version we've been using in npm/gr" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/125257 (owner: 10Krinkle) [23:20:55] (03PS1) 10AndyRussG: Trigger unit tests for the Campaigns extension [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125344 [23:22:42] (03CR) 10Adamw: [C: 031] "Hopefully this is enuf!" [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/125344 (owner: 10AndyRussG)