[08:40:08] <grrrit-wm>	 (03PS1) 10Hashar: fix parsoidsvc-jslint was not voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105151 
[08:41:50] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] "tested locally this time jslint job should vote =)" [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105151 (owner: 10Hashar)
[08:42:00] <grrrit-wm>	 (03Merged) 10jenkins-bot: fix parsoidsvc-jslint was not voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105151 (owner: 10Hashar)
[10:50:33] <grrrit-wm>	 (03PS1) 10Hashar: multigit: GERRIT_URL -> GERRIT_CHANGE [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105160 
[10:50:35] <grrrit-wm>	 (03PS1) 10Hashar: multigit: fetch branches from Gerrit, not Zuul [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105161 
[10:50:41] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] multigit: GERRIT_URL -> GERRIT_CHANGE [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105160 (owner: 10Hashar)
[10:50:44] <grrrit-wm>	 (03Merged) 10jenkins-bot: multigit: GERRIT_URL -> GERRIT_CHANGE [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105160 (owner: 10Hashar)
[10:52:45] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] multigit: fetch branches from Gerrit, not Zuul [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105161 (owner: 10Hashar)
[10:52:49] <grrrit-wm>	 (03Merged) 10jenkins-bot: multigit: fetch branches from Gerrit, not Zuul [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105161 (owner: 10Hashar)
[10:58:04] <Nikerabbit>	 ugh
[10:58:48] <Nemo_bis>	 ?
[10:58:58] <Nikerabbit>	 [12.58.42] Niklas Laxström: two visual editor browser tests have been running for over 3 hours blocking my translate job... not nice
[11:05:19] <Nikerabbit>	 sent email
[11:17:51] <hashar>	 Nikerabbit: bug fill it please :-D
[11:18:05] <hashar>	 definitely need a timeout wrapper I guess
[11:24:26] <hashar>	 mw-jenkinsbot: welcome :D
[11:24:26] <mw-jenkinsbot>	 hashar you may not issue bot commands in this chat!
[11:27:54] <hashar>	 be back in a few 
[12:21:07] <mayankmadan>	 hi zeljkof
[12:21:18] <mayankmadan>	 zeljkof, can we start working on that download_pdf patch
[12:21:35] <mayankmadan>	 the tests which are meant for chrome have chrome in scenario
[12:21:48] <mayankmadan>	 chrome renders pdf differently so it wont work the other way
[12:22:28] <mayankmadan>	 so how can i make  sure that tests for chrome dont execute when using firefox
[12:22:47] <mayankmadan>	 and firefox tests dont execute when using chrome
[12:22:50] <mayankmadan>	 zeljkof, ^?
[12:23:42] <zeljkof>	 mayankmadan: ok
[12:24:09] <zeljkof>	 so, the first step is to separate the commit into two commits
[12:24:22] <zeljkof>	 one commit for firefox tests, one commit for chrome tests
[12:24:26] <zeljkof>	 do you know how to do that?
[12:24:49] <mayankmadan>	 zeljkof, i cant do that, i have linked that commit in melange
[12:25:56] <zeljkof>	 mayankmadan: you do not have to abandon the commit
[12:26:07] <zeljkof>	 just remove the chrome code
[12:26:13] <zeljkof>	 and add that as a separate commit
[12:26:39] <zeljkof>	 you can also leave a comment in the commit message saying the commit is split into two, if you want to keep track for melange
[12:29:09] <zeljkof>	 mayankmadan: do you know what to do?
[12:29:13] <zeljkof>	 do you need help?
[12:29:21] <mayankmadan>	 i dont think so
[12:29:44] <zeljkof>	 you do not need help, or you do not know what to do?
[12:29:55] <mayankmadan>	 i am currently working on an android task(which is almost completed), ill do it after that
[12:30:02] <zeljkof>	 ok
[12:30:02] <mayankmadan>	 i dont think i need help
[12:30:07] <zeljkof>	 ping me if you get stuck
[13:16:12] <grrrit-wm>	 (03PS31) 10Mayankmadan: Added a test for downloading pdf from a random page [qa/browsertests] - 10https://gerrit.wikimedia.org/r/98160 
[13:35:02] <mayankmadan>	 zeljkof, i have updated the patchset
[13:35:11] <zeljkof>	 mayankmadan: ok, will take a look
[13:42:27] <grrrit-wm>	 (03PS8) 10Hashar: (WIP) (WIP) visualeditor-doitall (WIP) (WIP) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 
[13:46:24] <grrrit-wm>	 (03PS9) 10Hashar: (WIP) (WIP) visualeditor-doitall (WIP) (WIP) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 
[13:46:42] <grrrit-wm>	 (03PS1) 10Mayankmadan: Adding pdf tests for chrome [qa/browsertests] - 10https://gerrit.wikimedia.org/r/105179 
[13:47:30] <mayankmadan>	 zeljkof, added for chrome too
[13:47:45] <mayankmadan>	 wait
[13:47:53] <mayankmadan>	 forgot to add downloaded_pdf_page.rb in that
[13:49:46] <hashar>	 hello :-)
[13:49:54] <zeljkof>	 hashar: welcome back :)
[13:50:08] <hashar>	 zeljkof: I am looking again at browser tests for visual editor
[13:50:48] <mayankmadan>	 wait, downloaded pdf page is not needed for chrome tests
[13:54:20] <hashar>	 zeljkof: I found out we are using bundler 1.3.5  from integration/bundler but 1.5 is out
[13:54:27] <hashar>	 zeljkof: should we use bundler 1.5 ?
[13:54:31] <zeljkof>	 hashar: yes
[13:54:39] <zeljkof>	 where are we using the old version?
[13:54:39] <hashar>	 going to update it so :-]
[14:05:47] <grrrit-wm>	 (03PS1) 10Hashar: bump to bundler 1.5.1 [integration/bundler] - 10https://gerrit.wikimedia.org/r/105180 
[14:05:52] <hashar>	 zeljkof: ^^^ :-)
[14:06:49] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] "being used the integration-selenium-driver.pmtpa.wmflabs instance to run the browser tests :-D" [integration/bundler] - 10https://gerrit.wikimedia.org/r/105180 (owner: 10Hashar)
[14:07:12] <grrrit-wm>	 (03CR) 10Hashar: [V: 032] bump to bundler 1.5.1 [integration/bundler] - 10https://gerrit.wikimedia.org/r/105180 (owner: 10Hashar)
[14:08:11] <zeljkof>	 hashar: wow, you have to do all that to bump gem version?
[14:08:48] <grrrit-wm>	 (03PS1) 10Hashar: bump bundler to 1.5.1 [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105181 
[14:08:57] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] bump bundler to 1.5.1 [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105181 (owner: 10Hashar)
[14:09:00] <grrrit-wm>	 (03Merged) 10jenkins-bot: bump bundler to 1.5.1 [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105181 (owner: 10Hashar)
[14:09:14] <hashar>	 zeljkof: yup because we are providing everything needed to run bundler since we can't use it as a debian package
[14:12:47] <hashar>	 00:02:01.738 Using bundler (1.5.1)
[14:12:49] <hashar>	 niiiice
[14:13:02] <Krinkle>	 hashar: 
[14:13:15] <Krinkle>	 -dev:
[14:13:16] <Krinkle>	 Krinkle: hashar: ping
[14:13:16] <Krinkle>	 14:00 Krinkle: https://wikitech.wikimedia.org/w/index.php?title=Server_Admin_Log&diff=cur&oldid=93754
[14:13:17] <Krinkle>	 14:03 Inez
[14:13:41] <Krinkle>	 What's up with the vm? I've tried all I know how to.
[14:13:47] <Krinkle>	 It's still broken.
[14:13:50] <Krinkle>	 Has been for almost 24 hours now
[14:13:53] <hashar>	 integration-slave01 ? 
[14:13:54] <Krinkle>	 We need it back up asap
[14:13:57] <hashar>	 that is for the npm job isn't it ?
[14:14:03] <Krinkle>	 Yep
[14:14:13] <Krinkle>	 First failed 19 hours ago
[14:14:34] <Krinkle>	 https://integration.wikimedia.org/ci/job/VisualEditor-npm/27/console
[14:15:15] <Krinkle>	 I've rebooted the labs instance, no effect. I've disconnected and re-launched client agent from Jenkins config panel, no effect.
[14:15:24] <Krinkle>	 I can connect to it with ssh fine.
[14:15:37] <hashar>	 will look at it
[14:15:39] <Krinkle>	 The Java debug console jenkins provides in the web app also works fine
[14:15:48] <hashar>	 like now :D
[14:15:53] <Krinkle>	 Thx ):
[14:16:37] <hashar>	 Krinkle: I think I already had such an issue, might be rights related
[14:17:09] <Krinkle>	 When I ran git commands manually through ssh/proxy/root/jenkins-deploy, worked fine.
[14:19:30] <hashar>	 hmm
[14:19:44] <hashar>	 00:00:00.673 Caused by: java.lang.NoClassDefFoundError: hudson/plugins/git/GitSCMBackwardCompatibility
[14:19:44] <hashar>	 00:00:00.674 Caused by: java.lang.ClassNotFoundException: hudson.plugins.git.GitSCMBackwardCompatibility
[14:22:59] <hashar>	 Krinkle: going to restart jenkins
[14:28:40] <zeljkof>	 The page at https://www.mediawiki.org says: Error loading data from server: readonly: The wiki is currently in read-only mode. Would you like to retry?
[14:28:55] <zeljkof>	 hashar: ^
[14:29:06] <zeljkof>	 and anybody else, do you know what is going on?
[14:30:33] <zeljkof>	 editing works fine with source editor
[14:30:49] <zeljkof>	 ok, visual editor is back too
[14:31:02] <hashar>	 zeljkof: might happen from time to time
[14:31:08] <hashar>	 zeljkof: maybe a db connection got dropped
[14:35:30] <zeljkof>	 hashar: saw it for the first time
[15:10:51] <grrrit-wm>	 (03PS32) 10Zfilipin: Added a test for downloading pdf from a random page [qa/browsertests] - 10https://gerrit.wikimedia.org/r/98160 (owner: 10Mayankmadan)
[15:13:58] <grrrit-wm>	 (03CR) 10Zfilipin: [C: 04-1] Test downloading PDF from a random page (033 comments) [qa/browsertests] - 10https://gerrit.wikimedia.org/r/98160 (owner: 10Mayankmadan)
[15:17:46] <grrrit-wm>	 (03CR) 10Zfilipin: "Additionally, tests cause an deprecation warning:" [qa/browsertests] - 10https://gerrit.wikimedia.org/r/98160 (owner: 10Mayankmadan)
[15:25:20] <grrrit-wm>	 (03CR) 10Zfilipin: [C: 04-1] "Something is wrong with this commit:" [qa/browsertests] - 10https://gerrit.wikimedia.org/r/105179 (owner: 10Mayankmadan)
[16:01:37] <grrrit-wm>	 (03PS1) 10Hashar: mwconf: set $wgHTTPProxy based on wmf site [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105198 
[16:01:54] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] mwconf: set $wgHTTPProxy based on wmf site [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105198 (owner: 10Hashar)
[16:06:09] <grrrit-wm>	 (03Merged) 10jenkins-bot: mwconf: set $wgHTTPProxy based on wmf site [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105198 (owner: 10Hashar)
[16:14:09] <grrrit-wm>	 (03CR) 10Hashar: "deployed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/105198 (owner: 10Hashar)
[16:26:03] <grrrit-wm>	 (03PS1) 10Hashar: remove phpcs jobs from mw/core gating [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105204 
[16:26:16] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] remove phpcs jobs from mw/core gating [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105204 (owner: 10Hashar)
[16:27:42] <grrrit-wm>	 (03Merged) 10jenkins-bot: remove phpcs jobs from mw/core gating [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105204 (owner: 10Hashar)
[16:37:31] <grrrit-wm>	 (03CR) 10Subramanya Sastry: "Thanks!" [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/105151 (owner: 10Hashar)
[17:28:52] <Krinkle>	 hashar: Anything going on still?
[17:29:15] <Krinkle>	 I'm still getting various Jenkins problems that are causing trouble for VisualEditor, I could look into it but want to make sure Im not duplicating anything you intend to address already
[17:29:20] <Krinkle>	 https://integration.wikimedia.org/ci/job/mwext-VisualEditor-qunit/7146/console
[17:29:32] <Krinkle>	 and mwext-VisualEditor-npm is not doing the submodule update properly
[17:35:08] <hashar>	 Krinkle: hey
[17:35:13] <hashar>	 Krinkle: ah that is bad :/
[17:37:12] <hashar>	 Krinkle: wiping workspace and reserving the job 
[17:37:32] <Krinkle>	 Checked manually and the submodule doesn't exist at all on slave01
[17:37:36] <Krinkle>	 not initialised yet
[17:37:48] <Krinkle>	 maybe it does try to do an update, but that won't do anything if the module hasn't even been initialised
[17:37:57] <Krinkle>	 lib/ve is an empty directory.
[17:40:08] <hashar>	 Krinkle: might be https://issues.jenkins-ci.org/browse/JENKINS-21057
[17:40:19] <hashar>	 workarounds:
[17:40:20] <hashar>	 Downgrade git plugin to 1.5
[17:40:20] <hashar>	 Do not check "Recursively update submodules"
[17:40:21] <hashar>	 Do not checkout git repository with submodules
[17:41:04] <Krinkle>	 hashar: Do note that the "No remotes" bug is a recent regression that I see only on gallium with -qunit jobs
[17:41:17] <Krinkle>	 Not related to the equally important regression in -npm jobs on slave01 with submodules not being initialised/updated.
[17:41:47] <hashar>	 that is the same issue isn't it ?
[17:41:54] <Krinkle>	 the -npm jobs aren't fatalling anywhere in Jenkins land, but in test execution itself when it is found that some random file is misssing which is supposed to come from a submodule.
[17:41:58] <Krinkle>	 No, not even close.
[17:42:19] <Krinkle>	 https://integration.wikimedia.org/ci/job/mwext-VisualEditor-qunit/7146/console is on gallium, and fatally failing because it can't find any remotes, it never starts regular job execution.
[17:42:52] <Krinkle>	 https://integration.wikimedia.org/ci/job/mwext-VisualEditor-npm/156/console is on slave01, and goes through git setup and job execution fine
[17:43:12] <Krinkle>	 but is failing because some of the tests are failing because there are files missing (lib/ve is an empty directory instead of an initialised and updated git submodule)
[17:43:24] <Krinkle>	 Make sense?
[17:46:14] <hashar>	 Krinkle: well
[17:46:27] <hashar>	 Krinkle: the only fix up would be to back up the git plugins :D
[17:48:15] <hashar>	 which mean restartingjenkins :(
[17:48:41] <hashar>	 Krinkle: seems like a plan for you ?
[17:49:48] <Krinkle>	 hashar: Well, I don't know any of that. Sorry. All I'm seeing is that Jenkins was working fine when I last used it a few days ago, and now it isn't doing submodules anymore for VE (possibly breaking other projects as well).
[17:49:57] <Krinkle>	 Did we upgrade something recently?
[17:50:14] <Krinkle>	 The -npm job isn't showing an error.
[17:50:25] <Krinkle>	 it looks like it just forgets to do the submodule stuff
[17:52:27] <Krinkle>	 hashar: If restarting is likely to fix it, by all means. It's important that VE can merge commits. We're all starting to come back from holiday, and it seems right now it is unconditionally breaking due to this. And we've adapted our workflow to no longer allow overriding with V+2 (only ops can now), so we really need it to work, by whatever hacky means :)
[17:52:39] <Krinkle>	 maybe we can try something else first though?
[17:52:45] <Krinkle>	 by back up, do you mean downgrade?
[17:53:03] <Krinkle>	 which plugins?
[17:53:45] <hashar>	 git plugin
[17:55:16] <hashar>	 Krinkle: I can't fix that issue tonight
[17:55:59] <hashar>	 according to the bug report, we have to downgrade the Jenkins "git client plugin"
[17:56:17] <hashar>	 which doesn't seem to be easy to do, probably have to compile the old version and upload it on jenkins
[17:58:51] <hashar>	 ah I got it on labs :-D
[17:59:30] <hashar>	 Krinkle: I can't work anymore sorry, wife + crying daughter home.  The only workaround is to make the jobs non voting
[17:59:40] <hashar>	 will get the plugin downgraded tomorrow when it is quiet at home
[18:00:03] <Krinkle>	 hashar: How did this happen do? What caused it?
[18:00:09] <Krinkle>	 though*
[18:00:20] <hashar>	 jenkins upgraded
[18:00:36] <Krinkle>	 Can we make sure to test jobs like these then?
[18:00:41] <Krinkle>	 Goes without saying of course :)
[18:01:03] <hashar>	 not sure what is impacted exactly :(
[18:07:41] <Krinkle>	 hashar: Well, VisualEditor can't run any tests or merge commits right now. I'd rather not have that situation for another 24 hours.
[18:08:19] <Krinkle>	 Something basic as git submodules should be tested for. I don't think this is limited to mwext-VisualEditor-npm.
[18:09:03] <hashar>	 gotta restart Jenkins now
[18:52:08] <chrismcmahon>	 fyi, the Mobile team will be debugging an issue with centralauth tokens on beta in a while, I aborted the Mobile tests in the build queue
[19:39:03] <grrrit-wm>	 (03PS1) 10Krinkle: Use use-remote-zuul-no-submodules for lint jobs [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/105244 
[20:15:41] <grrrit-wm>	 (03CR) 10Mayankmadan: [C: 04-1] "Doesnt work for me too." [qa/browsertests] - 10https://gerrit.wikimedia.org/r/103497 (owner: 10M4tx)
[21:46:03] <mayankmadan_>	 hi zeljkof 
[21:46:09] <mayankmadan_>	 its pretty late for you :)
[21:46:15] <zeljkof>	 hi mayankmadan_
[21:46:15] <chrismcmahon>	 hi manybubbles, do you know if Solr indexing is maybe messed up in beta labs?  we expected the coords in the page at http://en.wikipedia.beta.wmflabs.org/wiki/Tucson to be picked up in a Mobile/Nearby search and that does not seem to be happening.  JeffHall thinks it should have been picked up a few minutes ago, as the page is over 30 minutes old. 
[21:46:25] <zeljkof>	 mayankmadan_: it is even later for you :)
[21:46:43] <manybubbles>	 chrismcmahon: I don't know anything about the solr stuff, unfortunately
[21:46:55] <chrismcmahon>	 manybubbles: just wondered, thanks
[23:29:13] <grrrit-wm>	 (03PS1) 10Zfilipin: Updated Ruby, RubyGems and Firefox [qa/browsertests] - 10https://gerrit.wikimedia.org/r/105410 
[23:32:02] <grrrit-wm>	 (03CR) 10Cmcmahon: [C: 032] "maintenance" [qa/browsertests] - 10https://gerrit.wikimedia.org/r/105410 (owner: 10Zfilipin)
[23:37:40] <grrrit-wm>	 (03PS1) 10Zfilipin: Updated Ruby [selenium] - 10https://gerrit.wikimedia.org/r/105412 
[23:42:38] <grrrit-wm>	 (03CR) 10Cmcmahon: [C: 032] Updated Ruby [selenium] - 10https://gerrit.wikimedia.org/r/105412 (owner: 10Zfilipin)