[04:30:53] heya, just letting people know Flow is getting API failures on beta labs that shouldn't happen, it's as if a server has stale APC cache or an old deploy. No response yet in #wikimedia-labs channel. [05:11:19] 3Wikimedia / 3Quality Assurance: test2wiki's User:Selenium_user page with redirect and ShortUrl makes Flow browser tests fail - 10https://bugzilla.wikimedia.org/67788 (10spage) 3NEW p:3Unprio s:3normal a:3None Flow tests on test2wiki have never worked. I looked at one failure, https://integration.wi... [08:31:18] (03PS1) 10Hashar: Add Mantle as a dependency to MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145249 (https://bugzilla.wikimedia.org/67752) [08:36:39] (03PS2) 10Hashar: Add Mantle as a dependency to MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145249 (https://bugzilla.wikimedia.org/67752) [08:38:55] (03CR) 10Hashar: [C: 032] "Tests pass on https://gerrit.wikimedia.org/r/#/c/129335/" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145249 (https://bugzilla.wikimedia.org/67752) (owner: 10Hashar) [08:40:44] (03Merged) 10jenkins-bot: Add Mantle as a dependency to MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145249 (https://bugzilla.wikimedia.org/67752) (owner: 10Hashar) [08:41:47] 3Wikimedia / 3Continuous integration: MobileFrontend QUnit tests don't link Mantle as a dependency - 10https://bugzilla.wikimedia.org/67752#c3 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX a:3Antoine "hashar" Musso I have added Mantle as a dependency to any job that have MobileFrontend Jobs pass on https... [08:44:48] (03PS1) 10Hashar: qunit job for Flow [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145251 (https://bugzilla.wikimedia.org/67734) [08:45:38] (03PS1) 10Hashar: qunit job for Flow [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145252 (https://bugzilla.wikimedia.org/67734) [08:47:08] (03CR) 10Hashar: [C: 032] qunit job for Flow [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145252 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [08:47:12] (03Merged) 10jenkins-bot: qunit job for Flow [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145252 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [08:50:56] (03PS1) 10Hashar: Make Flow qunit job non voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145253 (https://bugzilla.wikimedia.org/67734) [08:51:12] (03CR) 10Hashar: [C: 032] Make Flow qunit job non voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145253 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [08:51:16] (03Merged) 10jenkins-bot: Make Flow qunit job non voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145253 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [08:53:31] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c6 (10Antoine "hashar" Musso) 5PATC>3NEW The qunit job has 4 assertions failing https://integration.wikimedia.org/ci/job/mwext-Flow-qunit/1/console It is now triggered when a patch is pro... [08:53:39] (03CR) 10Hashar: [C: 032] qunit job for Flow [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145251 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [08:55:08] (03Merged) 10jenkins-bot: qunit job for Flow [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145251 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [10:26:36] hashar: may I remind you to review my jjb commits? :) [10:27:04] I promise to review and merge yours today, but I need my changes merged into master first, so I can rebase your commits [10:38:40] forgot about that zeljkof :D [10:38:46] I [10:39:24] lets look at https://gerrit.wikimedia.org/r/#/c/144940/ Deleted CentralAuth job [10:39:43] hashar: most of the commits, if not all are really simple [10:39:57] so that centralauth has been deleted already nice [10:40:23] next [10:40:23] (03CR) 10Hashar: [C: 032] "Already deleted :)" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144940 (owner: 10Zfilipin) [10:40:31] Moved mediawiki_user variable from jobs section to project section [10:40:33] https://gerrit.wikimedia.org/r/#/c/144942/ [10:41:10] (03CR) 10Hashar: [C: 032] "Good! Diff at https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/516/console confirms it is indeed a noop." [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144942 (owner: 10Zfilipin) [10:41:11] I like small commits [10:42:01] (03Merged) 10jenkins-bot: Deleted CentralAuth job [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144940 (owner: 10Zfilipin) [10:42:29] (03CR) 10Hashar: [C: 032] "Straightforward. Welcome Sauce!" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144944 (owner: 10Zfilipin) [10:42:57] (03Merged) 10jenkins-bot: Moved mediawiki_user variable from jobs section to project section [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144942 (owner: 10Zfilipin) [10:43:05] (03CR) 10Hashar: [C: 032] "Diff at https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/518/console confirms it is a noop" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144945 (owner: 10Zfilipin) [10:43:52] (03CR) 10Hashar: [C: 032] "Welcome Sauce!" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144956 (owner: 10Zfilipin) [10:44:04] (03Merged) 10jenkins-bot: Running all CirrusSearch jobs at Sauce Labs [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144944 (owner: 10Zfilipin) [10:45:02] (03Merged) 10jenkins-bot: Moved mediawiki_user variable from jobs section to project section [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144945 (owner: 10Zfilipin) [10:45:07] (03CR) 10Hashar: [C: 032] "The new job has the same failure the previous had. Not a big deal." [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144967 (owner: 10Zfilipin) [10:45:09] zeljkof: all straightforward [10:45:21] hashar: yes :) [10:45:21] zeljkof: I guess the ZeroBanner one can be migrated to sauce labs [10:45:34] sauce does not support phantomjs :) [10:45:41] https://saucelabs.com/platforms [10:45:45] (03Merged) 10jenkins-bot: WikiLove job runs at Sauce Labs [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144956 (owner: 10Zfilipin) [10:45:53] but maybe we can fake the headers with another browser, will try [10:46:18] well [10:46:32] maybe migrate it to firefox ? [10:46:43] hashar: probably [10:47:02] but I have forgot were we able to inject headers for another browser, or just for phantomjs [10:47:08] but it is on the list [10:48:38] (03Merged) 10jenkins-bot: Tests moved from ZeroRatedMobileAccess to ZeroBanner repository [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144967 (owner: 10Zfilipin) [10:50:35] zeljkof: heading out for a nap [10:51:01] hashar: I will rebase and merge your commits, they look good, but were conflicting with master [10:51:51] need to look at them carefully [10:51:55] the rebase might have some glitches [10:52:12] hashar: sure, will be carefull [11:29:01] 3Wikimedia / 3Quality Assurance: test2wiki's User:Selenium_user page with redirect and ShortUrl makes Flow browser tests fail - 10https://bugzilla.wikimedia.org/67788#c1 (10This, that and the other) (In reply to spage from comment #0) > 1. Can someone get rid of the redirect. Done. We can blame "User:Block... [12:03:03] (03PS2) 10Zfilipin: WIP Upload the language screenshots [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/144937 (https://bugzilla.wikimedia.org/62737) [12:06:43] zeljkof: [12:06:57] hashar: what's up? [12:07:02] added you to loomio :-] [12:07:08] hashar: thanks :) [12:07:12] don't you join the -staff irc channel? [12:08:13] hashar: I should be there [12:08:20] yeah sorry [12:10:55] hashar: in a meeting with vikas, so I might not answer :) [12:11:07] aharoni: vikas and me are in the hangout [12:16:39] hashar: any idea why qunit tests are failing here? https://gerrit.wikimedia.org/r/#/c/145271/ [12:16:43] no clue [12:16:47] I have no clue how qunit works [12:16:51] https://integration.wikimedia.org/ci/job/mwext-VisualEditor-qunit/10499/console [12:17:07] retry ? [12:17:26] just comment "recheck" [12:22:15] hashar: trying, thans [12:22:16] thanks [12:26:28] vikas: we can not hear you in the hangout cc aharoni [12:32:53] zeljkof aharoni on my way rebooting .... [14:00:25] hashar: how I get permission to build this job => https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/ [14:05:02] vikas: what is your labs account? [14:05:28] hashar: vikassy [14:06:33] vikas: refresh the page [14:06:41] on the left you should now have a link saying "Build Now" [14:07:22] hashar: yup ! thank you :) [14:07:36] I need a LDAP group [14:07:36] :D [14:11:32] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734 (10Antoine "hashar" Musso) p:5Unprio>3Normal s:5normal>3enhanc [14:23:02] 3Wikimedia / 3Continuous integration: Jenkins: Figure out a long term solution to maintaining Jenkins configuration - 10https://bugzilla.wikimedia.org/67027#c2 (10Antoine "hashar" Musso) One can probably puppetize a chunk of it based on https://github.com/jenkinsci/puppet-jenkins. But I am opposed to have th... [14:25:17] (03PS1) 10Hashar: Qunit jobs for Popups extension [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145321 (https://bugzilla.wikimedia.org/67022) [14:28:01] (03CR) 10Hashar: [C: 032] Qunit jobs for Popups extension [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145321 (https://bugzilla.wikimedia.org/67022) (owner: 10Hashar) [14:28:43] (03Merged) 10jenkins-bot: Qunit jobs for Popups extension [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145321 (https://bugzilla.wikimedia.org/67022) (owner: 10Hashar) [14:45:11] (03PS1) 10Hashar: Templates for mw ext qunit and qunit-mobile jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145326 [14:55:00] (03PS1) 10Hashar: Apply qunit template where relevant [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145328 [14:56:06] (03PS1) 10Hashar: Qunit jobs for Popups extension [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145329 (https://bugzilla.wikimedia.org/67022) [14:58:50] (03CR) 10Hashar: [C: 032] Qunit jobs for Popups extension [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145329 (https://bugzilla.wikimedia.org/67022) (owner: 10Hashar) [14:59:16] (03CR) 10Hashar: [C: 032] Apply qunit template where relevant [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145328 (owner: 10Hashar) [14:59:33] (03CR) 10Hashar: [C: 032] Templates for mw ext qunit and qunit-mobile jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145326 (owner: 10Hashar) [14:59:38] (03Merged) 10jenkins-bot: Templates for mw ext qunit and qunit-mobile jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145326 (owner: 10Hashar) [14:59:43] (03Merged) 10jenkins-bot: Apply qunit template where relevant [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145328 (owner: 10Hashar) [14:59:46] (03Merged) 10jenkins-bot: Qunit jobs for Popups extension [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/145329 (https://bugzilla.wikimedia.org/67022) (owner: 10Hashar) [15:03:18] 3Wikimedia / 3Continuous integration: Jenkins: Create QUnit job for Extension:Popups - 10https://bugzilla.wikimedia.org/67022#c7 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX a:3Antoine "hashar" Musso Fixed! Test change: https://gerrit.wikimedia.org/r/#/c/145330/ Successful build: https://integration.wi... [15:06:17] 3Wikimedia / 3Continuous integration: Jenkins: Create QUnit job for Extension:Popups - 10https://bugzilla.wikimedia.org/67022#c8 (10Antoine "hashar" Musso) Forgot: for any change already in Gerrit, you can have the jobs rerun (and thus the qunit one) by commenting in Gerrit: 'recheck' [15:14:55] (03PS2) 10Zfilipin: Fixed folder where tests are located for WikidataBrowserTests repository [selenium] - 10https://gerrit.wikimedia.org/r/143609 [15:15:42] (03CR) 10Zfilipin: "Can anybody merge this? I hate to self-merge. :)" [selenium] - 10https://gerrit.wikimedia.org/r/143609 (owner: 10Zfilipin) [15:16:45] zeljkof: You don't need to recheck on newly-rebased items that already have C+2; it will just magically apply. [15:17:10] James_F: where did I do that? :) [15:17:37] zeljkof: https://gerrit.wikimedia.org/r/144451 just now. [15:18:09] I just did a recheck there, did not do rebase [15:18:17] I see now that it needs a rebase too :) [15:18:50] James_F: ok, no auto rebase, conflicts [15:19:25] zeljkof: Yeah, well, it needs to wait for the other three to land first anyway. :-) [15:19:43] James_F: wait, what? [15:19:56] This one? https://gerrit.wikimedia.org/r/#/c/144451/ [15:20:08] it should not depend on anything [15:20:21] zeljkof: Yeah, and I'll need to re-work the qunit jobs too, I think. [15:20:43] zeljkof: https://gerrit.wikimedia.org/r/#/c/144606/ and https://gerrit.wikimedia.org/r/#/c/144607/ and https://gerrit.wikimedia.org/r/#/c/144608/ [15:20:52] James_F: that was my next question, how to fix the build after the rename :) [15:21:06] Don't worry, I'll get it done. [15:21:22] Just right now we're in the middle of landing a major refactor so we don't want to break that. :-) [15:21:36] James_F: no rush [15:22:00] this is not urgent, but it would be great if we did it one of these days [15:22:02] * James_F nods. [15:23:34] (03CR) 10KartikMistry: [C: 032] Fixed folder where tests are located for WikidataBrowserTests repository [selenium] - 10https://gerrit.wikimedia.org/r/143609 (owner: 10Zfilipin) [15:23:36] (03Merged) 10jenkins-bot: Fixed folder where tests are located for WikidataBrowserTests repository [selenium] - 10https://gerrit.wikimedia.org/r/143609 (owner: 10Zfilipin) [15:38:29] (03CR) 10Zfilipin: "Thanks Kartik. :)" [selenium] - 10https://gerrit.wikimedia.org/r/143609 (owner: 10Zfilipin) [15:56:39] (03PS2) 10Zfilipin: Normalize ZeroRatedMobileAccess job [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/131048 (owner: 10Hashar) [16:01:00] (03PS3) 10Zfilipin: Normalize ZeroRatedMobileAccess job [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/131048 (owner: 10Hashar) [16:07:09] (03PS4) 10Zfilipin: Normalize ZeroBanner job [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/131048 (owner: 10Hashar) [16:07:29] zeljkof: I am off for today [16:07:38] but can review that tomorrow if that can wait :D [16:07:41] else just merge hehe [16:07:50] hashar: tomorrow is fine [16:07:53] there is no rush [16:07:59] great! [16:07:59] I have just finished Zero job refactoring [16:08:07] I am also done for today [16:09:09] wonderful! [18:15:19] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c8 (10Jon) You're a superstar! I have no idea why they are failing though. They pass fine for me locally... any ideas? [19:00:17] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c9 (10Antoine "hashar" Musso) Oh I would be a superstar if I had proposed a patch to fix all of the tests :-D The job is setup to fetch some dependencies: AbuseFilter,SpamBlacklist,CheckUser,... [19:01:10] (03PS1) 10Hashar: Flow qunit job depends on MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145405 (https://bugzilla.wikimedia.org/67734) [19:04:24] (03CR) 10Hashar: [C: 032] Flow qunit job depends on MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145405 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [19:06:52] (03Merged) 10jenkins-bot: Flow qunit job depends on MobileFrontend [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/145405 (https://bugzilla.wikimedia.org/67734) (owner: 10Hashar) [19:08:17] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c12 (10Antoine "hashar" Musso) Adding MobileFrontend as a dependency does not change much https://integration.wikimedia.org/ci/job/mwext-Flow-qunit/16/consoleFull [19:47:19] I have updated Jenkins gearman plugin to 0.0.7 [19:47:23] might screw up something [20:03:33] "might screw up something" he says hours before leaving on vacation :) [20:10:01] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c13 (10Jon) MobileFrontend is not a dependency... the test that seems to fail is in core which is what is confusing me. Out of interest shouldn't we just run tests against the Flow group? [20:17:36] !log deployment-prep the db upgrade / scap jobs in Jenkins were broken somehow. Seems fixed now [20:17:39] bd808: ^^:D [20:18:03] * bd808 looks [20:18:11] at least scap is unbroken [20:18:14] Sam was scapping manually a bit a ago [20:18:26] there was an issue with Jenkins [20:18:36] I think the gearman plugin does not handle properly the multi configuration jobs [20:18:50] they end up locking the executors on the dpeloyment-bastion [20:19:02] and if there is no executor left, the jobs queue up :/ [20:19:30] will try the update database job [20:21:12] My guess would be that the 2 scap runs that failed happened between Sam merging https://gerrit.wikimedia.org/r/#/c/145404/ and him setting up the right files on beta. [20:26:38] I guess [20:27:28] bd808: I think it is all good now. We will see tomorrow morning when the browser tests scream [20:27:48] * bd808 waves [20:28:18] ho I am going to be around for a while. Doing my fiscal year accounting [21:39:31] 3Wikimedia / 3Continuous integration: Setup QUnit tests on Jenkins for Flow - 10https://bugzilla.wikimedia.org/67734#c14 (10Antoine "hashar" Musso) 5PATC>3NEW > Out of interest shouldn't we just run tests against the Flow group? I don't think we can do that. We will middle term have a common qunit job... [22:13:15] what is the process for requesting shell access to the beta cluster? [22:13:52] I want to test https://gerrit.wikimedia.org/r/#/c/135008/ [22:38:23] tgr: I added you to the beta project [22:38:35] bd808: thanks! [22:38:44] deployment-bastion is the "tin" there [22:39:42] I gave you admin rights so you can sudo whatever you need to [22:40:45] bd808: so bastion.wmflabs.org -> deployment-bastion -> deployment-XX? [22:41:02] I still get permission denied for deployment-bastion though [22:41:30] I just pushed the buttons a minute ago, maybe ldap cache somewhere? [22:41:33] nevermind, seems to be an ssh config issue [22:41:56] You can ssh from bastion.wmflabs directly to any of the deployment-* hsots [22:41:58] hosts [22:42:27] But if you want to mess with code/config that comes from deployment-bastion [22:42:43] logs are in /data/project/logs on all hosts (nfs) [23:05:43] bd808: I am trying to test thumbnail generation [23:06:05] https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/Help suggests the box for that should be called deployment-imagescaler01, but that does not seem to exist [23:06:20] do you maybe remember the correct name? [23:07:04] I *think* that in beta images are converted on deployment-apache0[12] [23:07:50] The thumbnail pipeline in beta is a bit of a mess [23:08:24] Ouch. That page you found is way out of date [23:43:09] bd808: I have loaded http://upload.beta.wmflabs.org/wikipedia/en/thumb/4/4d/Snowman.JPG/320px-Snowman.JPG ... [23:43:50] then ssh-ed to deployment-apache01/02, checked /data/project/upload7/wikipedia/en/thumb/4/4d/Snowman.JPG/ and the thumbnail is not there [23:43:54] what am I missing? [23:46:46] tgr: Hmm headers seem to indicate that `http://10.68.16.16/w/thumb.php?f=Snowman.JPG&width=320` populated varnish. Track down that host? [23:56:49] bd808: that's deployment-cache-text02.eqiad.wmflabs but it also does not have the thumbnail [23:58:15] tgr: Yeah. Login there and look at /etc/varnish/upload-backend.inc.vcl maybe. I started to but got pulled into a conversation with rob.la