[00:20:33] !log deployment-tin not accepting jobs for some time, ran through https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update, is back now [00:20:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [00:31:19] !log running puppet on beta to update scap to latest packaged version: sudo salt -b '10%' -G 'deployment_target:scap/scap' cmd.run 'puppet agent -t' [00:31:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [01:19:18] Hi it seems rake-jessie is not loading at https://integration.wikimedia.org/zuul/ and causing queue [01:25:27] 10Continuous-Integration-Infrastructure, 10Parsoid: Intermittent jenkins failures with tar and disk related errors. - https://phabricator.wikimedia.org/T128032#2062169 (10Arlolra) Just now https://integration.wikimedia.org/ci/job/parsoidsvc-source-npm-0.10/4250/console [01:25:55] paladox: it all looks good to me. maybe just a hiccup while new vms were spawned? [01:33:50] (03PS3) 10Legoktm: Add npm-node-4.3 test to experimental: in a few templates [integration/config] - 10https://gerrit.wikimedia.org/r/273136 (owner: 10Paladox) [01:34:39] (03CR) 10Legoktm: [C: 032] "I removed the mediawiki-core-npm-node-4.3 job because it doesn't exist in jjb yet, and it looked non-trivial to create. Plus I don't think" [integration/config] - 10https://gerrit.wikimedia.org/r/273136 (owner: 10Paladox) [01:35:37] (03Merged) 10jenkins-bot: Add npm-node-4.3 test to experimental: in a few templates [integration/config] - 10https://gerrit.wikimedia.org/r/273136 (owner: 10Paladox) [01:37:43] !log deploying https://gerrit.wikimedia.org/r/273136 [01:37:48] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [01:38:31] Change-Id: Id1dba501888b904656f8b70eb0f0ba923c5e1680 [01:38:31] grr [01:39:15] (03CR) 10Legoktm: "Actually deploying to zuul now :)" [integration/config] - 10https://gerrit.wikimedia.org/r/272955 (owner: 10Paladox) [01:39:36] !log deploying https://gerrit.wikimedia.org/r/272955 (undeployed) and https://gerrit.wikimedia.org/r/273136 [01:39:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [01:39:54] James_F: ^ you can test it now [01:41:36] legoktm: Ha. The OOjs and OOjs UI aren't in that. Works for mediawiki/extensions/TemplateData though. [01:41:47] * legoktm facepalms [01:42:48] (03PS1) 10Legoktm: Add 'npm-node-4.3' as experimental for the 'npm' template [integration/config] - 10https://gerrit.wikimedia.org/r/273170 [01:43:18] +INFO:zuul.IndependentPipelineManager: oojs/ui [01:43:18] +INFO:zuul.IndependentPipelineManager: [01:43:26] (03CR) 10Legoktm: [C: 032] Add 'npm-node-4.3' as experimental for the 'npm' template [integration/config] - 10https://gerrit.wikimedia.org/r/273170 (owner: 10Legoktm) [01:44:23] (03Merged) 10jenkins-bot: Add 'npm-node-4.3' as experimental for the 'npm' template [integration/config] - 10https://gerrit.wikimedia.org/r/273170 (owner: 10Legoktm) [01:44:31] !log deploying https://gerrit.wikimedia.org/r/273170 [01:44:33] legoktm: https://integration.wikimedia.org/ci/job/npm-node-4.3/47/ tah-dah. [01:44:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [01:45:05] James_F: oojs/ooui repos now have it [01:45:22] and yay :D [01:45:26] Yes. [01:45:29] https://integration.wikimedia.org/ci/job/npm-node-4.3/48/ [01:46:12] Yay, no "20:31:42 WARN engine hawk@3.1.3: wanted: {"node":">=0.10.32"} (current: {"node":"0.10.25","npm":"2.14.12"}) " etc. [01:50:29] legoktm: However, https://integration.wikimedia.org/ci/job/npm-node-4.3/48/consoleFull very definitely hasn't finished yet… [01:50:43] hmm. [01:50:56] well, there's a 10 minute timeout on it [01:51:00] and we're 5min in [01:51:03] https://integration.wikimedia.org/ci/job/npm/54773/consoleFull took 16 minutes though. [01:51:22] Why 10? [01:51:32] oh, er, the timeout is 10 minutes to clone from git [01:51:37] job timeout is 30 [01:51:48] <-- distracted [01:51:59] Kk. [01:52:14] Maybe the npm cache for 4.3 is not very warm? [01:53:08] probably [01:53:39] 01:45:16 + rsync --archive --compress --contimeout 3 rsync://castor.integration.eqiad.wmflabs:/caches/oojs-ui/master/npm-node-4.3/ /home/jenkins [01:53:39] 01:45:17 rsync: change_dir "/oojs-ui/master/npm-node-4.3" (in caches) failed: No such file or directory (2) [01:53:44] yeah, that sounds like there is no cache at all [01:53:57] Well, that may take a 'while' then. :-) [01:54:08] OOjs UI, constantly pushing the envelope. [01:54:41] For our next trick, CI integration with Phabricator-based code review. [01:55:35] legoktm: If it passes and you're not busy, throw the switch? Otherwise I'll pester hashar tomorrow. ;-) [01:55:55] sure [02:01:54] 01:45:40 npm WARN optional dep failed, continuing fsevents@1.0.8 [02:01:54] 01:58:06 npm WARN deprecated graceful-fs@3.0.8: graceful-fs version 3 and before will fail on newer node releases. Please update to graceful-fs@^4.0.0 as soon as possible. [02:01:57] it's getting somewhere [02:08:04] James_F|Away: lol, it doesn't work because the demos need composer to be installed. [02:20:02] (03CR) 10Legoktm: [C: 04-1] "https://integration.wikimedia.org/ci/job/npm-node-4.3/48/console failed because the jessie disposable slaves don't have composer installed" [integration/config] - 10https://gerrit.wikimedia.org/r/273135 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [02:50:52] 3Scap3, 10scap, 7WorkType-NewFunctionality: [Spike] Benchmark built-in HTTP server options for scap3 fanout - https://phabricator.wikimedia.org/T127733#2062349 (10thcipriani) Played around with an idea this evening. Wrote a python script that spun up 11 DigitalOcean Droplets. On droplet 1 I ran: ``` git cl... [03:06:06] 6Release-Engineering-Team, 3Scap3, 10scap: create an app to audibilize logstash events - https://phabricator.wikimedia.org/T123419#2062384 (10mmodell) 5Open>3stalled [03:12:02] Project beta-scap-eqiad build #91041: 04FAILURE in 7 min 20 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/91041/ [03:21:05] Yippee, build fixed! [03:21:06] Project beta-scap-eqiad build #91042: 09FIXED in 6 min 20 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/91042/ [03:38:29] 3Scap3, 10scap, 7WorkType-NewFunctionality: [Spike] Benchmark built-in HTTP server options for scap3 fanout - https://phabricator.wikimedia.org/T127733#2062418 (10mmodell) Wow that's pretty sad [08:22:33] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/273136 (owner: 10Paladox) [09:35:04] `git fetchreviews' is aliased to `fetch -v origin refs/notes/review:refs/notes/review' [09:35:04] `git codereview' is aliased to `log --decorate --notes=review' [09:35:27] so on a repo you can refresh review notes with 'git fetchreviews' [09:35:35] and use 'git codereview' as a substitute for git log [09:35:36] :D [09:48:16] hashar: thanks :) [09:51:02] hashar: what is the convention on cherry picks? [09:51:09] should I +2 this or you? :) https://gerrit.wikimedia.org/r/#/c/273190/ [09:51:13] it is all green [10:08:19] !log Jenkins upgraded T128006 [10:08:20] T128006: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006 [10:08:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:08:37] zeljkof: what do you mean by convention ? [10:09:03] for wmf branches, if you +2 it you deploy it :-} [10:16:10] hashar: hm [10:16:43] but this is a test, do we deploy that code' [10:16:46] ? [10:20:06] yeah [10:20:27] else whenever someone want to cherry pick / deploy a change for Multimedia viewer it will have that patch lying around [10:20:30] and wonder what it is about [10:20:45] at best it revert it or stop pending clarification [10:20:59] will look at how to do it [10:21:10] been a while since I have deployed any extension bump [10:33:49] hashar: did you just update jenkins? [10:33:57] yeah [10:34:02] zeljkof: what is broken ? [10:34:03] there is a new search box in the list of job runs [10:34:10] oh [10:34:17] nothing broken, new feature noticed :) [10:34:33] ahah [10:34:43] that is neat [10:58:26] hashar: I was able to reproduce "ve.init is undefined" for VE tests [10:58:33] it happens only when using sauce :| [10:58:33] great!!!!!!!!!!!!!!! [10:58:38] :( [10:58:55] maybe try locally with a cold cache ? [11:01:15] hashar: no, there is something in the code I wrote that breaks only when using sauce labs [11:01:19] investigating [11:03:09] 7Browser-Tests, 10VisualEditor, 13Patch-For-Review, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Disable VisualEditor scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily - https://phabricator.wikimedia.org/T94162#2062849 (10zeljkofilipin) [11:03:12] 7Browser-Tests, 10VisualEditor, 13Patch-For-Review, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix `ve.init is undefined` and `ve.init.target is undefined` error messages in VisualEditor browser tests - https://phabricator.wikimedia.org/T126966#2062847 (10zeljkofilipin) 5Resolved>3Open The problem is stil... [11:21:03] !log deleting workspace /mnt/jenkins-workspace/workspace/browsertests-Wikidata-WikidataTests-linux-firefox-sauce on slave-trusty-1015 [11:21:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:21:49] hashar: registration is broken at beta [11:21:51] reporting [11:22:10] https://integration.wikimedia.org/ci/view/BrowserTests/view/GettingStarted/job/browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/769/testReport/%28root%29/Modal%20on%20editable%20returnto%20page/User_sees_modal_call_to_action_after_registration/ [11:25:02] well, probably not broken [11:25:07] but main page after registration is [11:30:42] hashar: https://phabricator.wikimedia.org/T128063 [11:43:40] 10Beta-Cluster-Infrastructure: deployment-mediawiki02 lost memcached access at 11:51am UTC - https://phabricator.wikimedia.org/T127966#2062920 (10Joe) [11:43:43] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team, 6Operations, 13Patch-For-Review, 7Puppet: deployment-tin puppet Error 400 on SERVER: Failed to parse template nutcracker/nutcracker.yml.erb - https://phabricator.wikimedia.org/T127845#2062919 (10Joe) 5Open>3Resolved [11:43:45] 10Continuous-Integration-Infrastructure, 7Regression: Puppet failure: "Exec[install_alternative_php] path slave-scripts/bin/php doesn't exist" - https://phabricator.wikimedia.org/T126498#2062921 (10Aklapper) @Legoktm: Any news to share from the last two weeks? [11:47:10] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: missing configuration for `browser_useragent` (MediawikiSelenium::ConfigurationError) - https://phabricator.wikimedia.org/T128064#2062922 (10zeljkofilipin) [11:54:25] hashar: if you have a minute, this will fix a lot of MMV tests, I promise ™️ ;) https://gerrit.wikimedia.org/r/#/c/273203/ [11:55:36] again ? [11:55:37] ;D [11:55:45] still cherry picking / deploying the other change [11:56:14] well, the last commit did fix _a_ problem [11:56:22] this fixes another one, fixing a lot of jobs [11:56:29] there is still at least one problem left :( [11:56:36] but let's fix this one now [11:56:43] if merged in master, can you run it against beta cluster to confirm ? [11:56:54] then cherry pick to .wmf14 branch [11:57:01] will get it +2 / merged [11:57:37] hashar: yes, as soon as it is in master, I can trigger the tests [11:57:48] oh the patch is for master [11:58:16] why dont we have a default browser user agent set? [11:58:29] this one wants a custom user agent :| [11:58:30] that should be set per convention [11:58:42] oh the tests require it ? [11:58:55] yes, the tests require a custom user agent, not sure why [11:59:33] should it be set in the test instead? [12:00:02] well, the tests insist for the user agent to be set in environment [12:00:05] who knows why [12:00:20] not sure that it makes any sense, just want to get them green for now :) [12:02:23] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Infrastructure, 10Wikidata: Wikidata sauce browser tests time out - https://phabricator.wikimedia.org/T128065#2062946 (10adrianheine) [12:04:00] zeljkof: will look later [12:04:10] already doing too many context switching [12:04:29] hashar: no rush [12:04:34] when ever you have the time [12:10:46] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Infrastructure, 10Wikidata: Wikidata sauce browser tests time out - https://phabricator.wikimedia.org/T128065#2062971 (10hashar) Namely: | Build | Start | Duration |--|--|-- | [[https://integration.wikimedia.org/ci/job/browsertests-Wikidata-WikidataTe... [12:15:31] zeljkof: I am done with the previous patch deployment [12:16:00] hashar: feel free to jump on the next one :) [12:16:10] gotta run the test now :D [12:16:17] I am debugging why VE fails when using sauce :| [12:16:23] oh, yes, will do [12:17:01] https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/885/console [12:17:08] fails because of the browser_useragent stuff hehe [12:17:20] yes [12:17:39] to fix that, merge this https://gerrit.wikimedia.org/r/#/c/273203/ [12:18:01] I think mediawiki.org job will still fail, but that should be fixed with editing jjb job config [12:18:07] tests/browser/features/step_definitions/mmv_performance_steps.rb: browser_factory.override(browser_user_agent: env[:browser_useragent]) [12:18:21] can't we just hardcode test-user-agent there ? [12:19:00] seems to have been introduced in 66b0020525f38c62463830a8038003cb4c8b26cc [12:19:14] for https://phabricator.wikimedia.org/T125072 undefined method `test_name' for # (NoMethodError) [12:19:27] oh no it is from a previous change [12:20:10] hashar: we could do a lot of stuff [12:20:17] not sure why they want user agent to be configurable [12:20:27] that is from 2c3ac55a673ec5aba0080bb73789f2ea1110bd93 [12:20:35] - @browser = browser(test_name(@scenario), user_agent: ENV["BROWSER_USERAGENT"]) [12:20:35] + browser(test_name(@scenario), user_agent: env[:browser_useragent]) [12:20:46] but forgot to update the env thingy [12:21:13] I think that was a way for them to manually run using a different agent [12:21:33] and potentially previously we exposed BROWSER_USERAGENT [12:25:03] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 6Revision-Scoring-As-A-Service, 13Patch-For-Review: Deploy ORES extension to beta cluster - https://phabricator.wikimedia.org/T127661#2062975 (10Ladsgroup) I think so :) [12:26:06] zeljkof: can you look at the new commit message on https://gerrit.wikimedia.org/r/#/c/273203/ [12:26:08] I rephrased it [12:26:23] looking... [12:28:06] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Infrastructure, 10Wikidata: Notify about aborted jobs - https://phabricator.wikimedia.org/T128067#2062976 (10adrianheine) [12:31:56] hashar: +1 from me :) [12:32:03] care to +2 it? [12:32:59] and cherry picked [12:33:11] lets wait for beta to catchup [12:33:12] great, thanks [12:33:24] beta? [12:33:25] then we can run a beta labs browser test and see whether it turns green :-} [12:33:39] well on +2 the change get merged [12:33:44] why it needs to catch up? we just need to wait until the commit is merged [12:33:56] and beta runs a job every x minutes to git pull (a few minutes) and then scap (a few more minutes) [12:34:06] but we do not need to update beta [12:34:07] so it might take up to say 20 minutes for a change to actually land on beta [12:34:10] oh [12:34:11] just the tests [12:34:11] yeah [12:34:12] ... [12:34:20] you are paying more attention than me [12:34:25] :D [12:34:38] I think mw.org MMV job will still fail [12:34:52] because our shell script only cares about beta and test2 [12:34:52] so got merged [12:34:59] time to trigger a beta labs browser test !!! [12:35:00] so we will have to update jjb shell script [12:35:03] will do [12:36:52] the jobs are running [12:42:09] and there is a nice one from adrian https://phabricator.wikimedia.org/T128065 :( [12:42:56] oh you triggered them all [12:43:07] I usually run a single one first to confirm [12:43:15] that emits less spam this way [12:47:32] hashar: I am _so_ sure they will all pass ;) [12:47:46] mw.org one failed, as expected https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/ [12:54:02] yeah tis missing the cherry pick [13:02:38] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2061064 (10hashar) I have screwed up my !log ... So that message is a notifier for recent changes which are sent over UDP to some service which, in production, relay to irc.wikime... [13:05:11] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063052 (10hashar) For beta we have in InitialiseSettings-labs.php: ``` '-wmgRC2UDPPrefix' => array( 'default' => false, ), ``` References: * {T62013}... [13:09:08] Yippee, build fixed! [13:09:08] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce build #330: 09FIXED in 33 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce/330/ [13:09:09] Yippee, build fixed! [13:09:09] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #887: 09FIXED in 33 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/887/ [13:09:50] hashar: MMV is greener already! :) https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/ [13:09:56] (some jobs still running) [13:10:08] Yippee, build fixed! [13:10:08] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #708: 09FIXED in 34 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/708/ [13:10:23] Yippee, build fixed! [13:10:24] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #735: 09FIXED in 34 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/735/ [13:15:34] ah great [13:16:29] Yippee, build fixed! [13:16:30] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #728: 09FIXED in 40 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/728/ [13:16:39] zeljkof: merging in the wmf.14 patch https://gerrit.wikimedia.org/r/#/c/273205/ [13:16:47] only two failing! [13:17:03] will disable scenarios that fail using IE [13:17:10] and that will move us to only one failing [13:17:24] and the last one should not be hard to fix too [13:17:24] which ones are failing ? [13:17:41] IE8 and mw.org [13:17:42] https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/ [13:18:02] timed out after 30 seconds, waiting for {:class=>"mw-mmv-close", :tag_name=>"div"} to become present (Watir::Wait::TimeoutError) [13:18:05] pff [13:18:10] maybe it is some timeout [13:18:13] it is [13:18:16] ie8 https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/722/testReport/junit/(root)/Multimedia%20Viewer%20performance/MMV_with_warm_cache_and_small_browser_window/ [13:18:25] who knows what is the problem, will disable for now [13:18:30] and let the team fix it [13:18:33] hold on [13:18:36] lets rebuild it [13:18:49] maybe ie8 just does not work anymore ;-} [13:18:58] probably :P [13:19:18] relaunched at https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/723/consoleFull [13:19:27] some javascript error [13:19:34] yeah [13:19:40] might be worth reaching them and see whether they still support IE 8 / might want to fix it [13:20:03] will report the problem [13:21:05] the wmf.14 cherry pick will land soonish [13:22:19] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily - https://phabricator.wikimedia.org/T94150#2063106 (10zeljkofilipin) [13:23:06] hashar: Could you install composer on npm-node-4.3 please. Per https://gerrit.wikimedia.org/r/#/c/273135/ [13:23:17] paladox: no [13:23:26] hashar: Oh ok. [13:23:28] npm node 4.3 is to run 'npm test' on nodepool instances [13:23:29] not composer [13:23:37] and the instances dont even have PHP! [13:24:18] zeljkof: gahhhh https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/888/console [13:24:28] And I am at a wiki article with at least two embedded pictures # features/step_definitions/mmv_steps.rb:3 [13:24:28] 00:00:08.320 757: unexpected token at ' [13:24:30] 00:00:08.320 [13:24:30] 00:00:08.320 301 Moved Permanently [13:24:33] 00:00:08.320

The document has moved here.

[13:24:51] hashar: Oh just that oojs/ui fails with the composer message at https://integration.wikimedia.org/ci/job/npm-node-4.3/48/consoleFull [13:25:12] too bad for them :-} [13:25:26] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily - https://phabricator.wikimedia.org/T94150#2063112 (10zeljkofilipin) [13:25:29] hashar: Oh ok. [13:25:58] hashar: I know how to fix it [13:25:59] will do [13:27:44] James_F: oojs-ui carn't be upgraded to nodejs 4.3 because it uses composer and that is no longer supported in this new test. Per failure https://integration.wikimedia.org/ci/job/npm-node-4.3/48/console https://github.com/wikimedia/oojs-ui/blob/master/Gruntfile.js#L385 [13:29:20] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063116 (10hashar) Should be good now. Need confirmation though. [13:29:39] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2061064 (10hashar) [13:29:47] James_F You may want to create a seperate test for doing the compoer stuff so the stuff that runs composer from npm should use 0.10 and the other stuff that doesent should use the newer node version. [13:31:21] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2061064 (10hashar) ``` hashar@deployment-tin:~$ mwscript eval.php --wiki=enwiki > var_dump( $wmgUseRC2UDP ); bool(false) > ``` [13:31:24] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063123 (10JeanFred) >>! In T128006#2063116, @hashar wrote: > Should be good now. Need confirmation though. Confirmed. Thanks! [13:31:43] 10Beta-Cluster-Infrastructure: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063124 (10JeanFred) 5Open>3Resolved [13:31:51] 10Beta-Cluster-Infrastructure, 7WorkType-Maintenance: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063127 (10hashar) a:3hashar [13:33:00] paladox: that is an experimental job right ? [13:33:15] hashar: Yes. [13:33:21] paladox: good catch ;-} [13:33:27] since james asked to migrate yesterday [13:33:51] hashar: Most mediawiki repos should be able to check if they upgrade as off today by writing check experimental. [13:34:31] hashar: I think using the old npm test for certain code in grunt should only be run throuhg npm 0.10 since composer wont work on the new experimental test. [13:39:12] (03PS1) 10Paladox: Add oojs-ui-npm to experimental: [integration/config] - 10https://gerrit.wikimedia.org/r/273226 [13:39:41] hashar: Ive created a special test to work around composer not being install on there https://gerrit.wikimedia.org/r/#/c/273226/ [13:39:53] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce Jenkins job broken - https://phabricator.wikimedia.org/T128072#2063153 (10zeljkofilipin) [13:43:03] hashar: It seems there is a big queue at https://integration.wikimedia.org/zuul/ [13:43:43] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce Jenkins job broken - https://phabricator.wikimedia.org/T128072#2063185 (10zeljkofilipin) [13:47:21] hashar: https://gerrit.wikimedia.org/r/#/c/273227/ will fix https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/ [13:52:35] * zeljkof is out of lunch ;) [13:59:17] zeljkof: that does not fix anything really :D [13:59:23] just hide stuff under the carpet ! ;D [14:11:48] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team, 6Operations, 13Patch-For-Review, 7Puppet: deployment-tin puppet Error 400 on SERVER: Failed to parse template nutcracker/nutcracker.yml.erb - https://phabricator.wikimedia.org/T127845#2063230 (10hashar) Thank you @joe ! [14:13:26] 10Beta-Cluster-Infrastructure, 7WorkType-Maintenance: Internal error when assigning user rights on Commons Beta - https://phabricator.wikimedia.org/T128006#2063234 (10hashar) [14:14:22] 3Scap3, 10scap: create an app to audibilize logstash events - https://phabricator.wikimedia.org/T123419#2063240 (10hashar) [14:14:56] (03PS2) 10Hashar: Replace jslint test with jshint and jsonlint [integration/config] - 10https://gerrit.wikimedia.org/r/273128 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [14:15:33] (03CR) 10Hashar: [C: 032] Replace jslint test with jshint and jsonlint [integration/config] - 10https://gerrit.wikimedia.org/r/273128 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [14:17:34] (03Merged) 10jenkins-bot: Replace jslint test with jshint and jsonlint [integration/config] - 10https://gerrit.wikimedia.org/r/273128 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [14:20:50] hashar: well, it _does_ fix the job ;) [14:23:27] (03CR) 10Hashar: "I have deleted jobs in Jenkins." [integration/config] - 10https://gerrit.wikimedia.org/r/273128 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [14:28:01] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: Fix MultimediaViewer scenarios that fail at en.wikipedia.beta.wmflabs.org or disable them from running daily - https://phabricator.wikimedia.org/T94157#2063275 (10zeljkofilipin) [14:29:08] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: Fix MultimediaViewer scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94157#1156464 (10zeljkofilipin) [14:33:32] hashar: what is wrong with job config history? https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/jobConfigHistory/ [14:33:39] there is not history for this job [14:35:49] (03PS1) 10Zfilipin: WIP Fix browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce [integration/config] - 10https://gerrit.wikimedia.org/r/273234 (https://phabricator.wikimedia.org/T94157) [14:37:41] zeljkof: it is garbage collected I guess [14:37:55] :( [14:37:58] I did a change recently to keep the history up to 90 days [14:38:30] https://integration.wikimedia.org/ci/configure look for "history" [14:38:40] then click Advanced button [14:41:47] thanks, makes sense [14:41:56] not sure when was the job last updated anyway [14:42:02] might be more than 90 days [14:47:53] zeljkof: I am off for rest of afternoon [14:48:02] might check in a bit later in the evening, nothing sure [14:48:03] hashar: enjoy! [14:48:10] and thanks for help! [14:48:17] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/273128 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [14:53:50] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2063344 (10zeljkofilipin) [14:57:06] (03PS1) 10Paladox: Add npm-node-4.3 test to experimental: in mediawiki/core [integration/config] - 10https://gerrit.wikimedia.org/r/273235 [14:57:23] hashar: Could you merge https://gerrit.wikimedia.org/r/273235 [14:57:23] please. [14:57:38] its todo with adding nodejs 4.3 to mediawiki/core to experiment with [14:58:07] paladox: he is off for the rest of the day [14:58:09] since we have it in experimental: for mediawiki/repos not all but most that use the extension unit tests templates or npm templates. [14:58:23] zeljkof: Ok thanks for telling me. [14:58:59] (03CR) 10Hashar: [C: 04-1] "I need to create a new zuul cloner macro that takes advantage of the local mirror existing on Nodepool instance." [integration/config] - 10https://gerrit.wikimedia.org/r/273235 (owner: 10Paladox) [14:59:06] paladox: it is a bit more complicated :D [14:59:13] got that in mind, will do soonish ™ [15:00:17] hashar: Oh ok, legoktm did say that he could not add that to mediawiki/core but i thought that was because i named it wrong but thanks for telling me. Is it because it has to be prefixed with mediawiki-core and run different to the normal tests. [15:00:44] apparently, he is still around ;) [15:01:02] hashar: Should i abandon the patch or keep it open. [15:04:29] hashar: If you have time could you merge this https://gerrit.wikimedia.org/r/#/c/273226/ please. [15:09:32] paladox: yeah just abandon [15:09:51] for the rest, poke other folk ;-} I am off for real now [15:09:53] hashar: Ok, hashar would this package https://www.npmjs.com/package/getcomposer work for oojs-ui [15:09:59] hashar: Ok. [15:10:23] (03Abandoned) 10Paladox: Add npm-node-4.3 test to experimental: in mediawiki/core [integration/config] - 10https://gerrit.wikimedia.org/r/273235 (owner: 10Paladox) [15:11:37] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-MultimediaViewer: browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce fails with "757: unexpected token" - https://phabricator.wikimedia.org/T128079#2063388 (10zeljkofilipin) [15:12:19] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#1446477 (10zeljkofilipin) [15:15:16] 7Browser-Tests, 6Collaboration-Team-Backlog, 10Flow, 13Patch-For-Review, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix Flow scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94153#2063427 (10zeljkofilipin) [15:15:58] 10Browser-Tests-Infrastructure, 10Reading-Web, 5WMF-deploy-2016-02-09_(1.27.0-wmf.13), 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix MobileFrontend scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94156#2063429 (10zeljkofilipin) [15:16:44] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer, 13Patch-For-Review, 5WMF-deploy-2016-02-16_(1.27.0-wmf.14), 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): missing configuration for `browser_useragent` (MediawikiSelenium::ConfigurationError) - https://phabricator.wikimedia.org/T128064#2063430 (10zeljkofi... [15:16:47] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer, 13Patch-For-Review: Fix MultimediaViewer scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94157#2063431 (10zeljkofilipin) [15:17:35] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2063432 (10zeljkofilipin) [15:18:23] 7Browser-Tests, 10VisualEditor, 13Patch-For-Review, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix VisualEditor scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94162#2063435 (10zeljkofilipin) [15:19:12] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#1670906 (10zeljkofilipin) [15:20:14] 10MediaWiki-Codesniffer, 10Possible-Tech-Projects: Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T89682#1103586 (10SCHENKERX) Hello, I am a student who are willing to join your CodeSniffer improvement project in the GSoC 2016. I have read the project source code from the Gi... [15:21:20] 6Release-Engineering-Team, 10scap: sync-wikiversions doesn't sync co-masters - https://phabricator.wikimedia.org/T128082#2063458 (10demon) [15:27:04] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2063495 (10zeljkofilipin) [15:27:05] 7Browser-Tests, 10MediaWiki-extensions-RelatedArticles, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix failed RelatedArticles Selenium Jenkins jobs - https://phabricator.wikimedia.org/T126589#2063494 (10zeljkofilipin) 5Open>3Resolved [15:29:35] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-RelatedArticles: browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce job fails with: unknown environment `beta` (MediawikiSelenium::ConfigurationError) - https://phabricator.wikimedia.org/T128084#2063500 (10zeljkofilipin) [15:30:26] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#1670936 (10zeljkofilipin) [15:49:14] 10MediaWiki-Codesniffer, 10Possible-Tech-Projects: Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T89682#2063597 (10EBernhardson) >>! In T89682#2055776, @Sumit wrote: >>>! In T89682#2046748, @EBernhardson wrote: >> In addition to phpcs, which is a frankly very minor start, I... [15:54:02] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs npm CI job to Nodepool - https://phabricator.wikimedia.org/T128089#2063605 (10Jdforrester-WMF) [15:55:08] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs-UI, 7WorkType-NewFunctionality: Migrate OOjs UI npm CI job to Nodepool - https://phabricator.wikimedia.org/T128091#2063634 (10Jdforrester-WMF) [15:55:36] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs-UI, 7WorkType-NewFunctionality: Provide composer on the nodepool servers so OOjs UI can use it in the npm job - https://phabricator.wikimedia.org/T128092#2063648 (10Jdforrester-WMF) [15:56:04] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10VisualEditor, 7WorkType-NewFunctionality: Migrate VisualEditor core npm CI job to Nodepool - https://phabricator.wikimedia.org/T128093#2063661 (10Jdforrester-WMF) [15:57:00] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 7WorkType-NewFunctionality: Provide the Chrome binary on the nodepool servers so OOjs can use it in testing - https://phabricator.wikimedia.org/T128090#2063675 (10Jdforrester-WMF) [15:57:03] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10VisualEditor, 7WorkType-NewFunctionality: Migrate VisualEditor core npm CI job to Nodepool - https://phabricator.wikimedia.org/T128093#2063661 (10Jdforrester-WMF) [15:57:08] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 7WorkType-NewFunctionality: Provide the Chrome binary on the nodepool servers so OOjs, VE core can use it in testing - https://phabricator.wikimedia.org/T128090#2063621 (10Jdforrester-WMF) [15:59:45] (03PS2) 10Jforrester: Migrate OOJS repos to npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/273135 (https://phabricator.wikimedia.org/T128089) (owner: 10Paladox) [16:00:08] (03CR) 10Jforrester: [C: 04-1] "Both parts of this are blocked – see the tasks. :-(" [integration/config] - 10https://gerrit.wikimedia.org/r/273135 (https://phabricator.wikimedia.org/T128089) (owner: 10Paladox) [16:01:32] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs npm CI job to Nodepool - https://phabricator.wikimedia.org/T128089#2063605 (10Paladox) Yep this is blocked. Reason, composer will not be installed on nodepool ins... [16:02:10] James_F: https://integration.wikimedia.org/ci/job/npm-node-4.3/59/console [16:03:09] paladox: … yes? [16:04:00] paladox: That's OOjs, which is blocked on Chrome or Chromium being available. [16:04:57] James_F: I think that will be easy to fix unless it was not included on purpose in the first place. Im not sure if the binary was installed or if we used the shell script from jenkins. [16:05:18] paladox: It's almost certainly an accident that Firefox is included already, but yeah. [16:07:21] James_F: Oh ok looking in macro.yaml, in npm it is set with a shell script and npm 4.3 dosent use that script. [16:08:20] Its npm-oid actually sorry for npm 4.3. But if we include the shell script it should bring chromium back. [16:08:41] * James_F shrugs. [16:08:46] I'm just a CI user, not a CI expert. :-) [16:08:58] James_F Ok. [16:16:34] (03PS1) 10Zfilipin: WIP Fix RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce [integration/config] - 10https://gerrit.wikimedia.org/r/273258 (https://phabricator.wikimedia.org/T128084) [16:17:06] Project browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #1: 04FAILURE in 22 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/1/ [16:21:20] Yippee, build fixed! [16:21:20] Project browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #2: 09FIXED in 47 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/2/ [16:21:36] Yippee, build fixed! [16:21:37] Project browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #54: 09FIXED in 1 min 5 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/54/ [16:22:58] 10MediaWiki-Codesniffer, 10Possible-Tech-Projects: Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T89682#2063755 (10polybuildr) This task needs a few updates. PHPCS is now voting in several repositories (including core, iirc). [16:24:05] I ordered some swag that I will bring to the hackathon -- https://phab.wmfusercontent.org/file/data/uocvg33dmxmplmqc3htx/PHID-FILE-aml3tjgbm2zdiyld5l5c/3owdnfnoa3vk3nzq/672349_r.png [16:26:25] RECOVERY - Puppet failure on integration-slave-precise-1012 is OK: OK: Less than 1.00% above the threshold [0.0] [16:28:11] Project browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #3: 04FAILURE in 10 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/3/ [16:28:17] (03PS2) 10Zfilipin: Fix RelatedArticles Selenium Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/273258 (https://phabricator.wikimedia.org/T128084) [16:29:57] 10Continuous-Integration-Config, 10Mathoid, 15User-mobrovac: Enable node4 for mathoid tests - https://phabricator.wikimedia.org/T124447#2063769 (10mobrovac) >>! In T124447#2050952, @hashar wrote: > The CI part itself seems to be done. I have no idea why `mediawiki/services/mathoid/deploy` fails tests though... [16:31:04] (03PS1) 10Paladox: Add npm-setup.sh script to npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) [16:35:38] (03CR) 10Bmansurov: [C: 031] Fix RelatedArticles Selenium Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/273258 (https://phabricator.wikimedia.org/T128084) (owner: 10Zfilipin) [16:38:18] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 13Patch-For-Review, 7WorkType-NewFunctionality: Provide the Chrome binary on the nodepool servers so OOjs, VE core can use it in testing - https://phabricator.wikimedia.org/T128090#2063621 (10Paladox) The patch above just a... [16:40:02] 10Continuous-Integration-Infrastructure: Run 'npm' job with Node 4 (instead of Node 0.10) - https://phabricator.wikimedia.org/T126774#2063807 (10Paladox) Please use check experimental Which will run nodejs 4.3 on mediawiki/* repos. Mediawiki/core does not support that yet. But any repo using the npm template c... [16:41:09] 10Browser-Tests-Infrastructure, 10Wikidata: Merge tests/browser/environments.yml and tests/browser/config/config.yml in WikidataBrowserTests - https://phabricator.wikimedia.org/T128097#2063809 (10zeljkofilipin) [16:50:24] (03PS1) 10Paladox: Add npm-node-4.3 and parsoidsvc-deploy-npm-node-4.3 to experimental: [integration/config] - 10https://gerrit.wikimedia.org/r/273271 (https://phabricator.wikimedia.org/T126992) [17:09:47] 10Browser-Tests-Infrastructure, 10Wikidata: Merge tests/browser/environments.yml and tests/browser/config/config.yml in WikidataBrowserTests - https://phabricator.wikimedia.org/T128097#2063887 (10zeljkofilipin) [17:15:19] Project browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #55: 04FAILURE in 8.3 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/55/ [17:33:46] bd808: hmm, sloppy cdb timestamp compare just went out, didn't seem to fix the cdbs being rebuilt :\ [17:34:01] see most recent sync-file in -operations [17:34:14] Have you done 2? [17:34:24] I kind of expected the first to still suck [17:34:39] nope, just did one [17:35:06] why would the first one suck? [17:35:53] going again now. Still seems to be rebuilding on mira :\ [17:51:35] bd808: blerg :( opposite this https://github.com/wikimedia/scap/blob/master/scap/tasks.py#L391 [17:53:06] opposite? Is isclose() not based on absolute value? [17:53:15] oh! [17:53:17] ffs [17:53:19] yeah [17:53:34] for want of a "not" the war was lost [17:54:17] I take it you didn't test that in beta cluster ;) [17:54:58] I pulled it to beta last night, but, no, didn't verify. Just made sure nothing broke catastrophically. [17:55:09] thcipriani: here's your sticker -- https://phab.wmfusercontent.org/file/data/uocvg33dmxmplmqc3htx/PHID-FILE-aml3tjgbm2zdiyld5l5c/papyzw667luy6uad/672349_r.png [17:55:19] now you have to actually fix [17:55:49] "The link you followed to view this file is invalid or expired." [17:56:04] * thcipriani does the needful [17:56:48] thcipriani: https://phabricator.wikimedia.org/M147 [17:57:04] ah, ha [17:58:02] Project browsertests-Wikidata-WikidataTests-linux-firefox build #115: 15ABORTED in 4 hr 0 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-WikidataTests-linux-firefox/115/ [18:25:26] (03CR) 10Krinkle: [C: 04-1] "Avoid hardcoding these commands. Use the 'npm-install' or 'npm' macro instead, which already does all of this." [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) (owner: 10Paladox) [18:28:58] (03PS2) 10Paladox: Re use builder npm for npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) [18:29:11] (03CR) 10Paladox: "Ok done." [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) (owner: 10Paladox) [18:29:45] (03PS3) 10Paladox: Re use builder npm for npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) [18:37:23] (03PS2) 10Krinkle: [SimpleSurvey] Archive repo [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:37:32] (03PS3) 10Krinkle: [SimpleSurvey] Repo is archived [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:37:48] (03CR) 10Krinkle: [C: 032] [SimpleSurvey] Repo is archived [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:38:02] (03PS4) 10Paladox: [SimpleSurvey] Repo is archived [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) [18:38:26] 3Scap3, 10scap, 7WorkType-NewFunctionality: [Spike] Benchmark built-in HTTP server options for scap3 fanout - https://phabricator.wikimedia.org/T127733#2064280 (10mmodell) aiohttp seems to have decent performance, from someone else's testing: https://groups.google.com/forum/#!topic/aio-libs/3TaOnHqryn0 It... [18:38:32] (03CR) 10Paladox: "@Krinkle needs rebase before +2 since it will come back with gerrit error." [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:39:04] (03CR) 10Krinkle: [C: 032] [SimpleSurvey] Repo is archived [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:39:19] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [18:41:47] (03Merged) 10jenkins-bot: [SimpleSurvey] Repo is archived [integration/config] - 10https://gerrit.wikimedia.org/r/273129 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [19:46:54] 6Release-Engineering-Team, 10DBA, 10MediaWiki-Configuration, 6Operations, and 3 others: codfw is in read only according to mediawiki - https://phabricator.wikimedia.org/T124795#1966220 (10jcrespo) p:5Triage>3High [19:47:12] 6Release-Engineering-Team, 10DBA, 10MediaWiki-Configuration, 6Operations, and 3 others: codfw is in read only according to mediawiki - https://phabricator.wikimedia.org/T124795#1966220 (10jcrespo) a:3jcrespo [20:00:56] PROBLEM - Free space - all mounts on deployment-fluorine is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine.diskspace._srv.byte_percentfree (<37.50%) [20:54:34] Yippee, build fixed! [20:54:35] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #936: 09FIXED in 28 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/936/ [20:55:14] zeljkof: one more build fixed! Flow this time :-} [20:59:05] hashar: :) [20:59:21] no idea how it magically got fixed [21:01:21] hashar: where can I find log of jobs for beta? [21:01:40] It seems jobs for ORES extension is not done [21:02:22] (I would really appreciate if I get access to beta dbs to check them too) [21:02:30] even temporarily [21:03:40] send me an email hashar, please [21:13:30] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team: Grant Amir "ladsgroup" Sarabadani shell access to beta cluster - https://phabricator.wikimedia.org/T128130#2064754 (10hashar) [21:14:15] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team, 10MediaWiki-extensions-ORES, 10ores: Grant Amir "ladsgroup" Sarabadani shell access to beta cluster - https://phabricator.wikimedia.org/T128130#2064766 (10hashar) [21:16:28] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team, 10MediaWiki-extensions-ORES, 10ores: Grant Amir "ladsgroup" Sarabadani shell access to beta cluster - https://phabricator.wikimedia.org/T128130#2064770 (10Legoktm) +1 for me :) [21:19:41] PROBLEM - Puppet failure on deployment-cache-text04 is CRITICAL: CRITICAL: 75.00% of data above the critical threshold [0.0] [21:21:31] 10Beta-Cluster-Infrastructure, 6Release-Engineering-Team, 10MediaWiki-extensions-ORES, 10ores: Grant Amir "ladsgroup" Sarabadani shell access to beta cluster - https://phabricator.wikimedia.org/T128130#2064787 (10Krenair) 5Open>3Resolved a:3Krenair [21:22:55] Yippee, build fixed! [21:22:56] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #728: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/728/ [21:36:20] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Trebuchet: git clone of /srv/deployment/jobrunner/jobrunner failing on deployment-jobrunner01 - https://phabricator.wikimedia.org/T126846#2064847 (10hashar) With `puppet agent -tv --debug`: ``` Debug: Exec[ensure_php_hhvm](provider=posix): Executing chec... [21:37:34] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Trebuchet: git clone of /srv/deployment/jobrunner/jobrunner failing on deployment-jobrunner01 - https://phabricator.wikimedia.org/T126846#2064849 (10hashar) [21:43:12] legoktm: A work around for repos not working with composer is to use skip-if in zuul. Its a workaround but allows us to turn composer on global wide. [21:48:05] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Trebuchet: git clone of /srv/deployment/jobrunner/jobrunner failing on deployment-jobrunner01 - https://phabricator.wikimedia.org/T126846#2064877 (10hashar) I did a git deploy from tin: ``` $ git deploy start Deployment started. $ git deploy sync Repo: j... [21:49:14] !log beta did a git-deploy of jobrunner/jobrunner hoping to fix puppet run on deployment-jobrunner01 and apparently it did! T126846 [21:49:15] T126846: git clone of /srv/deployment/jobrunner/jobrunner failing on deployment-jobrunner01 - https://phabricator.wikimedia.org/T126846 [21:49:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:51:50] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 10Trebuchet: git clone of /srv/deployment/jobrunner/jobrunner failing on deployment-jobrunner01 - https://phabricator.wikimedia.org/T126846#2064886 (10hashar) 5Open>3Resolved a:3hashar puppet run. Not sure how the git clone issue got fixed, maybe Ty... [21:56:34] !log beta: stopped jobchron / jobrunner on deployment-jobrunner01 and restarting them by running puppet [21:56:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:08:35] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2064970 (10Paladox) Also for mediawiki this is blocked because the 4.3 version could only be run o... [22:11:47] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:12:25] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:12:28] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs-UI, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs UI npm CI job to Nodepool - https://phabricator.wikimedia.org/T128091#2065000 (10Paladox) [22:12:32] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:12:35] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10VisualEditor, 7WorkType-NewFunctionality: Migrate VisualEditor core npm CI job to Nodepool - https://phabricator.wikimedia.org/T128093#2065002 (10Paladox) [22:12:43] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:12:45] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10VisualEditor, 7WorkType-NewFunctionality: Migrate VisualEditor core npm CI job to Nodepool - https://phabricator.wikimedia.org/T128093#2063661 (10Paladox) [22:13:05] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:13:08] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs-UI, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs UI npm CI job to Nodepool - https://phabricator.wikimedia.org/T128091#2063634 (10Paladox) [22:13:44] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:13:46] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs-UI, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs UI npm CI job to Nodepool - https://phabricator.wikimedia.org/T128091#2063634 (10Paladox) [22:13:54] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2064983 (10Paladox) [22:13:57] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10VisualEditor, 7WorkType-NewFunctionality: Migrate VisualEditor core npm CI job to Nodepool - https://phabricator.wikimedia.org/T128093#2063661 (10Paladox) [22:14:01] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2065018 (10Paladox) [22:14:04] 10Continuous-Integration-Infrastructure, 5Continuous-Integration-Scaling, 10OOjs, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate OOjs npm CI job to Nodepool - https://phabricator.wikimedia.org/T128089#2065017 (10Paladox) [22:16:15] 10Continuous-Integration-Infrastructure: Run 'npm' job with Node 4 (instead of Node 0.10) - https://phabricator.wikimedia.org/T126774#2065026 (10Paladox) [22:16:17] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Jessie should have the ability to run serval tests at once like UbuntoTrusty can - https://phabricator.wikimedia.org/T128135#2065027 (10Paladox) [22:23:43] awesome hashar ! [22:23:45] !log beta: jobrunner01 had apache/hhvm killed somehow .... Blame me [22:23:46] thanks :) [22:23:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:23:54] Amir1|insomnia: the jobrunner is broken on beta I think [22:24:09] no clue how that crap works though [22:24:19] hmm, let me take a look at it [22:24:24] or maybe it is just very slow at processing jobs [22:24:38] do you think we need puppets? [22:24:46] no idea [22:24:47] RECOVERY - Puppet failure on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [22:25:03] but I am for sure going to bed since I am already half asleep [22:25:16] I have filled a task about giving you shell access to beta cluster. No idea what is the policy [22:27:30] hashar: I got it [22:27:32] thanks [22:27:41] well jobs seems to be processed [22:27:49] but we probably want to get another instances [22:27:51] instance [22:27:58] cause that barely move the job queue down apparently [22:28:15] and I havent found statsd metrics for beta ;( [22:33:05] Amir1: jobs are processed https://graphite.wikimedia.org/render/?width=966&height=395&_salt=1456439567.944&yMin=0&from=-24hours&target=MediaWiki.jobqueue.size [22:33:12] somehow that is the size of the queue [22:33:36] Awesome, I check it [22:34:17] yes, it's updated now [22:34:18] http://en.wikipedia.beta.wmflabs.org/w/index.php?title=Special:RecentChanges&hidenondamaging=1 [22:34:39] but why it takes this long, is it related to beta itself? [22:35:38] not enough power I guess [22:37:09] hashar: Im going to do more migration from jslint. I think we will complete the migration really soon. Ive probly migrated 100+ tests this week maybe less but deffintly higher then 60 i think. [22:37:37] Amir1: I might have fixed it though [22:38:05] Amir1: if you head to https://grafana.wikimedia.org/dashboard/db/labs-project-board you can pick a labs project ( deployment-prep is for beta) and servers ( look for deployment-jobrunner01 ) [22:38:26] awesome [22:38:29] in short load / cpu have rise since I have started tweaking it [22:38:48] we can test it by making an edit [22:38:51] let me do an edit [22:38:55] !log beta: maybe deployment-jobunner01 is processing jobs a bit faster now. Seems like hhvm went wild [22:38:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:39:07] it is going to take a while to catchup imho [22:39:26] oh, okay [22:39:29] paladox: yeah i noticed, that is awesome [22:39:43] hashar: ok. [22:39:43] paladox: thank you very much for leading that jshint/jsonlint ! [22:39:53] hashar: Your welcome. :) [22:40:10] Amir1: feel free to poke around for folks that knows about the job queue. Maybe beta can use a second instance to run them [22:40:29] okay cool :) [22:40:41] hashar: It seems on jessie it only runs one test at a time. [22:41:00] Causes a big queue on zuul if mutiple operation patches are uploaded. [22:43:34] paladox: ah yeah pool of instances get exhausted [22:43:40] too many patches [22:43:47] will look at getting more instances next week [22:44:09] hashar: Oh, Once we start migrating from npm we could possibly use some of those resources for node pool. [22:44:16] hashar: Thanks. [22:44:34] andrewbogott: do we have labs capacity to bump the Nodepool pool? We have started migrated bunch of jobs to it. Not sure what is the capacity planning looking like [22:45:14] hashar: I’m due to measure capacity again. Probably best if you hold off for a bit until I have numbers. [22:45:58] andrewbogott: I am a bit worried by the amount of resources beta cluster and CI are consuming [22:47:14] hashar: is there any help to check logs, db, etc. in beta? [22:47:22] I couldn't find any [22:47:36] andrewbogott: I will craft a task about it with some infos about current situation. Thx! [22:47:47] I want to run some db queries, if possible [22:48:06] https://logstash-beta.wmflabs.org/ for logs [22:48:53] will have to ask around how to use it [22:49:07] for the rest guess you need shell [22:49:57] No model available for [damaging] !! [22:50:15] Amir1: try searching for 'ores' [22:50:16] I have already [22:50:18] *have it [22:50:53] when you click on a line you get all fields [22:50:58] I searched, it was an old bug IIRC [22:50:58] legoktm fixes it [22:51:02] *fixed [22:51:05] and can then filter / refine [22:51:22] come on legoktm you are too fast [22:51:42] Amir1: do you guys have an irc channel ? [22:51:54] not that I am out of pet projects to lurk at ... [22:52:03] for ores and stuff? #wikimedia-ai [22:52:26] so once the fix is merged, a bunch of jenkins jobs trigger to git pull code and push it on instances (that is "scap") [22:52:39] might take sometime. I think the git pull is every 10 minutes [22:52:56] https://integration.wikimedia.org/ci/view/Beta/ list them [22:52:59] what about db queries? [22:53:09] I've access [22:53:14] need shell which we should probably grant you [22:53:29] " ssh deployment-bastion.eqiad.wmflabs" works for me [22:53:44] great [22:53:51] Krenair gave it to me a couple of hours ago [22:53:53] so you can close the task I filled earlier and sent you by email [22:54:00] good thank you Krenair ! [22:54:03] he already did [22:54:11] use deployment-tin.deployment-prep.eqiad.wmflabs instead [22:54:13] :) [22:54:18] kk [22:54:21] deployment-bastion will be shutdown eventually [22:54:24] I just did it and closed the task [22:54:26] we rebuild it [22:54:46] Krenair: awesome. Great / barnstar / your favorite beverage etc... [22:54:51] :) [22:54:57] Amir1: so from there we have a shell wrapper "sql" [22:55:03] and I think you can just: sql enwiki [22:55:04] and bam [22:55:20] awesome [22:55:24] thanks :) [22:55:25] dont EVER do any schema changes / crazy write queries on it [22:55:34] that should be done via code / sql patches etc [22:55:37] point taken [22:55:44] or that will mess thing up [22:55:52] sure [22:55:56] and eventually you might end up with the same issue in production later on [22:55:59] same goes for code [22:56:11] the code is git pulled so your live hack would get erased [22:56:23] again... hack / gerrit review / merge / auto magic deploy ... [22:56:26] rinse and repeat [22:56:45] and we have an habit of abusing feature flags like $wgOresMyCrazyFeatureForAmir1 = false; [22:57:24] which you can then enable for beta cluster via operations/mediawiki-config.git wmf-config/*-labs.php files and set it there with wgOresMyCrazyFeatureForAmir1' => array( 'default' => true ) ); [22:57:27] or something like that [22:57:46] and later when that land in production you can wgOresMyCrazyFeatureForAmir1 = true solely testwiki [22:57:57] or progressively roll your new feature on specific wikis [22:58:04] Amir1: ^^ ;-) [22:58:37] Amir1: oh and in your ~/.bashrc do export MYSQL_PS1='(\u@\h) [\d]> ' [22:58:40] fancy prompt [22:58:40] sure :))) [22:59:08] gives you: (wikiadmin@deployment-db2) [enwiki]> [22:59:26] to run mediawiki maintenance script use the mwscript wrapper [22:59:43] mwscript showJobs.php --wiki=enwiki --group [22:59:53] or fancier: mwscript showJobs.php --wiki=enwiki --group|column -t [23:00:06] awesome [23:00:08] thanks [23:00:10] :) [23:00:21] that no more shows any ORES jobs so maybe they ran [23:00:22] * Amir1 writes them down [23:00:37] that stuff might be on wikitech somewhere [23:00:40] 10Deployment-Systems, 10MediaWiki-Debug-Logger: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086#2065147 (10bd808) p:5High>3Low [23:00:55] else ask your favorite deployers / wikimedia cluster folks [23:01:00] I should check them by making edits and checking jobs [23:01:23] also the job emits metrics to statsd [23:01:25] I'll bother you ;) [23:01:29] so you could get some kind of graphs [23:01:53] i.e. stuff like https://grafana.wikimedia.org/dashboard/db/job-queue-rate [23:01:56] (that is for production) [23:02:01] dont EVER do any schema changes / crazy write queries on it [23:02:01] that should be done via code / sql patches etc [23:02:14] So I actually changed `sql` so it connects to a slave by default [23:02:22] I don't remember whether the slave in beta is read-only or not [23:02:28] no idea [23:02:42] that seems like a good thing. I am not sure how the sql script is pushed though [23:02:53] might be from puppet or something that is entirely local to beta [23:02:56] But you should always use `sql --write dbname` for actual writes [23:03:03] it's from puppet hashar [23:03:10] just like prod [23:03:12] so you changed it in puppet ? [23:04:27] Amir1: and in short do ask around. beta cluster is close to production env [23:04:44] sure, [23:04:45] Krenair: I ran out of barnstars can't give you any more [23:04:53] :) [23:04:56] yes hashar [23:04:59] we should sit down and work on running the ores service in beta too [23:05:00] \O/ [23:05:09] I did it for prod primarily [23:05:09] I have no idea how [23:05:12] I assume it works on beta [23:05:25] but beta's slave db server might be set as writable [23:05:30] Amir1: maybe we can migrate the existing instances from the ores labs project under beta [23:05:30] so don't rely on it [23:05:39] Amir1: or just have beta to point on the ores labs project [23:06:12] Krenair: yeah maybe. I have no clue really I dont ever write to db and only SELECT once in a while [23:06:31] :) [23:06:49] Amir1: assuming Ores is all in puppet (which is a requirement for production) that should be straightforward to create a new ores on beta [23:07:04] that is even a good exercise, let you polish puppet / config files etc [23:07:05] It's not AFAIK [23:07:28] when I have created services, I would set it up, amend / pil up puppet patches until it worked [23:07:40] they are probably working on it [23:07:40] then would delete the instances, recreate them and apply the puppet [23:07:59] until puppet would be able to provision the service without errors. At that point I would send it to ops for review / merge [23:08:20] if you get a point of contact among wmf operations yeah that will be fine ;-} [23:08:36] and from there it should be trivial to create a new ores on beta cluster [23:09:07] and you can use it as an integration platform before updating prod.  \O/ [23:09:31] damn past midnight. sleeeep [23:10:03] hashar: it'll be yuvi and akosiaris [23:11:36] Amir1: two, perfect! you are well covered ;-} [23:11:56] but both of them are in vacation [23:12:06] so I have to wait until the next week [23:13:20] I checked jobs work perfectly fine (at least for now) [23:17:22] Amir1: great!!! [23:17:36] that is going to be a long journey of baby steps trust me ;-D [23:17:42] ostriches, let me know when a good time to do the Beta thing is. I re-tested locally. [23:18:07] matt_flaschen: good luck with the external store thing, and thank you for leading that effort! [23:18:21] I hope so [23:18:36] on that. I am officially sleeping after saying so at least three time! Have good hacks. [23:19:32] hashar, no problem. Have a good night. [23:19:50] you too [23:19:55] sleep tight [23:22:10] 10Beta-Cluster-Infrastructure, 10Staging, 10DBA, 3Collaboration-Team-Current: Use External Store on Beta Cluster - https://phabricator.wikimedia.org/T95871#2065217 (10Mattflaschen) >>! In T95871#2058661, @demon wrote: > Yep, that about summarizes it. Ping me on IRC when you wanna do this so you can have a... [23:34:27] legoktm or Krinkle im migrating jslint to jshint and jsonlint but ive got to template extension-checks which uses jslint, what do i do, can i convert it or should i keep it as it is. [23:37:23] legoktm or Krinkle not many tests use that template. Im just going to remove the template and replace with mw-checks-test or mw-checks. [23:37:53] getting rid of the template sounds good [23:39:39] legoktm: I checked the job runner in fa.wikiepdia.beta [23:39:49] it works like a charm [23:39:59] everything is set [23:40:28] legoktm: Ok. I will do that now after i upload the patch for the next migration set. [23:42:05] (03PS1) 10Paladox: Replace jslint test with jshint and jsonlint tests [integration/config] - 10https://gerrit.wikimedia.org/r/273394 (https://phabricator.wikimedia.org/T127362) [23:46:56] Amir1: awesome! [23:59:00] legoktm: Shoulden jslint reports delete after 15 days.