[00:41:55] would it be okay for me to clone https://github.com/subbuss/parsoid_visual_diffs into integration/visualdiff in gerrit? (note the integration/ prefix, hence the question here :) ) [00:42:52] tim has previously created integration/uprightdiff .. so i figured integration/visualdiff would fit in nicely since the two are going to be used together. [00:52:47] subbu: yes, I don't think anyone will mind :) [00:53:39] great. i'm going to intialize it with mediawiki/services/parsoid as the owner but we can add any other groups later to that repo. [00:54:01] i meant 'mediawiki-services-parsoid' as the owner [00:54:13] can you also add the integration group? [00:55:19] i was going to use this command line: "ssh -p 29418 gerrit.wikimedia.org gerrit create-project --require-change-id --owner='mediawiki-service-parsoid' --parent='integration' --description='"Visual diffing utilities"' integration/visualdiff [00:55:32] i suppose i can add it after from the gui unless i can change that command to do it. [00:57:08] i will go ahead and we can tweak it from the gui. [00:58:35] legoktm, looks like i don't have the permissions actually. :) https://www.mediawiki.org/wiki/Git/Creating_new_repositories .. do you have permissions to create the repo? [01:00:35] subbu: I do, but I've never done it before and apparently it's really easy to get it wrong, you can add a request to https://www.mediawiki.org/wiki/Git/New_repositories/Requests and qchris will get to it, usually within a day or two [01:00:58] okay. thanks. [02:29:20] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-eqiad's timer trigger broken, only triggers on mediawiki-config merges - https://phabricator.wikimedia.org/T123701#1936073 (10Catrope) 3NEW [02:29:41] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-eqiad's timer trigger broken, only triggers on mediawiki-config merges - https://phabricator.wikimedia.org/T123701#1936080 (10Catrope) [02:39:10] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-eqiad's timer trigger broken, only triggers on mediawiki-config merges - https://phabricator.wikimedia.org/T123701#1936091 (10bd808) From https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/88651/consoleFull: > 02:24:... [02:49:00] !log Trying to fix submodules in deployment-bastion:/srv/mediawiki-staging/php-master/extensions for T123701 [02:49:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [02:54:43] Yippee, build fixed! [02:54:44] Project beta-code-update-eqiad build #88654: 09FIXED in 1 min 42 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/88654/ [02:54:44] Project beta-scap-eqiad build #86308: 04FAILURE in 0.53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86308/ [03:00:53] RoanKattouw_away: I think I've got the jenkins job fixed but it looks like you have a manual scap running so I can't test the full thing [03:01:33] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-eqiad's timer trigger broken, only triggers on mediawiki-config merges - https://phabricator.wikimedia.org/T123701#1936105 (10bd808) The SpellingDictionary submodule was added in Yippee, build fixed! [03:16:17] Project beta-scap-eqiad build #86310: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86310/ [03:26:36] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #943: 04FAILURE in 44 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/943/ [03:38:38] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure, 15User-bd808: beta-scap-eqiad's timer trigger broken, only triggers on mediawiki-config merges - https://phabricator.wikimedia.org/T123701#1936115 (10bd808) 5Open>3Resolved a:3bd808 [05:31:10] bd808: Thanks for fixing that [08:26:54] (03CR) 10Paladox: "@Hashar this can be merged please since https://gerrit.wikimedia.org/r/#/c/263390/ was merged." [integration/config] - 10https://gerrit.wikimedia.org/r/263409 (owner: 10Paladox) [11:04:23] 10Continuous-Integration-Infrastructure, 10Gerrit, 10GitHub-Mirrors, 10ValueView, and 2 others: [Task] Redirect unused extensions/ValueView repository to data-values/value-view - https://phabricator.wikimedia.org/T123624#1936360 (10thiemowmde) https://wikitech.wikimedia.org/wiki/Gerrit#Deleting_repositorie... [11:07:58] 10Continuous-Integration-Infrastructure: Phase out gallium.wikimedia.org - https://phabricator.wikimedia.org/T95757#1936365 (10hashar) [11:38:57] 3releng-201516-q2: [keyresult] Deprecate gitblit in favor of Diffusion - https://phabricator.wikimedia.org/T111465#1936423 (10MoritzMuehlenhoff) [11:58:13] 10Beta-Cluster-Infrastructure, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1936501 (10MoritzMuehlenhoff) [12:17:57] !log restarting Jenkins for plugins updates [12:18:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [12:37:01] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #910: 15ABORTED in 5 min 0 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/910/ [12:55:06] Yippee, build fixed! [12:55:07] Project browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #721: 09FIXED in 1 min 5 sec: https://integration.wikimedia.org/ci/job/browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/721/ [14:20:56] Project beta-update-databases-eqiad build #5813: 04FAILURE in 55 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/5813/ [14:33:43] Yippee, build fixed! [14:33:43] Project beta-update-databases-eqiad build #5814: 09FIXED in 2 min 12 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/5814/ [14:38:38] Project browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce build #385: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce/385/ [14:46:04] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #911: 04STILL FAILING in 31 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/911/ [16:12:40] PROBLEM - Host deployment-urldownloader02 is DOWN: CRITICAL - Host Unreachable (10.68.23.133) [17:41:29] ostriches: Found a way to solve the gitblit/gerrit replication issue?= [17:42:40] Kill gitblit faster :p [17:43:24] it's apparently blocking dev for some wmde work :/ [17:43:52] That's an unrelated problem [17:46:29] Missing unknown 3e588b1f9e4b7e39bf38e43e9b21ccf9f2cb2c94. [17:46:39] But that object doesn't exist in either repo anywhere.... [17:46:45] oh [17:46:54] gerrit or github [18:22:02] ostriches: We can test to enable a ssh push to gitblit. If ssh works, this is easy, if not, it is difficult, so we should choose other steps [18:22:41] I'm not going to enable that when I'm actively trying to kill the infrastructure around gitblit. [18:25:39] 10Continuous-Integration-Infrastructure, 10Gerrit, 10GitHub-Mirrors, 10ValueView, and 2 others: [Task] Redirect unused extensions/ValueView repository to data-values/value-view - https://phabricator.wikimedia.org/T123624#1937256 (10demon) [18:56:57] 10Continuous-Integration-Infrastructure, 10Gerrit, 10GitHub-Mirrors, 10ValueView, and 3 others: [Task] Redirect unused extensions/ValueView repository to data-values/value-view - https://phabricator.wikimedia.org/T123624#1937331 (10demon) > # Delete https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki... [18:58:49] Ostri https://secure.phabricator.com/T10115 [18:59:08] ostriches: ^ [19:00:37] Hmm :) [19:23:05] This seems https://integration.wikimedia.org/zuul/ to have frozen again. [19:25:48] paladox: changes are moving around the queues for me [19:26:26] greg-g: Yes seems to have un frozen now. [20:03:29] (03PS1) 10Hashar: (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 [20:05:20] (03CR) 10jenkins-bot: [V: 04-1] (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (owner: 10Hashar) [20:09:14] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (owner: 10Hashar) [20:12:33] (03PS2) 10Hashar: (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 [20:14:05] the random friday evening video https://www.youtube.com/watch?v=Lo7BYF_q9Ys (safe for work) [20:17:13] hashar: Would removing --quick from the update.php script in jenkins for SemanticMediaWiki work. Since it keeps failing at the update part. [20:17:40] paladox: have you read what update.php --quick does? :-} [20:18:10] hashar: No but was looking at the code. [20:18:33] hashar: I asked on the github repo for SemanticMediaWiki which said to use --quick which we do. [20:18:54] paladox: look at mediawiki maintenance/update.php and search for --quick :-} [20:19:06] it is unrelated to SemanticMediaWiki failure for usre [20:19:09] hashar: Would skip countdown sequence cause it to not update the sql that is in SemanticMediawiki [20:19:35] hashar: Oh. How would i fix the failure in SemanticMediaWiki for jenkins [20:19:55] have you seen what happens when --quick is enabled ? [20:20:04] if ( !$this->hasOption( 'quick' ) ) { [20:20:04] $this->output( "Abort with control-c in the next five seconds " [20:20:05] . "(skip this countdown with --quick) ... " ); [20:20:05] wfCountDown( 5 ); [20:20:07] } [20:20:14] i.e. --quick skips the wfCountDown(5) call [20:20:19] which just waits 5 seconds [20:20:25] Yes that is what i was looking at. [20:20:47] so there is absolutely no way removing --quick will solve the semantic mediawiki issue :-} [20:20:51] whatever it can be [20:20:56] So that dosent have anything to do with the problem this repo is having. Could it be because it isent using LoadExtensionSchemaUpdates hook to load the sql. [20:21:29] hashar: [20:25:01] paladox: where is the build failure? [20:25:09] https://integration.wikimedia.org/ci/job/mwext-testextension-zend-composer/672/console [20:25:22] https://gerrit.wikimedia.org/r/#/c/264174/ [20:27:39] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#1937648 (10hashar) I gave it a try. The few projects triggering the `npm` job tends to have a ton of depend... [20:28:45] hashar: Where as here https://integration.wikimedia.org/ci/job/mwext-SemanticMediaWiki-testextension-zend/44/consoleFull it passes the database update but fails at when it tests the extension and says missing classes. [20:30:09] 00:01:12.617 Error: 1146 Table 'jenkins_u3_mw.smw_object_ids' doesn't exist (127.0.0.1:3306) [20:30:13] bah looks like a table isn't created [20:30:38] I have seen that for another extnesion [20:30:52] so when one the updater, the extensions are already loaded in [20:30:59] and the update create the main page with a default content [20:31:07] BEFORE the database schema has been updated [20:31:24] and if an edit to the main page triggers code that rely on new db schema, it is dead [20:31:29] yes. [20:31:31] that one is worth reporting to semantic folks [20:31:43] pretty sure wikibase had a similar issue [20:32:03] hashar: What should i say to them. I have a bug opend here https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1375 [20:32:23] hashar: wikibase how was it fixed. [20:32:52] paladox: can't remember [20:33:00] hashar: Oh. [20:33:29] maybe it wasn't even wikibase :D [20:36:46] hashar: Im not sure why it passes for them on github. But hey use these configuation. https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/tests/travis/update-configuration-settings.sh [20:36:54] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/tests/travis/update-configuration-settings.sh [20:37:10] not sure if one of those configuation in those files will fix the issues for us. [20:37:30] oh [20:37:33] and the other job https://integration.wikimedia.org/ci/job/mwext-SemanticMediaWiki-testextension-zend/44/consoleFull [20:37:36] is slightly different [20:37:44] it runs install.php first [20:37:48] then inject extensions [20:37:51] and run update.php after [20:38:01] so the main page is created already [20:38:24] paladox: so maybe mwext-testextension-zend-composer should be adjusted to install.php first [20:38:27] then inject extensions [20:38:32] and run update.php with extensions [20:38:41] I dont know really [20:39:37] hashar: We should try that. it may help. If it dosent we can always revert. [20:40:47] hashar: What should i change in it since it looks the same as the generic one just some differences. [20:41:20] hashar: Maybe composwr [20:41:57] paladox: hmm [20:42:00] hashar: Maybe - composer-update: [20:42:00] dir: 'src' should be run after - mw-install-mysql [20:42:00] - mw-apply-settings [20:42:05] paladox: does Semantic even develop on Gerrit ? [20:42:21] if they have entirely switched to github we might as well drop them entirely from ci [20:42:25] paladox: They only do i18n updates. [20:42:48] so there is probably little need to have duplicate effort [20:42:49] hashar: But i am thinking off the other extensions that require this one. Some [20:43:10] that at least deserves a bug [20:43:17] hashar: Some of the semantic extensions are on gerrit and are not on github and require SemanticMediaWiki. [20:43:19] both jobs are slightly different [20:43:22] hashar: Yes. [20:43:27] and I am not sure which approach we should adopt [20:43:31] it is either: [20:43:44] A) install mediawiki , run install , inject extensions, run update [20:43:57] B) install mediawiki, inject extensions, run install [20:44:06] (err ignore install mediawiki) [20:45:06] hashar: Maybe a looks good. But im not sure maybe we can create a seperate test for Semantic Extensions. So that A) install mediawiki , run install , inject extensions, run update [20:45:26] Oh wait maybe A) install mediawiki , run install , inject extensions, run update would work in just the composer test. [20:47:04] (03PS1) 10Paladox: Doing some testing with prepare-mediawiki-zuul-project-no-vendor test [integration/config] - 10https://gerrit.wikimedia.org/r/264333 [20:47:05] or we can make it the standard way for all jobs [20:47:44] hashar: Yes i think we should make it standard.] [20:47:53] Hashar: Not sure if https://gerrit.wikimedia.org/r/#/c/264333/ would work. [20:49:15] hashar: Looking at https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/.travis.yml i think that A) install mediawiki , run install , inject extensions, run update was how they did it too, [20:49:18] . [20:53:24] which probably makes sense [21:00:30] (03PS2) 10Paladox: Doing some testing with prepare-mediawiki-zuul-project-no-vendor test [integration/config] - 10https://gerrit.wikimedia.org/r/264333 [21:01:10] hashar: Please could you review https://gerrit.wikimedia.org/r/264333 i would like to test to see if that works. Because - builder: [21:01:10] name: prepare-mediawiki [21:01:42] is set differently to the rest because - shell: "cp deps.txt src/extensions_load.txt" is set in between - mw-install-mysql and - mw-apply-settings [21:05:29] paladox: sorry can't dig into that tonight [21:05:32] :/ [21:05:40] hashar: Oh ok. [21:06:29] (03PS3) 10Paladox: Doing some testing with prepare-mediawiki-zuul-project-no-vendor test [integration/config] - 10https://gerrit.wikimedia.org/r/264333 [21:06:39] ideally [21:06:46] we will want to remove all the cruft we have in JJB [21:06:57] and create a new software glue that takes care of everything [21:07:04] hashar: Yes. [21:07:32] hashar: We should SemanticFormsInputs move this back to mediawiki-extensions.yaml [21:07:42] file since it is failing in ext file. [21:08:05] But that would be temporarily until extension.json is created for that repo. [21:13:03] (03PS1) 10Paladox: Move SemanticFormsInputs back to mediawiki-extensions.yaml file [integration/config] - 10https://gerrit.wikimedia.org/r/264349 [21:16:25] (03PS1) 10Paladox: Add mwext-{name}-testextension-hhvm to experimental: for some templates [integration/config] - 10https://gerrit.wikimedia.org/r/264355 [21:19:38] (03PS1) 10Paladox: Add some experimental tests to extension-checks template [integration/config] - 10https://gerrit.wikimedia.org/r/264358 [21:24:01] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#1937918 (10greg) [21:24:02] 5Continuous-Integration-Scaling, 7Tracking: [tracking] Disposable VMs need a cache for package managers - https://phabricator.wikimedia.org/T112560#1937919 (10greg) [21:24:23] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 5Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#1819116 (10greg) Added T112560 as a blocker for this, then. Correct me if I'm wrong :) [21:26:59] (03PS3) 10Hashar: (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 [21:42:58] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (owner: 10Hashar) [21:51:25] (03PS4) 10Hashar: (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 [22:01:03] (03PS5) 10Hashar: (WIP) add castor [integration/config] - 10https://gerrit.wikimedia.org/r/264327 [22:24:35] btw: I though the your SAL was big enough, all stuff before 2016 is now archived, so people with slow internet have a change now^^ [22:30:58] eh? on wiki? [22:33:21] looks like it [22:33:26] (diff | hist) . . N! Release Engineering/SAL/Archive 1‎; 22:22 . . (+211,610)‎ . . ‎Luke081515 (Talk | contribs | block)‎ (archived) [22:33:26] (diff | hist) . . Release Engineering/SAL‎; 22:22 . . (-211,537)‎ . . ‎Luke081515 (Talk | contribs | block)‎ (time for archiving...) [rollback] [22:33:44] was really big [22:34:33] The Obama page is bigger :P [22:35:24] Project beta-scap-eqiad build #86418: 04FAILURE in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86418/ [22:35:37] but you don't have to scroll such a long way :P [22:40:43] Luke081515: thanks [22:41:19] btw, I think we mostly use the ES-backed SAL: https://tools.wmflabs.org/sal/releng [22:41:24] (at least I do) [22:42:48] ah, ok [22:43:15] I did not found this tool yet^^ [22:44:07] see the /topic [22:44:08] :) [22:44:46] Yippee, build fixed! [22:44:47] Project beta-scap-eqiad build #86419: 09FIXED in 7 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/86419/ [22:44:54] whew [22:54:05] ostriches: Congratulations! :) [22:54:26] ostriches: Whee! [22:57:17] (03PS6) 10Hashar: castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) [22:58:28] I am soo stupid [22:58:42] ? [22:58:58] I forgot about firewall rules [22:59:46] (03CR) 10jenkins-bot: [V: 04-1] castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) (owner: 10Hashar) [23:03:04] (03PS7) 10Hashar: castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) [23:12:51] (03PS8) 10Hashar: castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) [23:14:36] (03PS9) 10Hashar: castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) [23:14:50] I should stop before I break something [23:16:48] :) :) [23:17:20] castor-save #68 completed. Result was SUCCESS [23:17:20] 00:01:44.689 Finished: SUCCESS [23:17:23] looks greenish :-} [23:18:12] yay [23:18:14] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) (owner: 10Hashar) [23:18:22] I am out of scotch though [23:18:37] err [23:18:39] wrong word [23:19:02] that scotch http://ecx.images-amazon.com/images/I/81mmbQrmGkL._SL1500_.jpg [23:20:45] heh, "sure..." [23:22:49] ah cache in action [23:22:58] build time went from 2 minutes to 3 ! [23:25:30] ostriches: congratulations on the title bump [23:25:41] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) (owner: 10Hashar) [23:26:40] Ty [23:31:31] (03CR) 10Hashar: [C: 04-2] "Seems to work" [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) (owner: 10Hashar) [23:32:55] 5Continuous-Integration-Scaling, 5Patch-For-Review, 7Tracking: [tracking] Disposable VMs need a cache for package managers - https://phabricator.wikimedia.org/T112560#1938443 (10hashar) Did a first pass using a cache store/restore system based on rsync. Investigated as part of T116017 Currently deployed on... [23:33:50] 5Continuous-Integration-Scaling, 5Patch-For-Review, 7Tracking: Investigate using a cache store/restore system for package managers - https://phabricator.wikimedia.org/T116017#1938450 (10hashar) See the proof of concept https://gerrit.wikimedia.org/r/264327 based on rsync and a central rsync cache. Lets foll... [23:40:57] (03PS10) 10Hashar: castor: package managers cache storage [integration/config] - 10https://gerrit.wikimedia.org/r/264327 (https://phabricator.wikimedia.org/T112560) [23:40:59] (03PS1) 10Hashar: Macro to archive tox-logs [integration/config] - 10https://gerrit.wikimedia.org/r/264410 [23:52:32] one [23:52:35] done [23:52:36] end of breakage [23:52:39] merry xmas!