[00:01:43] Ok sorry won't happen again [00:02:57] Thanks :) [00:20:04] Project beta-update-databases-eqiad build #8765: 04FAILURE in 3.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8765/ [00:39:48] ejegg hi could you abort https://integration.wikimedia.org/ci/job/mediawiki-extensions-qunit/44297/console please has been running along time and seems to be stuck [00:41:26] legoktm jzerebecki Dereckson ^^ [01:20:41] Yippee, build fixed! [01:20:42] Project beta-update-databases-eqiad build #8766: 09FIXED in 39 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8766/ [01:34:03] 03Scap3: Update l10nupdate to be compatible with scap3 - https://phabricator.wikimedia.org/T136157#2325200 (10Dereckson) [01:57:00] Project browsertests-Wikidata-WikidataTests-Group0-SmokeTests-linux-firefox-sauce build #58: 04FAILURE in 16 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-WikidataTests-Group0-SmokeTests-linux-firefox-sauce/58/ [02:07:49] 10Deployment-Systems, 03Scap3, 13Patch-For-Review: Update l10nupdate to be compatible with scap3 - https://phabricator.wikimedia.org/T136157#2325245 (10Dereckson) [02:09:28] 10Deployment-Systems, 03Scap3, 13Patch-For-Review: Update l10nupdate to be compatible with scap3 - https://phabricator.wikimedia.org/T136157#2325200 (10Dereckson) [02:09:30] 10Deployment-Systems, 06Release-Engineering-Team, 06Operations, 03Scap3 (Scap3-MediaWiki-MVP): Completely port l10nupdate to scap - https://phabricator.wikimedia.org/T133913#2325246 (10Dereckson) [02:18:17] Project selenium-QuickSurveys » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #33: 04FAILURE in 5 min 16 sec: https://integration.wikimedia.org/ci/job/selenium-QuickSurveys/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/33/ [02:38:21] PROBLEM - Parsoid on deployment-parsoid06 is CRITICAL: Connection refused [03:44:44] 06Release-Engineering-Team, 05Release: MW-1.28.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T135559#2325282 (10mmodell) a:03mmodell [06:49:35] (03PS1) 10Madhuvishy: Add job that allows for updating analytics refinery artifacts with latest source jars [integration/config] - 10https://gerrit.wikimedia.org/r/290640 (https://phabricator.wikimedia.org/T130123) [06:51:09] (03PS2) 10Madhuvishy: Add maven release job template and analytics-refinery-release project [integration/config] - 10https://gerrit.wikimedia.org/r/290597 (https://phabricator.wikimedia.org/T132182) [06:52:06] (03CR) 10jenkins-bot: [V: 04-1] Add maven release job template and analytics-refinery-release project [integration/config] - 10https://gerrit.wikimedia.org/r/290597 (https://phabricator.wikimedia.org/T132182) (owner: 10Madhuvishy) [07:08:14] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2325564 (10mobrovac) Now I can't deploy Mathoid in beta as it is asking me for a password: ``` mobrovac@deployment-tin:/srv/deployment/mathoid/deploy$ scap deploy -f... [08:22:06] Yippee, build fixed! [08:22:07] Project selenium-QuickSurveys » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #34: 09FIXED in 5 min 25 sec: https://integration.wikimedia.org/ci/job/selenium-QuickSurveys/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/34/ [08:47:28] (03CR) 10Hashar: [C: 032] fix error message in deploy-promote [tools/release] - 10https://gerrit.wikimedia.org/r/290503 (owner: 1020after4) [08:48:23] (03Merged) 10jenkins-bot: fix error message in deploy-promote [tools/release] - 10https://gerrit.wikimedia.org/r/290503 (owner: 1020after4) [08:58:04] 05Continuous-Integration-Scaling, 06Services, 07Tracking, 07WorkType-NewFunctionality: [tracking] Migrate services CI jobs to node 4.3 - https://phabricator.wikimedia.org/T126991#2325662 (10hashar) [08:58:53] hashar hi [08:59:06] Im going to migrate qunit to trusty nodepool [09:01:14] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2325666 (10hashar) [09:01:26] James_F: yt? [09:01:47] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2029093 (10hashar) Updated the task description since there are no more jobs using NodeJS 0.8... [09:01:54] paladox: not the qunit ones please [09:02:01] O [09:02:04] Oh [09:02:04] there is a bunch of provisioning that needs to be done on the image first [09:02:09] with bunch of puppet work [09:02:12] + image rebuilding etc [09:02:31] hashar: it seems qunit kept failing last night [09:02:33] twice [09:02:35] eek [09:02:42] mysql had some troubles recently [09:02:56] Oh, i doint think it was mysql since it just froze. [09:03:00] No error shown [09:03:19] do you have a job example? [09:04:36] hashar: https://integration.wikimedia.org/ci/job/mwext-qunit/16611/console [09:04:43] Oh it does look like something with mysql [09:05:04] Sorry my mistake didnt see that before probaly the job had to be aborted for us to see what happened [09:05:28] https://integration.wikimedia.org/ci/job/mwext-qunit/16611/ shows it failed after 31 minutes [09:05:40] Yep because we had to abort it [09:05:48] It was stuck on it testing [09:05:52] and it got manually cancelled [09:05:53] hmm [09:06:04] Yes maybe we should add something like an inactive timer [09:06:13] while doing the mysql install [09:06:17] so if a test get's stuck the job will automatically abort [09:08:05] that instance has some troubles [09:08:10] Oh [09:08:11] the resolv.conf list bad domains [09:08:26] Should we kill it and open another one [09:09:55] 10Continuous-Integration-Infrastructure: integration-slave-trusty-1013 has a bad resolv.conf - https://phabricator.wikimedia.org/T136170#2325678 (10hashar) [09:10:53] 10Continuous-Integration-Infrastructure: integration-slave-trusty-1013 has a bad resolv.conf - https://phabricator.wikimedia.org/T136170#2325690 (10hashar) That is since `May 16 19:52` /etc/resolv.conf :( [09:11:34] !log restarting puppetmaster on integration-puppetmaster ( memory leak / can not fork) [09:11:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:14:36] 10Continuous-Integration-Infrastructure: integration-slave-trusty-1013 has a bad resolv.conf - https://phabricator.wikimedia.org/T136170#2325693 (10hashar) I have manually changed resolv.conf and restarted the nscd (Name Service Cache Daemon) service. [09:15:00] hashar: It looks like [09:15:01] ## THIS FILE IS MANAGED BY PUPPET [09:15:01] ## [09:15:01] ## source: modules/base/resolv.conf.labs.erb [09:15:01] ## from: base::resolving [09:15:01] domain <%= @labsproject %>.<%= @site %>.<%= @labs_tld %> [09:15:03] search <%= @labsproject %>.<%= @site %>.<%= @labs_tld %> <%= @site %>.<%= @labs_tld %> [09:15:05] <% scope.lookupvar('::nameservers').each do |nsip| -%> [09:15:07] nameserver <%= nsip %> [09:15:09] <% end -%> [09:15:11] options timeout:2 ndots:2 [09:15:12] !log Fixed resolv.conf on integration-slave-trusty-1013 and force running puppet to catch up with change since May 16 19:52 [09:15:15] yeah [09:15:15] Should we change it back too [09:15:15] ## THIS FILE IS MANAGED BY PUPPET [09:15:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:15:17] ## [09:15:19] ## source: modules/base/resolv.conf.labs.erb [09:15:21] ## from: base::resolving [09:15:21] looks like @labs_tld wasn't set somehow [09:15:25] domain <%= @labsproject %>.<%= @site %>.wmflabs [09:15:27] search <%= @labsproject %>.<%= @site %>.wmflabs <%= @site %>.wmflabs [09:15:29] <% scope.lookupvar('::nameservers').each do |nsip| -%> [09:15:31] nameserver <%= nsip %> [09:15:33] <% end -%> [09:15:35] options timeout:2 ndots:2 [09:15:37] Woops should have used paste. [09:15:39] hashar: Yep [09:16:32] * 7aeab22 - Set the proper .tld in labtest resolv.conf (Mon May 16 19:44:29 2016 +0000) [09:16:32] :D [09:16:32] hashar: https://github.com/wikimedia/operations-puppet/search?utf8=%E2%9C%93&q=labs_tld&type=Code [09:17:24] hashar: Should we revert it. [09:17:28] no [09:17:32] Oh ok [09:17:33] it is a transient effect I guess [09:17:38] @labs_tld is provided by hiera [09:17:43] I guess it failed to return a value somehow [09:17:47] Oh, how would we fix it. [09:17:54] but for all the other instances it worked just fine [09:18:00] so looks like a one time race condition [09:18:23] Oh ok [09:18:31] But it keeps happening. [09:18:33] puppet is running just fine [09:18:40] Ok [09:19:15] 10Continuous-Integration-Infrastructure: integration-slave-trusty-1013 has a bad resolv.conf - https://phabricator.wikimedia.org/T136170#2325700 (10hashar) 05Open>03Resolved a:03hashar Might have been a side effect of 7aeab22b66e554060b626254bfd62c57429ae4e3 and a hiera lookup that failed somehow. All CI... [09:22:19] 10Continuous-Integration-Infrastructure: mwext-qunit fails during Mysql install (and maybe teardown) - https://phabricator.wikimedia.org/T136171#2325704 (10hashar) [09:22:59] paladox: I have filled the qunit / mysql issue at https://phabricator.wikimedia.org/T136171 [09:23:10] hashar: Thanks [09:27:57] 10Continuous-Integration-Infrastructure: mwext-qunit fails during Mysql install (and maybe teardown) - https://phabricator.wikimedia.org/T136171#2325720 (10hashar) Nothing of interest in syslog / dmesg :( [09:30:41] !log Clearing git-sync-upstream script on integration-slave-trusty1013 and integration-slave-trusty-1017. That is only supposed to be on the puppetmaster [09:30:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:36:44] 10Continuous-Integration-Infrastructure: mwext-qunit fails during Mysql install (and maybe teardown) - https://phabricator.wikimedia.org/T136171#2325722 (10hashar) Ah /var/log/upstart/mysql.log.1.gz had some content: ``` 160523 19:33:13 [Warning] Using unique option prefix key_buffer instead of key_buffer_size i... [09:38:54] 10Continuous-Integration-Infrastructure: mwext-qunit fails during Mysql install (and maybe teardown) - https://phabricator.wikimedia.org/T136171#2325723 (10hashar) 05Open>03Resolved a:03hashar Could not find anything and there is only one build that failed this way. Calling it a one off / transient issue. [09:39:08] paladox: I gave up on the qunit / mysql issue. No evidence :( [09:41:37] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 06Reading-Web-Backlog: Don't send alerts when beta cluster is in known states that make tests fail - https://phabricator.wikimedia.org/T136172#2325727 (10Jhernandez) [09:49:26] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 06Reading-Web-Backlog: Don't send alerts when beta cluster is in known states that make tests fail - https://phabricator.wikimedia.org/T136172#2325727 (10phuedx) An alternative would be to skip a scenario that required write access to a wiki that wa... [10:11:35] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 06Reading-Web-Backlog: Don't send alerts when beta cluster is in known states that make tests fail - https://phabricator.wikimedia.org/T136172#2325786 (10phuedx) tl;dr: It's likely better to tag tests that can be run with the wiki in read-only mode... [10:13:20] 05Continuous-Integration-Scaling, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate PHP extensions building jobs to Nodepool - https://phabricator.wikimedia.org/T134381#2325800 (10hashar) p:05Normal>03Low [10:15:34] 10Staging, 06Project-Admins, 15User-greg: Create "Staging" project to track the creation of a staging cluster - https://phabricator.wikimedia.org/T88298#2325826 (10Danny_B) [10:43:04] (03PS1) 10Hashar: Migrate parsoidsvc-php-parsertests to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) [10:43:49] (03CR) 10jenkins-bot: [V: 04-1] Migrate parsoidsvc-php-parsertests to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [10:54:34] (03PS2) 10Hashar: Migrate parsoidsvc-php-parsertests to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) [10:56:15] (03CR) 10Hashar: [C: 032] "tested and works. Will update the Nodepool image so they come with netpbm installed." [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [10:57:39] (03Merged) 10jenkins-bot: Migrate parsoidsvc-php-parsertests to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [10:58:33] !log Updating Nodepool ci-jessie-wikimedia snapshot image to get netpbm package installed into it. T126992 https://gerrit.wikimedia.org/r/290651 [10:58:34] T126992: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992 [10:58:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:02:48] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2325990 (10hashar) ``` Notice: /Stage[main]/Packages::Netpbm/Package[netpbm]/ensure: ensure ch... [11:04:11] (03CR) 10Paladox: "Hi you forgot to update parameter_functions. Since it needs to have this test added due to it not ending in -Jessie or node-4.3 and hhvm n" [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:08:53] (03CR) 10Hashar: "No need to update parameter function, it applies the nodepool parameter to any job suffixed with '-jessie':" [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:10:26] (03CR) 10Paladox: "Woops sorry." [integration/config] - 10https://gerrit.wikimedia.org/r/290651 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:13:38] (03PS4) 10Paladox: Add npm-run-doc-jsduck-node-4.3 test [integration/config] - 10https://gerrit.wikimedia.org/r/290564 (https://phabricator.wikimedia.org/T109005) [11:15:06] hashar hi could you review https://gerrit.wikimedia.org/r/#/c/290564/ please [11:15:35] Since we should should create separate npm tests that run gem install jsduck [11:15:58] That way when we create a package jsduck for Jessie we can then use it. [11:19:18] (03PS1) 10Hashar: zuul: trigger the parsoid parser tests [integration/config] - 10https://gerrit.wikimedia.org/r/290657 (https://phabricator.wikimedia.org/T126992) [11:19:42] (03CR) 10Hashar: [C: 032] zuul: trigger the parsoid parser tests [integration/config] - 10https://gerrit.wikimedia.org/r/290657 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:20:32] (03Merged) 10jenkins-bot: zuul: trigger the parsoid parser tests [integration/config] - 10https://gerrit.wikimedia.org/r/290657 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:24:45] (03PS1) 10Hashar: parsoidsvc-hhvm-parsertests-jessie now archive log dir [integration/config] - 10https://gerrit.wikimedia.org/r/290659 (https://phabricator.wikimedia.org/T126992) [11:24:56] (03CR) 10Hashar: [C: 032] parsoidsvc-hhvm-parsertests-jessie now archive log dir [integration/config] - 10https://gerrit.wikimedia.org/r/290659 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:26:23] (03Merged) 10jenkins-bot: parsoidsvc-hhvm-parsertests-jessie now archive log dir [integration/config] - 10https://gerrit.wikimedia.org/r/290659 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:29:22] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326033 (10hashar) [11:39:59] (03PS1) 10Hashar: parsoidsvc: migrate jshint/jsonlint to npm [integration/config] - 10https://gerrit.wikimedia.org/r/290663 (https://phabricator.wikimedia.org/T126992) [11:42:36] (03CR) 10Hashar: [C: 032] "Slight change: it will always triggers. But it is not much of an issue really. Looks like almost all patches touch javascript files." [integration/config] - 10https://gerrit.wikimedia.org/r/290663 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:43:50] (03Merged) 10jenkins-bot: parsoidsvc: migrate jshint/jsonlint to npm [integration/config] - 10https://gerrit.wikimedia.org/r/290663 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [11:53:36] 10Beta-Cluster-Infrastructure, 10DBA, 10Flow, 03Collab-Team-2016-Apr-Jun-Q4: Run Flow External Store migration in dry-run mode on Beta - https://phabricator.wikimedia.org/T119567#2326090 (10jcrespo) The backup on /mnt/mysql_backup.sql.bz2 has finally finished (there were issues with labswiki). You can cont... [11:54:26] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326093 (10hashar) [12:42:17] (03PS1) 10Hashar: Migrate parsoid test jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290675 (https://phabricator.wikimedia.org/T126992) [12:44:57] (03CR) 10Hashar: [C: 032] Migrate parsoid test jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290675 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [12:45:03] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326152 (10hashar) [12:46:09] (03Merged) 10jenkins-bot: Migrate parsoid test jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290675 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [12:59:45] https://integration.wikimedia.org/ci/job/mwext-qunit/16650/console this job seems stuck and blocking other jobs... can something be done about it? [13:01:33] hashar: ? [13:01:48] Nikerabbit: let me look at it [13:01:58] Nikerabbit: it is happening from time to time [13:02:36] Nikerabbit: I noticed that yesterday, going to debug it on the instance [13:03:14] ok [13:08:10] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2326202 (10hashar) [13:10:14] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2326205 (10hashar) lsof for the jenkins-deploy user: {P3176} [13:10:31] Nikerabbit: something is locked but I have really no clue what :( [13:12:27] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2326219 (10hashar) After the Jenkins 30 minutes timeout is reached, the job is aborted/fails: ``` 13:09:48 Build timed out (after 30 minutes). Marking the build as failed.... [13:13:39] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326222 (10hashar) parsoidsvc-source-npm-0.10 is blocked pending `jsduck`. [13:20:56] hashar: ok [13:25:05] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2326243 (10hashar) 72b0d9da08d6ea8c1842d76aaefe8e57046a2026 bumped the karma related tools to 1.0.x on May 19th. Build that had a timeout: https://integration.wikimedi... [13:31:19] (03PS7) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (owner: 10Paladox) [13:39:54] (03PS8) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (owner: 10Paladox) [13:46:31] Project selenium-VisualEditor » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #31: 04FAILURE in 2 min 29 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/31/ [13:47:43] (03PS9) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (owner: 10Paladox) [13:49:41] (03CR) 10Hashar: "Did a few very minor tweak such as renaming 'npm-legacy' Zuul template to 'npm-node-0.10'." [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (owner: 10Paladox) [13:52:50] (03PS10) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [14:10:26] (03PS11) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [14:10:28] (03PS1) 10Hashar: dib: add g++ for NodeJS native modules compilation [integration/config] - 10https://gerrit.wikimedia.org/r/290689 (https://phabricator.wikimedia.org/T119143) [14:10:30] (03PS1) 10Hashar: jjb: default to clone to /src/ [integration/config] - 10https://gerrit.wikimedia.org/r/290690 (https://phabricator.wikimedia.org/T130207) [14:11:49] (03CR) 10Hashar: "Thanks a ton Paladox" [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [14:12:02] (03CR) 10Hashar: [C: 032] dib: add g++ for NodeJS native modules compilation [integration/config] - 10https://gerrit.wikimedia.org/r/290689 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [14:12:39] !log Regenerating Nodepool snapshot to include g++ which is required by some NodeJS native modules T119143 [14:12:40] T119143: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143 [14:12:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [14:13:05] (03Merged) 10jenkins-bot: dib: add g++ for NodeJS native modules compilation [integration/config] - 10https://gerrit.wikimedia.org/r/290689 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [14:32:42] PROBLEM - puppet last run on gallium is CRITICAL: CRITICAL: Puppet has 1 failures [14:33:50] Yippee, build fixed! [14:33:51] Project selenium-WikiLove » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #32: 09FIXED in 1 min 49 sec: https://integration.wikimedia.org/ci/job/selenium-WikiLove/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/32/ [14:46:52] PROBLEM - Puppet run on integration-slave-precise-1002 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [14:51:02] hashar: Ok :) [14:51:07] thanks for working on it [14:51:49] hashar: Im thinking for jsduck just create a seperate npm template for this [14:52:07] It will allow us to find another solution to installing jsduck [14:53:33] Yes it sounds hacky but it will be behind the scenes making it less hacky for users who maintain a repo [14:53:51] We could export the path to the jsduck bin [14:54:21] It just sounds like a better soulution until we create a deb jsduck package for jessie. [14:58:02] (03PS1) 10Hashar: dib: ensure_package('g++') does not work [integration/config] - 10https://gerrit.wikimedia.org/r/290698 (https://phabricator.wikimedia.org/T119143) [14:58:10] (03CR) 10Hashar: [C: 032] dib: ensure_package('g++') does not work [integration/config] - 10https://gerrit.wikimedia.org/r/290698 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [14:58:15] ostriches: Ive contacted github and yes they do have a limit. [14:58:17] phuedx: Sorry, here now. Can I help? [14:58:58] (03Merged) 10jenkins-bot: dib: ensure_package('g++') does not work [integration/config] - 10https://gerrit.wikimedia.org/r/290698 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [14:59:07] paladox: I think I will give up and just install jsduck on the CI slaves :D [14:59:18] blindly using gem install :-} [14:59:24] hashar: Ok yes :) [14:59:35] that is becoming non sense / complicated [14:59:44] Will you do it globally meaning it is already installed but add the path to jsduck. [14:59:53] apparently jsduck is stall / legacy and hasn't released anything new since 2013 [14:59:58] so we can live with a stall version [15:00:02] yeah globally [15:00:08] Ok thanks :) [15:00:08] that would install it as /usr/local/bin/jsduck [15:00:12] and fix up the issue [15:00:14] ;-D [15:00:19] Ok :) [15:00:26] Do you know when we will do that please [15:01:02] i am polishing up / testing your nice node-0.10 template [15:01:13] hashar: Thanks :) [15:01:18] the patch was all fine [15:01:44] hashar: Im going to have a look at what else we can migrate like existing tests that are customised but there generic have already been migrated [15:01:45] Thanks [15:01:48] if I manage to deploy / test it by end of today [15:01:54] Ok [15:02:09] the bulk has been moved [15:02:18] qunit is going to be next for sure [15:02:21] hashar: Ok, Do you know when we will do that please for jsduck please. [15:02:23] hashar: Yep [15:02:25] that needs some puppet work though [15:02:42] then we can probably migrate the mwext-mw-selenium job (that runs selenium against a local mediawiki install) [15:03:01] I was changing the template and adding the param when you said not do do it yet [15:03:03] !sal [15:03:03] https://tools.wmflabs.org/sal/releng [15:03:37] my only problem is that I ended up loosing track of what I was doing an hour or so ago :( [15:03:49] hashar: Oh [15:04:01] hashar: I thought we already did most of the things to get qunit to work [15:04:03] ah yeah [15:04:10] migrating the node-0.10 jobs :D [15:04:15] Yep :) [15:04:36] npm-legacy will be gone [15:04:43] which mean no more npm jobs running on the permanent slaves \O/ [15:04:50] hashar: Those two tests that failed in it can be migrated to npm node 4.3 [15:04:55] since we added g++ [15:05:12] fundraising/dash requires node 0.10 [15:05:18] that is what they are running on production [15:05:18] and also you have a patch here https://gerrit.wikimedia.org/r/#/c/290690/ [15:05:24] pywikibot/i18n, yeah we could [15:05:33] though that would need the node 4.3 job to git clone to /src/ [15:05:41] which in turn impact all the repositories :-D [15:05:54] hashar: Oh, we can have two node jobs running for fundraising/dash [15:05:59] Oh [15:06:02] yeah potentially [15:06:09] one for node 0.10 and another for 4.3 [15:06:15] You could do an if statement for detecting the project repo [15:06:18] yep [15:06:22] so whenever they migrate to Jessie in production they will know it pass [15:06:37] yep [15:07:11] hashar: https://gerrit.wikimedia.org/r/#/c/280665/ [15:07:15] !log g++ added to Jessie and Trusty Nodepool instances | T119143 [15:07:15] T119143: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143 [15:07:20] Image ci-jessie-wikimedia-1464188494 in wmflabs-eqiad is ready [15:07:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [15:07:55] Image ci-trusty-wikimedia-1464188447 in wmflabs-eqiad is ready [15:09:32] hashar: This https://github.com/strongloop/modern-syslog/blob/master/.travis.yml supports both 0.10 and 4.3 [15:09:53] where as node-syslog dosent it only supports 0.10 [15:10:17] ejegg|away hi could you review https://gerrit.wikimedia.org/r/#/c/280665/ please [15:10:48] yeah ejegg|away did some work on that [15:11:37] hashar: https://gerrit.wikimedia.org/r/#/c/277768/ [15:11:38] yep [15:11:54] hashar: Could you merge https://gerrit.wikimedia.org/r/#/c/280665/ please. [15:12:01] Since https://gerrit.wikimedia.org/r/#/c/277768/ was merged [15:12:14] So we can add the new npm test and keep old npm test. [15:12:17] please [15:12:20] it fails :( https://integration.wikimedia.org/ci/job/npm-node-0.10/6/console [15:12:21] bah [15:13:43] It says [15:13:43] 15:11:09 gyp ERR! stack Error: connect ENETUNREACH [15:14:30] who knows what it can mean :D [15:14:41] network unreacheable [15:14:49] but that does not yield the hostname / url [15:14:55] anyway a retry worked [15:15:48] (03PS2) 10Hashar: jjb: default to clone to /src/ [integration/config] - 10https://gerrit.wikimedia.org/r/290690 (https://phabricator.wikimedia.org/T130207) [15:16:06] hashar: Yep [15:16:10] (03CR) 10Hashar: [C: 032] "Proven to be working properly via the next change." [integration/config] - 10https://gerrit.wikimedia.org/r/290690 (https://phabricator.wikimedia.org/T130207) (owner: 10Hashar) [15:16:31] (03PS12) 10Hashar: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [15:17:16] (03Merged) 10jenkins-bot: jjb: default to clone to /src/ [integration/config] - 10https://gerrit.wikimedia.org/r/290690 (https://phabricator.wikimedia.org/T130207) (owner: 10Hashar) [15:17:18] (03CR) 10Hashar: [C: 032] "I have confirmed that pywikibot/i18n works fine as well as wikimedia/fundraising/dash on both master and deployment branches." [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [15:18:26] 05Continuous-Integration-Scaling, 03releng-201516-q4, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2326599 (10hashar) [15:18:28] 05Continuous-Integration-Scaling, 10Pywikibot-i18n, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate pywikibot/i18n npm job to Nodepool - https://phabricator.wikimedia.org/T130207#2326596 (10hashar) 05Open>03Resolved a:03hashar The job got migrated to a job that clone to `/src/` (npm-node-0.1... [15:18:47] (03Merged) 10jenkins-bot: Migrate npm legacy test to trusty nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290486 (https://phabricator.wikimedia.org/T119143) (owner: 10Paladox) [15:18:57] Thanks ^^ [15:20:17] 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling, 07WorkType-NewFunctionality: Jobs should checkout source in a subdirectory of the workspace to avoid polluting it with CI related files - https://phabricator.wikimedia.org/T130208#2326606 (10hashar) Since https://gerrit.wikimedia.org/... [15:21:03] good enough for now [15:21:05] I have a meeting [15:21:18] then that will be the end of the day ;-) [15:21:24] will keep doing migration tomorrow ;-} [15:21:31] there is a bunch of tasks that are now easy to solve! [15:21:46] ok [15:21:57] hashar: Do you know when we will do jsduck please. [15:22:05] (03PS1) 10Paladox: Add npm-node-4.3 test to experimental: in pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290699 [15:22:13] hashar ^^ [15:22:32] (03PS2) 10Paladox: Add npm-node-4.3 test to experimental: in pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290699 [15:29:26] hashar i fixed pywikibot/i18n here https://gerrit.wikimedia.org/r/290700 [15:36:39] well [15:36:44] it is fixed already ;-} [15:37:44] hashar: Oh, i am going to add cd src to node 4.3 [15:37:49] like you did to node 0.10 [15:39:22] node 4.3 is a bit more complicated [15:39:34] have to verify that all repositories triggering it would still pass properly [15:42:04] hashar Oh. [15:42:53] (03PS1) 10Paladox: In node-4.3 clone under src [integration/config] - 10https://gerrit.wikimedia.org/r/290702 [15:43:10] hashar: unless we create a customised version for pywikibot for now [15:45:11] Project selenium-MobileFrontend » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #27: 04FAILURE in 23 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/27/ [15:46:53] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005#2326725 (10hashar) Given how jsduck is present in a wide range of jobs, I am just going to provision jsduck from ruby gems with somethin... [15:47:21] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005#2326727 (10hashar) [15:47:23] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326726 (10hashar) [15:50:27] (03PS1) 10Paladox: Add pywikibot-npm-node-4.3 experimental job to pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290703 (https://phabricator.wikimedia.org/T130207) [15:50:31] hashar: ^^ [15:50:59] Ive created an experimental job for pywikibot to try it then make it it's test job if it passes [15:50:59] (03PS1) 10Hashar: dib: add jsduck to Nodepool instances [integration/config] - 10https://gerrit.wikimedia.org/r/290704 (https://phabricator.wikimedia.org/T109005) [15:51:00] :) [15:52:53] (03Abandoned) 10Paladox: Add npm-node-4.3 test to experimental: in pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290699 (owner: 10Paladox) [15:52:55] (03Merged) 10jenkins-bot: dib: add jsduck to Nodepool instances [integration/config] - 10https://gerrit.wikimedia.org/r/290704 (https://phabricator.wikimedia.org/T109005) (owner: 10Hashar) [15:53:04] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2326768 (10thcipriani) >>! In T132747#2325564, @mobrovac wrote: > Any ideas? There was a problem with the keys in the `secrets` module after https://gerrit.wikimedia... [15:54:09] hashar: Should we also https://gerrit.wikimedia.org/r/#/c/290704/1/dib/puppet/ciimage.pp do browsers [15:54:13] for trusty [15:59:35] 05Continuous-Integration-Scaling, 10MobileFrontend, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate MobileFrontend npm job to Nodepool - https://phabricator.wikimedia.org/T130587#2326817 (10Jdforrester-WMF) [15:59:37] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2326818 (10Jdforrester-WMF) [15:59:40] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005#2326815 (10Jdforrester-WMF) 05Open>03Resolved a:03Jdforrester-WMF [15:59:46] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005#1537429 (10Jdforrester-WMF) a:05Jdforrester-WMF>03hashar [16:11:45] (03PS1) 10Hashar: dib: on Trusty set gem to gem2.0 [integration/config] - 10https://gerrit.wikimedia.org/r/290708 (https://phabricator.wikimedia.org/T109005) [16:11:54] (03CR) 10Hashar: [C: 032] dib: on Trusty set gem to gem2.0 [integration/config] - 10https://gerrit.wikimedia.org/r/290708 (https://phabricator.wikimedia.org/T109005) (owner: 10Hashar) [16:12:47] (03Merged) 10jenkins-bot: dib: on Trusty set gem to gem2.0 [integration/config] - 10https://gerrit.wikimedia.org/r/290708 (https://phabricator.wikimedia.org/T109005) (owner: 10Hashar) [16:18:01] jsduck is crap :/ [16:18:08] Oh [16:18:15] It failed [16:18:35] hashar ^^ [16:19:27] Or does it work but was hard to get it working. [16:20:56] (03PS1) 10Hashar: dib: install jsduck with a .deb package on Trusty [integration/config] - 10https://gerrit.wikimedia.org/r/290709 (https://phabricator.wikimedia.org/T109005) [16:21:33] oops [16:21:35] I pushed that one [16:21:55] paladox: jsduck should be on Jessie and hence available to the npm-node-4.3 jobs [16:22:06] Oh [16:22:07] for trusty and node-0.10 the image is rebuilding [16:22:12] we will see [16:22:15] Ok, :) [16:22:18] will finish that up tmorrow [16:22:19] Thanks for working on it [16:26:18] 2016-05-25 16:24:35,491 INFO nodepool.image.build.wmflabs-eqiad.ci-trusty-wikimedia: Notice: /Stage[main]/Main/Package[ruby-jsduck]/ensure: ensure changed 'purged' to 'present' [16:26:20] :D [16:26:35] !log 2016-05-25 16:24:35,491 INFO nodepool.image.build.wmflabs-eqiad.ci-trusty-wikimedia: Notice: /Stage[main]/Main/Package[ruby-jsduck]/ensure: ensure changed 'purged' to 'present' T109005 [16:26:35] T109005: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005 [16:26:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [16:27:44] the trusty image is being published [16:27:56] whenever that is complete, new instances will start having jsduck [16:28:01] that takes a while though [16:28:52] that is all for today [16:33:56] Ok thanks [16:42:14] 07Browser-Tests, 06Reading-Web-Backlog: Run browser tests on beta cluster from desktop domain - https://phabricator.wikimedia.org/T130429#2327138 (10Jhernandez) p:05Triage>03Normal [17:06:22] ejegg hi could you review https://gerrit.wikimedia.org/r/#/c/280665/ please [17:08:25] sure, I think I can just merge that [17:08:48] lemme just test again locally [17:09:56] ejegg: Ok thanks. [17:11:56] hmm, failing. I'll try to fix it [17:15:03] Ok thanks [17:15:06] probably the ancient version of node we need to support [17:16:13] Oh [18:47:27] ACKNOWLEDGEMENT - puppet last run on gallium is CRITICAL: CRITICAL: Puppet has 1 failures daniel_zahn https://gerrit.wikimedia.org/r/#/c/290723/1 [18:55:01] I never knew there was a website called http://paladox.com/ spotted by twentyafterfour [19:00:50] hashar: I never knew there was a website called http://paladox.com/ spotted by twentyafterfour [19:05:04] magic :)) [19:05:45] Yep [19:06:19] hashar: Would you be able to review https://gerrit.wikimedia.org/r/#/c/290703/ and merge please [19:06:37] (03PS2) 10Paladox: Add pywikibot-npm-node-4.3 experimental job to pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290703 (https://phabricator.wikimedia.org/T130207) [19:06:43] tgr: so I had to rush earlier sorry ( re: https://gerrit.wikimedia.org/r/#/c/290269/ (mediawiki_api gem) ) [19:14:03] hashar: nw, I know my timing is poor [19:14:58] I am really tempted to have https://phabricator.wikimedia.org/T135884 flagged as a blocker for 1.27 [19:15:12] and whatever 1.28-wmf.X which would enable the AuthManager [19:15:23] since that will break a lot of Selenium tests [19:15:53] I am not sure whether zeljkof had a look at it yet [19:16:20] hashar: it is a blocker for enabling AuthManager in master, certainly [19:16:43] and we definitely want it in 1.27 since it is LTS [19:16:45] as a last resort we could enable in 1.27 but not master, but that's really messy [19:16:53] indeed [19:16:58] hashar: did not yet, will do in an hour or so :) [19:17:00] yup that looks akward [19:17:36] hashar: I don't remember if I asked this already, but do you know of any other CI account creation code? [19:17:57] tgr: from a quick glance at it a couple days ago, my worry is that create_account() no more supports token = nil extra param . Might just require a new release of the gem and adjusting all repos [19:18:08] tgr: and I havent really looked at the spec you wrote (thanks for that! [19:18:31] tgr: I dontknow of any other cases [19:18:47] hashar: I assumed that's just used recursively (can't imagine anyone actually passing in a token) [19:18:56] karma / qunit are hitting a locally installed mw made available over http://localhost/ but I dont think they create any account [19:19:46] (oh and I learned how to unblock/reset an iPod: press menu + select for a while and bam ) [19:19:58] it probably has lost a few years of uptime in the process :/ [19:21:33] hashar: Yes [19:21:48] I keep having to check an apple doc to do that [19:22:05] But for an iphone and ipad now. Not ipod touch that broke. [19:22:43] wikihow has pictures http://www.wikihow.com/Hack-an-iPod-Nano-3G (warning potential data loss) [19:23:19] Oh thats old [19:23:39] I had the ipod touch 4g. but now i doint use it. [19:23:50] I use apples new password touch id. [19:24:11] So quick and easy you doint even need to move your hand. [19:24:33] hashar ^^ [19:24:48] 06Release-Engineering-Team, 05Release: MW-1.28.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T135559#2327594 (10mmodell) [19:27:20] it is usually my wife acting as the home DJ / play list conductor [19:27:31] she has a much larger musical culture than I [19:28:08] oh, i just use spotify. [19:28:22] I just "darling, what about some music?" [19:28:31] oh [19:28:42] I use the soundbar bluetooth. [19:30:46] a few local hackers ended up building their own hifi amp.. [19:30:49] not for the dummies though [19:31:01] Oh [19:31:24] I got a new samsung tv a few months ago and also got a smart soundbar. [19:31:39] It turns on everytime i select it on my iphones bluetooth. [19:32:37] https://github.com/chanmix51/SE_EL360/blob/master/schematic/schematic.svg :D [19:33:08] Oh, thats complicated [19:33:32] they get their lamp off eBay.ru from russian army surplus :D [19:33:49] funny thing they have been made back in the 70's [19:33:56] anyway [19:34:02] lets process with some jsduck related stuff :D [19:34:33] Oh [19:34:36] Ok [19:35:08] hashar: But since i had touch id i use that all the time. The only thing i doint use it for is the app store. [19:35:22] 10Continuous-Integration-Config, 06Operations, 13Patch-For-Review: Switch CI from jsduck deb package to a gemfile/bundler system - https://phabricator.wikimedia.org/T109005#2327620 (10hashar) Trusty: JSDuck 5.3.4 (Ruby 1.9.3) Jessie: JSDuck 5.3.4 (Ruby 2.1.5) [19:35:40] But my battery does last all day on the ipad pro. More then the 10+hrs [19:35:44] 10Beta-Cluster-Infrastructure, 10Math, 10VisualEditor, 10VisualEditor-MediaWiki, 07Beta-Cluster-reproducible: Math nodes are not getting rendered for the first time in a session - https://phabricator.wikimedia.org/T132620#2327621 (10Ryasmeen) [19:40:25] 06Release-Engineering-Team, 05Release: MW-1.28.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T135559#2303065 (10Tgr) [19:52:30] https://integration.wikimedia.org/ci/job/mwext-qunit/16682/console got aborted? [19:55:44] paladox: npm is all happy for parsoid on both repos and both 4.3 and 0.10 \O/ [19:55:47] Krinkle: I have killed it [19:55:55] I see. [19:55:55] hashar: :) [19:55:58] What happened? [19:56:13] Krinkle: there is some unknown issue that started roughly 7 days ago where karma/chromium whatever ends up stall waiting for Chromium to exit [19:56:29] took some traces at https://phabricator.wikimedia.org/T136188 [19:56:30] Is there a task? [19:56:56] with links to console that froze over the job history ( 30 days iirc ) [19:57:19] might be due to chromium 50 or the recent bump of karma related modules to 1.0.x [19:58:49] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2327849 (10hashar) With jsduck it is now much easier and: | Repo | Status | Build |--|--|-- |... [20:01:31] (03PS1) 10Hashar: parsoid: switch 'npm test' to Nodepool + Node 4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/290744 (https://phabricator.wikimedia.org/T126992) [20:02:29] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2327878 (10hashar) [20:04:23] (03CR) 10Hashar: [C: 032] parsoid: switch 'npm test' to Nodepool + Node 4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/290744 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [20:06:10] (03Merged) 10jenkins-bot: parsoid: switch 'npm test' to Nodepool + Node 4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/290744 (https://phabricator.wikimedia.org/T126992) (owner: 10Hashar) [20:07:07] 05Continuous-Integration-Scaling, 06Services, 07Tracking, 07WorkType-NewFunctionality: [tracking] Migrate services CI jobs to node 4.3 - https://phabricator.wikimedia.org/T126991#2327896 (10hashar) [20:07:15] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2327893 (10hashar) 05Open>03Resolved a:03hashar Complete! With a lot of help from @Palad... [20:07:33] 05Continuous-Integration-Scaling, 06Services, 07Tracking, 07WorkType-NewFunctionality: [tracking] Migrate services CI jobs to node 4.3 - https://phabricator.wikimedia.org/T126991#2029072 (10hashar) [20:08:47] 05Continuous-Integration-Scaling, 03releng-201516-q4, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2327905 (10hashar) [20:08:50] 05Continuous-Integration-Scaling, 06Services, 07Tracking, 07WorkType-NewFunctionality: [tracking] Migrate services CI jobs to node 4.3 - https://phabricator.wikimedia.org/T126991#2029072 (10hashar) 05Open>03Resolved All identified repositories have been migrated. Parsoid has been done a minute or so ag... [20:14:32] (03Abandoned) 10Hashar: Add npm-run-doc-jsduck-node-4.3 test [integration/config] - 10https://gerrit.wikimedia.org/r/290564 (https://phabricator.wikimedia.org/T109005) (owner: 10Paladox) [20:15:15] (03Abandoned) 10Hashar: Update parsoid test [integration/config] - 10https://gerrit.wikimedia.org/r/290541 (owner: 10Paladox) [20:15:57] (03Abandoned) 10Hashar: If jsduck is not avilable install jsduck through bundle install [integration/config] - 10https://gerrit.wikimedia.org/r/290524 (owner: 10Paladox) [20:18:25] (03Abandoned) 10Hashar: Migrate visualeditors npm-run-doc to node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/290520 (owner: 10Paladox) [20:18:45] (03PS4) 10Hashar: Add MGChecker to test pipeline (Verified+2) group [integration/config] - 10https://gerrit.wikimedia.org/r/288514 (owner: 10Florianschmidtwelzow) [20:18:45] hashar ^^ thanks [20:19:22] (03CR) 10Hashar: [C: 032] "Thank you for the patch and reviews!!!!!!!" [integration/config] - 10https://gerrit.wikimedia.org/r/288514 (owner: 10Florianschmidtwelzow) [20:20:09] (03Merged) 10jenkins-bot: Add MGChecker to test pipeline (Verified+2) group [integration/config] - 10https://gerrit.wikimedia.org/r/288514 (owner: 10Florianschmidtwelzow) [20:20:36] Project beta-update-databases-eqiad build #8785: 04FAILURE in 35 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8785/ [20:27:02] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2029093 (10Arlolra) Thanks @hashar and @Paladox! [20:27:34] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs to Nodepool and add Node 4.3 support - https://phabricator.wikimedia.org/T126992#2327960 (10Paladox) Your welcome. :) [20:36:24] (03PS1) 10Hashar: [oojs/ui] migrate npm doc/demos to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290784 (https://phabricator.wikimedia.org/T128091) [20:36:48] (03CR) 10Hashar: [C: 04-1] "Pending result of jobs on https://gerrit.wikimedia.org/r/#/c/285972/" [integration/config] - 10https://gerrit.wikimedia.org/r/290784 (https://phabricator.wikimedia.org/T128091) (owner: 10Hashar) [20:40:24] hashar: It passes https://integration.wikimedia.org/ci/job/oojs-ui-npm-run-demos-node-4.3/7/console [20:40:26] :) [20:40:47] hashar: I think migrating qunit to nodepool will resolve the issues we keep having with it. [20:41:16] https://integration.wikimedia.org/ci/job/oojs-ui-npm-run-doc-node-4.3/7/console [20:48:07] (03CR) 10Hashar: [C: 032] [oojs/ui] migrate npm doc/demos to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290784 (https://phabricator.wikimedia.org/T128091) (owner: 10Hashar) [20:48:25] (03CR) 10Hashar: "Both doc and demos passed on Jessie / Node 4.3 :-} https://gerrit.wikimedia.org/r/#/c/285972/" [integration/config] - 10https://gerrit.wikimedia.org/r/290784 (https://phabricator.wikimedia.org/T128091) (owner: 10Hashar) [20:49:29] (03Merged) 10jenkins-bot: [oojs/ui] migrate npm doc/demos to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290784 (https://phabricator.wikimedia.org/T128091) (owner: 10Hashar) [20:50:10] 05Continuous-Integration-Scaling, 03releng-201516-q4, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2328018 (10hashar) [20:50:13] 05Continuous-Integration-Scaling, 10OOjs-UI, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate OOjs UI npm, npm-run-doc and npm-run-demos CI jobs to Nodepool - https://phabricator.wikimedia.org/T128091#2328015 (10hashar) 05Open>03Resolved We had some experimental jobs added via 5ff89eeb6104cece2... [20:50:30] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2328019 (10Krinkle) Possibly related: * https://github.com/karma-runner/karma/issues/1822 * https://github.com/karma-runner/karma/issues/411 We may wanna create a new ups... [20:50:39] hashar: ^ [20:51:14] 05Continuous-Integration-Scaling, 03releng-201516-q4, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#1987977 (10hashar) [20:51:29] 05Continuous-Integration-Scaling, 03releng-201516-q4, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#1988025 (10hashar) **Status as of 2016-05-25** Bulk of jobs have been migrated. Have to audit what is left to migrate. [20:58:12] (03PS1) 10Hashar: Drop old mwext-VisualEditor-npm job [integration/config] - 10https://gerrit.wikimedia.org/r/290787 (https://phabricator.wikimedia.org/T119143) [21:01:20] (03CR) 10Hashar: [C: 032] Drop old mwext-VisualEditor-npm job [integration/config] - 10https://gerrit.wikimedia.org/r/290787 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:03:07] (03PS1) 10Paladox: Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 [21:03:11] hashar ^^ [21:03:42] hashar: Im not sure about mediawiki-core-jsduck-publish [21:03:43] test [21:04:42] (03Merged) 10jenkins-bot: Drop old mwext-VisualEditor-npm job [integration/config] - 10https://gerrit.wikimedia.org/r/290787 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:05:25] (03CR) 10jenkins-bot: [V: 04-1] Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 (owner: 10Paladox) [21:07:37] (03PS2) 10Paladox: Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 [21:08:05] (03PS3) 10Paladox: Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 [21:08:44] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2328098 (10hashar) > Does anything stand out from the perspective of resource consumption? E.g. do the stalled processes use a lot of CPU? Checked on one that got stalled... [21:08:59] (03CR) 10jenkins-bot: [V: 04-1] Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 (owner: 10Paladox) [21:10:01] (03PS4) 10Paladox: Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 [21:11:14] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling: Tracking: Track what tests we have left to convert to nodepool - https://phabricator.wikimedia.org/T136249#2328123 (10Paladox) [21:11:21] hashar: ive created https://phabricator.wikimedia.org/T136249 [21:11:36] to help us with identifying the last tests to migrate to nodepool [21:11:37] :) [21:12:53] 10MediaWiki-Codesniffer: Community bonding evaluation for Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T133829#2328138 (10EBernhardson) 05Open>03Resolved * Detailed plan: After review i think the plan in the original proposal is sufficiently detailed to work off of for th... [21:12:56] 10MediaWiki-Codesniffer, 03Google-Summer-of-Code-2016: [GSoC 2016 Proposal] Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T130574#2139795 (10EBernhardson) [21:13:15] (03CR) 10Zfilipin: [C: 032] Update account creation code for AuthManager [ruby/api] - 10https://gerrit.wikimedia.org/r/290269 (https://phabricator.wikimedia.org/T135884) (owner: 10Gergő Tisza) [21:13:51] 10MediaWiki-Codesniffer: Community bonding evaluation for Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T133829#2328147 (10EBernhardson) [21:14:07] 10MediaWiki-Codesniffer: Community bonding evaluation for Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T133829#2246443 (10EBernhardson) The first meeting will be May 27th [21:14:18] (03PS1) 10Hashar: Make mwext-qunit to timeout earlier and not fail [integration/config] - 10https://gerrit.wikimedia.org/r/290794 (https://phabricator.wikimedia.org/T119143) [21:15:44] (03CR) 10Paladox: [C: 031] Make mwext-qunit to timeout earlier and not fail [integration/config] - 10https://gerrit.wikimedia.org/r/290794 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:17:20] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling: Tracking: Track what tests we have left to convert to nodepool - https://phabricator.wikimedia.org/T136249#2328179 (10Paladox) [21:18:16] (03CR) 10Hashar: "Krinkle: setting the timeout high enough and not to fail would be a workaround. That might have some nasty side effects such as a faillin" [integration/config] - 10https://gerrit.wikimedia.org/r/290794 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:20:39] Yippee, build fixed! [21:20:39] Project beta-update-databases-eqiad build #8786: 09FIXED in 38 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8786/ [21:24:01] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling: Tracking: Track what tests we have left to convert to nodepool - https://phabricator.wikimedia.org/T136249#2328233 (10hashar) We have some meta tasks already and bulk of the work is tracked solely via... [21:24:43] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling: Tracking: Track what tests we have left to convert to nodepool - https://phabricator.wikimedia.org/T136249#2328237 (10hashar) Oh and there is the very old {T47499} [21:24:45] paladox: tasks are mess really :-} [21:24:46] (03PS1) 10Zfilipin: Release minor version 0.6.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/290801 [21:24:55] hashar: Oh [21:25:33] zeljkof: that is lovely :-} [21:25:44] hashar: :) [21:25:59] zeljkof: might want to add a changelog entry in the readme :-} [21:26:09] I don't usually code this late, but when I do, I release gems :) [21:26:14] tgr/anomie are going to be happy [21:26:15] hashar: are qunit timeouts like https://integration.wikimedia.org/ci/job/mwext-qunit/16690/consoleFull caused by the patch or is that some tooling issue? [21:26:17] hashar: I did [21:26:25] that session manager rework has been crippling them for months :-(((( [21:26:33] https://gerrit.wikimedia.org/r/#/c/290801/1/README.md,cm [21:26:35] zeljkof: thanks a lot for working on this so late! [21:26:38] tgr: infra issue :( [21:27:00] tgr: no problem, it is my fault [21:27:04] tgr: I mean it is unlikely it the patch you sent in Gerrit, most probably an issue with Karma / Chromium . We had both upgraded [21:27:08] I should have done it long ago [21:27:13] but was busy with other stuff [21:27:13] tgr: https://phabricator.wikimedia.org/T136188 [21:27:43] (03CR) 10Zfilipin: [C: 032] Release minor version 0.6.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/290801 (owner: 10Zfilipin) [21:27:43] zeljkof: we will want to bump the version in all repos [21:28:00] hashar: will test with MF, that is where tgr noticed the problem [21:28:04] since whenever master branches enable the feature all selenium jobs relying on account creation will explode :D [21:28:27] I will have to bump the dependency on new api gem in mw-selenium gem [21:29:33] oh, why is the api gem waiting for core commit to merge?! https://integration.wikimedia.org/zuul/ [21:29:54] ah, it is in mediawiki queue [21:30:48] (03PS1) 10Paladox: Migrate mediawiki-phpunit-phpflavour-composer to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290806 (https://phabricator.wikimedia.org/T119138) [21:30:51] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2328262 (10hashar) Maybe we could rollback the karma version bump 72b0d9da08d6ea8c1842d76aaefe8e57046a2026 ? Then Chromium 50 got pushed to Trusty and automatically inst... [21:30:52] hashar: ^^ [21:31:22] zeljkof: https://gerrit.wikimedia.org/r/#/c/290805/ should trigger the selenium errors [21:32:07] hashar: Maybe we can now reduce some trusty instances and add them to nodepool [21:32:35] and reduce some precise [21:32:37] tgr: great [21:32:49] please [21:32:56] I am releasing gems, and we can test if updating gemfile fixes the commit [21:34:09] (03Merged) 10jenkins-bot: Release minor version 0.6.0 [ruby/api] - 10https://gerrit.wikimedia.org/r/290801 (owner: 10Zfilipin) [21:35:27] 05Continuous-Integration-Scaling, 03releng-201516-q4, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2328292 (10hashar) [21:37:27] thcipriani: hey, I have an issue with scap3. ORES beta was down when I was deploying because the service restart was made before another check named "setup_virtualenv" (https://phabricator.wikimedia.org/T135677#2308180). See our config: https://github.com/wiki-ai/ores-wikimedia-config/blob/master/scap/checks.yaml the restart is another check and not set in [21:37:27] scap.cfg. What would be the best way to tell scap not to run the restart before another check [21:37:38] 07Browser-Tests, 10Wikidata, 07Tracking: [tracking] make Wikidata browsertests non-flaky - https://phabricator.wikimedia.org/T92619#2328300 (10Jdlrobson) [21:37:40] 10Browser-Tests-Infrastructure, 10MobileFrontend, 06Reading-Web-Backlog, 07Upstream: Upstream: Issue with Chrome driver with resizing window - https://phabricator.wikimedia.org/T88288#2328298 (10Jdlrobson) 05stalled>03Resolved I can only assume one year later that given this hasn't come up again this i... [21:37:41] (03PS5) 10Paladox: Migrate mediawiki core jsduck test to jessie [integration/config] - 10https://gerrit.wikimedia.org/r/290791 (https://phabricator.wikimedia.org/T119143) [21:37:59] 05Continuous-Integration-Scaling, 03releng-201516-q4, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2328303 (10hashar) What is left invoking `npm` and not on Nodepool instances? From the XML Jenkins jobs: ``` $ grep -... [21:38:04] 10Continuous-Integration-Infrastructure, 07Regression: Jobs sometimes fail with "/usr/local/bin/npm: No such file or directory" - https://phabricator.wikimedia.org/T129617#2328306 (10hashar) [21:38:06] 10releng-201516-q2: [keyresult] Migrate majority of CI jobs to Nodepool (part 1) - https://phabricator.wikimedia.org/T114315#2328308 (10hashar) [21:38:08] 10Continuous-Integration-Infrastructure: Run 'npm' job with Node 4 (instead of Node 0.10) - https://phabricator.wikimedia.org/T126774#2328307 (10hashar) [21:38:10] 05Continuous-Integration-Scaling, 03releng-201516-q4, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2328304 (10hashar) 05Open>03Resolved a:03hashar [21:38:23] 10releng-201516-q2: [keyresult] Migrate majority of CI jobs to Nodepool (part 1) - https://phabricator.wikimedia.org/T114315#1691169 (10hashar) [21:39:22] Amir1: so the issue is restart is running before setup_virtualenv? [21:39:28] tgr: it did not fail :) https://gerrit.wikimedia.org/r/#/c/290805/ [21:39:35] thcipriani: yup [21:39:55] or maybe before the check ends [21:40:12] hashar: We may want to up 40 to 50 or 60 at https://phabricator.wikimedia.org/T133911 [21:40:26] Since we are migrating alot of tests. [21:40:53] Amir1: one second, digging in code to verify my assumption [21:41:08] (03PS2) 10Paladox: Migrate mediawiki-phpunit-phpflavour-composer to nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/290806 (https://phabricator.wikimedia.org/T119138) [21:41:10] legoktm: can I change a core global from extension.json config? [21:41:16] https://gerrit.wikimedia.org/r/#/c/290805/1/extension.json doesn't seem to work [21:42:14] okay thanks :) [21:42:43] I guess I can just use Depends-On [21:42:47] 10Continuous-Integration-Config, 05Continuous-Integration-Scaling, 10releng-201516-q3, 07WorkType-NewFunctionality: [keyresult] Migrate as many misc CI jobs as possible to Nodepool - https://phabricator.wikimedia.org/T119140#2328334 (10hashar) [21:42:49] 10releng-201516-q2: [keyresult] Migrate majority of CI jobs to Nodepool (part 1) - https://phabricator.wikimedia.org/T114315#2328330 (10hashar) 05Open>03Resolved a:03hashar Python/Ruby/Javascript jobs have been almost all migrated. There are a few left over but they need some architecture work or are jobs... [21:43:01] greg-g: ^^^finally closable and closed [21:43:17] basically ruby/javascript/python are all on Nodepool as of today [21:43:22] :) [21:45:19] YAY [21:45:39] and key result #2 is already half done probably [21:45:43] it has PHP in it [21:45:49] which I tracked in some other task :D [21:45:51] 05Continuous-Integration-Scaling, 10releng-201516-q3, 13Patch-For-Review: [keyresult] Migrate majority of CI jobs to Nodepool (part 2) - https://phabricator.wikimedia.org/T119138#2328347 (10hashar) Would need to audit the history, but a bulk of PHP jobs have been migrated fairly recently when I managed to pr... [21:46:18] * hashar blames paladox for fixing and migrating so many things *smiles* [21:46:29] :) [21:46:32] :) [21:46:44] ah yeah [21:46:45] https://phabricator.wikimedia.org/T119139 [21:46:47] that is the evil task [21:46:51] [keyresult] Migrate php (Zend and HHVM) CI jobs to Nodepool [21:47:05] Oh [21:47:11] it has a bunch of closed one at least :D [21:47:17] :) [21:47:46] I really like Phabricator Maniphest [21:47:54] but for project management it ends up being terrible [21:48:07] or I can't really manage to organize both at task level and at workboard level [21:48:24] Yep, maybe open a task in phabricator so when funding open ups we could propose that task [21:48:29] at least it is super flexible [21:48:46] unlike most issue trackers I have used [21:48:53] Yep. I created this https://phabricator.wikimedia.org/D234 [21:48:54] :) [21:49:24] o really [21:49:57] Yep. :) works too http://www.test-random-wikisaur.tk/diffusion/2/ [21:50:02] I havent done modern PHP in ages nor I am familiar with the Phabricator code base :( [21:50:03] See ^^ hashar [21:50:17] hashar: Oh, it is based on php 5.2 [21:51:26] maybe upstream would be interested [21:51:34] or some other orgs using Phabricator [21:51:43] there are quite a few now [21:51:44] hashar: Oh probaly [21:52:59] hashar: twentyafterfour will try and test arcyd which will drop users requirement of needing arc. [21:53:05] Since it will allow us to use git. [21:53:10] And land it. [21:53:20] 10Continuous-Integration-Infrastructure: Install php7 and the php-ast extension so etsy/phan can be run from jenkins - https://phabricator.wikimedia.org/T132636#2328374 (10EBernhardson) After playing around with dotdeb on labs, it's possible but there are potential inconsistencies. After looking around more i l... [21:53:26] yeah that is exciting [21:53:34] Yep [21:53:51] Also i got Luke081515 to migrate a few repos to use diffusion mirroring [21:53:59] Such as integration-config [21:54:04] which is upstream https://secure.phabricator.com/T5000 [21:54:09] which you can now view refs/changes/ on github. [21:54:10] :) [21:54:29] Yep that one [21:54:34] epriestley replied there and it seems he is more willing to support github like fork + pull request model [21:54:38] so one could fork a repo [21:54:43] then push as needed (plain git) [21:54:47] Yep, but 80 hrs of work [21:54:51] then use differential to ask for a merge [21:54:57] Only if someone pays so 150 dollars per hour [21:55:00] Yep [21:55:16] that is like 80*200 = 16 000 [21:55:25] I dont mind funding part of it personally [21:55:38] Oh ok. [21:55:40] :) [21:55:52] like say a thousand * a dozen over people and a few orgs and it is done -} [21:56:07] given 80 hours is an accurate estimation [21:56:12] Yep. [21:56:13] s/over/other/ [21:56:13] sorry [21:57:04] hashar: We did https://phabricator.wikimedia.org/T135327 [21:57:28] so some phabricator improvements are being funded. [21:57:35] zeljkof: https://gerrit.wikimedia.org/r/#/c/290805 now fails, but I have no idea what to update [21:57:41] https://phabricator.wikimedia.org/T136213 [21:58:06] mediawiki_api is probably a dependency of mediawiki_selenium [21:58:12] tgr: great, doing the needful [21:58:14] hashar: Maybe open a task in phabricator.wikimedia.org about that. [21:58:16] so does that need a new release as well? [21:58:16] will ping you when done [21:58:33] yes, releasing mw-selenium, will update MF repo [21:59:41] why was this event cancelled? https://phabricator.wikimedia.org/E196 [22:00:12] (03PS1) 10Zfilipin: Bump mediawiki_api dependency to 0.6 [selenium] - 10https://gerrit.wikimedia.org/r/290813 [22:00:35] twentyafterfour ^^ [22:00:35] (03CR) 10EBernhardson: "recheck" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/275747 (https://phabricator.wikimedia.org/T130004) (owner: 10Lethexie) [22:01:06] greg-g im not sure but twentyafterfour: replied in #wikimedia-codereview to say that it will still happend tomarror. [22:01:24] 05Continuous-Integration-Scaling, 10releng-201516-q3: [keyresult] Migrate majority of CI jobs to Nodepool (part 2) - https://phabricator.wikimedia.org/T119138#1819068 (10hashar) [22:01:56] paladox: weird, thanks [22:02:00] Ok [22:02:18] 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling, 07WorkType-NewFunctionality: Jobs should checkout source in a subdirectory of the workspace to avoid polluting it with CI related files - https://phabricator.wikimedia.org/T130208#2328405 (10hashar) [22:02:34] greg-g ive reopeneded it since it should be up to twentyafterfour or you. [22:02:36] Amir1: could you rename the checks with prefixes like: 01_setup_virtualenv, 02_restart_worker, 03_restart_web. This seems to work in practice; however, I think this may have more to do with the yaml module than scap code. I will make a patch to make this more explicit. [22:03:24] (03Abandoned) 10Paladox: Add pywikibot-npm-node-4.3 experimental job to pywikibot/i18n [integration/config] - 10https://gerrit.wikimedia.org/r/290703 (https://phabricator.wikimedia.org/T130207) (owner: 10Paladox) [22:03:27] (03PS2) 10Paladox: In node-4.3 clone under src [integration/config] - 10https://gerrit.wikimedia.org/r/290702 [22:03:42] (03PS3) 10Paladox: In node-4.3 clone under src [integration/config] - 10https://gerrit.wikimedia.org/r/290702 (https://phabricator.wikimedia.org/T130208) [22:04:00] (03CR) 10jenkins-bot: [V: 04-1] Add the SpaceBeforeControlStructureBraceSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/275747 (https://phabricator.wikimedia.org/T130004) (owner: 10Lethexie) [22:04:09] thcipriani: thanks [22:05:34] anyway it is past midnight [22:05:45] Oh [22:05:48] :) [22:05:48] and my cron "@midnight shutdown" is bugged [22:05:54] its only 11:05pm [22:05:57] here [22:05:57] have sweet dreams [22:06:06] perfide Albion!!!!!!!!!!! [22:06:11] Ok [22:06:25] ( https://en.wikipedia.org/wiki/Perfidious_Albion ) :D [22:06:26] no reason to cancel code review, it's still happening [22:06:45] which is about the most friendly sentence a french can say to english people [22:06:48] twentyafterfour: Yep ive re openeded it. [22:06:54] Oh [22:06:55] (03CR) 10Zfilipin: [C: 032] Bump mediawiki_api dependency to 0.6 [selenium] - 10https://gerrit.wikimedia.org/r/290813 (owner: 10Zfilipin) [22:07:24] 06Release-Engineering-Team, 05Release: MW-1.28.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T135559#2328435 (10mmodell) [22:08:45] thcipriani: https://github.com/wiki-ai/ores-wikimedia-config/commit/b633f8700c2a87b720f7176e1fb1648c4a657484 [22:08:50] let me try deploying [22:09:10] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review, 07Tracking: Track what tests we have left to convert to nodepool }tracking] - https://phabricator.wikimedia.org/T136249#2328437 (10Danny_B) [22:09:29] Amir1: kk, that's what worked in my testing, but there's definitely nothing in scap ensuring that ordering right now. [22:09:53] it would be great to have it thcipriani [22:10:07] I don't think that would be hard [22:10:19] let me add it to my to-do list [22:10:26] Amir1: agreed. [22:11:04] the logic is all in: scap/checks.py load method [22:11:58] yeah, I'm trying to deploy, it can't connect to sca01 again [22:12:33] (03Merged) 10jenkins-bot: Bump mediawiki_api dependency to 0.6 [selenium] - 10https://gerrit.wikimedia.org/r/290813 (owner: 10Zfilipin) [22:13:20] Amir1: :( [22:13:42] we've got a patch cherry picked on deployment-tin for keyholder that might be the issue here. [22:13:57] * thcipriani checks [22:14:00] It's okay [22:14:36] we can wait if you have some WIP patches about keholder/ssh stuff [22:15:09] the keyholder patch is about ready to land, as soon as operations folks are willing to merge it [22:15:25] it should be 100% done as of now. I commented on the ores patch mentioning that [22:15:57] the keyholder patch probably needs to be updated on deployment-puppetmaster [22:16:51] Amir1: should work now, puppet is broken on deployment-sca01 :(, didn't replace the public key like it should have [22:17:30] oh [22:17:40] (03PS1) 10Zfilipin: Release patch version 1.7.1 [selenium] - 10https://gerrit.wikimedia.org/r/290817 [22:17:43] twentyafterfour: can we cherry pick it? If yes, I do [22:17:53] thcipriani: thanks [22:17:59] let me try [22:18:50] connected successfully, let's say if it can deploy [22:20:52] 10Beta-Cluster-Infrastructure, 06Revision-Scoring-As-A-Service: ores-beta is down - https://phabricator.wikimedia.org/T135677#2328519 (10Ladsgroup) With [[https://github.com/wiki-ai/ores-wikimedia-config/commit/b633f8700c2a87b720f7176e1fb1648c4a657484|b633f87]], we moved forward but still #scap3 doesn't unders... [22:21:17] (03CR) 10Zfilipin: [C: 032] Release patch version 1.7.1 [selenium] - 10https://gerrit.wikimedia.org/r/290817 (owner: 10Zfilipin) [22:21:21] 03Scap3: Scap3 checks should act based on alphabetical name of checks - https://phabricator.wikimedia.org/T136253#2328523 (10Ladsgroup) [22:22:28] 03Scap3, 06Revision-Scoring-As-A-Service: Scap3 checks should act based on alphabetical name of checks - https://phabricator.wikimedia.org/T136253#2328523 (10Ladsgroup) [22:22:30] (03Merged) 10jenkins-bot: Release patch version 1.7.1 [selenium] - 10https://gerrit.wikimedia.org/r/290817 (owner: 10Zfilipin) [22:25:09] Still errors [22:25:16] but in something else [22:25:28] fatal: reference is not a tree: 4e0a05ee7f7252af61fadbb3ff0454a0c1a36245 [22:26:42] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review, 07Tracking: Track what tests we have left to convert to nodepool (tracking) - https://phabricator.wikimedia.org/T136249#2328553 (10Danny_B) [22:27:34] I fixed it but now I can't connect to sca01 [22:27:42] 22:27:09 ['/usr/bin/scap', 'deploy-local', '-v', '--repo', 'ores/deploy', '--force', '-g', 'worker', 'fetch'] on deployment-sca01.deployment-prep.eqiad.wmflabs returned [255]: Permission denied (publickey,keyboard-interactive). [22:29:01] Amir1: hmm, puppet changed the key back. Even though it's showing: Error 400 on SERVER [22:29:14] hmm [22:29:18] tgr: looks good to me so far [22:29:18] https://gerrit.wikimedia.org/r/#/c/290805/ [22:29:23] https://integration.wikimedia.org/ci/job/mwext-mw-selenium/6793/console [22:29:58] it's the public_key_source parameter in the ores patch. conflicts with the new keyholder patch I think. [22:30:04] maybe my cherry-picked commit in puppetmaster is causing this? [22:30:29] ish a combination of cherry-picked commits [22:30:53] thcipriani: where is public_key_source used? [22:31:22] twentyafterfour: in some of my commits, cherry-picked in beta [22:31:23] it's in /etc/puppet/modules/ores/manifests/scapdeploy.pp [22:32:16] which I *think* is actually out-of-date with the current patch-set [22:32:46] zeljkof: awesome! I'll go through the projects with a gemfile then [22:32:55] thcipriani: no, because there's another commit I made [22:32:59] the latest version of that patch doesn't have the public_key_source, so beta is just out of date [22:33:04] and out of date at the same time [22:33:07] should I do anything special? like 'check experimental' to make sure all tests are run? [22:33:25] git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/03/280403/51 && git cherry-pick FETCH_HEAD [22:33:46] twentyafterfour: may have to run puppet on the puppetmaster if there's no public_key_source in that file. [22:33:58] if the problem persists, may just need a puppetmaster restart [22:34:43] thcipriani: want me to take care of the cherry picks on deployment-puppetmaster? [22:34:59] twentyafterfour: if you're there, please :) [22:35:14] tgr: over and out for this week, from now on the problems should be solved by updating the gemfile, like here https://gerrit.wikimedia.org/r/#/c/290805/ [22:35:34] thcipriani: on it [22:36:40] zeljkof: ack, thank you so much for working past midnight on this! [22:37:01] thcipriani: twentyafterfour there was a commit I made, '[LOCAL HACK] change ores key' I deleted it [22:38:11] tgr: no problem, if I have managed my time better... :) [22:38:51] !log running puppet agent manually on sca01 [22:38:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:39:31] yup [22:39:34] it's fixed now [22:40:23] neat. [22:50:26] twentyafterfour: it seems a "public_key_source" is left in modules/service/manifests/deploy/scap.pp [22:50:41] or I'm wrong [22:51:52] you deleted it but probably the puppetmaster is not updated [22:56:12] I realized what was the problem, git rebase conflict [22:56:15] got fixed [22:56:21] oh [22:56:29] 10Continuous-Integration-Config, 10MediaWiki-General-or-Unknown, 10MobileFrontend, 06Reading-Web-Backlog, 03Reading-Web-Sprint-73-One day we'll finish the EventLogging card: MediaWiki core and MobileFrontend break branches REL1_25 and fundraising/REL1_25 ... - https://phabricator.wikimedia.org/T135906#2328615 [22:58:08] and now another error: "Error 400 on SERVER: secret(): invalid secret keyholder/deploy-service.pub" [22:59:13] I think it should be "servicedeploy_rsa" [23:02:38] (03PS1) 10Jdlrobson: Drop Gather from testing infrastructure [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) [23:12:44] (03CR) 10Paladox: [C: 04-1] Drop Gather from testing infrastructure (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson) [23:18:49] 10Deployment-Systems, 06Release-Engineering-Team, 06Operations, 03Scap3 (Scap3-MediaWiki-MVP): Completely port l10nupdate to scap - https://phabricator.wikimedia.org/T133913#2328633 (10Dereckson) [23:18:52] 10Deployment-Systems, 03Scap3, 13Patch-For-Review, 15User-bd808: Update l10nupdate to be compatible with scap3 - https://phabricator.wikimedia.org/T136157#2328629 (10Dereckson) 05Open>03Resolved p:05Triage>03Normal a:03bd808 Done in 6f10f415. [23:26:33] 03Scap3: Scap hangs when a server isn't responsive - https://phabricator.wikimedia.org/T136255#2328639 (10Dereckson) [23:28:15] 10Continuous-Integration-Infrastructure: mwext-qunit job has karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2328654 (10Paladox) @hashar chrome 51 was released today. I'm wondering is it this https://chromium.googlesource.com/chromium/src/+/5672f89725063a8e2edadef7e76448a57f869e... [23:35:37] (03PS2) 10Jdlrobson: Drop Gather from testing infrastructure [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) [23:37:43] (03CR) 10jenkins-bot: [V: 04-1] Drop Gather from testing infrastructure [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson) [23:44:18] (03CR) 10Paladox: [C: 04-1] "Hi you need to remove the rest of the test in that repo but leave only archived template." [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson) [23:44:55] (03PS2) 10Madhuvishy: Add job that allows for updating analytics refinery artifacts with latest source jars [integration/config] - 10https://gerrit.wikimedia.org/r/290640 (https://phabricator.wikimedia.org/T130123) [23:46:40] (03PS3) 10Paladox: Drop Gather from testing infrastructure [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson) [23:47:00] (03CR) 10Paladox: "I removed all the other tests from the repo only leaving archived." [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson) [23:58:58] (03CR) 10Paladox: [C: 031] Drop Gather from testing infrastructure [integration/config] - 10https://gerrit.wikimedia.org/r/290822 (https://phabricator.wikimedia.org/T136176) (owner: 10Jdlrobson)