[00:06:01] Project beta-scap-eqiad build #102398: 04STILL FAILING in 1 min 9 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102398/ [00:09:16] PROBLEM - Puppet run on deployment-cache-upload04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:11:08] For the beta-scap-eqiad issues: we commited a revert change. [00:15:53] Project beta-scap-eqiad build #102399: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102399/ [00:22:14] hmmm...I don't seem to be able to get into many of the deployment-prep servers :\ [00:23:14] oh, nvmd, just saw email about ldap breakage. [00:25:53] Project beta-scap-eqiad build #102400: 04STILL FAILING in 1 min 8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102400/ [00:26:35] Project selenium-Flow » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #17: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/17/ [00:27:26] Project selenium-Flow » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #17: 04FAILURE in 11 min: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/17/ [00:32:29] Project beta-scap-eqiad build #102401: 04STILL FAILING in 1 min 9 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102401/ [00:34:18] PROBLEM - Puppet run on deployment-cache-upload04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [00:35:52] Project beta-scap-eqiad build #102402: 04STILL FAILING in 1 min 9 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102402/ [00:45:53] Project beta-scap-eqiad build #102403: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102403/ [00:47:27] sudo: ldap_start_tls_s(): Can't contact LDAP server [00:47:36] is there some problem with LDAP on labs? [00:47:50] yes SMalyshev, it's on the labs list now [00:48:17] ah, ok, thanks... looks like I[m not on that list, will read the archive [00:48:38] labs-l is the one you want [00:49:32] it says it should be fixed within 30 mins from 00:22 which is like 4 mins from now :) [00:49:38] I'll patiently wait [00:52:06] which instance are you having problems with? [00:52:27] e.g. wdq-beta.eqiad.wmflabs is broken [00:52:37] i.e. ldap is broken there [00:53:30] wdq-beta? is that an instance? [00:53:55] Krenair should i revert https://github.com/wikimedia/mediawiki/commit/f7dad57c64db3eb1296894c2d3ae97b9f7f27c4c#diff-268cf4a97e2c7483271ee0e4356afe8d since it is causing https://phabricator.wikimedia.org/T135199 [00:55:28] I'm not the person to ask about that [00:55:46] Oh ok [00:55:54] Project beta-scap-eqiad build #102404: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102404/ [00:55:59] Krenair: yes. https://wikitech.wikimedia.org/wiki/Nova_Resource:Wdq-beta.wikidata-query.eqiad.wmflabs [00:56:18] yep still can't log in... [00:56:18] Okay, wda-beta.wikidata-query.eqiad.wmflabs [00:56:27] deployment-prep instances have the same problem, I'm asking now [00:56:38] should still work as root@ if you have a root key set up [00:57:06] Krenair: I'm logged in there, so I still can use existing sessions, but not new logins and no scp anymore [00:57:35] and no sudo: sudo: unknown uid 10977: who are you? [00:58:53] sudo is based on ldap [01:01:54] yeah I guessed so :) so it looks like it's still broken [01:02:34] looks like everything with self-hosted puppetmasters is broken [01:05:05] Krenair: may be. so would updating the puppetmaster fix it? [01:05:40] updating the ldap config on the puppetmaster maybe, we're looking at it [01:05:53] Project beta-scap-eqiad build #102405: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102405/ [01:06:48] ah, funny thing - I can't log in to puppetmaster either of course :) [01:07:50] needs an existing root [01:07:54] mostly all ops [01:15:54] Project beta-scap-eqiad build #102406: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102406/ [01:24:59] Project beta-scap-eqiad build #102407: 04STILL FAILING in 16 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102407/ [01:34:41] Project beta-scap-eqiad build #102408: 04STILL FAILING in 0.45 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102408/ [01:34:43] Yippee, build fixed! [01:34:43] Project browsertests-Wikidata-SmokeTests-linux-firefox-sauce build #624: 09FIXED in 17 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce/624/ [01:44:41] Project beta-scap-eqiad build #102409: 04STILL FAILING in 0.16 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102409/ [01:55:57] Project beta-scap-eqiad build #102410: 04STILL FAILING in 1 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102410/ [02:05:52] Project beta-scap-eqiad build #102411: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102411/ [02:15:58] Project beta-scap-eqiad build #102412: 04STILL FAILING in 1 min 16 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102412/ [02:25:54] Project beta-scap-eqiad build #102413: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102413/ [02:35:52] Project beta-scap-eqiad build #102414: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102414/ [02:38:20] PROBLEM - Parsoid on deployment-parsoid06 is CRITICAL: Connection refused [02:38:46] Project beta-scap-eqiad build #102415: 04STILL FAILING in 1 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102415/ [02:40:53] I have no time to look into why beta-scap-eqiad is failing [02:41:10] clearly scap's ssh is broken somehow [02:45:53] Project beta-scap-eqiad build #102416: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102416/ [02:55:55] Project beta-scap-eqiad build #102417: 04STILL FAILING in 1 min 14 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102417/ [03:05:53] Project beta-scap-eqiad build #102418: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102418/ [03:15:54] Project beta-scap-eqiad build #102419: 04STILL FAILING in 1 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102419/ [03:25:58] Project beta-scap-eqiad build #102420: 04STILL FAILING in 1 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102420/ [03:30:08] 10Deployment-Systems, 03Scap3: Scap scripts on mw1017 are incorrect - https://phabricator.wikimedia.org/T135206#2291696 (10thcipriani) [03:30:14] 10Deployment-Systems, 03Scap3: Scap scripts on mw1017 are incorrect - https://phabricator.wikimedia.org/T135206#2291708 (10thcipriani) p:05Triage>03High [03:35:54] Project beta-scap-eqiad build #102421: 04STILL FAILING in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102421/ [03:45:52] Project beta-scap-eqiad build #102422: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102422/ [03:47:34] !log ldap failure has created a shadow mwdeploy user on beta, deleted using vipw [03:47:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [03:55:55] Project beta-scap-eqiad build #102423: 04STILL FAILING in 1 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102423/ [03:56:24] thcipriani|afk: blugh, thanks [03:56:59] yeah. still a problem getting to tmh01 I can't figure out. [03:57:08] mostly done. [03:59:38] which is ridiculous since it's the exact same public key as on the other machines. blerg. [04:01:58] I guess you just have to restart ssh then try a few more times? nobody else on the box. Solar radiation flipping bits. [04:05:57] Yippee, build fixed! [04:05:57] Project beta-scap-eqiad build #102424: 09FIXED in 1 min 14 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102424/ [04:06:30] yippee! [04:06:36] alright, g'night thcipriani|afk [04:06:37] !log changed ownership of mwdeploy public keys post shadow mwdeploy user removal is important [04:06:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [04:06:59] greg-g: o/ have a good night [04:17:48] Yippee, build fixed! [04:17:48] Project selenium-MultimediaViewer » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #10: 09FIXED in 21 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/10/ [05:11:03] this is a new one [05:11:04] 04:50:21 git.exc.GitCommandError: 'git remote update origin' returned with exit code 1 [05:11:04] 04:50:21 stderr: 'fatal: unable to look up current user in the passwd file: no such user [05:11:04] 04:50:21 Unexpected end of command stream [06:13:50] PROBLEM - Puppet run on deployment-mediawiki01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:32:01] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs from node 0.8/0.10 to 4.3 - https://phabricator.wikimedia.org/T126992#2029093 (10mobrovac) @hashar is there anything left here to do? For the transition period, is it possible... [06:53:54] RECOVERY - Puppet run on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [06:58:36] PROBLEM - Puppet run on deployment-restbase02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:00:44] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2291812 (10mmodell) scap::target no longer allows the key to be specified. It's now all-or-nothing, either it manages the user and the key or it does neither. [07:18:36] RECOVERY - Puppet run on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:50] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs from node 0.8/0.10 to 4.3 - https://phabricator.wikimedia.org/T126992#2291839 (10Paladox) @mobrovac yes but one problem though jsduck is not installed on nodepool Jessie which... [07:25:22] 10Deployment-Systems, 03Scap3: Scap scripts on mw1017 are incorrect - https://phabricator.wikimedia.org/T135206#2291841 (10mmodell) How could that happen? [07:33:07] PROBLEM - Puppet run on deployment-pdf02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [07:52:57] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs from node 0.8/0.10 to 4.3 - https://phabricator.wikimedia.org/T126992#2291878 (10mobrovac) >>! In T126992#2291839, @Paladox wrote: > @mobrovac yes but one problem though jsduc... [07:58:56] 05Continuous-Integration-Scaling, 10Parsoid, 06Services, 13Patch-For-Review, 07WorkType-NewFunctionality: Migrate Parsoid CI jobs from node 0.8/0.10 to 4.3 - https://phabricator.wikimedia.org/T126992#2291882 (10Paladox) @mobrovac Yes @Hashar did that in https://gerrit.wikimedia.org/r/#/c/280906/ But I a... [08:05:12] Project beta-scap-eqiad build #102448: 04FAILURE in 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102448/ [08:08:06] RECOVERY - Puppet run on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0] [08:15:58] Yippee, build fixed! [08:15:59] Project beta-scap-eqiad build #102449: 09FIXED in 1 min 14 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102449/ [09:32:36] (03PS2) 10Hashar: [NewUsersList] Add npmt test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287784 (owner: 10Paladox) [09:32:38] (03PS2) 10Hashar: [Negref] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287770 (owner: 10Paladox) [09:32:40] (03PS2) 10Hashar: [NoTitle] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287787 (owner: 10Paladox) [09:32:42] (03PS2) 10Hashar: [News] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287775 (owner: 10Paladox) [09:32:44] (03PS2) 10Hashar: [NetworkAuth] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287773 (owner: 10Paladox) [09:32:46] (03PS2) 10Hashar: [NamespaceHTML] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287762 (owner: 10Paladox) [09:32:48] (03PS2) 10Hashar: [NewsBox] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287779 (owner: 10Paladox) [09:32:50] (03PS2) 10Hashar: [NamespaceRelations] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287767 (owner: 10Paladox) [09:32:52] (03PS2) 10Hashar: [NewSignupPage] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287781 (owner: 10Paladox) [09:32:54] (03PS2) 10Hashar: [NamespacePaths] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/287765 (owner: 10Paladox) [09:32:56] (03PS1) 10Hashar: Merge composer/npm on 10 extensions [integration/config] - 10https://gerrit.wikimedia.org/r/288583 [09:38:30] (03CR) 10Hashar: [C: 032] Merge composer/npm on 10 extensions [integration/config] - 10https://gerrit.wikimedia.org/r/288583 (owner: 10Hashar) [09:59:05] !log Deleting non nodepool mediawiki PHPUnit jobs for T135001 (mediawiki-phpunit-hhvm mediawiki-phpunit-parsertests-hhvm mediawiki-phpunit-parsertests-php55 mediawiki-phpunit-php55) [09:59:06] T135001: Migrate PHPUnit MediaWiki core jobs to Nodepool - https://phabricator.wikimedia.org/T135001 [09:59:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:00:17] 10Continuous-Integration-Config, 05Continuous-Integration-Scaling, 10releng-201516-q3, 03releng-201516-q4, and 2 others: Migrate PHPUnit MediaWiki core jobs to Nodepool - https://phabricator.wikimedia.org/T135001#2292036 (10hashar) [10:02:33] (03PS1) 10Hashar: Remove obsolete MediaWiki jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288586 (https://phabricator.wikimedia.org/T135001) [10:05:21] (03CR) 10Hashar: [C: 032] Remove obsolete MediaWiki jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288586 (https://phabricator.wikimedia.org/T135001) (owner: 10Hashar) [10:07:05] (03CR) 10jenkins-bot: [V: 04-1] Remove obsolete MediaWiki jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288586 (https://phabricator.wikimedia.org/T135001) (owner: 10Hashar) [10:10:35] (03PS2) 10Hashar: Remove obsolete MediaWiki jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288586 (https://phabricator.wikimedia.org/T135001) [10:10:37] (03PS1) 10Hashar: Migrate mw php5 job in 'php5' pipe to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/288587 [10:14:26] (03CR) 10Hashar: [C: 032] Migrate mw php5 job in 'php5' pipe to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/288587 (owner: 10Hashar) [10:15:45] (03Merged) 10jenkins-bot: Migrate mw php5 job in 'php5' pipe to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/288587 (owner: 10Hashar) [10:15:47] (03Merged) 10jenkins-bot: Remove obsolete MediaWiki jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288586 (https://phabricator.wikimedia.org/T135001) (owner: 10Hashar) [10:16:51] hu? fatal: unable to look up current user in the passwd file: no such user [10:17:12] eeek [10:17:18] jzerebecki: sounds lab has some issue :/ [10:18:12] seems only to be on integration-slave-trusty-1004 [10:18:20] filling a task [10:19:15] yea instance seems broken, asking for a password on ssh [10:19:30] thx [10:19:53] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292072 (10hashar) [10:20:49] !log Put integration-slave-trusty-1004 offline. Ssh/passwd is borked T135217 [10:20:50] T135217: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217 [10:20:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:22:13] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292089 (10hashar) At least it is reachable via salt though it is duplicated: ``` salt -v 'integration-slave-trusty-1004*... [10:26:40] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292106 (10hashar) ``` $ id jenkins-deploy id: jenkins-deploy: no such user ``` And from syslog: ``` May 13 10:25:01... [10:35:01] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292119 (10hashar) salt -v 'integration-slave-trusty-1004*' cmd.run 'sed -i -e "s/undef/ldap-labs.eqiad.wikimedia.org/g"... [10:38:12] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292125 (10hashar) Ran the exact same thing for the integration-puppetmaster. [10:44:19] 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure: 'fatal: unable to look up current user in the passwd file: no such user - https://phabricator.wikimedia.org/T135217#2292132 (10hashar) 05Open>03Resolved a:03hashar Puppet agent managed to run on the integration-puppetmaster and it... [10:44:29] jzerebecki: solved ;-} [10:49:15] nice :) [11:25:07] 03releng-201516-q4, 10Malu (Malu-Prototype), 07Surveys, 15User-zeljkofilipin: Send out browser testing user satisfaction survey - https://phabricator.wikimedia.org/T131123#2292198 (10zeljkofilipin) [12:48:54] RECOVERY - Puppet run on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0] [13:33:00] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2292449 (10Ladsgroup) >>! In T132747#2289809, @thcipriani wrote: >>>! In T132747#2289617, @mobrovac wrote: >> Ping? I reported this problem over a month ago and it st... [13:37:59] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2292467 (10mobrovac) >>! In T132747#2292449, @Ladsgroup wrote: >>>! In T132747#2289809, @thcipriani wrote: >>>>! In T132747#2289617, @mobrovac wrote: >>> Ping? I repo... [13:41:26] PROBLEM - jenkins_zmq_publisher on gallium is CRITICAL: Connection refused [13:42:24] ^I wasn't aware that was a thing? [13:42:38] twentyafterfour: or hashar is that relevant? [13:43:26] RECOVERY - jenkins_zmq_publisher on gallium is OK: TCP OK - 0.000 second response time on port 8888 [13:49:09] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2292478 (10thcipriani) >>! In T132747#2292449, @Ladsgroup wrote: > > It's because ORES is going to be at SCB in prod so basically I wanted the beta environment as si... [13:49:19] (03PS1) 10Hashar: zuul: remove 'npm' from experimental [integration/config] - 10https://gerrit.wikimedia.org/r/288602 [13:51:31] (03CR) 10Hashar: [C: 032] zuul: remove 'npm' from experimental [integration/config] - 10https://gerrit.wikimedia.org/r/288602 (owner: 10Hashar) [13:52:29] (03Merged) 10jenkins-bot: zuul: remove 'npm' from experimental [integration/config] - 10https://gerrit.wikimedia.org/r/288602 (owner: 10Hashar) [13:56:01] (03PS1) 10Hashar: Drop lenient puppet-lint job [integration/config] - 10https://gerrit.wikimedia.org/r/288606 [13:59:29] (03CR) 10Hashar: [C: 032] Drop lenient puppet-lint job [integration/config] - 10https://gerrit.wikimedia.org/r/288606 (owner: 10Hashar) [14:01:06] (03Merged) 10jenkins-bot: Drop lenient puppet-lint job [integration/config] - 10https://gerrit.wikimedia.org/r/288606 (owner: 10Hashar) [14:12:08] (03PS1) 10Hashar: Convert integration-composer-check* to a job template [integration/config] - 10https://gerrit.wikimedia.org/r/288608 [14:15:13] Project beta-scap-eqiad build #102485: 04FAILURE in 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102485/ [14:22:22] (03PS1) 10Hashar: Normalize analytics maven jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288611 [14:22:33] (03CR) 10Hashar: [C: 032] Normalize analytics maven jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288611 (owner: 10Hashar) [14:23:06] (03PS2) 10Hashar: Normalize analytics maven jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288611 [14:25:15] (03CR) 10Hashar: [C: 032] Normalize analytics maven jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288611 (owner: 10Hashar) [14:26:02] Yippee, build fixed! [14:26:02] Project beta-scap-eqiad build #102486: 09FIXED in 1 min 17 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/102486/ [14:26:57] (03Merged) 10jenkins-bot: Normalize analytics maven jobs [integration/config] - 10https://gerrit.wikimedia.org/r/288611 (owner: 10Hashar) [14:38:29] (03PS2) 10Florianschmidtwelzow: Add MGChecker to test pipeline (Verified+2) group [integration/config] - 10https://gerrit.wikimedia.org/r/288514 [14:39:51] !log remove shadow l10nupdate user from deployment-tin and mira in beta [14:39:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [14:53:06] (03CR) 10Paladox: [C: 04-1] "It seems to remove the builder from it which means this tests nothing now. Unless I'm wrong." [integration/config] - 10https://gerrit.wikimedia.org/r/288608 (owner: 10Hashar) [14:53:51] (03CR) 10Paladox: [C: 031] "Oh I did read it wrong sorry." [integration/config] - 10https://gerrit.wikimedia.org/r/288608 (owner: 10Hashar) [14:54:13] (03CR) 10Hashar: [C: 032] Convert integration-composer-check* to a job template [integration/config] - 10https://gerrit.wikimedia.org/r/288608 (owner: 10Hashar) [14:54:16] paladox: ;)) [15:02:00] 10Deployment-Systems, 03Scap3: Scap scripts on mw1017 are incorrect - https://phabricator.wikimedia.org/T135206#2292783 (10thcipriani) blerg. Because there are old binstubs in the system path that are shadowing the package installed ones! These don't seem to exist in Puppet anywhere, they may just be old cruf... [15:34:12] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2292867 (10Ladsgroup) I also need to mention that taking out ORES won't solve the issue, you can't login to sca-01 via keyholder when the key is not the key ORES defi... [15:38:32] hashar: :) [15:39:33] Yippee, build fixed! [15:39:33] Project selenium-MobileFrontend » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #12: 09FIXED in 17 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/12/ [15:53:38] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2292923 (10greg) [17:18:29] 05Gerrit-Migration, 03releng-201516-q4, 10Differential, 07Documentation, 07WorkType-NewFunctionality: Initial documentation of example Differential workflows (with Gerrit equivalents) - https://phabricator.wikimedia.org/T117058#2293107 (10greg) [17:18:32] 05Gerrit-Migration, 10Differential, 07Documentation, 07Upstream, 15User-greg: Document best practices to amend a change written by another contributor - https://phabricator.wikimedia.org/T121751#2293104 (10greg) 05Open>03Resolved a:03greg I think we're good here now, yes? https://www.mediawiki.org/... [17:40:14] 05Gerrit-Migration, 07Epic: Align Wikimedia and Phabricator code review processes - https://phabricator.wikimedia.org/T167#2293138 (10greg) [17:42:02] 05Gerrit-Migration, 07Epic: Align Wikimedia and Phabricator code review processes - https://phabricator.wikimedia.org/T167#2245 (10greg) (I just quickly responded to the various points in the descriptions with answers or tasks.) [17:42:40] 05Gerrit-Migration, 07Epic: Align basics of current Gerrit code-review process with Differential - https://phabricator.wikimedia.org/T167#2293147 (10greg) [18:02:51] 06Release-Engineering-Team, 15User-greg, 15User-zeljkofilipin: Plan 2016 RelEng team offsite - https://phabricator.wikimedia.org/T134830#2278940 (10greg) [18:03:27] 06Release-Engineering-Team, 15User-greg, 15User-zeljkofilipin: Plan 2016 RelEng team offsite - https://phabricator.wikimedia.org/T134830#2278940 (10greg) p:05Normal>03Low >>! In T134830#2279688, @zeljkofilipin wrote: > Can we make the task private? we could make it a security issue, which is kind of wro... [18:17:03] 06Release-Engineering-Team, 10Security-Data-Mapping: Release Engineering Data Map - https://phabricator.wikimedia.org/T133239#2293288 (10greg) [18:19:31] 06Release-Engineering-Team, 10Security-Data-Mapping: Release Engineering Data Map - https://phabricator.wikimedia.org/T133239#2293292 (10greg) p:05Triage>03Normal [18:46:58] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2292710 (10Dzahn) confirmed the trusty file works: https://releases.wikimedia.org/debian/dists/... [18:48:33] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2293402 (10Dzahn) don't see error in apache log either on the backend (bromine). i think we must... [18:49:00] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2293403 (10Dzahn) P.S. still no reason that the files are owned by "5038" [20:22:41] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations, 10Parsoid: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2293782 (10hashar) on bromine, both files look exactly the same with same weird uid... [20:22:55] 10MediaWiki-Releasing, 06Release-Engineering-Team, 06Operations, 10Parsoid: This file can not be downloaded: https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease - https://phabricator.wikimedia.org/T135238#2293784 (10hashar) [20:26:23] 03Scap3, 10scap, 13Patch-For-Review: scap::target shouldn't allow users to redefine the user's key - https://phabricator.wikimedia.org/T132747#2293791 (10mmodell) @ladsgroup: There was a problem when two different scap::targets defined a key for the same user. The key would be defined by whichever scap::targ... [21:43:32] 10Beta-Cluster-Infrastructure, 10Deployment-Systems, 03Scap3, 06Operations, 13Patch-For-Review: Automate the generation deployment keys (keyholder-managed ssh keys) - https://phabricator.wikimedia.org/T133211#2294115 (10mmodell) >>! In T133211#2275973, @faidon wrote: > I think having a flat service deplo... [21:54:49] 05Gerrit-Migration, 10Differential: Consider documenting the Arcanist installer for Windows that Paladox created - https://phabricator.wikimedia.org/T131939#2294139 (10Luke081515) Actually I can confirm, that this installer is useful for users, he installs arc and php for you, so you only have to use it ;). I... [22:05:01] 06Release-Engineering-Team, 13Patch-For-Review, 15User-greg: Make a table of access levels per service RelEng maintains per person - https://phabricator.wikimedia.org/T135187#2291186 (10hashar) Using https://gerrit.wikimedia.org/r/288711 and something like: `python matrix.py dduvall demon gjg hashar thcipr... [22:35:10] 06Release-Engineering-Team, 13Patch-For-Review, 15User-greg: Make a table of access levels per service RelEng maintains per person - https://phabricator.wikimedia.org/T135187#2294328 (10greg) :) nicely done. Now I just need to manually think of all the other places like Phabricator #repository-admins and th... [22:53:05] 10Beta-Cluster-Infrastructure: deployment-tin ssh: Connection closed by UNKNOWN - https://phabricator.wikimedia.org/T134777#2294368 (10Mattflaschen) p:05Triage>03High