[00:12:59] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Enable CI jobs on the new CiviCRM repos - https://phabricator.wikimedia.org/T118604#3450882 (10Eileenmcnaughton) We have some repos that hold the uncompiled code- but in the end I'v... [00:17:21] Yippee, build fixed! [00:17:21] Project selenium-Flow » chrome,beta,Linux,BrowserTests build #456: 09FIXED in 1 min 19 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/456/ [00:17:37] Yippee, build fixed! [00:17:38] Project selenium-Flow » firefox,beta,Linux,BrowserTests build #456: 09FIXED in 1 min 36 sec: https://integration.wikimedia.org/ci/job/selenium-Flow/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/456/ [00:37:29] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Use phpcs.xml of core in mediawiki-core-phpcs-jessie - https://phabricator.wikimedia.org/T170159#3450903 (10Fomafix) a:03Fomafix [01:20:54] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.2 - https://phabricator.wikimedia.org/T156120#3451094 (10Paladox) [01:24:54] 10Gerrit, 10Release-Engineering-Team (Backlog), 10Patch-For-Review: Update gerrit to 2.14.2 - https://phabricator.wikimedia.org/T156120#3451127 (10Paladox) [01:55:26] 10Deployment-Systems, 10Release-Engineering-Team (Kanban), 10Scap (Scap3-Adoption-Phase1), 10scap2, and 2 others: Deploy jobrunner with scap3 (Trebuchet jobrunner/jobrunner) - https://phabricator.wikimedia.org/T129148#3451167 (10Krinkle) >>! In T129148#3450253, @Krinkle wrote: > Just now I tried to deploy... [03:22:42] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [04:02:45] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [04:26:27] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,BrowserTests build #457: 04FAILURE in 30 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=BrowserTests/457/ [04:41:57] Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #457: 04FAILURE in 45 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/457/ [06:39:45] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Reading-Web-Backlog (Tracking): Create Phame blog for Readers department - https://phabricator.wikimedia.org/T170793#3451389 (10phuedx) 🤘 Thanks, @mmodell! [07:22:14] I still can't login into Gerrit, it says invalid password but it works on WikiTech. [07:23:41] Reception123: use your wikitech account, not the email address [07:23:43] :) [07:23:52] I can't login to gerrit, is it related to my rights being revoked? [07:23:59] Oh, thank you [07:23:59] Legal told me my gerrit is back [07:24:10] Now I'm getting an old error, "Cannot assign user name "reception123" to account 5117; name already in use." [07:24:16] which I thought it was taken care of? [07:24:19] I can login to phabricator though [07:24:24] Amir1: I had your account deactivated last Sunday or so [07:25:12] hashar: but legal told me last hour that phab and gerrit should be back [07:25:49] Amir1: they should have said something like: "I Am NOT An Engineer but your gerrit should be back" ;-) [07:26:20] :D [07:26:24] Reception123: try with an upper case R: Reception123 ? [07:26:39] So what should I do now to have it again? [07:26:44] I remember trying that last time and it not working but I'll try again [07:27:08] nope, same thing, just goes back to lowercase for the message [07:27:11] Reception123: else you are hit by the bug https://phabricator.wikimedia.org/T152640 [07:27:29] Yeah, I know it was that bug, but I remember someone saying it was fixed [07:27:36] but apparently I'm wrong though [07:29:24] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451423 (10hashar) From T169996 > Cannot assign user name "reception123" to account 5067; name already in use. Which is still happening today:... [07:29:43] thanks hashar [07:33:45] will catch up with all the above later [07:33:48] have someone at home [07:44:30] Amir1: gerrit should be fine if your wikitech is logging in [07:45:07] Amir1: what error message is gerrit giving you when you try to log in? [07:45:42] p858snake|L: It says my password and / or username is wrong [07:46:38] Amir1: I recommend opening a task, or sending RainbowSprinkles a quick email [07:48:54] There is already a task [08:01:43] Amir1: yeah I got your account deactivated. Will look at your task this morning and reenable it [08:01:50] i am busy right now [08:02:15] Okay, thanks. :) [08:05:01] afk for lunch [08:17:19] Reception123: I dont have a way to fix your Gerrit login issue. T152640 suggests to try with both lower and upper case: reception123 and Reception123 [08:17:20] T152640: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640 [08:17:46] most probably the database get confused somehow [08:25:37] hashar: Lower-case is the same thing. Wouldn't the database need to be checked? [08:28:15] Reception123: most probably. can you comment on https://phabricator.wikimedia.org/T152640 that you attempted to login with each lower and upper case please ? [08:28:29] Reception123: I might dig into it later this morning. Maybe there is a magic sql command to fix up your account [08:29:27] sure, and thanks [08:30:20] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451505 (10Reception123) For the record, I've tried both upper-case and lower-case and I get the exact same error. ``` Cannot assign user name "... [08:30:51] Reception123: ellse you will have to wait for Chad who would be around in 6-7 hours from now ( RainbowSprinkles ) [08:31:39] hashar: I PMed you about the reason why I needed to login to Gerrit [08:36:36] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451517 (10Paladox) It’s because he is missing an record from the external id table select * from accounts where full_name = ''; Then look... [08:37:19] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451518 (10Paladox) @hashar ^^ [08:41:07] paladox: hi! Apparently one of your comment is not complete https://phabricator.wikimedia.org/T152640#3451517 :D [08:41:22] paladox: and there is no need to ping me on the task, I am already subscribed to it! [08:42:44] hashar: It's been edited [09:26:53] paladox: thanks for the gerrit sql thing. Though I dont have database access so gotta wait for Chad to show up [09:26:59] Reception123:^^ [09:31:01] Ok, I'll wait then [09:32:38] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451623 (10hashar) >>! In T152640#2856931, @hashar wrote: > > Login with the upper case first letter seems to generate in Gerrit error log:... [09:58:42] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Regression, 10Upstream: Cannot log into Gerrit as of recent upgrade - https://phabricator.wikimedia.org/T152640#3451674 (10Paladox) @hashar this issue is related to a missing entry in the external_id table. It should fix the users problem once the missing en... [09:59:47] hashar your welcome :) [09:59:55] hashar it's a missing entry in the db [10:00:02] as this happened to marco [10:02:42] 10Scap (Scap3-Adoption-Phase1), 10releng-201516-q4, 10releng-201718-q1, 10Trebuchet: [keyresult] Migrate remaining trebuchet deployed services - https://phabricator.wikimedia.org/T129290#3451683 (10fgiunchedi) [10:02:44] 10Deployment-Systems, 10Scap (Scap3-Adoption-Phase1), 10scap2, 10monitoring, and 2 others: Deploy statsv with scap3 - https://phabricator.wikimedia.org/T129139#3451681 (10fgiunchedi) 05Open>03Resolved Completed ``` tin:/srv/deployment/statsv/statsv$ scap deploy 10:01:07 Started deploy [statsv/statsv@0... [10:03:30] Reception123 hashar i doint know the exact command chad uses to fix the entry but i belive it's a missing gerrit: in the db [10:03:48] (the gerrit: is under the external_id table which needs re added) [10:19:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog): operations-puppet-cache-update-jessie should run on a regular basis - https://phabricator.wikimedia.org/T171037#3451754 (10hashar) [10:22:24] paladox: yeah he will catch up with that later today I guess [10:22:29] ok :) [10:59:33] PROBLEM - Puppet errors on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:21:20] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Kanban), 10MW-1.30-release-notes (WMF-deploy-2017-07-11_(1.30.0-wmf.9)), 10Patch-For-Review, 10User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3451914 (10zeljkofilipin) This is all is ne... [11:39:52] (03PS1) 10Zfilipin: Use RelatedArticles' LocalSettings.php when running Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/366236 (https://phabricator.wikimedia.org/T164721) [11:42:16] Amir1: I am back around [11:42:38] hashar: okay, do you have a minute to check what's going on? [11:43:05] Amir1: I have no clue why it is still inactive :/ [11:43:44] hashar: is it inactive or it is and I'm using wrong password? it says "Invalid username or password." [11:44:21] 'ladsgroup' failed to sign in: Authentication error, account inactive [11:44:21] on the server side [11:44:40] okay, so nothing is wrong with the password [11:44:43] (03CR) 10Zfilipin: "Tested for core" [integration/config] - 10https://gerrit.wikimedia.org/r/366236 (https://phabricator.wikimedia.org/T164721) (owner: 10Zfilipin) [11:45:03] should we wait for others to chime in? [11:46:31] I have been using gerrit set-account Ladsgroup --active [11:46:42] which supposedly should reactivate the account, but apparently that does not work :-( [11:47:14] hashar: what do you think of retrying with "ladsgroup" (no capital letter)? I know gerrit is crazy about these [11:47:24] yeah I did both [11:47:52] would need to check the Gerrit database but I do not have access to it [11:47:52] :( [11:48:20] That's really bad, you should [11:48:29] Should we talk to Ops about that? [11:48:58] I barely know gerrit internal, so it is probably for me to not have access to it :D [11:49:46] okay :/ [11:52:55] (03PS1) 10AnotherLadsgroup: Revert "Remove myself from tests whitelist" [integration/config] - 10https://gerrit.wikimedia.org/r/366240 [11:53:38] hashar: for now, this would be useful to some degrees: https://gerrit.wikimedia.org/r/#/c/366240/ (I use one email address for both accounts, so full tests will be ran when I need to) [11:54:12] (03CR) 10Hashar: [C: 032] Revert "Remove myself from tests whitelist" [integration/config] - 10https://gerrit.wikimedia.org/r/366240 (owner: 10AnotherLadsgroup) [11:54:23] Thanks! [11:54:30] zeljkof: checking your patch https://gerrit.wikimedia.org/r/#/c/366236/ [11:55:32] (03Merged) 10jenkins-bot: Revert "Remove myself from tests whitelist" [integration/config] - 10https://gerrit.wikimedia.org/r/366240 (owner: 10AnotherLadsgroup) [11:55:35] hashar: thanks! should be trivial to review, and I have tested it at a test job the best I could, should not break anything [11:55:50] zeljkof: yeah hmm [11:55:53] zeljkof: though that is a hack for a single repo [11:56:48] zeljkof: yes, not sure how to do it in another way [11:56:58] hashar: ^ [11:57:13] so there is potentially another way [11:57:54] in CI we append bunch of snippets at the end of LocalSettings.php they are in integration/jenkins.git in mediawiki/conf.d/* [11:58:02] one of them set $wgWikimediaJenkinsCI = true; [11:58:20] which we introduced for the Wikibase extension, so they can finely tweaks settings when being run on Jenkins [11:59:03] example usage: https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/Wikibase.php#L37-L40 [11:59:22] So potentially in RelatedArticles.php we can do something like: [11:59:28] if ( isset( $wgWikimediaJenkinsCI ) && $wgWikimediaJenkinsCI == true ) ) { [11:59:43] require_once __DIR__ . '/tests/browsers/LocalSettings.php'; [11:59:44] } [12:00:20] (though that would trigger for any job, including the PHPUnit job. But maybe it will be all fine) [12:01:18] hashar: so, that is something that should be done in RelatedArticles repo? [12:01:42] Amir1: deployed [12:01:52] Thanks! [12:02:05] zeljkof: yup [12:02:29] hashar: well, let me try [12:02:50] zeljkof: though we are migrating toward extension.json [12:03:03] hashar: what does that even mean? :) [12:03:04] so we would need a hook [12:07:00] zeljkof: I am trying something :D [12:10:12] hashar: something like this? https://gerrit.wikimedia.org/r/#/c/366244/ [12:11:26] what is the equivalent of `npm test` or `rake test` for php? `composer test`? [12:11:29] * zeljkof is trying [12:12:26] https://www.mediawiki.org/wiki/Continuous_integration/Entry_points#PHP [12:13:07] zeljkof: something like https://gerrit.wikimedia.org/r/366245 [12:13:34] I am not sure whether it is a good design pattern to adapt though [12:13:59] another way is to add a PHP snippet in integration/jenkins mediawiki.d/ [12:14:11] which would glob( 'extensions/*/tests/browser/LocalSettings.php' ); [12:14:14] and include them automatically [12:14:34] hashar: ok, I have no clue what would be the PHP or mediawiki way [12:14:53] but at least we know CI is ready for RelatedArticles (a possibly) other repost [12:14:54] repos [12:15:14] but for RelatedArticles we need to do this tweak [12:16:14] hashar: almost all tests failed for your commit :D [12:17:04] looks like there is something wrong with json [12:17:12] amended [12:22:03] hashar i think it's because gerrit needs restarting after re activating an account (i think) [12:22:54] zeljkof: and of course the PHPUnit tests fail now https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm-jessie/11259/console [12:23:52] hashar: can you fix it? [12:24:21] zeljkof: how do we load the custom settings file with the rake based jobs? [12:24:48] hashar: I remember seeing a script, but it basically does the same thing I do [12:24:50] ah mw-selenium-setup [12:24:51] let me find it [12:25:08] yeah that is a slave script [12:25:34] invoked from the mw-selenium-nodepool macro [12:25:37] does something like [12:25:50] cd src/extensions/RelatedArticles/tests/browser [12:25:57] if [ -f LocalSettings ] ; // append it [12:26:00] hashar: that is where I got the inspiration for my patch [12:26:25] but since for node selenium tests are executed from core, I had to do it in the core [12:27:28] jenkins/bin/mw-selenium-setup.sh [12:27:35] so potentially you should be able to do: [12:27:39] - shell: | [12:27:41] cd src/extensions/$EXT_NAME/tests/browser [12:28:00] . . '/srv/deployment/integration/slave-scripts/bin/mw-selenium-setup.sh' [12:28:03] and that would do the magic [12:28:09] https://phabricator.wikimedia.org/diffusion/CIJE/browse/master/bin/mw-selenium-setup.sh [12:28:20] i.e. append LocalSettings of the triggered extension [12:28:26] yeah that is the script [12:28:32] but the only repo that has LocalSettings.php in tests/selenium is RelatedArticles [12:28:37] none of the other repos need it [12:28:57] so, the workaround is for RelatedArticles however we do it [12:29:05] hashar: ^ [12:29:07] MobileFrontend/tests/browser/LocalSettings.php [12:29:11] MultimediaViewer/tests/browser/LocalSettings.php [12:29:11] Popups/tests/browser/LocalSettings.php [12:29:11] QuickSurveys/tests/browser/LocalSettings.php [12:29:11] RelatedArticles/tests/browser/LocalSettings.php [12:29:16] there are a few more :} [12:29:22] oh [12:29:22] :) [12:29:41] but I don't think they use it [12:29:46] but then, I could be wrong [12:30:24] it might be a copy/paste, or it might be a pattern mobile web team really uses [12:30:50] na it is not [12:30:54] each have different content [12:31:20] with random tweaks :} [12:31:38] so yeah I would just port the behavior [12:32:52] Amir1 hashar http://gerrit.wikimedia.org/r/accounts/ladsgroup/active shows amir1 account is ok [12:32:58] so it's been reactivated [12:33:13] https://gerrit-review.googlesource.com/Documentation/rest-api-accounts.html#get-active [12:33:42] Amir1 did you enter the password correctly? [12:34:01] can you login into wikitech with the account? [12:34:02] I even changed the password and tried again [12:34:06] yes I can [12:34:10] ok [12:34:20] Amir1: still around? [12:34:22] i think gerrit needs a restart (needs the cache flushed) [12:34:22] Amir1: can you try again? [12:34:27] sure [12:34:35] then I guess I will clear the user cache and see whether it changes anything [12:34:37] hashar: I will try that after lunch and swat [12:35:17] hashar: yes, i was able to login now [12:35:24] :) [12:35:25] why it took so much time [12:35:30] fucking gerrit [12:35:54] I did gerrit set-account --active Ladsgroup [12:36:03] i.e. swapped '--active' and 'ladsgroup' [12:36:07] maybe that did the trick [12:36:15] lol case insensitive? [12:36:27] I have read the doc for set-account [12:36:34] which states the username should be last option [12:36:44] when the gerrit set-account --help says it should be first [12:36:50] maybe that is the reason [12:36:51] or a cache expired [12:36:53] anyway solved! [12:36:58] :) [12:41:13] (03CR) 10Hashar: [C: 032] [UserExport] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/366042 (owner: 10Umherirrender) [12:42:39] (03Merged) 10jenkins-bot: [UserExport] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/366042 (owner: 10Umherirrender) [12:44:43] PROBLEM - Puppet errors on deployment-etcd-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [12:49:21] 10Continuous-Integration-Infrastructure, 10Wikipedia-Android-App-Backlog: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452243 (10hashar) Ah nice. I did play with the job yesterday and it got confused because android-25 apparently has no ABI depende... [12:54:43] !log Gerrit: created repo integration/jenkinsci/android-emulator-plugin.git owned by access group integration-jenkinsci-android-emulator-plugin which has Mholloway - T170904 [12:54:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:54:50] T170904: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904 [12:59:43] 10Continuous-Integration-Infrastructure, 10Wikipedia-Android-App-Backlog: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452273 (10hashar) And I made a fork of the repository in Gerrit. Should be clonal via: git clone ssh://gerrit.wikimedia.org... [13:02:56] (03PS1) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [13:04:47] PROBLEM - Puppet errors on deployment-conf03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:04:51] (03CR) 10jerkins-bot: [V: 04-1] Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) (owner: 10Gergő Tisza) [13:09:58] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 3 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3452312 (10pmiazga) a:03pmiazga [13:11:48] 10Continuous-Integration-Infrastructure, 10Wikipedia-Android-App-Backlog: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452317 (10Mholloway) 05Open>03Resolved a:03Mholloway Got it! Yes, let's follow up there. Thank you, @hashar! [13:12:20] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3452321 (10pmiazga) https://gerrit.wikimedia.org/r/3662... [13:18:43] (03PS1) 10Mholloway: Update system image component name construction for APIs >= 25 [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) [13:46:43] Yippee, build fixed! [13:46:44] Project selenium-VisualEditor » firefox,beta,Linux,BrowserTests build #464: 09FIXED in 2 min 41 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/464/ [13:51:23] (03PS2) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [13:52:48] (03CR) 10jerkins-bot: [V: 04-1] Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) (owner: 10Gergő Tisza) [13:55:35] !log Jenkins: added JDK "Debian - OpenJdk 7" with JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 [13:55:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:58:07] (03PS3) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [13:58:21] (03PS4) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [14:04:22] (03CR) 10jerkins-bot: [V: 04-1] Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) (owner: 10Gergő Tisza) [14:12:10] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3452528 (10hashar) [14:12:43] (03PS1) 10Hashar: Maven job for Jenkins plugins [integration/config] - 10https://gerrit.wikimedia.org/r/366257 (https://phabricator.wikimedia.org/T170904) [14:14:05] hashar: any idea what's wrong with https://gerrit.wikimedia.org/r/#/c/366248/ ? the error message is completely cryptic to me [14:14:35] tgr: have you tried ROT13 ? [14:15:08] "Non MediaWiki projects must not have jobs in common with the mediawiki queue." [14:15:22] that is because some non mediawiki project ends up having a job that is use by a mediawiki project [14:15:35] thus the non mediawiki project will end up in the same pipeline as mediawiki core/extensions/skins [14:16:05] yeah but all I am doing is adding a new mediawiki project [14:16:24] (03CR) 10Hashar: Set up CI for ReadingLists extension (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) (owner: 10Gergő Tisza) [14:16:26] is there some way of declaring it as such that I missed? [14:16:35] tgr: npm -> mwgate-npm https://gerrit.wikimedia.org/r/#/c/366248/4/zuul/layout.yaml@5100 [14:16:38] and that will do the trick [14:17:07] all the other extensions use plain npm though [14:17:11] we have the mediawiki repos in an isolated pipe. If we add the 'npm' job to it, that would merge in that pipe any repo triggering that 'npm' job [14:17:25] so we have a job named differently to keep the isolation [14:17:42] it is a bit messy though :( [14:18:27] (03CR) 10Hashar: [C: 032] Maven job for Jenkins plugins [integration/config] - 10https://gerrit.wikimedia.org/r/366257 (https://phabricator.wikimedia.org/T170904) (owner: 10Hashar) [14:18:42] every single mediawiki extension in that config file uses npm [14:18:55] tgr: they use "mwgate-npm" [14:19:15] - name: mediawiki/extensions/ReassignEdits [14:19:15] template: [14:19:15] - name: jshint [14:19:15] - name: jsonlint [14:19:15] - name: extension-unittests-generic [14:19:17] - name: npm [14:19:22] same file, right next block [14:19:38] yeah [14:19:48] I dont have the same view! [14:19:52] (03Merged) 10jenkins-bot: Maven job for Jenkins plugins [integration/config] - 10https://gerrit.wikimedia.org/r/366257 (https://phabricator.wikimedia.org/T170904) (owner: 10Hashar) [14:19:58] line 5102 of your patch it shows mwgate-npm [14:20:23] ugh, am I looking at something outdated? [14:20:28] (03CR) 10Hashar: "recheck" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [14:21:02] I guess yes [14:21:08] we did the switch few months ago [14:21:15] where are you looking at ? [14:21:15] sorry, that was stupid [14:21:36] just forgot to pull before writing the patch apparently [14:22:04] ah and the rebase caught up with the changes :D [14:22:07] that explains it [14:22:53] (03PS5) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [14:28:21] (03CR) 10jerkins-bot: [V: 04-1] Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) (owner: 10Gergő Tisza) [14:33:49] Yippee, build fixed! [14:33:49] Project selenium-WikiLove » firefox,beta,Linux,BrowserTests build #459: 09FIXED in 1 min 48 sec: https://integration.wikimedia.org/ci/job/selenium-WikiLove/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/459/ [14:38:33] RainbowSprinkles: Guess who's got a back-port for REL1_29? :-( What's the process for non-security patches - just merge it and tell people to wait for .1? [14:43:04] !log Jenkins: uploaded a patched android-emulator plugin for T150623 and restarting Jenkins [14:43:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:43:08] T150623: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623 [14:43:55] Project beta-scap-eqiad build #164771: 15ABORTED in 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164771/ [14:46:08] (03CR) 10Hashar: "I have build the plugin with this patch and updated our Jenkins instance." [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [14:48:44] PROBLEM - Puppet errors on deployment-sca04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [14:52:47] hashar: what's the gated version of running composer test? there is no mwgate-composer-test [14:55:12] PROBLEM - Puppet errors on deployment-sca02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:00:41] PROBLEM - Puppet errors on deployment-trending01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:03:30] (03CR) 10Hashar: "I have changed the OS Version from android-24 to android-25 and that fails with:" [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366253 (https://phabricator.wikimedia.org/T150623) (owner: 10Mholloway) [15:04:56] tgr: for mediawiki extensions? There is none [15:05:13] tgr: composer test is run in the job that runs the PHPUnit tests if there is a composer.json file [15:05:20] so they are mixed in together [15:05:23] (composer test + PHPUnit [15:05:31] I see, thanks [15:06:15] (03PS6) 10Gergő Tisza: Set up CI for ReadingLists extension [integration/config] - 10https://gerrit.wikimedia.org/r/366248 (https://phabricator.wikimedia.org/T168975) [15:10:17] PROBLEM - Puppet errors on deployment-changeprop is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:12:42] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:13:22] PROBLEM - Puppet errors on deployment-mcs01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:13:36] PROBLEM - Puppet errors on deployment-sca01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:15:48] PROBLEM - Puppet errors on deployment-sca03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:24:12] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3452755 (10hashar) `gdb wait-for-devices` complained with: ``` Your emulator is out of date, plea... [15:25:19] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Use phpcs.xml of core in mediawiki-core-phpcs-jessie - https://phabricator.wikimedia.org/T170159#3452757 (10Fomafix) a:05Fomafix>03None [15:27:30] PROBLEM - Puppet errors on deployment-eventlogging03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:45:01] RainbowSprinkles i did what you suggested and added a preference for the language + fallback to english if the lang string does not exist https://gerrit-review.googlesource.com/#/c/114092/ :) [15:59:36] 10Continuous-Integration-Infrastructure (Little Steps Sprint), 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: For MediaWiki extensions, merge rubocop inside mwext-mw-selenium-jessie - https://phabricator.wikimedia.org/T164479#3452906 (10zeljkofilipin) Testing job: https://in... [16:13:12] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452966 (10greg) a:05Mholloway>03hashar No offense, @Mholloway, b... [16:14:10] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452971 (10Mholloway) Ah, no worries, looks like Phab just auto-assig... [16:14:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: Create a Wikimedia fork of the Jenkins android emulator plugin - https://phabricator.wikimedia.org/T170904#3452972 (10greg) No worries, that usually is a good approximate :) [16:17:34] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [16:27:08] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikipedia-Android-App-Backlog, 10Jenkins, and 2 others: Upgrade CI emulator to API 25 - https://phabricator.wikimedia.org/T150623#3453003 (10Mholloway) And with the API 24 image we're back to boot timeouts, it seems. That's an... [16:52:32] RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0] [17:05:08] James_F: Yes. Just merge it and it'll make .1 [17:06:09] paladox: That's not quite how fallbacks work. You don't want your users to *have* to choose a secondary language. [17:06:30] RainbowSprinkles yeh, working on doing an update [17:06:36] managed to fix it so it works now :) [17:06:56] Everything has to be in polymer({}) for it to work due to it getting it from the rest api. [17:07:40] So easy to implement french. [17:08:04] I did this changes: 'Changements', for french only to see if it fallsback to english [17:08:14] and does since i did not translate any other text :) [17:12:02] 10Deployment-Systems, 10Scap (Scap3-Adoption-Phase1), 10scap2, 10monitoring, and 2 others: Deploy statsv with scap3 - https://phabricator.wikimedia.org/T129139#3453166 (10thcipriani) >>! In T129139#3451681, @fgiunchedi wrote: > Completed \o/ [17:13:57] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453176 (10Jdlrobson) In standup we decided we will ski... [17:22:31] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW-1.30.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T167893#3453245 (10thcipriani) 05Open>03Resolved [17:25:36] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Discovery, 10Discovery-Analysis, 10Operations: Setup a mirror for R language dependencies (CRAN) - https://phabricator.wikimedia.org/T170995#3453250 (10mpopov) @hashar Thank you for making this ticket and emailin... [17:39:25] 10Scap (Scap3-MediaWiki-MVP), 10scap2, 10Operations: Depool proxies temporarily while scap is ongoing to avoid taxing those nodes - https://phabricator.wikimedia.org/T125629#3453277 (10demon) 05Open>03Resolved [17:40:10] RainbowSprinkles https://gerrit-review.googlesource.com/#/c/114092/ looks better now :) [17:41:41] I wish polymer would allow us to access the propeties outside of polymer({}) but it dosen't seem possible. [17:42:53] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator: setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3453304 (10RobH) So it seems this system is offline, and loaded into the installer. It had been fully installed before, so I'm now investigating. [17:45:54] 10Beta-Cluster-Infrastructure, 10Discovery, 10Wikidata, 10Wikidata-Query-Service: recommendation_api module breaking beta labs puppet - https://phabricator.wikimedia.org/T171075#3453328 (10awight) [17:53:36] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator: setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3453367 (10RobH) a:05RobH>03Cmjohnson So this tries to load into the installer, and fails for sdb: ``` ┌─────────────┤ [!!] Partition disks ├─... [17:53:45] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator: replace sdb and then setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3453369 (10RobH) [17:54:57] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator: replace sdb and then setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3215445 (10RobH) This could also simply be a bad or lose cable on the drive bay! Basic troubleshooting of swapping sda and sdb to see if the e... [17:57:48] 10Beta-Cluster-Infrastructure, 10Discovery, 10Wikidata, 10Wikidata-Query-Service: recommendation_api module breaking beta labs puppet - https://phabricator.wikimedia.org/T171075#3453376 (10schana) @mobrovac Do you know anything about this? [18:18:59] 10Beta-Cluster-Infrastructure, 10Recommendation-API: recommendation_api module breaking beta labs puppet - https://phabricator.wikimedia.org/T171075#3453515 (10Smalyshev) Doesn't see to be related to WDQS - recommendation API is just calling it. [18:23:45] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453560 (10phuedx) The change that skips the test is {1... [18:24:34] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453563 (10phuedx) @Jdlrobson, @pmiazga: Now that {1323... [18:40:27] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator: replace sdb and then setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3453652 (10RobH) dell dispatch SR951188562 for the replacement hard disk, shipping to eqiad. [19:23:32] Project selenium-MinervaNeue » firefox,beta,Linux,BrowserTests build #15: 04FAILURE in 34 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/15/ [19:33:21] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453895 (10pmiazga) @phuedx - there is still one error... [19:42:55] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453941 (10Jdlrobson) We are still blocking merges in a... [19:45:53] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-extensions-Scribunto, and 5 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3453947 (10Jdlrobson) Looks like the... [19:53:58] 10Release-Engineering-Team (Kanban), 10Wikidata, 10Release, 10Train Deployments, 10Wikimedia-log-errors: Class undefined: \Wikibase\Lib\JsonUnitStorage - https://phabricator.wikimedia.org/T171097#3453979 (10demon) [19:55:30] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-extensions-Scribunto, and 5 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3454002 (10Jdlrobson) Tests in quest... [20:02:02] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3454029 (10Anomie) This does not seem to have anything... [20:03:16] 10Release-Engineering-Team (Kanban), 10Wikidata, 10Train Deployments, 10Wikimedia-log-errors: Class undefined: \Wikibase\Lib\JsonUnitStorage - https://phabricator.wikimedia.org/T171097#3454052 (10demon) [20:11:07] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MinervaNeue, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3454109 (10Jdlrobson) > This does not seem to have anyt... [20:11:09] Project beta-scap-eqiad build #164804: 04FAILURE in 7 min 18 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164804/ [20:14:20] Project beta-scap-eqiad build #164805: 04STILL FAILING in 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164805/ [20:14:57] 20:14:20 sudo: ldap_start_tls_s(): Connect error [20:14:57] 20:14:20 sudo: a password is required [20:16:18] Reedy: -operations is looking into it [20:16:20] ci will start failing now [20:16:25] 10Release-Engineering-Team, 10Africa-Wikimedia-Developers, 10Operations, 10Composer: Setup a Composer Repository (Packagist) for MediaWiki Extensions - https://phabricator.wikimedia.org/T170897#3454169 (10D3r1ck01) [20:20:04] Project beta-update-databases-eqiad build #18577: 04FAILURE in 1.5 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/18577/ [20:20:05] PROBLEM - Puppet errors on deployment-mediawiki05 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:22:03] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0] [20:22:39] PROBLEM - Puppet errors on jenkinstest is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:23:43] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [20:23:47] Project beta-scap-eqiad build #164806: 04STILL FAILING in 0.6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164806/ [20:23:47] PROBLEM - Puppet errors on deployment-jobrunner02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [20:23:47] PROBLEM - Puppet errors on deployment-prometheus01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [20:25:04] (03PS4) 10Ejegg: Tests for DonationInterface/vendor submodule [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [20:26:50] (03CR) 10jerkins-bot: [V: 04-1] Tests for DonationInterface/vendor submodule [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [20:33:56] Project beta-scap-eqiad build #164807: 04STILL FAILING in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164807/ [20:34:32] why does puppet fail with PROBLEM - puppet on phab-tin.phabricator.eqiad.wmflabs is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 1 minute ago with 2 failures. Failed resources (up to 3 shown): File[/srv/patches],File[/srv/deployment] when ldap is down. [20:34:51] (03PS2) 10Umherirrender: Run extension-unittests-generic along with extension-gate [integration/config] - 10https://gerrit.wikimedia.org/r/364505 [20:43:53] Project beta-scap-eqiad build #164808: 04STILL FAILING in 0.59 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164808/ [20:47:01] 10Scap (Scap3-Adoption-Phase1), 10Cassandra, 10RESTBase-Cassandra, 10Patch-For-Review, 10Services (doing): Deploy cassandra metrics collector via scap3 - https://phabricator.wikimedia.org/T137371#3454274 (10mobrovac) [20:49:56] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10Reading-Web-Backlog, and 3 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3454281 (10Jdlrobson) [20:50:22] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10Reading-Web-Backlog, and 4 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3445966 (10Jdlrobson) [20:53:53] Project beta-scap-eqiad build #164809: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164809/ [21:02:55] 10Release-Engineering-Team (Kanban), 10Wikidata, 10Wikimedia-log-errors: Class undefined: \Wikibase\Lib\JsonUnitStorage - https://phabricator.wikimedia.org/T171097#3454325 (10greg) [21:03:55] Project beta-scap-eqiad build #164810: 04STILL FAILING in 0.6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164810/ [21:04:53] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW 1.30.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T168050#3454350 (10demon) [21:08:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): Improve the Jenkins output to the QA-Alerts mail list - https://phabricator.wikimedia.org/T169779#3454360 (10Jrbranaa) 05Open>03Invalid After further investigation, it appears that this isn't an output issue from Jenkins. I was... [21:13:46] Project beta-scap-eqiad build #164811: 04STILL FAILING in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164811/ [21:16:38] 10Scap (Scap3-Adoption-Phase1), 10releng-201516-q2, 10releng-201516-q3, 10releng-201516-q4, 10scap2: [keyresult] Migrate all Service team owned services and MW to scap - https://phabricator.wikimedia.org/T109926#3454380 (10greg) [21:16:40] 10Scap (Scap3-Adoption-Phase1), 10scap2, 10Cassandra, 10RESTBase-Cassandra: Deploy logstash logback encoder with scap3 - https://phabricator.wikimedia.org/T116340#3454379 (10greg) [21:20:01] Project beta-update-databases-eqiad build #18578: 04STILL FAILING in 0.37 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/18578/ [21:23:44] Project beta-scap-eqiad build #164812: 04STILL FAILING in 0.62 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164812/ [21:26:06] RainbowSprinkles will gerrit know that ldap has a new cert? [21:27:00] Looks like nope [21:27:41] Oh, so gerrit needs restarting? [21:29:09] I'm doing it [21:31:17] ok thanks [21:32:40] PROBLEM - Puppet errors on jenkinstest is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:40] PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:42] PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:44] PROBLEM - Puppet errors on deployment-elastic06 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [21:32:44] PROBLEM - Puppet errors on deployment-fluorine02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:46] PROBLEM - Puppet errors on deployment-salt02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:49] PROBLEM - Puppet errors on deployment-kafka03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:49] PROBLEM - Puppet errors on integration-saltmaster is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:32:50] PROBLEM - Puppet errors on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:04] PROBLEM - Puppet errors on integration-slave-docker-1000 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:04] PROBLEM - Puppet errors on deployment-db03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [21:33:11] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:13] PROBLEM - Puppet errors on deployment-aqs01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [21:33:21] PROBLEM - Puppet errors on deployment-mcs01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:24] PROBLEM - Puppet errors on castor is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:24] PROBLEM - Puppet errors on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:31] PROBLEM - Puppet errors on deployment-kafka04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:33] PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:35] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Not possible for a skin to have PHPUnit tests (PHPunit tests for skin do not seem to be run) - https://phabricator.wikimedia.org/T170880#3454455 (10Jdlrobson) Mm... after debugging th... [21:33:39] PROBLEM - Puppet errors on deployment-sca01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:39] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:44] PROBLEM - Puppet errors on deployment-sca04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:44] PROBLEM - Puppet errors on deployment-puppetdb01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:44] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:44] PROBLEM - Puppet errors on integration-slave-docker-1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:45] Project beta-scap-eqiad build #164813: 04STILL FAILING in 0.63 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164813/ [21:33:46] PROBLEM - Puppet errors on deployment-prometheus01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:49] PROBLEM - Puppet errors on saucelabs-02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:49] PROBLEM - Puppet errors on deployment-jobrunner02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:49] PROBLEM - Puppet errors on deployment-elastic05 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:51] PROBLEM - Puppet errors on deployment-ircd is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:56] PROBLEM - Puppet errors on deployment-cache-upload04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:33:57] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:01] PROBLEM - Puppet errors on deployment-stream is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:06] PROBLEM - Puppet errors on deployment-zotero01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:06] PROBLEM - Puppet errors on deployment-kafka01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:14] PROBLEM - Puppet errors on saucelabs-03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:16] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:19] PROBLEM - Puppet errors on deployment-urldownloader is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:19] PROBLEM - Puppet errors on deployment-ms-be04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:23] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:23] PROBLEM - Puppet errors on integration-r-lang-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:25] PROBLEM - Puppet errors on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:30] PROBLEM - Puppet errors on deployment-poolcounter04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:33] PROBLEM - Puppet errors on deployment-tmh01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:35] PROBLEM - Puppet errors on integration-slave-jessie-android is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:43] PROBLEM - Puppet errors on deployment-kafka05 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:44] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:34:54] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3454457 (10Jdlrobson) [21:35:01] PROBLEM - Puppet errors on deployment-eventlogging04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:02] PROBLEM - Puppet errors on deployment-memc04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:05] PROBLEM - Puppet errors on deployment-mediawiki05 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:14] PROBLEM - Puppet errors on deployment-sca02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:17] PROBLEM - Puppet errors on deployment-parsoid09 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:18] PROBLEM - Puppet errors on deployment-changeprop is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:23] PROBLEM - Puppet errors on deployment-conf03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:26] PROBLEM - Puppet errors on deployment-elastic07 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:27] PROBLEM - Puppet errors on deployment-mediawiki04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:29] PROBLEM - Puppet errors on deployment-restbase02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:34] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:34] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:37] PROBLEM - Puppet errors on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:41] PROBLEM - Puppet errors on saucelabs-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:41] PROBLEM - Puppet errors on deployment-secureredirexperiment is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:41] PROBLEM - Puppet errors on deployment-trending01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:48] PROBLEM - Puppet errors on integration-publishing is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:48] PROBLEM - Puppet errors on deployment-etcd-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:48] PROBLEM - Puppet errors on deployment-sca03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:54] PROBLEM - Puppet errors on integration-slave-docker-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:35:55] PROBLEM - Puppet errors on deployment-apertium02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:36:00] PROBLEM - Puppet errors on deployment-memc05 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:36:20] PROBLEM - Puppet errors on deployment-db04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:36:20] PROBLEM - Puppet errors on deployment-sentry01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:36:48] PROBLEM - Puppet errors on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:37:22] PROBLEM - Puppet errors on integration-puppetmaster01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:37:23] PROBLEM - Puppet errors on deployment-ms-be03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:38:24] twentyafterfour does phabricator need restarting for the new ldap cert? [21:38:32] should we.. .. heh. ^ that [21:39:06] it should only need an puppet run and an apache restart [21:39:53] (03PS1) 10Jdlrobson: Include Vector in phpunit tests for MobileFrontend [integration/config] - 10https://gerrit.wikimedia.org/r/366470 (https://phabricator.wikimedia.org/T170880) [21:40:45] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3454477 (10Jdlrobson) ^ short term solution..? (untested)... [21:41:15] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3454478 (10Jdlrobson) cc @hashar your help here would be ap... [21:43:44] Project beta-scap-eqiad build #164814: 04STILL FAILING in 0.63 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164814/ [21:51:03] 10Scap (Scap3-Adoption-Phase1), 10scap2, 10Cassandra, 10RESTBase-Cassandra, and 2 others: Deploy logstash logback encoder with scap3 - https://phabricator.wikimedia.org/T116340#3454505 (10mobrovac) [21:51:55] Project beta-scap-eqiad build #164815: 04STILL FAILING in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164815/ [21:53:46] Project beta-scap-eqiad build #164816: 04STILL FAILING in 0.73 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164816/ [21:53:54] Project beta-scap-eqiad build #164817: 04STILL FAILING in 0.45 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164817/ [21:54:18] RainbowSprinkles Reedy does scap on beta need restarting ^^ [21:54:24] to pickup the new ldap cert [21:54:24] ? [21:54:32] You don't restart scap [21:54:39] It's not a daemon [21:54:43] woops i meant apache2 [21:55:06] I don't know, busy [21:55:10] It's just beta [21:55:29] ok [21:55:40] why would restarting apache fix that? [21:56:27] 21:51:55 21:51:55 scap failed: KeyError 'getpwuid(): uid not found: 2947' (duration: 00m 00s) [21:56:41] Probably ncsld or whatev needs kicking? [21:57:48] (03CR) 10Ejegg: "Hmm, phplint is unsafe for check?" [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [21:58:10] 10Continuous-Integration-Config, 10Wikimedia-Site-requests: Enforce our PHP coding standards on operations/mediawiki-config via composer? - https://phabricator.wikimedia.org/T162835#3454530 (10Jdforrester-WMF) 05Open>03Resolved a:03Reedy OK, this has been done. We want to drive up the standards over time... [21:58:36] I can't seem to get into deployment-tin, ugh [22:00:59] (03CR) 10Awight: "I think this regex needs updating," [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [22:03:01] (03PS5) 10Paladox: Tests for DonationInterface/vendor submodule [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [22:03:10] (03CR) 10Paladox: "> I think this regex needs updating," [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [22:03:46] Project beta-scap-eqiad build #164818: 04STILL FAILING in 0.59 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164818/ [22:05:11] 10Continuous-Integration-Config, 10Wikimedia-Site-requests: Enforce our PHP coding standards on operations/mediawiki-config via composer? - https://phabricator.wikimedia.org/T162835#3454585 (10Reedy) [22:08:28] Project beta-scap-eqiad build #164819: 04STILL FAILING in 0.39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164819/ [22:11:43] (03CR) 10jerkins-bot: [V: 04-1] Tests for DonationInterface/vendor submodule [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight) [22:13:46] Project beta-scap-eqiad build #164820: 04STILL FAILING in 0.6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164820/ [22:17:23] beta updates are still failing, LDAP? [22:17:47] scap failed: KeyError 'getpwuid(): uid not found: 2947' (duration: 00m 00s) [22:18:01] something need to be kicked? [22:18:06] greg-g: can you try to run puppet and restart nscd and nslcd there? [22:18:08] greg-g perhaps a restart? [22:18:11] it may be missed by salt or something [22:18:19] does deployment-prep have its own salt master? [22:19:07] * greg-g dunnos [22:20:01] Project beta-update-databases-eqiad build #18579: 04STILL FAILING in 0.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/18579/ [22:20:37] deployment-* on openstack? [22:21:15] don't restart the hosts (I don't believe you have that perm though) [22:22:23] i was looking to see if it had its own salt and unless ldap broke that bad i dont [22:22:24] paladox: I don't know? [22:22:31] ok [22:23:44] Project beta-scap-eqiad build #164821: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164821/ [22:23:56] no deployment-prep expert but deployment-salt02.deployment-prep.eqiad.wmflabs may be a salt master? [22:24:56] indeed [22:25:46] I'm working on it [22:26:04] andrewbogott is it safe to assume cert is active on toolforge? [22:26:20] I was trying to get do it from local for you guys but I did a number on my lcoal ssh config trying to debug initial issues [22:26:22] thanks andrewbogott [22:26:32] Zppix: as far as I know [22:26:40] andrewbogott alright ty [22:26:56] deployment stuff should be better now, I think [22:27:57] paladox: twentyafterfour: i checked and had restarted apache on phab2001 but there was no point. the Apache config doesnt nuse LDAP, if anything then Phab service would need a restart [22:28:21] oh [22:28:35] https://phabricator.wikimedia.org/P5768 [22:28:37] mutante: It should really be stateless [22:28:51] the phd service shouldn't need ldap it's only for logins, right? [22:29:13] yea, afaik [22:29:27] and it was only because Apache decides to throw 500 error in this case [22:29:37] instead of saying nicely "CA expired" or something [22:30:30] twentyafterfour: what got replaced here is the CA that signed the certs for LDAP servers [22:30:57] so puppet had to run to put the new CA file on all the ..cleints [22:30:59] clients [22:51:48] question: how does one makes a config where one WMF version should have one value and another should have another value? [22:51:50] e.g. T171107 [22:51:50] T171107: Class undefined: \Wikibase\Lib\JsonUnitStorage - https://phabricator.wikimedia.org/T171107 [22:52:20] class name changed between wmf.9 and wmf.10 - how this should be handled? [22:59:46] That's shit [22:59:53] $wgVersion? [22:59:58] It's not gonna sit around for long [23:02:04] RECOVERY - Puppet errors on integration-slave-jessie-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [23:02:38] RECOVERY - Puppet errors on jenkinstest is OK: OK: Less than 1.00% above the threshold [0.0] [23:02:50] RECOVERY - Puppet errors on integration-saltmaster is OK: OK: Less than 1.00% above the threshold [0.0] [23:03:33] Project beta-scap-eqiad build #164822: 04STILL FAILING in 29 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164822/ [23:03:42] RECOVERY - Puppet errors on integration-slave-docker-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [23:05:41] RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [23:09:21] RECOVERY - Puppet errors on integration-r-lang-01 is OK: OK: Less than 1.00% above the threshold [0.0] [23:10:47] RECOVERY - Puppet errors on integration-publishing is OK: OK: Less than 1.00% above the threshold [0.0] [23:13:46] RECOVERY - Puppet errors on saucelabs-02 is OK: OK: Less than 1.00% above the threshold [0.0] [23:14:12] RECOVERY - Puppet errors on saucelabs-03 is OK: OK: Less than 1.00% above the threshold [0.0] [23:15:52] RECOVERY - Puppet errors on integration-slave-docker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [23:17:50] RECOVERY - Puppet errors on integration-slave-trusty-1004 is OK: OK: Less than 1.00% above the threshold [0.0] [23:20:21] (03PS1) 10Mholloway: [Test] Stop forcing QEMU 1 use with the "-engine classic" flag [integration/jenkinsci/android-emulator-plugin] - 10https://gerrit.wikimedia.org/r/366484 [23:20:35] 10MediaWiki-Codesniffer: phpcbf doesn't remove space before ; after removing () from require call - https://phabricator.wikimedia.org/T171125#3454841 (10Reedy) [23:22:01] Yippee, build fixed! [23:22:01] Project beta-update-databases-eqiad build #18580: 09FIXED in 2 min 0 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/18580/ [23:22:09] Project beta-scap-eqiad build #164823: 04STILL FAILING in 17 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164823/ [23:27:12] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3454858 (10Jdlrobson) [23:29:52] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MW-1.30-release-notes, 10MediaWiki-Core-Tests, and 6 others: Parser tests fail if default Skin for unit tests makes use of doEditSectionLink - https://phabricator.wikimedia.org/T170880#3454862 (10Jdlrobson) [23:32:04] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: MW 1.30.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T168050#3454883 (10Smalyshev) [23:48:46] Project beta-scap-eqiad build #164824: 04STILL FAILING in 25 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/164824/ [23:49:56] (03CR) 10Awight: [C: 031] "Thanks for the rebase & fixes!" [integration/config] - 10https://gerrit.wikimedia.org/r/304875 (https://phabricator.wikimedia.org/T143025) (owner: 10Awight)