[00:00:51] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.84 ms [00:19:23] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [00:22:41] (03CR) 10Legoktm: [C: 032] Update squizlabs/php_codesniffer to 2.5.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/258778 (owner: 10Paladox) [00:27:07] (03Merged) 10jenkins-bot: Update squizlabs/php_codesniffer to 2.5.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/258778 (owner: 10Paladox) [00:30:54] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.81 ms [01:04:01] (03PS1) 10Legoktm: Release 0.5.1 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261302 [01:07:50] (03CR) 10Legoktm: [C: 032] Release 0.5.1 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261302 (owner: 10Legoktm) [01:08:30] (03Merged) 10jenkins-bot: Release 0.5.1 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261302 (owner: 10Legoktm) [01:10:43] tag pushed [01:12:17] (03CR) 10Paladox: "Thanks." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/258778 (owner: 10Paladox) [01:22:23] Hi it seems that https://phabricator.wikimedia.org/diffusion/MCSN/ is not updating since patches have been merged but arn't showing in diffusion. It may be happening to other repos but not sure. [01:35:40] (03PS1) 10Legoktm: Add gen-changelog.sh util to assist with updating HISTORY.md [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261308 [01:40:50] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [02:23:31] 10Continuous-Integration-Config, 10MediaWiki-Codesniffer: Make mw-tools-codesniffer-mwcore-testrun job more like the actual mediawiki-core-phpcs job - https://phabricator.wikimedia.org/T116348#1907634 (10Legoktm) Also, it would be nice if I could see the *diff* in the phpcs output after applying the new commit. [02:24:15] 10Continuous-Integration-Config, 10MediaWiki-Codesniffer: Make mw-tools-codesniffer-mwcore-testrun job more like the actual mediawiki-core-phpcs job - https://phabricator.wikimedia.org/T116348#1907635 (10Legoktm) From https://github.com/squizlabs/PHP_CodeSniffer/releases/tag/2.5.0: * PHPCS will now use a phpcs... [03:30:54] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.82 ms [06:15:51] (03PS1) 10Gergő Tisza: Add JadeMaveric to the test runner whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/261322 [06:34:45] (03CR) 10JadeMaveric: [C: 031] "cool, so that's how you whitelist someone." [integration/config] - 10https://gerrit.wikimedia.org/r/261322 (owner: 10Gergő Tisza) [07:13:33] (03CR) 10Polybuildr: [C: 032] Add gen-changelog.sh util to assist with updating HISTORY.md [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261308 (owner: 10Legoktm) [07:14:14] (03Merged) 10jenkins-bot: Add gen-changelog.sh util to assist with updating HISTORY.md [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261308 (owner: 10Legoktm) [07:21:22] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [07:40:38] Yippee, build fixed! [07:40:39] Project beta-scap-eqiad build #84207: 09FIXED in 7 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84207/ [08:36:04] Yippee, build fixed! [08:36:04] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #828: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/828/ [10:15:42] (03PS2) 10Gilles: Add new thumbor repos [integration/config] - 10https://gerrit.wikimedia.org/r/261114 (https://phabricator.wikimedia.org/T120205) [10:16:16] Project beta-scap-eqiad build #84220: 04FAILURE in 2 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84220/ [10:26:37] Yippee, build fixed! [10:26:37] Project beta-scap-eqiad build #84221: 09FIXED in 8 min 16 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84221/ [10:33:42] 10MediaWiki-Codesniffer: Add support for a blacklist in MW codesniffer - https://phabricator.wikimedia.org/T122575#1907888 (10Paladox) 3NEW [11:04:34] 10MediaWiki-Codesniffer: Add support for a blacklist in MW codesniffer - https://phabricator.wikimedia.org/T122575#1907922 (10Paladox) 5Open>3declined [11:43:50] 5Gerrit-Migration, 10Analytics-Tech-community-metrics: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#1907950 (10Qgil) According to https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201516Q3,... [11:45:51] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 1.01 ms [12:42:37] 10MediaWiki-Codesniffer: Add sniff to detect double empty lines in php code - https://phabricator.wikimedia.org/T120570#1908068 (10polybuildr) > If I remeber correctly there exists a sniff for this, but I have not the name at the moment. I looked through all the sniffs in the PHPCS repo and the closest one I fo... [13:37:21] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [14:36:44] Project beta-scap-eqiad build #84243: 04FAILURE in 3 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84243/ [14:40:51] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.59 ms [14:45:57] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [14:46:20] Yippee, build fixed! [14:46:20] Project beta-scap-eqiad build #84244: 09FIXED in 7 min 20 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84244/ [14:56:36] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 1.64 ms [15:43:32] (03PS4) 10Bmansurov: Setup RelatedArticles browser tests job [integration/config] - 10https://gerrit.wikimedia.org/r/260764 (https://phabricator.wikimedia.org/T120715) [15:44:40] Reedy: i'm looking for the metavid SVN repo that we had at some point. Might you happen to know where that disappeared to ? [15:50:20] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [16:35:50] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.57 ms [16:55:50] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [16:58:45] (03PS2) 10Gergő Tisza: Add JadeMaveric to the test runner whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/261322 [17:00:33] (03CR) 10Gergő Tisza: "Oops, an extra tab." [integration/config] - 10https://gerrit.wikimedia.org/r/261322 (owner: 10Gergő Tisza) [17:00:51] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 1.15 ms [17:47:27] thedj: did you find this? -- https://phabricator.wikimedia.org/diffusion/SVN/browse/trunk/extensions/MetavidWiki/ [17:49:11] twentyafterfour: can you help me understand what’s happening with my phab login? I logged out for the first time in ages and now can’t find my way back. [17:50:04] When I log in with (I thought) my old login/password I get the ‘ldap registration’ page [18:05:29] twentyafterfour: nevermind, sorted. [18:57:28] hey! Is there anyone around that could look into network problems on CI? Example: https://integration.wikimedia.org/ci/job/wikidata-query-rdf/772/console [18:57:41] all tests for this project now fail because of the network issues [19:01:13] 6Release-Engineering-Team: WDQS builds fail due to network issues - https://phabricator.wikimedia.org/T122594#1908476 (10Smalyshev) 3NEW [19:07:29] 6Release-Engineering-Team: WDQS builds fail due to network issues - https://phabricator.wikimedia.org/T122594#1908504 (10Andrew) [19:08:44] SMalyshev: I just linked that to a recent bug which I think is the cause. I don’t know if you can read the related ticket, but I’d suggest you ask godog or paravoid about it in #wikimedia-operations [19:09:29] andrewbogott: can't read it. I'll ping them [19:10:43] I’ll look at your logs a bit more, too, to make sure that’s what it is... [19:11:44] 10Continuous-Integration-Infrastructure, 10MediaWiki-General-or-Unknown: Update grunt-jsonlint to 1.0.7 on all repos, on git Wikimedia or todo with Wikimedia - https://phabricator.wikimedia.org/T122460#1908513 (10Legoktm) a:3Legoktm https://gerrit.wikimedia.org/r/#/q/status:open+topic:bump-dev-deps,n,z [19:13:39] actually... [19:13:42] SMalyshev: https://oss.sonatype.org/content/groups/jetty/org/glassfish/javax.el/maven-metadata.xml [19:13:52] that’s one of the files it’s trying to load? Because that’s a straightforward 404 [19:14:44] And the other files I can wget just fine on integration-slave-trusty-1023 [19:14:51] so I don’t think it’s that bug, sorry for the red herring [19:15:58] 6Release-Engineering-Team: WDQS builds fail due to network issues - https://phabricator.wikimedia.org/T122594#1908515 (10Andrew) [19:19:18] 6Release-Engineering-Team: WDQS builds fail due to network issues - https://phabricator.wikimedia.org/T122594#1908528 (10Andrew) bd808 suggests that there may be some cruft (leftover from when the tests ran on production servers) that relied on the old http proxy on Carbon. That would definitely be broken now d... [19:21:38] andrewbogott: there's a bunch of files it tried to d/l... I'm not sure what exactly as it's internal maven dependencies, but ultimately it tried to locate some java packages [19:21:49] so maybe CI needs some maven settings there? [19:22:35] some of them may be 404s, it's normal for package managers [19:22:42] maybe — I don’t know much about what goes on inside the CI tests, probably they use a proxy but don’t need to. [19:23:38] andrewbogott: well, this one just runs maven... so I imagine the proxy settings depend on local maven config probably [19:23:46] or local Java config [19:24:24] is it possible to install these packages into local maven repo on CI maybe? I'm not sure how that works there [19:25:08] * andrewbogott doesn’t know what maven is [19:26:26] andrewbogott: that's make for Java, only worse :) [19:27:09] andrewbogott: so the ticket link you left in https://phabricator.wikimedia.org/T122594 links back to https://phabricator.wikimedia.org/T122594 :) [19:27:46] oops :) [19:34:37] 10Continuous-Integration-Infrastructure, 6operations: Test mwext-qunit-composer database disk image is malformed - https://phabricator.wikimedia.org/T122599#1908567 (10Paladox) 3NEW [19:51:41] !log Fixed git remote of integration-puppetmaster.integration:/var/lib/git/labs/private to use https instead of old ssh method [19:51:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:53:08] !log Cherry-picked https://gerrit.wikimedia.org/r/#/c/261476/ to integration-puppetmaster for testing [19:53:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:53:49] 10Continuous-Integration-Infrastructure: Merge tests even though a repo is being tested before it - https://phabricator.wikimedia.org/T122602#1908616 (10Paladox) 3NEW [19:59:24] PROBLEM - Puppet failure on wmfbranch is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [20:07:19] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [20:10:54] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.82 ms [20:18:06] (03PS1) 10Polybuildr: Add sniff to detect consecutive empty lines in a file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) [20:18:34] 10MediaWiki-Codesniffer, 5Patch-For-Review: Add sniff to detect double empty lines in php code - https://phabricator.wikimedia.org/T120570#1908714 (10polybuildr) a:3polybuildr [20:18:58] 10MediaWiki-Codesniffer, 5Patch-For-Review: Add sniff to detect double empty lines in php code - https://phabricator.wikimedia.org/T120570#1856627 (10polybuildr) Submitted a patch - it's an edited version of `Squiz.WhiteSpace.SuperfluousWhitespace`. [20:21:19] Project beta-scap-eqiad build #84276: 04FAILURE in 5 min 29 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84276/ [20:25:08] This is the second time tests haven't run on my patch. [20:25:16] https://gerrit.wikimedia.org/r/#/c/261572/ is one [20:26:36] oh wait, the other time was not my patch, it was https://gerrit.wikimedia.org/r/#/c/257817/ [20:26:46] I can't seem to find anything in https://integration.wikimedia.org/zuul/ [20:27:02] I can run a recheck, but if someone here can throw light on why the tests didn't run? [20:29:15] PROBLEM - App Server Main HTTP Response on deployment-mediawiki02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:30:10] Okay, recheck-ing. [20:30:20] (03CR) 10Polybuildr: "recheck" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:31:46] okay... I think the tests still aren't running. [20:31:46] Yippee, build fixed! [20:31:46] Project beta-scap-eqiad build #84277: 09FIXED in 5 min 25 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/84277/ [20:34:03] RECOVERY - App Server Main HTTP Response on deployment-mediawiki02 is OK: HTTP OK: HTTP/1.1 200 OK - 39339 bytes in 0.628 second response time [20:34:45] RECOVERY - Puppet failure on wmfbranch is OK: OK: Less than 1.00% above the threshold [0.0] [20:36:07] 6Release-Engineering-Team, 5Patch-For-Review: WDQS builds fail due to network issues - https://phabricator.wikimedia.org/T122594#1908781 (10bd808) Looks like the cherry-pick of the puppet patch works -- https://integration.wikimedia.org/ci/job/wikidata-query-rdf/782/console This ticket should stay open until... [20:38:06] (03CR) 10BryanDavis: "recheck" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:39:42] polybuildr: not rechecking for me either so likely not a problem with whitelisting. Not sure how to debug further [20:43:21] (03PS2) 10Florianschmidtwelzow: Add sniff to detect consecutive empty lines in a file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:45:41] bd808: hmm. odd. also, won't run check or test pipelines. Then again, for whitelisted people, it won't try to run the check pipeline at all I guess. [20:45:50] Should this be moved to another channel?.. [20:46:46] this is the right place to whine about CI issues, but I'm afraid that it is the wrong week to get fast answers [20:47:21] I'm surprised there's so much activity. :P [20:47:58] there are other things working through zuul I think. Maybe the problem is confined to this repo? [20:48:14] * bd808 tries to remember where to go to see zuul error logs [20:48:22] addshore is busy CR+2 spamming, so yeah, some things work :P [20:48:33] sorry :/ [20:48:45] I can't see any V+2s by Jenkins though [20:48:52] there is quite a queue currently, but everything is still moving [20:48:59] addshore: no need to be :o [20:49:11] and there [20:49:18] https://integration.wikimedia.org/zuul/ now shows codesniffer [20:49:32] after FlorianSW's rebase? [20:49:58] apparently [20:50:00] yeh, https://integration.wikimedia.org/zuul/ is a bit behind, there are like 50 other things in the gate-submit queue currently [20:50:06] and probably 20 still in the test queue [20:50:12] polybuildr: at least the change was not mergeable, and I check with a click on "rebase" before commenting that a manual rebase is needed :) [20:50:23] Sorry, if this interrupt your work :/ [20:50:25] bd808: thx !!! [20:50:40] bd808: this proves once more how much phabricator search sucks I guess.... [20:50:57] addshore: oh. that probably explains it. [20:51:03] thedj: sadly yes [20:51:21] I found it by clicking a link on the mw.o page for the extension though ;) [20:51:34] addshore: It's still running the test pipeline for PS1. PS2 isn't anywhere, so yeah, behind I guess. why though? [20:51:51] https://gerrit.wikimedia.org/r/#/q/status:open+topic:bump-dev-deps,p,003a22c10003ca49 [20:51:55] bd808: oh. they all redirected to /diffusion for me. not postfixes... [20:52:13] FlorianSW: oh, not a problem. thanks for the rebase! I thought you'd fixed an issue with CI tests not running, but apparently it's just because there's a really big queue :P [20:52:40] but see in #wikimedia-dev polybuildr and things are still moving :) [20:52:47] addshore: I noticed that. :P but why is the zuul web monitor behind? [20:52:51] shouldn't it just show a queue? [20:53:15] polybuildr: I dont know, I have always found it misses things [20:53:37] maybe there is a limit on how many things it will display in a pipe? Or maybe even the pipe has a limited size? [20:53:40] I see. [20:53:45] yeah, I guessed that [20:53:53] but at the time, gate-and-submit was full, but test was nearly empty [20:53:57] so it can't be per pipeline [20:54:01] maybe a limit totally? [20:54:32] well, there will definatly be a limit on totall running processes / jobs [20:55:01] um, this is weird. :P [20:55:12] but per https://integration.wikimedia.org/ci/ it looks like there is loads of execution space free :P [20:55:17] is it just me or is https://gerrit.wikimedia.org/r/#/c/261572/ not showing the green tick for V+2, but has a comment of V+2? [20:55:57] +2V was for PS1 though, and the current version is PS2 ;) [20:55:58] polybuildr: the comment is for PS1 [20:56:00] so no tick for you ;) [20:56:01] oh right. [20:56:11] yeah okay :P [20:56:25] I shall hide in a corner until it review PS2. [20:56:39] Thanks for the help! [20:57:28] it looks to me like zuul thinks a lot more slaves are in use than Jenkins does. I'm not sure what that means [20:57:43] * bd808 is doing things from https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Debugging [20:59:13] (03PS1) 10Smalyshev: Add wikimedia/textcat testing [integration/config] - 10https://gerrit.wikimedia.org/r/261577 [21:11:57] (03CR) 10BryanDavis: [C: 032] Add wikimedia/textcat testing [integration/config] - 10https://gerrit.wikimedia.org/r/261577 (owner: 10Smalyshev) [21:12:13] bd808: thank you! [21:12:28] I haven't done one of these in a while. :) [21:13:15] bd808: one of what? [21:13:26] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [21:14:11] polybuildr: deploying a new zuul config [21:14:39] bd808: ah, so after CR+2, also updated the deployed version? [21:15:21] yeah, I'll be pulling the new config to gallium and loading it [21:15:52] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.75 ms [21:15:52] once zuul gets around to merging it ... [21:16:42] bd808: I've only seen that operation from this end, where the person deploying updates the server admin log from IRC :P [21:17:08] it's not too complex -- https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Update_configuration [21:17:31] the process is very similar to how we apply production changes to the wiki config [21:23:37] (03CR) 10Polybuildr: "Finally. :P" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [21:25:57] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [21:26:03] PROBLEM - Puppet failure on integration-dev is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0] [21:26:18] (03Merged) 10jenkins-bot: Add wikimedia/textcat testing [integration/config] - 10https://gerrit.wikimedia.org/r/261577 (owner: 10Smalyshev) [21:26:37] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 1.53 ms [21:32:48] !log Updated zuul with https://gerrit.wikimedia.org/r/#/c/261577/ [21:32:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:33:46] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:34:22] SMalyshev: wikimedia/textcat should be hooked up to zuul now [21:35:12] (03PS3) 10BryanDavis: Add JadeMaveric to the test runner whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/261322 (owner: 10Gergő Tisza) [21:35:18] (03CR) 10BryanDavis: [C: 032] Add JadeMaveric to the test runner whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/261322 (owner: 10Gergő Tisza) [21:36:32] (03Merged) 10jenkins-bot: Add JadeMaveric to the test runner whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/261322 (owner: 10Gergő Tisza) [21:38:37] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 31457 bytes in 1.043 second response time [21:42:21] !log Updated zuul with https://gerrit.wikimedia.org/r/#/c/261322/ [21:42:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:45:02] (03PS2) 10BryanDavis: GCI: Whitelist Victorbarbu (GCI student) [integration/config] - 10https://gerrit.wikimedia.org/r/261163 (owner: 10Florianschmidtwelzow) [21:46:40] (03CR) 10BryanDavis: [C: 032] GCI: Whitelist Victorbarbu (GCI student) [integration/config] - 10https://gerrit.wikimedia.org/r/261163 (owner: 10Florianschmidtwelzow) [21:48:52] (03Merged) 10jenkins-bot: GCI: Whitelist Victorbarbu (GCI student) [integration/config] - 10https://gerrit.wikimedia.org/r/261163 (owner: 10Florianschmidtwelzow) [21:51:08] !log Updated zuul with https://gerrit.wikimedia.org/r/#/c/261163/ [21:51:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:53:03] (03PS3) 10BryanDavis: Add new thumbor repos [integration/config] - 10https://gerrit.wikimedia.org/r/261114 (https://phabricator.wikimedia.org/T120205) (owner: 10Gilles) [21:53:09] (03CR) 10BryanDavis: [C: 032] Add new thumbor repos [integration/config] - 10https://gerrit.wikimedia.org/r/261114 (https://phabricator.wikimedia.org/T120205) (owner: 10Gilles) [21:54:32] (03Merged) 10jenkins-bot: Add new thumbor repos [integration/config] - 10https://gerrit.wikimedia.org/r/261114 (https://phabricator.wikimedia.org/T120205) (owner: 10Gilles) [21:56:06] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:56:54] !log Updated zuul with https://gerrit.wikimedia.org/r/#/c/261114/ [21:57:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:06:28] RECOVERY - Puppet failure on integration-dev is OK: OK: Less than 1.00% above the threshold [0.0] [22:25:24] PROBLEM - Host integration-t102459 is DOWN: CRITICAL - Host Unreachable (10.68.16.67) [22:31:35] RECOVERY - Host integration-t102459 is UP: PING OK - Packet loss = 0%, RTA = 0.89 ms [23:01:00] Yippee, build fixed! [23:01:00] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-firefox-sauce build #281: 09FIXED in 1 min 59 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-firefox-sauce/281/ [23:06:54] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer: Fix failed MultimediaViewer browsertests Jenkins jobs - https://phabricator.wikimedia.org/T94157#1909192 (10Jdlrobson) @zeljkofilipin would you be willing to fix this since I am not too familiar with the inner workings of the framework? [23:11:21] thedj: https://phabricator.wikimedia.org/diffusion/SVN/browse/trunk/extensions/MetavidWiki/ [23:13:17] bd808: do you know who owns wikimedia packagist repo? I'd like to add https://github.com/wikimedia/wikimedia-textcat to packagist [23:14:16] SMalyshev: I can take care of it [23:14:55] bd808: cool, thanks! [23:16:51] https://packagist.org/packages/wikimedia/textcat [23:16:51] bd808: excellent, thanks! [23:17:29] SMalyshev: you might want to mark the conposer.json for it as replacing https://packagist.org/packages/smalyshev/textcat [23:18:22] bd808: how I do that? [23:18:26] bd808: is jenkins/zuul crapping out again? [23:18:35] and btw, how do I tag in gerrit? [23:20:32] SMalyshev: tags in gerrit are just normal tag pushes to the repo but you need to have the rights added for your user to do that on the repo [23:20:32] I don't remember the exact right needed.... [23:20:33] andrewbogott: Not sure. It has been falling behind and then catching up all day [23:20:33] ah, ok, I think it works. hopefully... [23:21:44] SMalyshev: for replace in composer see -- https://getcomposer.org/doc/04-schema.md#replace [23:22:08] bd808: aha, thanks [23:22:31] "replace": { "smalyshev/textcat": "*" } should do it [23:23:04] I think there is a way to mark the project in the packagist interface as abandoned too and point to the new name [23:23:39] bd808: maybe this is even better way... [23:27:27] 10MediaWiki-Codesniffer, 5Patch-For-Review: Add sniff to detect double empty lines in php code - https://phabricator.wikimedia.org/T120570#1909286 (10Legoktm) Do you also want to submit an upstream PR? [23:45:12] andrewbogott: So it looks like all the dynamic jessie slaves for Jenkins have disappeared from the master :/ [23:45:49] bd808: I must be behind… is the live CI system using the dynamic slaves? Or is that still a parallel/beta system? [23:46:04] it's live. [23:46:32] what’s the project name? [23:46:36] the rake-jessie job is pinned to them at least and that seems to be what is blocking zuul things right now [23:46:46] * bd808 looks [23:48:08] andrewbogott: contintcloud I think [23:48:32] so when you say ‘disappeared from the master' [23:49:13] not showing up at https://integration.wikimedia.org/ci/computer/ [23:50:34] I see 60 instances in wikitech's view but none connected into the jenkins master... [23:52:23] I'm not seeing obvious instructions on https://www.mediawiki.org/wiki/Continuous_integration/Architecture/Isolation for troubleshooting [23:53:04] and no hashar to ping (23:00 local time is a reasonable time to be off-line) [23:53:30] chasemp: what do you remember about nodepool? [23:54:02] loaded question :) um [23:54:16] not much about internal workings [23:54:20] if the service is up [23:54:26] that's what I know atm [23:55:34] chasemp: is it safe to ‘service nodepool restart’? [23:55:52] yes totally afaik it keeps no state outside of the db [23:56:01] "If you send a SIGINT to the nodepoold process, Nodepool will wait for diskimages to finish building (if any) and disconnect all its internal process." -- http://docs.openstack.org/infra/nodepool/operation.html [23:58:44] well, I ran root@labnodepool1001:~# service nodepool restart [23:58:47] and it’s hanging [23:58:52] the logs say that it stopped [23:58:56] guess I’ll… wait [23:59:47] ok, there we go