[00:17:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Add some more m4executor docker slaves for Jenkins - https://phabricator.wikimedia.org/T201972 (10Catrope) >>! In T201972#4503663, @Jdforrester-WMF wrote: > Or even "most of"? Time to start punishing code that's not switched over? Or rather,... [03:53:31] legoktm: I'm not familiar with the latest, but RE: priority, it might just be higher priority in that it has its own queue, so it's less likely to be behind other changes to be merged together. [03:54:11] Having said that, afaik Zuul doesn't have changes behind each other unless the jobs have limited concurrency, given we use anticipatory zuul-merger [04:00:34] hm [04:04:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Add some more m4executor docker slaves for Jenkins - https://phabricator.wikimedia.org/T201972 (10Krinkle) In numbers: * – It seems that, most hours, we're 100% on Docker jobs alread... [04:52:48] 10Gerrit, 10Release-Engineering-Team (Kanban): Gerrit has created duplicate accounts for some users - https://phabricator.wikimedia.org/T197083 (10mmodell) This should **hopefully** be resolved, at least for @kaldari, on Thursday. Again, I apologize for the long delay. [06:12:38] 10Beta-Cluster-Infrastructure: Persistently high maxlag on wikidata.beta.wmflabs.org - https://phabricator.wikimedia.org/T201983 (10Dalba) [06:19:31] 10Continuous-Integration-Infrastructure: Add pre-commit hook that does basic checks like php -l - https://phabricator.wikimedia.org/T201778 (10Aklapper) >>! In T201778#4497478, @Legoktm wrote: > Aryeh is/was a contractor on our team, so I think it's OK if he puts stuff on our board. Ah, thanks (and I'm sorry!).... [06:26:02] 10Beta-Cluster-Infrastructure, 10Wikidata: Persistently high maxlag on wikidata.beta.wmflabs.org - https://phabricator.wikimedia.org/T201983 (10Legoktm) Based on https://wikidata.beta.wmflabs.org/wiki/Special:DispatchStats it looks like dispatch hasn't run for 12 days? [06:35:11] (03CR) 10WMDE-Fisch: [C: 031] Use extension-quibble-composer for Mailgun extension testing [integration/config] - 10https://gerrit.wikimedia.org/r/452580 (owner: 10Phantom42) [10:54:25] 10Phabricator (Upstream), 10Upstream: Wikimedia Phabricator's "Developer Log In" should set the focus on username - https://phabricator.wikimedia.org/T201231 (10Aklapper) `phabricator/src/applications/auth/provider/*` [12:07:06] (03PS1) 10Reedy: Re-instate "Use new operations-puppet image with Lua support" [integration/config] - 10https://gerrit.wikimedia.org/r/452926 [12:15:34] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Wikimedia Portals Update and European Mid-day SWAT windows at the same time on Mondays - https://phabricator.wikimedia.org/T201932 (10Jdrewniak) @zeljkofilipin yup, I think moving the portals deploy 30 minutes earlier would be good. [12:17:20] (03CR) 10Reedy: [C: 032] "Stupid caching" [integration/config] - 10https://gerrit.wikimedia.org/r/452926 (owner: 10Reedy) [12:19:08] (03Merged) 10jenkins-bot: Re-instate "Use new operations-puppet image with Lua support" [integration/config] - 10https://gerrit.wikimedia.org/r/452926 (owner: 10Reedy) [12:53:40] legoktm: late reply, but g&s and g&s-swat have the same ("high") precedence. Having its own queue means that wmf/* patches are not stuck behind master patches [16:20:05] Project beta-update-databases-eqiad build #27596: 04FAILURE in 5.5 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/27596/ [16:24:10] !log beta-update-databases-eqiad broken due to ooui patch not being merged by jerkins [16:24:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:44:20] 10MediaWiki-Releasing, 10Service-Architecture, 10Services (later): Distribution strategy option: Use Debian packages - https://phabricator.wikimedia.org/T88154 (10Arlolra) [17:20:05] Project beta-update-databases-eqiad build #27597: 04STILL FAILING in 5.5 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/27597/ [18:23:22] Yippee, build fixed! [18:23:22] Project beta-update-databases-eqiad build #27598: 09FIXED in 3 min 21 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/27598/ [19:06:37] thcipriani|afk: hmm, but as for prioritization, zuul will give it the same priority as gate-and-submit, so how is it deciding which patch to run the jobs for first? [21:13:37] legoktm: I think it works as a FIFO within groups of jobs of the same precedence (but I don't know for sure). The problem that the g&s-swat queue side-steps is the dependent pipeline; that is, a lot of times during SWAT I'd end up waiting on a bunch of patches to master to be merged even though the tests for the patch I was trying to merge to a wmf/* branch had already completed. [21:17:44] thcipriani|afk: yeah...I was thinking we could patch zuul to add a "very high" priority or something? [21:18:11] IIRC this is a limitation of gearman [21:18:14] oh [21:20:42] https://github.com/gearman/gearmand/blob/master/PROTOCOL#L165-L167 is the quickest thing I could find that jives with my memory [21:21:48] !log krinkle@deployment-deploy01: Removing php-master/StartProfiler.php for T201782. [21:21:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:21:55] T201782: Remove use of StartProfiler.php in wmf production - https://phabricator.wikimedia.org/T201782 [21:23:27] figures [21:25:08] 10Continuous-Integration-Infrastructure, 10Upstream, 10Zuul: Change force merged cause a deadlock in Zuul gate-and-submit pipeline - https://phabricator.wikimedia.org/T93812 (10Legoktm) [21:25:10] 10Continuous-Integration-Infrastructure: Remove pywiki* and integration* from 'mediawiki' gate queue. - https://phabricator.wikimedia.org/T93304 (10Legoktm) [21:25:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10WorkType-Maintenance: Re-evaluate use of "Dependent Pipeline" in Zuul for gate-and-submit in the short term - https://phabricator.wikimedia.org/T94322 (10Legoktm) 05declined>03Open I'm re-opening this gi... [21:25:20] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10WorkType-Maintenance: Re-evaluate use of "Dependent Pipeline" in Zuul for gate-and-submit - https://phabricator.wikimedia.org/T94322 (10Legoktm) [21:30:52] thcipriani|afk: Do you know if there's a task or known cause yet for beta scap now taking > 35min to run? [21:31:03] It used to be much quicker, right? [21:32:41] Krinkle: I think it's probably still https://phabricator.wikimedia.org/T191921 but it's been a while since I looked at that [21:33:15] It took 14min on August 10 and most days before/after that. [21:33:23] I'm trying to narrow down now when it regressed [21:34:16] Hm.. maybe it's intermittent [21:34:20] FWIW if there are not changes to l10n it will run fast [21:34:23] but in recent history, this seems the first one that took 45 min [21:34:24] https://integration.wikimedia.org/ci/job/beta-scap-eqiad/219324/ [21:34:25] *no changes [21:35:58] Hm.. maybe, yeah, that could be it. [21:36:06] I don't see a correlation yet, but it's possible. [21:36:07] https://integration.wikimedia.org/ci/job/beta-scap-eqiad/219316/ a few hours earlier [21:46:46] thcipriani|afk: Yeah, since the switch to deploy01 with stretch, I'd expect that to be faster now. [21:47:17] Although enabling JIT still seems worthwhile [21:47:50] 10Continuous-Integration-Config: CI: Minerva PHPUnit tests should be included in shared extension gate job - https://phabricator.wikimedia.org/T202030 (10Jdlrobson) [21:48:06] 10Continuous-Integration-Config, 10Release-Engineering-Team: CI: Minerva PHPUnit tests should be included in shared extension gate job - https://phabricator.wikimedia.org/T202030 (10Jdlrobson) [22:58:12] 10Gerrit: Place holder task for Gerrit 2.16 upgrade - https://phabricator.wikimedia.org/T200739 (10Paladox) Looks like a google employee has chimed in so we may see 2.16 branched really soon. https://groups.google.com/forum/#!topic/repo-discuss/Ra7ZCBNYYmQ