[00:00:38] 10Continuous-Integration-Infrastructure, 10Test-Coverage, 10Wikimedia-production-error (Shared Build Failure): coverage test mwext-phpunit-coverage-patch-docker produces PHP errors - https://phabricator.wikimedia.org/T223605 (10Krinkle) [00:01:05] 10Continuous-Integration-Infrastructure, 10Test-Coverage, 10Wikimedia-production-error (Shared Build Failure): coverage test mwext-phpunit-coverage-patch-docker produces PHP errors - https://phabricator.wikimedia.org/T223605 (10Krinkle) This doesn't cause build failures that prevent merges (in that, this job... [00:26:39] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10mcruzWMF) >>! In T223102#5199089, @Aklapper wrote: >>>! In T223102#5199076, @mcruzWMF wrote: >> 1. Can... [01:21:36] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10Aklapper) Hmm. For which exact URL do you get which exact error message? [01:25:39] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10mcruzWMF) >>! In T223102#5199188, @Aklapper wrote: > Hmm. For which exact URL do you get which exact e... [02:55:54] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<10.00%) [05:13:40] (03PS1) 10Tim Starling: wmerrors is moving to PHP 7 exclusively [integration/config] - 10https://gerrit.wikimedia.org/r/511627 (https://phabricator.wikimedia.org/T187147) [06:25:57] PROBLEM - Puppet errors on integration-slave-jessie-1002 is CRITICAL: (Service Check Timed Out) [06:50:53] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:00:02] Project mediawiki-core-code-coverage-docker build #4262: 04FAILURE in 4 hr 0 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/4262/ [07:10:24] (03PS1) 10Legoktm: Bump mediawiki-core-code-coverage-docker timeout to 5 hours [integration/config] - 10https://gerrit.wikimedia.org/r/511635 [07:10:56] !log manually rebuilding mediawiki-core-code-coverage-docker with 5 hour timeout [07:10:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:15:19] ehhhhhhh [07:18:10] hehehe [07:19:13] 10Continuous-Integration-Config, 10Test-Coverage: https://doc.wikimedia.org/cover/mediawiki-core/clover.xml.bz2 is gone - https://phabricator.wikimedia.org/T223955 (10Legoktm) p:05Triage→03High [07:19:31] literally cannot stop finding rabbit holes to get trapped in [07:19:44] <3 Thanks for all the rabbit meat though [07:20:38] (03CR) 10Legoktm: [C: 03+2] Bump mediawiki-core-code-coverage-docker timeout to 5 hours [integration/config] - 10https://gerrit.wikimedia.org/r/511635 (owner: 10Legoktm) [07:20:39] legoktm: I'm hoping to dive into T223752 soon, if you ever feel like discussing other people's rabbit problems. [07:20:40] T223752: Decouple Quibble planning and execution phases - https://phabricator.wikimedia.org/T223752 [07:21:38] The fun parts will be * choice of command pattern representation, and * high-level workflows which are expanded into detailed steps... at some point. [07:21:43] hmm [07:22:02] I'll make a draft patch soon so there's something concrete to discuss... [07:22:32] I'm glad we're at the stage where we're thinking about refactoring and cleanliness rather than just trying to stuff features into quibble [07:22:48] (03Merged) 10jenkins-bot: Bump mediawiki-core-code-coverage-docker timeout to 5 hours [integration/config] - 10https://gerrit.wikimedia.org/r/511635 (owner: 10Legoktm) [07:22:56] :D one step beyond basic survival needs [07:23:40] It's too bad all the "pioneer" analogies are so fraught with colonial baggage. [07:26:00] awight: my main question is do you think that's what will help speed up the main bottlenecks in CI? [07:26:13] my gut feeling is that there's more low-hanging fruit elsewhere [07:26:46] (03CR) 10Legoktm: [C: 03+2] wmerrors is moving to PHP 7 exclusively [integration/config] - 10https://gerrit.wikimedia.org/r/511627 (https://phabricator.wikimedia.org/T187147) (owner: 10Tim Starling) [07:28:14] (03Merged) 10jenkins-bot: wmerrors is moving to PHP 7 exclusively [integration/config] - 10https://gerrit.wikimedia.org/r/511627 (https://phabricator.wikimedia.org/T187147) (owner: 10Tim Starling) [07:28:41] legoktm: My thought is that it'll make it much easier to do peformance refactoring. [07:29:09] !log deployed https://gerrit.wikimedia.org/r/511627 [07:29:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:30:22] (03CR) 10Noa wmde: "This is a proposed solution to help fix T222200. The corresponding change to the query service gui is https://gerrit.wikimedia.org/r/c/wik" [integration/config] - 10https://gerrit.wikimedia.org/r/511308 (https://phabricator.wikimedia.org/T222200) (owner: 10Noa wmde) [07:32:22] 10Release-Engineering-Team (Backlog), 10Quibble (marble): CI should run composer test/npm test BEFORE running Quibble - https://phabricator.wikimedia.org/T211703 (10awight) How does this relate to {T211702}? It seems that they are redundant, if we do this task then the lint steps should be skipped in Quibble,... [07:34:11] 10Release-Engineering-Team (Backlog), 10Quibble (marble): CI should run composer test/npm test BEFORE running Quibble - https://phabricator.wikimedia.org/T211703 (10Legoktm) I disagree with the premise of this task, since it's optimizing for the case when someone writes code that fails linters, which should be... [07:40:58] awight: I mean, I think if we made phpunit tests faster or selenium less flaky, those would be bigger wins IMO. that isn't to say refactoring quibble isn't important of course :) [07:44:39] legoktm: I'm sure you're right--my interest is partly academic, and partly a knee-jerk reaction to trying to make the changes for T199116. [07:44:40] T199116: Quibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests - https://phabricator.wikimedia.org/T199116 [07:46:50] ohhhhhhhhh [07:46:56] I'd also be interested in getting reengaged in CI via more useful tasks, feel free to @ me if you see high-value stuff that needs extra people. [07:46:57] that makes much more sense now [07:47:08] Hehe I've been burned before ;-) [07:47:41] IMO the decoupling will be key for T211702 also, since the alternative is probably some extreme tech debt. [07:47:42] T211702: Quibble initialize step should only clone the target repository - https://phabricator.wikimedia.org/T211702 [07:49:36] More of a badger den than a rabbithole. [07:50:03] https://phabricator.wikimedia.org/T193824#5109607 is going to change how we do dependencies in CI FYI [07:51:49] Yes! I'd love to be involved in the bigger containerization project, too. Thanks for the heads-up. [07:54:33] Probably, individual pieces of our toolchain should be written using microservice principles, e.g. a script in will take a list of desired of mw-extensions, and produce a JSON document with the dependency tree. [07:55:16] But maybe not. Maybe we do just want to fit into composer like this task suggests. [07:55:47] the reason the script is part of MediaWiki is because that's still the authoritative way to read extension.json/skin.json and will never break [07:56:20] +1 as long as it produces intermediate outputs that we can use from other tools, that seems fine! [07:57:00] It would be unfortunate though, if we needed to abuse composer to check out specific Gerrit patchsets, etc. [08:24:00] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Reconfiguring jenkins job mwext-php70-phan-docker" [integration/config] - 10https://gerrit.wikimedia.org/r/511447 (https://phabricator.wikimedia.org/T223397) (owner: 10Hashar) [08:24:36] !log Updated phan jobs to no more install mediawiki development dependencies (potentially conflicting with the extensions code) https://gerrit.wikimedia.org/r/#/c/integration/config/+/511447/ # T223397 [08:24:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:24:41] T223397: phan issue due to a composer dev dependency - https://phabricator.wikimedia.org/T223397 [08:26:22] (03Merged) 10jenkins-bot: phan: do not install mediawiki dev dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/511447 (https://phabricator.wikimedia.org/T223397) (owner: 10Hashar) [08:45:54] 10Continuous-Integration-Config, 10Test-Coverage: https://doc.wikimedia.org/cover/mediawiki-core/clover.xml.bz2 is gone - https://phabricator.wikimedia.org/T223955 (10hashar) 05Open→03Resolved a:03hashar I have removed bzip2 in favor of gzip when migrating the job to a Docker container. 2ffe38c714bb0192... [08:49:28] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10phan: phan issue due to a composer dev dependency - https://phabricator.wikimedia.org/T223397 (10hashar) | EventLogging (+2 already) | https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventLogging/+/511449/ |... [09:23:56] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10Wikidata, and 5 others: Introduce wikidata termbox SSR to kubernetes - https://phabricator.wikimedia.org/T220402 (10Tarrow) @mobrovac and @akosiaris Sorry to drop in late to the conversation but I've been on holiday and then at the hackathon.... [09:34:43] (03CR) 10Hashar: Codehealth: Ignore error from npm run-script test:unit (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/510957 (owner: 10Kosta Harlan) [09:40:22] Yippee, build fixed! [09:40:23] Project mediawiki-core-code-coverage-docker build #4263: 09FIXED in 2 hr 29 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/4263/ [10:11:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Reading-Infrastructure-Team-Backlog, 10Wikimedia-production-error (Shared Build Failure): Jenkins jobs for MediaWiki failing with 'npm: shasum check failed' - https://phabricator.wikimedia.org/T203506 (10hashar) We have not switched the... [10:15:05] (03PS1) 10Hashar: Make castor use rsync --delay-updates [integration/config] - 10https://gerrit.wikimedia.org/r/511672 (https://phabricator.wikimedia.org/T203506) [10:15:51] (03CR) 10Hashar: "Note that this only updated a few legacy jobs. The rest of jobs are switched by updating the container being used: https://gerrit.wikimed" [integration/config] - 10https://gerrit.wikimedia.org/r/479558 (https://phabricator.wikimedia.org/T203506) (owner: 10Thcipriani) [10:18:16] (03Abandoned) 10Hashar: Bump mediawiki-core-php70-phan-docker to mediawiki-phan:0.1.13 [integration/config] - 10https://gerrit.wikimedia.org/r/498304 (owner: 10Legoktm) [10:32:14] 10Continuous-Integration-Infrastructure: Investigate slow down of mediawiki-core-code-coverage-docker on some Jenkins instances - https://phabricator.wikimedia.org/T223971 (10hashar) [10:55:49] 10Continuous-Integration-Infrastructure: Investigate slow down of mediawiki-core-code-coverage-docker on some Jenkins instances - https://phabricator.wikimedia.org/T223971 (10hashar) https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/4262/ started at May 21, 2019 3:00:00 AM and took 4 h... [10:56:55] 10Release-Engineering-Team (Kanban), 10MediaWiki-Database, 10MW-1.34-notes, 10Regression: 1.34.0-wmf.3 generating lots of temporary tables on MySQL slaves - https://phabricator.wikimedia.org/T223978 (10Marostegui) [10:58:15] 10Release-Engineering-Team (Kanban), 10MediaWiki-Database, 10MW-1.34-notes, 10Regression: 1.34.0-wmf.3 generating lots of temporary tables on MySQL slaves - https://phabricator.wikimedia.org/T223978 (10Marostegui) p:05Triage→03Normal [11:17:13] 10Continuous-Integration-Infrastructure: Investigate slow down of mediawiki-core-code-coverage-docker on some Jenkins instances - https://phabricator.wikimedia.org/T223971 (10hashar) Which leaves me with the CPU that would be slower on 1055 and 1056? ` hashar@integration-cumin:~$ sudo cumin --trace --force 'nam... [11:20:27] 10Continuous-Integration-Infrastructure: wmf-quibble-vendor-mysql-hhvm-docker job sometime take 40+ minutes to run - https://phabricator.wikimedia.org/T222023 (10hashar) And eventually a similar issue happens with the mediawiki core coverage job which I am tracking at T223971 Seems the reason for the slow down... [11:21:12] 10Continuous-Integration-Infrastructure: Investigate slow down of mediawiki-core-code-coverage-docker on some Jenkins instances - https://phabricator.wikimedia.org/T223971 (10hashar) A few weeks ago, I have deleted `integration-slave-docker-1037` since it was notoriously slow when running the job wmf-quibble-ven... [11:23:18] !log Depooling integration-slave-docker-1055 and integration-slave-docker-1056 : CPU is too slow # T223971 [11:23:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:23:22] T223971: Investigate slow down of mediawiki-core-code-coverage-docker on some Jenkins instances - https://phabricator.wikimedia.org/T223971 [11:28:20] (03CR) 10Hashar: "It should take 2h / 2h30 based on https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/buildTimeTrend" [integration/config] - 10https://gerrit.wikimedia.org/r/511635 (owner: 10Legoktm) [11:40:16] 10Release-Engineering-Team (Kanban), 10MediaWiki-Database, 10MW-1.34-notes, 10MW-1.34-release, 10Regression: 1.34.0-wmf.3 generating lots of temporary tables on MySQL slaves - https://phabricator.wikimedia.org/T223978 (10Marostegui) [14:10:39] PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [140.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [14:21:43] RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 30.00% above the threshold [90.0] https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [14:49:58] hashar: hey, you're the train conductor on the week after hackathon. My condolences. [14:50:34] F [14:51:18] 10Deployments, 10Release-Engineering-Team (Kanban): mwdebug2002.codfw.wmnet and mwdebug1002.eqiad.wmnet need more vCPU: scap-cdb-rebuild is too slow - https://phabricator.wikimedia.org/T224026 (10hashar) 05Open→03Declined The previous task was {T212955}. I have confirmed they all have 4 vCPU so that match... [14:52:32] speaking of the train [14:52:36] I gotta deploy to the new group [14:52:56] But seriously, I want to write a script to periodically cleans "Patch-For-Review" from tasks that don't need it. Hopefully it saves lots of dev time. Is it fine for you and twentyafterfour [14:53:16] The bot supports removing project tags [14:53:40] But we haven’t configured it I guess because a task could have multiple changes [14:53:51] Herald? It does but it can't understand when to do it properly [14:54:07] that's why it needs a bot to interact with the API [14:54:11] No, GerritBot [14:54:26] Aka its-phabricator [14:54:38] hmm, fixing it should be easy [14:54:42] I will work on it [14:54:55] https://knowyourmeme.com/memes/press-f-to-pay-respects To understand Lucas_WMDE's comment [14:58:06] Amir1: that's fine with me. We should try to make sure it doesn't send a bunch of notificatrion spam in the process [14:59:17] yeah, is it possible to do so for example for a certain bot? [15:00:00] Amir1: I've been wanting to do that for a long time. What it should really do is track the patches at the top next to @mentions and show the status of each patch in the phab ui. phabricator could query gerrit to find the status of each one. the code should probably live in phabricator though instead of gerritbot [15:00:05] 10Continuous-Integration-Infrastructure, 10Quibble: Decouple Quibble planning and execution phases - https://phabricator.wikimedia.org/T223752 (10awight) I ran into another complication: the zuul cloner takes a huge number of parameters, plus reading from the environment. The command abstraction shouldn't exp... [15:00:10] Amir1: maybe [15:00:22] we previously had a hack like that but it got broken by upstream changes [15:01:12] I see, yeah but changing phabricator is way outside of my knowledge :( [15:01:24] Amir1: I can help with that part :) [15:02:45] Amir1: having that tag go away instead of manually cleaning it up would be So awesome [15:02:47] twentyafterfour: nice, Is there a documentation [15:03:28] apergos: yeah, I get all emails but lots of devs removing those tags [15:04:38] I didn't even think about the spam factor [15:04:38] I didn’t even know we’re supposed to clean it up, I just ignore it [15:04:47] */but/that [15:04:50] for clinic duty it's one of the things we do for sre [15:05:00] as well as look at outstanding patches and comment where we can [15:05:39] patch for review tag is a poor substitute for proper integration between phab and gerrit [15:05:59] I've never worked on proper integration because gerrit's future has always been in question [15:07:03] and yet here we are still using it and with a lot of work invested in it [15:07:12] I mean, of we knew it would be gone in six months, sure [15:07:34] but it seems like for better or worse, it's our tool for at least a couple years [15:07:38] apergos: right, it may be worth investing more into gerrit at this point [15:08:07] especially if dev tooling is supposed to be a priority [15:08:23] it's hard to know how much to invest in something when everything is uncertain and the decisions are out of my hands [15:08:31] that's not to say it need to land on your plate, everyone's got lots to do already [15:08:37] yep, I hear that [15:08:39] but I want the dev experience to be a lot better than it is [15:08:55] apergos: well, this is my "wheel house" so to speak [15:09:02] it's something I want to work on [15:09:04] I thought the decision against Differential was pretty clear at this point? https://phabricator.wikimedia.org/T191182 [15:09:05] I would like a one stop shop for people to get set up in gerrit/phab/wikitech [15:09:09] IMO, dev time is expensive and we are wasting lots of it by random devs removing those tags [15:09:11] and I’m not aware of any other alternatives at the moment [15:09:24] Lucas_WMDE: decision against differential doesn't mean that gerrit is forever [15:09:29] I wrote a little talk on all the steps and it was so offputting to the audience (at a hackathon) that only like 2 people managed to get through it [15:09:38] and I never actually saw a good argument against differential [15:09:46] but there's I'm sure lots more urgent items... [15:10:02] Lucas_WMDE: other alternatives : one is gitlab [15:10:08] now let's not re-open that can of worms (differential) [15:10:38] hahah [15:10:51] alternatives *that we’re currently looking into, I meant [15:10:52] it's more than a can of worms, it's like a tank full of them [15:11:00] of poisonous snakes [15:11:02] gerrit isn’t forever, but I don’t see any replacement in the near future [15:11:30] so investing into it isn’t in immediate danger of being wasted time, I think [15:11:36] it's what we all use every day, might as well get higher-ups to get behind it until [15:11:41] Lucas_WMDE: that's why I just said maybe it's time to invest in fixing phabricator->gerrit integration [15:11:46] we are mid-migration to the new thing! [15:11:57] agreed :) [15:12:03] look I don't like gerrit very much at all but it's even worse to keep gerrit and not fix the warts [15:12:22] it has its quirks but I can work with it. but it could be much better [15:12:37] and I can work with it != random volunteer can work with it easily [15:12:55] apergos: yeah gerrit is extremely hostile to new users IMO [15:13:06] I've only gotten used to it after 5 years at the foundation [15:13:34] that's one heck of a learning curve [15:13:57] well I learned it faster than that but still hated it ;) [15:14:03] :-D [15:14:09] now I'm just resigned to it [15:14:16] awwww [15:14:29] that's the world's saddest short developer story... [15:14:36] over the last year, gerrit's improved it's ui for new users. [15:14:44] I don't think anyone here likes gerrit [15:14:50] * paladox does [15:14:57] yes the ui improved in some ways but not as much as I'd like to see it [15:14:58] the ui has improved over time [15:15:23] anyway some small changes in phabricator can make big difference in everyone's productivity [15:15:26] * Amir1 is shocked [15:15:31] Lucas_WMDE: apergos: we are sticking to Gerrit for code review. At least for the foreseeable future [15:15:37] now that I’ve gotten used to it, I think I actually like Gerrit a lot [15:15:39] fine by me [15:15:41] especially with the new UI [15:15:43] like adding a proper patch integration instead of patch-for-review tag [15:16:05] just let's get folks who can decide, to make sure there are resources for maintaining it/making it better [15:16:08] et tu Lucas? [15:16:14] and as Paladox said, we will soon force the new UI to everyone when we upgrade Gerrit from 2.15 to 2.16 (2.16 no more has the old ui) [15:16:30] 2.16 has the old ui [15:16:32] yay for forcing new things on users [15:16:38] just that the new ui is enforced as the default [15:16:45] 3.0 dropped the old ui [15:16:51] and it already has been decided that we do not have time / resources / envy / whatever to improve Gerrit UI. I guess because it is not in WMF area of interests? [15:17:06] or maybe we never made a use case to get ui improvement funded [15:17:07] I thought the gerrit ui has been undergoing improvements [15:17:11] yup [15:17:13] yeah by upstreams [15:17:16] pala dox worked directly with upstream on some things [15:17:21] hashar: the ui isn't the only problem with gerrit [15:17:32] https://gerrit.git.wmflabs.org/r/ <-- is what it will look like for us. [15:17:35] and I recall being part of a test group checking out the new ui [15:17:45] maintaining it has gotten very time consuming as of late [15:17:46] twentyafterfour upstream have a prototype feature to bring pull requests to gerrit. [15:18:06] paladox: uhm, I'm not sure how I feel about that [15:18:44] https://groups.google.com/forum/#!topic/repo-discuss/92PqVTBTbXI [15:18:57] I don't really mind the gerrit workflow, as apposed to pull requests... they both have their advantages [15:19:13] differential supported either kind of workflow but people hate arcanist [15:19:30] the one real annoyance is when there are too many people in puppet at the same time and we get into rebase hell [15:19:35] I dislike differential that it's all CLI based [15:19:36] (as far as the workflow) [15:19:52] apergos: but that's due to the settings on the puppet repo [15:19:58] apergos: we don't have that issue with other repos [15:19:59] gerrit lets me use git exclusively from the command line [15:20:02] differential does not [15:20:10] yeah I know, fast-forward [15:20:15] and I'm reluctant to relax that [15:20:17] apergos: right, that is a distinct advantage [15:20:36] (I don't use git-review even. just git :-P) [15:20:46] * twentyafterfour doesn't use git-review either [15:20:51] high five! [15:20:53] don't see the point in it [15:21:01] * twentyafterfour high-fives apergos [15:21:03] lol [15:21:06] :-D [15:21:09] twentyafterfour me too [15:21:19] I think at least 50% of us don't use git-review [15:21:33] nice [15:21:41] not even for downloading? [15:21:53] it's another layer of 'things that can go wrong' and another tool to learn its idiosyncratic ways [15:21:58] also another thing against phabricator, is how can you even contribute? signups are locked upstream. [15:22:00] rather invest that into git and be done [15:22:05] Lucas_WMDE: for that I just copy/paste a link from the ui [15:22:08] it's maintained by really only one person. [15:22:11] ah, ok [15:22:27] same, get the link from the drop-down [15:22:32] apergos: the puppet rebase hell, we can fix it up. Iam pretty sure I got a task about it :] [15:23:09] one thing that will ease the presure is moving db config changes to etcd [15:23:14] can't wait for that [15:24:06] hmm [15:24:20] google's putting more funding into gerrit to progress it furthur. This is in a form of a ESC (Engineering Steering Committee), community manager and mentorships. [15:24:40] including making gerrit transparent [15:24:47] apergos: https://phabricator.wikimedia.org/T166888#3311916 and following comments which talk about the merge strategy configured for puppet.git [15:24:54] but that should deserve a standalone task [15:25:09] https://gerrit-review.googlesource.com/q/hashtag:%22proposal-for-better-collaboration%22+(status:open%20OR%20status:merged) [15:25:18] https://groups.google.com/forum/#!msg/repo-discuss/6L67g41DPgQ/TAvsDG0GAwAJ [15:26:08] (03PS1) 10Thcipriani: Gerrit v2.15.13 [software/gerrit] (deploy/wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/511735 [15:26:53] (03CR) 10Paladox: [C: 03+2] Gerrit v2.15.13 [software/gerrit] (deploy/wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/511735 (owner: 10Thcipriani) [15:27:44] looking at the linkfest here :-) [15:32:08] yeah I read that ticket back in the day [15:34:32] I'm in faidon's camp more or less [15:36:08] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10hdothiduc) I can confirm, I see the same as María. [15:40:41] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10mmodell) I think editing forms is limited to admins [15:43:19] 10Continuous-Integration-Config, 10Operations: Fix operations/puppet.git "rebase hell" - https://phabricator.wikimedia.org/T224033 (10hashar) [15:43:35] apergos: for puppet.git rebase hell, I have filled https://phabricator.wikimedia.org/T224033 [15:43:45] apergos: but I am not sure I will champion it among SRE team :D [15:46:39] 10Continuous-Integration-Config, 10Operations: Fix operations/puppet.git "rebase hell" - https://phabricator.wikimedia.org/T224033 (10hashar) + @faidon and @catrope who participated on T166888#3311916 and following. [15:47:40] also if anyone is interested in https://groups.google.com/forum/#!topic/repo-discuss/IDHj43VwpmE [15:49:08] interested to see whether the committee means more attention paid to gerrit improvements, paladox [15:49:12] we'll see [15:49:27] yup [15:49:49] and hashar is gone before I could thank him [15:49:52] but I have subscribed [16:03:29] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Create service-pipeline job aware of .pipeline/config.yaml - https://phabricator.wikimedia.org/T224035 (10thcipriani) [16:03:40] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Create service-pipeline job aware of .pipeline/config.yaml - https://phabricator.wikimedia.org/T224035 (10thcipriani) p:05Triage→03Normal [16:07:49] (03PS2) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 [16:07:51] (03PS2) 10Dduvall: service-pipeline: Define pipeline builder jobs for blubber [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) [16:08:50] (03PS3) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) [16:08:54] (03PS3) 10Dduvall: service-pipeline: Define pipeline builder jobs for blubber [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) [16:26:12] (03PS1) 10Awight: [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [16:27:02] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [16:48:10] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 4 others: Kask integration testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10thcipriani) [16:49:34] 10Release-Engineering-Team, 10Operations, 10Release Pipeline, 10serviceops, and 4 others: Kask integration testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10thcipriani) p:05Triage→03Normal It seems that the cassandra subchart already exists for cask (via... [16:54:41] 10Continuous-Integration-Config, 10Operations: Fix operations/puppet.git "rebase hell" - https://phabricator.wikimedia.org/T224033 (10Volans) I'm assuming that in cases in which the rebase fails because of conflicts or the CI fails after the rebase Jenkins would vote -1 and the patch would be out of the mergin... [16:55:00] 10Continuous-Integration-Config, 10Operations: Fix operations/puppet.git "rebase hell" - https://phabricator.wikimedia.org/T224033 (10Volans) p:05Triage→03Normal [16:58:01] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10Quiddity) Note: I'm an admin (and on their team) and can edit the form in the future as needed. [17:04:19] (03PS2) 10Awight: [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [17:04:59] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [17:05:14] 10Continuous-Integration-Infrastructure, 10cloud-services-team (Kanban): Old cloudvirt (with Intel Xeon) are twice slower than new ones (Intel Sky Lake) - https://phabricator.wikimedia.org/T223971 (10hashar) I guess what would help is to run the same benchmark directly on the cloudvirt hosts: ` apt -y install... [17:07:36] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10mcruzWMF) >>! In T223102#5201629, @Quiddity wrote: > Note: I'm an admin (and on their team) and can ed... [17:09:13] (03CR) 10Lars Wirzenius: [C: 03+1] "Without a deeper understanding of this I'm reluctant to +2, but it looks OK to me." [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [17:10:09] (03CR) 10Lars Wirzenius: [C: 03+1] "LGTM" [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [17:12:48] 10Release-Engineering-Team (Kanban), 10Operations, 10SRE-Access-Requests, 10User-Urbanecm, 10User-greg: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830 (10Urbanecm) [17:15:22] (03PS3) 10Awight: [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [17:16:01] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [17:17:18] 10Release-Engineering-Team (Kanban), 10Operations, 10SRE-Access-Requests, 10User-Urbanecm, 10User-greg: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830 (10Urbanecm) I hereby confirm authenticity of my SSH key (`ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... [17:26:09] 10Project-Admins: Create a project for mediawiki extension Passwordless Login - https://phabricator.wikimedia.org/T223893 (10Florian) > TODO: Creating an extension homepage on mediawiki.org (and then updating the project description) is welcome. :) Done! :) https://www.mediawiki.org/wiki/Extension:PasswordlessL... [17:30:29] 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Release, 10Train Deployments: 1.34.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T220731 (10Reedy) [17:50:02] (03CR) 10Kosta Harlan: Codehealth: Ignore error from npm run-script test:unit (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/510957 (owner: 10Kosta Harlan) [17:52:55] kostajh: good morning :) [17:53:11] hi hashar! in London this week :) [17:54:51] hashar: thanks for the puppet rebase ticket, subscribed :-) [17:55:20] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Reconfiguring jenkins job mwext-codehealth-master-non-voting" (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/510957 (owner: 10Kosta Harlan) [17:55:34] kostajh: i have updated the codehealth thingie jobs! [17:55:51] apergos: please do feel free to talk about it with other SREs folks :-] [17:56:09] :-D [17:56:11] I am not sure what would be the solution, but we can surely improve! [17:57:15] hashar: thx! at hackathon a few project maintainers were interested to opt-in, and I've added some patches to enable the pipeline for them. I'll wait to see that this works with Echo though [17:57:37] (03Merged) 10jenkins-bot: Codehealth: Ignore error from npm run-script test:unit [integration/config] - 10https://gerrit.wikimedia.org/r/510957 (owner: 10Kosta Harlan) [17:58:53] I don't know either but at least looking at it might get some suggestions [17:58:58] even 'let's split the repo into..' [17:58:59] dunno [18:09:21] (03CR) 10Thcipriani: "inline question" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [18:15:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Discovery-Search: quibble-vendor-mysql-hhvm-docker for WikibaseCirrusSearch takes over 40 minutes - https://phabricator.wikimedia.org/T222757 (10hashar) Hey @Smalyshev , sorry I have delayed my reply to this request. Beside what Greg men... [18:27:54] (03CR) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [18:29:18] (03PS4) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) [18:29:20] (03PS4) 10Dduvall: service-pipeline: Define pipeline builder jobs for blubber [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) [18:34:57] (03PS5) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) [18:34:59] (03PS5) 10Dduvall: service-pipeline: Define pipeline builder jobs for blubber [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) [18:36:16] (03CR) 10Dduvall: service-pipeline: Templates for projects using .pipeline/config.yaml (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [18:41:13] (03CR) 10Thcipriani: [C: 03+2] service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [18:43:34] (03Merged) 10jenkins-bot: service-pipeline: Templates for projects using .pipeline/config.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/510601 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [19:03:24] (03CR) 10Thcipriani: [V: 03+2] Gerrit v2.15.13 [software/gerrit] (deploy/wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/511735 (owner: 10Thcipriani) [19:04:08] hashar: Here's a working draft, to illustrate what I was thinking about Quibble: https://gerrit.wikimedia.org/r/#/c/511749/ [19:17:26] (03PS4) 10Awight: [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [19:18:01] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [19:25:54] thcipriani: thanks for T224041 (I think, I still don't understand much of this) [19:25:58] T224041: Kask integration testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 [19:27:21] urandom: marxarelli and I were talking about it this morning. From the looks of it we'll just need to figure out a way to ensure the cassandra container (from your existing chart) is deployed, and then we should be able to run whatever the test entrypoint is during the pipeline (is the hope). [19:27:27] what is the test entrypoint by-the-by? [19:28:33] thcipriani: OK, is this entrypoint in deployment pipeline terminology? [19:29:07] I just mean how do you run your tests that need cassandra? [19:30:07] thcipriani: `make functional-test` [19:31:43] cool, so I guess the first step will be getting kask deployed in minikube along with cassandra and ensuring that test works as expected. [19:32:02] then we can figure out what's missing in the pipeline to make that happen [19:32:31] thcipriani: ala helm? [19:32:55] I haven't looked/tried, is it straightforward to use our helm charts with minikube? [19:33:42] that seems like it'd be a pretty useful thing to be able to do...generally [19:33:48] they're supposed to be built for local dev first (hence https://gerrit.wikimedia.org/r/509102/ ) [19:33:56] awight: I am very unlikely to jump into that quibble change this week ;/ [19:34:13] but I haven't tried for anything more complicated than mathoid yet, so TBD :) [19:35:04] oh I see, first step get it working in minikube, then promote it to the real thing? [19:35:47] on a related but disjoint note: should it be possible to use our docker repository directly? [19:36:03] you mean the wikimedia docker registry? [19:36:06] yes [19:36:33] yeah, you should be able to use kask images from there locally: https://tools.wmflabs.org/dockerregistry/wikimedia/mediawiki-services-kask/tags/ [19:37:06] aha, cool [19:41:20] How does tagging there work? [19:41:32] I see there is no latest [19:41:59] Wondering what I'd do for documentation [19:45:09] oh, tagging is the date+time and sha1 of the repo [19:45:31] in some cases there may also be *-candidate tags there [19:45:41] those are tagged for testing only [19:45:54] so 2019-05-10-162420-production [19:45:57] is latest [19:46:04] right, but nothing stable [19:46:31] 2019-05-10-162420-production is latest only until something new is added :) [19:46:41] indeed [19:46:50] yeah, docker is weird with latest [19:47:11] and we cannot add tags, correct? [19:47:38] like if we wanted to manually create one called `recommended` or somesuch and refer to it in documentation [19:47:56] you can add tags to your repo. If you tag a release in your repo, and push the tag to gerrit, it will create an image tagged as whatever you've called your tag [19:48:49] hrmm [19:48:50] hrm, I'm not sure about that use case though. Docker tags get confusing and local versions of the "latest" tag and upstream versions of the "latest" tag can diverge [19:49:21] and lead to ever greater confusion [19:49:38] semantic versioning of tags might be the best thing, really [19:50:02] I was just looking for a way to avoid referring to say 2019-05-10-162420-production in documentation, knowing that at some point that will be wrong [19:50:23] outdated, unavailable, etc [19:51:11] sure. For the most part we've avoided having a "latest" since that's caused headaches for folks in the past. [19:52:32] here's the task where we have been talking through image tagging and arguing generally about "latest" :)https://phabricator.wikimedia.org/T209088 [19:55:09] pro tip: 1) do not use latest [19:55:12] 2) do not use Docker [19:55:24] the tl;dr: my opinion is "latest" is kind of baked in to the docker workflow for end users, so we should support it but also never use it in production because it leads to weird/hard-to-reason-about state [19:55:36] hashar: :D [19:55:48] that's a next-level tip [19:57:23] oh my reason for not using the "latest" tag is that its equivalent to running Debian unstable or adding dependencies without version limits [19:57:54] and hmm . Docker build is similar in that respect. I would guess I am partly fine with docker to run containers, I just learned to hate docker build ;D [19:57:54] hashar: or master on a git repo [19:58:24] urandom: yeah same issue :] [19:58:52] but it's useful to have something stable to refer to sometimes [19:59:03] hashar: to use you Debian example, 'stable' [19:59:20] as in the stable release [19:59:29] and the stable name that refers to it [19:59:30] :) [19:59:36] whch itself is not necessarly stable enough. But I digress! [19:59:39] there is a Yo Dawg meme here somewhere [19:59:56] I heard you liked stable so I put some stable in your stable [20:00:11] :) [20:02:43] haha [20:12:11] (03PS5) 10Awight: [WIP] Rough out plan-execution decoupling [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [20:13:06] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10Aklapper) >>! In T223102#5201311, @mmodell wrote: > I think editing forms is limited to admins. That o... [20:26:46] (03CR) 10Brennen Bearnes: "Noting here that this has been used to publish everything under the /dev namespace in the WMF Docker registry." [releng/dev-images] - 10https://gerrit.wikimedia.org/r/510619 (https://phabricator.wikimedia.org/T223328) (owner: 10Brennen Bearnes) [20:51:06] 10Release-Engineering-Team (Kanban), 10Operations, 10SRE-Access-Requests, 10User-Urbanecm, 10User-greg: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830 (10RStallman-legalteam) Updated NDA for Urbanecm is fully signed and on file. [21:03:51] 10Continuous-Integration-Config, 10Librarization, 10Performance-Team, 10RunningStat: Publish Doxygen for RunningStat library - https://phabricator.wikimedia.org/T185724 (10aaron) a:05aaron→03None [21:10:24] twentyafterfour: hey, around? I'm done with the script [21:12:42] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: CI runs same tests twice - https://phabricator.wikimedia.org/T223725 (10hashar) T105474 is to prevent patches that got a CR+2 from being added to test. So that is really similar. I have identified a bunch of issues in doing so. But that... [21:12:52] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: CI runs same tests twice - https://phabricator.wikimedia.org/T223725 (10hashar) [21:12:56] 10Continuous-Integration-Config, 10Release-Engineering-Team (Backlog), 10Zuul, 10Patch-For-Review, 10Upstream: 'recheck' on a CR+2 patch should trigger gate-and-submit, not test - https://phabricator.wikimedia.org/T105474 (10hashar) [21:14:36] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10puppet-compiler: Puppet catalog compiler - increasing max concurrent jobs - https://phabricator.wikimedia.org/T221969 (10hashar) @herron any thoughts about my proposals? ;) [21:15:16] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10puppet-compiler: Puppet catalog compiler - increasing max concurrent jobs - https://phabricator.wikimedia.org/T221969 (10hashar) [21:26:58] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Wikimedia-Incident: CI is unavailable since around 10:00 UTC - https://phabricator.wikimedia.org/T222605 (10Ladsgroup) [21:27:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Operations: contint1001: DISK WARNING - free space: /srv 88397 MB (10% inode=94%): - https://phabricator.wikimedia.org/T219850 (10Ladsgroup) [21:28:36] https://phabricator.wikimedia.org/p/Ladsgroup/ [21:28:41] I'm removing those [21:28:52] 10MediaWiki-Releasing, 10MW-1.32-release: Unknown key used to sign MediaWiki 1.32.0 tarball - https://phabricator.wikimedia.org/T213521 (10Ladsgroup) [21:30:48] 10Continuous-Integration-Infrastructure, 10Operations: docker-registry is returnning HTTP 403 Forbidden for all requests - https://phabricator.wikimedia.org/T201737 (10Ladsgroup) [21:31:00] 10Release-Engineering-Team (Kanban), 10Performance-Team: RepoGroup exceptions due to "false" being passed as a key to MapCacheLRU - https://phabricator.wikimedia.org/T200026 (10Ladsgroup) [21:31:06] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban): Some Jenkins jobs lack a timeout (ex: mediawiki-phpunit-coverage-patch) - https://phabricator.wikimedia.org/T197519 (10Ladsgroup) [21:31:08] 10Scap, 10Operations, 10Wikimedia-Incident: Update Debian Package for Scap3 to 3.8.3-1 - https://phabricator.wikimedia.org/T198277 (10Ladsgroup) [21:34:55] 10Continuous-Integration-Infrastructure: integration-slave-jessie-1001 and integration-slave-jessie-1002 out of disk space - https://phabricator.wikimedia.org/T189587 (10Ladsgroup) [21:35:19] 10Release-Engineering-Team (Kanban), 10MinervaNeue, 10Readers-Web-Backlog, 10Vector, 10User-zeljkofilipin: Vector browser test blocking merge in Minerva - https://phabricator.wikimedia.org/T188553 (10Ladsgroup) [21:55:40] (03PS1) 10Dduvall: pipeline: Include .pipeline/config.yaml [blubber] - 10https://gerrit.wikimedia.org/r/511784 [21:57:06] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [blubber] - 10https://gerrit.wikimedia.org/r/511784 (owner: 10Dduvall) [21:57:45] (03PS2) 10Dduvall: pipeline: Include .pipeline/config.yaml [blubber] - 10https://gerrit.wikimedia.org/r/511784 [21:58:58] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [blubber] - 10https://gerrit.wikimedia.org/r/511784 (owner: 10Dduvall) [22:02:56] Amir1, reg your wikitech-l .. how interesting .. because just this morning, I remarked on #mediawiki-parsoid: "[10:36:15] it would be good if there were a way for gerritbot to auto-remove the patch-for-review tag from a phab task .. but i suppose it would then have to track all gerrit patches that reference that patch .. so, not a quick fix then.[10:36:22] *auto-remove on merge" [22:03:35] subbu: haha, it's happening now [22:03:37] :D [22:04:27] I know .. i just found it an interesting coincidence. :) [22:04:30] subbu well it [22:04:34] *it's supported [22:05:06] * paladox added that support. [22:05:58] !log generating new blubber-pipeline-* and trigger-blubber-pipeline-* jenkins jobs defined by https://gerrit.wikimedia.org/r/c/integration/config/+/510602 [22:05:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:06:06] thanks paladox :) [22:06:31] and with gerrit 2.16 you can configure gerritbot to do different actions per repo. [22:06:58] nice [22:07:54] (03CR) 10Dduvall: "I've gone ahead and generated the jobs for testing against https://gerrit.wikimedia.org/r/c/blubber/+/511784" [integration/config] - 10https://gerrit.wikimedia.org/r/510602 (https://phabricator.wikimedia.org/T224035) (owner: 10Dduvall) [22:09:47] well, i think we may be able to extend this so that GerritBot checks gerrit and sees if all changes are merged before removing the tag. [22:23:20] (03PS6) 10Awight: Separate planning and execution phases [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [22:26:47] 10Release-Engineering-Team (Kanban), 10Operations, 10SRE-Access-Requests, 10User-Urbanecm, 10User-greg: Requesting access to production for SWAT deploy for Urbanecm - https://phabricator.wikimedia.org/T192830 (10Dzahn) [22:31:30] 10Phabricator, 10CommRel-Design, 10CommRel-Internals: Create Phabricator form for CommRel-Design and Comms requests and add a link to it in the "Star" dropdown - https://phabricator.wikimedia.org/T223102 (10Aklapper) >>! In T223102#5199076, @mcruzWMF wrote: > 2. We should also create a form that is just for... [22:54:06] 10Continuous-Integration-Config, 10Librarization, 10Performance-Team, 10RunningStat, 10patch-welcome: Publish Doxygen for RunningStat library - https://phabricator.wikimedia.org/T185724 (10Krinkle) [22:55:52] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10dduvall) [22:56:14] (03PS7) 10Awight: Separate planning and execution phases [integration/quibble] - 10https://gerrit.wikimedia.org/r/511749 (https://phabricator.wikimedia.org/T223752) [22:56:53] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<20.00%) [23:00:12] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10dduvall) [23:04:48] (03PS1) 10Dduvall: pipeline: Execute "configure" stage on "blubber" nodes [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 [23:05:15] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 (owner: 10Dduvall) [23:05:43] (03CR) 10Dduvall: [C: 03+2] pipeline: Execute "configure" stage on "blubber" nodes [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 (owner: 10Dduvall) [23:06:06] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 (owner: 10Dduvall) [23:06:08] (03Merged) 10jenkins-bot: pipeline: Execute "configure" stage on "blubber" nodes [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 (owner: 10Dduvall) [23:06:41] (03CR) 10jenkins-bot: pipeline: Execute "configure" stage on "blubber" nodes [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511788 (owner: 10Dduvall) [23:38:44] (03PS1) 10Dduvall: pipeline: Support Zuul based SCM for configure stage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 [23:39:27] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:45:48] (03PS2) 10Dduvall: pipeline: Support Zuul based SCM for configure stage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 [23:46:04] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:46:37] (03CR) 10Dduvall: [C: 03+2] pipeline: Support Zuul based SCM for configure stage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:47:03] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:47:05] (03Merged) 10jenkins-bot: pipeline: Support Zuul based SCM for configure stage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:47:25] (03CR) 10jenkins-bot: pipeline: Support Zuul based SCM for configure stage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/511790 (owner: 10Dduvall) [23:51:46] (03PS3) 10Dduvall: pipeline: Include .pipeline/config.yaml [blubber] - 10https://gerrit.wikimedia.org/r/511784 [23:52:58] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [blubber] - 10https://gerrit.wikimedia.org/r/511784 (owner: 10Dduvall)