[01:19:16] <wikibugs>	 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 10User-brennen: Ocassionally pushing to GitLab takes very long - https://phabricator.wikimedia.org/T332967 (10Legoktm) Pulling 2 commits in `repos/mwbot-rs/toolforge` took 3+ minutes today:  `lines=10 time GIT_TRACE=1 git pull 18:14:24.266517 git...
[02:45:02] <wmf-insecte>	 Project beta-code-update-eqiad build #454819: 04FAILURE in 2 min 1 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454819/
[02:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454820: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454820/
[03:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454821: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454821/
[03:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454822: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454822/
[03:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454823: 04STILL FAILING in 0.56 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454823/
[03:33:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454824: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454824/
[03:43:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454825: 04STILL FAILING in 0.62 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454825/
[03:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454826: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454826/
[04:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454827: 04STILL FAILING in 0.63 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454827/
[04:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454828: 04STILL FAILING in 0.59 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454828/
[04:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454829: 04STILL FAILING in 0.56 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454829/
[04:33:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454830: 04STILL FAILING in 0.54 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454830/
[04:43:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454831: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454831/
[04:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454832: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454832/
[05:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454833: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454833/
[05:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454834: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454834/
[05:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454835: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454835/
[05:33:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454836: 04STILL FAILING in 0.65 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454836/
[05:43:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454837: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454837/
[05:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454838: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454838/
[06:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454839: 04STILL FAILING in 0.6 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454839/
[06:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454840: 04STILL FAILING in 0.61 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454840/
[06:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454841: 04STILL FAILING in 0.55 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454841/
[06:33:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454842: 04STILL FAILING in 0.56 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454842/
[06:43:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454843: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454843/
[06:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454844: 04STILL FAILING in 0.54 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454844/
[07:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454845: 04STILL FAILING in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454845/
[07:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454846: 04STILL FAILING in 0.56 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454846/
[07:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454847: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454847/
[07:33:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454848: 04STILL FAILING in 0.59 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454848/
[07:43:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454849: 04STILL FAILING in 0.61 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454849/
[07:51:53] <wikibugs>	 (03PS1) 10Robert Vogel: Review access change [extensions/HeaderFooter] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/942728
[07:52:03] <wikibugs>	 (03CR) 10Robert Vogel: [V: 03+2 C: 03+2] Review access change [extensions/HeaderFooter] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/942728 (owner: 10Robert Vogel)
[07:53:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454850: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454850/
[08:03:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454851: 04STILL FAILING in 0.57 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454851/
[08:13:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454852: 04STILL FAILING in 0.58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454852/
[08:23:01] <wmf-insecte>	 Project beta-code-update-eqiad build #454853: 04STILL FAILING in 0.56 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454853/
[08:23:58] <jnuche>	 !log temporarily disabled failing beta-code-update-eqiad job
[08:23:59] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:13:56] <wikibugs>	 10GitLab (Auth & Access), 10Release-Engineering-Team (Radar), 10CAS-SSO, 10Infrastructure-Foundations, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10Jelto) OIDC is enabled instance-wide now.   >>! In T320390#9052117, @dancy wrote: > In https://gitlab...
[09:15:11] <jnuche>	 _joe_, claime: morning 👋
[09:15:21] <jnuche>	 I'm looking into a problem with the Jenkins beta jobs, `/srv` partition on the Jenkins beta node is out of space
[09:15:27] <_joe_>	 uh hello
[09:15:28] <claime>	 yello
[09:15:28] <jnuche>	 I've seen that in `/srv/deployment-charts`, `.git` is taking up 14G
[09:15:34] <claime>	 oof
[09:15:36] <_joe_>	 wat
[09:15:39] <jnuche>	 last locally checked out commit there goes back to May so I'm guessing we are not using those charts in beta atm?
[09:15:51] <jnuche>	 tried running `git gc` but it cannot reach some commits and fails, fetching changes has the same problem
[09:16:00] <jnuche>	 I was thinking of deleting the dir altogether, unless you think it's actually being used
[09:16:04] <_joe_>	 jnuche: I don't understand
[09:16:23] <_joe_>	 it's 38 M on my computer and I do have 100s of dead branches I'm sure
[09:16:30] <_joe_>	 but yes that makes sense
[09:16:44] <_joe_>	 so, if it is, a puppet run should bring it back
[09:16:49] <jnuche>	 jnuche@deployment-deploy03:/srv/deployment-charts$ du -sh .git
[09:16:49] <jnuche>	 14G	.git
[09:16:59] <_joe_>	 jnuche: I am not saying you're wrong
[09:17:03] <_joe_>	 I'm saying I'm surprised
[09:17:19] <jnuche>	 I know, just wanted to show what I was seeing :)
[09:17:27] <_joe_>	 for comparison
[09:17:29] <_joe_>	 deploy1002:/srv/deployment-charts $ du -sh .git
[09:17:31] <_joe_>	 22M	.git
[09:17:34] <claime>	 I agree with _joe_, just remove it, if it's needed puppet will put it back
[09:17:47] <wikibugs>	 10GitLab (Auth & Access), 10Release-Engineering-Team (Radar), 10CAS-SSO, 10Infrastructure-Foundations, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10CodeReviewBot) jelto opened https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_request...
[09:17:48] <claime>	 I don´t  really see what it's doing on beta
[09:17:49] <_joe_>	 wait one sec
[09:17:58] <jnuche>	 waiting
[09:17:59] <_joe_>	 I want to look at that git directory
[09:18:40] <jnuche>	 the repo doesn't seem to be in that node's puppet catalog though: https://puppet-compiler.wmflabs.org/output/942646/42724/deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud/prod.deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud.pson.gz
[09:21:46] <_joe_>	 jnuche: sorry I'm having some issues accessing the cloud bastion for some reason
[09:22:03] <claime>	 cgoubert@deployment-deploy03:/srv/deployment-charts/.git/objects/pack$ ls -l | wc -l
[09:22:04] <claime>	 38692
[09:22:08] <claime>	 times 552K
[09:22:12] <claime>	 That's a lotta files
[09:22:34] <jnuche>	 yup
[09:24:00] <claime>	 That's all tmp_pack files though
[09:25:02] <ihurbain>	 o/ not sure what the status of logspam-watch is, but since there was some concerns on slack a couple of weeks ago around a noisy rt-testing on parsoid, just wanted to mention that we've had a very noisy rt-testing again on friday, so if that pops up there, it's safe to ignore :)
[09:26:25] <jnuche>	 ihurbain: thanks for the heads-up! gonna cross-post that to slack
[09:26:52] <ihurbain>	 thanks :)
[09:27:19] <claime>	 git fsck complains about broken links all over
[09:27:34] <claime>	 Honestly I think all of it needs to die in a fire and get recloned
[09:27:55] <jnuche>	 I like your style
[09:28:02] <claime>	 But I'm also questioning why that clone is there in the first place
[09:28:27] <claime>	 Is that jenkins host running ci for deployment-charts ?
[09:28:35] <_joe_>	 no 
[09:28:41] <claime>	 afaik there's no beta deployments of deployment-charts
[09:28:42] <_joe_>	 that is the deployment server for beta
[09:28:43] <claime>	 so wth
[09:29:04] <jnuche>	 maybe there was some attempt in the past to run some K8s stuff there? no idea
[09:29:08] <claime>	 mebbe
[09:29:26] <_joe_>	 no
[09:29:33] <_joe_>	 it's just the puppetization I guess
[09:30:08] <claime>	 probably yeah
[09:30:12] <_joe_>	 and indeed it seems to be stuck in may
[09:30:27] <claime>	 _joe_: On a git template commit from alex
[09:30:33] <claime>	 I say we say it's his fault
[09:30:34] <jnuche>	 the node doesn't seem to be a `role::deployment_server::kubernetes` AFAICS though
[09:30:35] <claime>	 :p
[09:59:01] <wikibugs>	 (03PS1) 10Robert Vogel: Add new test dependency to BlueSpicePageAccess [integration/config] - 10https://gerrit.wikimedia.org/r/943514
[10:00:33] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Add new test dependency to BlueSpicePageAccess [integration/config] - 10https://gerrit.wikimedia.org/r/943514 (owner: 10Robert Vogel)
[10:02:37] <wikibugs>	 10GitLab (Pipeline Services Migration🐤), 10collaboration-services: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171 (10LSobanski)
[10:03:56] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Radar), 10DBA, 10Data-Persistence-Backup, and 2 others: phabricator->phorge migration - database handling - https://phabricator.wikimedia.org/T335080 (10LSobanski) @brennen How's this going, do you need anything else from us or Data Persistence to proceed?
[10:20:06] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Escape Goats🐐), 10collaboration-services: Sync ldap/ops users to group on GitLab and define permissions for repos/sre - https://phabricator.wikimedia.org/T343035 (10LSobanski)
[10:34:59] <TheresNoTime>	 jnuche: hi, how's the beta job looking? (only now seen it was failing)
[10:38:47] <jnuche>	 TheresNoTime: I think it should recover once I free some disk space
[10:38:56] <jnuche>	 _joe_: what do you think? chart-ago delenda est?
[10:39:13] <_joe_>	 jnuche: please go on
[10:39:19] <jnuche>	 thx
[10:39:30] <_joe_>	 something broke there in ways I don't understand 
[10:39:51] <jnuche>	 yeah, it's pretty weird
[10:40:13] <wikibugs>	 10Beta-Cluster-Infrastructure: deployment-deploy03 /srv at 100% usage - https://phabricator.wikimedia.org/T343114 (10TheresNoTime)
[10:40:24] <TheresNoTime>	 ^ just for the record I guess
[10:41:45] <jnuche>	 TheresNoTime: thanks, gonna update that ticket
[10:45:32] * TheresNoTime should have checked https://www.isbetabroken.com /j
[10:46:16] <wikibugs>	 10Beta-Cluster-Infrastructure: deployment-deploy03 /srv at 100% usage - https://phabricator.wikimedia.org/T343114 (10jnuche) Deleted `/srv/deployment-charts` to free some space. `.git` dir there had grown to 14G.  The repo wasn't puppetized as far as I could see and hadn't gotten any new commits from the remote...
[10:47:06] <wikibugs>	 10Beta-Cluster-Infrastructure: deployment-deploy03 /srv at 100% usage - https://phabricator.wikimedia.org/T343114 (10jnuche) After deletion:  ` jnuche@deployment-deploy03:/srv$ df -h Filesystem      Size  Used Avail Use% Mounted on udev            3,9G     0  3,9G   0% /dev tmpfs           799M   81M  718M  11%...
[10:47:42] <wikibugs>	 10Beta-Cluster-Infrastructure: deployment-deploy03 /srv at 100% usage - https://phabricator.wikimedia.org/T343114 (10jnuche) 05Open→03Resolved a:03jnuche
[10:49:39] <TheresNoTime>	 *that* is going to be a big `beta-code-update-eqiad` run..
[10:52:15] <wmf-insecte>	 Yippee, build fixed!
[10:52:15] <wmf-insecte>	 Project beta-code-update-eqiad build #454854: 09FIXED in 3 min 51 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/454854/
[10:52:35] <jnuche>	 neat!
[10:52:55] <TheresNoTime>	 thanks jnuche! :D
[10:53:03] <jnuche>	 !log reenabled beta-code-update-eqiad job after freeing up some disk space. See: https://phabricator.wikimedia.org/T343114 
[10:53:05] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:53:13] <jnuche>	 np!
[10:57:12] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Escape Goats🐐): deployment-deploy03 /srv at 100% usage - https://phabricator.wikimedia.org/T343114 (10jnuche)
[11:06:43] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] Disable the build pipeline in zuul for datahub [integration/config] - 10https://gerrit.wikimedia.org/r/942689 (https://phabricator.wikimedia.org/T341194) (owner: 10Btullis)
[11:08:31] <wikibugs>	 (03Merged) 10jenkins-bot: Disable the build pipeline in zuul for datahub [integration/config] - 10https://gerrit.wikimedia.org/r/942689 (https://phabricator.wikimedia.org/T341194) (owner: 10Btullis)
[11:09:33] <James_F>	 !log Zuul: [analytics/datahub] Mark as archived, moved to GitLab in T341194
[11:09:35] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[11:09:36] <stashbot>	 T341194: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194
[11:16:10] <James_F>	 !log jjb: Manually deleted ~30 jjb definitions for analytics/datahub, plus the now-empty datahub view in Jenkins, for T341194
[11:16:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[11:16:13] <stashbot>	 T341194: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194
[11:16:45] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10Data Engineering and Event Platform Team, 10Data-Platform-SRE: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194 (10BTullis) Oh, I'm getting a crashloop from the mce-container: It looks l...
[11:38:49] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10Data Engineering and Event Platform Team, 10Data-Platform-SRE, 10Patch-For-Review: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194 (10CodeReviewBot) btullis opened https://gitlab.wiki...
[13:00:26] <wmf-insecte>	 Yippee, build fixed!
[13:00:26] <wmf-insecte>	 Project beta-scap-sync-world build #114135: 09FIXED in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/114135/
[13:18:01] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10Data Engineering and Event Platform Team, 10Data-Platform-SRE, 10Patch-For-Review: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194 (10CodeReviewBot) btullis merged https://gitlab.wiki...
[13:59:15] <wikibugs>	 10Release-Engineering-Team (Escape Goats🐐), 10Release, 10Train Deployments: 1.41.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T340248 (10Lucas_Werkmeister_WMDE) ##### Risky Patch!  🚂🔥  * **Change**:  https://gerrit.wikimedia.org/r/c/mediawiki/core/+/940328 * **Summary**: ** Refactoring in...
[14:29:00] <wikibugs>	 (03PS2) 10Robert Vogel: Add new test dependency to BlueSpicePageAccess [integration/config] - 10https://gerrit.wikimedia.org/r/943514
[15:04:50] <isaranto>	 dduvall: I opened an MR for appending PYTHONPATH in blubber https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/49. I'm pinging here as it seems I can't add a reviewer on gitlab
[15:12:05] <wikibugs>	 10GitLab (Auth & Access), 10Release-Engineering-Team (Radar), 10CAS-SSO, 10Infrastructure-Foundations, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10CodeReviewBot) dancy merged https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_request...
[15:13:17] <wikibugs>	 10GitLab (Infrastructure), 10collaboration-services: Fix gitlab-runner registration token in gitlab-runners wmcs project - https://phabricator.wikimedia.org/T342654 (10LSobanski) a:03eoghan
[16:12:07] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Phatality: Link to git blame URLs for each stracktrace frame reported by Phatality - https://phabricator.wikimedia.org/T342400 (10dduvall) 05In progress→03Resolved
[16:12:42] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10Data Engineering and Event Platform Team, 10Data-Platform-SRE: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194 (10BTullis) This is now done. We have fully deployed the new images genera...
[16:13:09] <dduvall>	 isaranto: excellent! i'll have a look. re: "can't add a reviewer on gitlab" what do you mean by that exactly?
[16:16:27] <isaranto>	 dduvall: I cannot edit the reviewers field on the MR to add someone (permissions issue). I don't know if what I'm saying makes sense but most of the times I just assign someone through there. Anyway I can always mention someone in a comment
[16:19:03] <dduvall>	 isaranto: oy. that's annoying and strange. brennen do you happen to know why ^ would be. afaict it only should take higher than a guest role to assign a reviewer. from https://docs.gitlab.com/ee/user/permissions.html "Merge requests: Assign reviewer	 	✓	✓	✓	✓"
[16:19:53] <dduvall>	 (that paste is probably not helpful since the roles are not present)
[16:19:54] <brennen>	 isaranto: is the person you're trying to add as a reviewer a member of the project?
[16:20:14] <brennen>	 oh, it's blubber, so...  hrm.
[16:22:18] <isaranto>	 brennen: yeah it is dduvall. it is not that I cannot add him, I cannot edit the field on the MR at all. 
[16:22:50] <brennen>	 yeah, just checking this out by impersonating your account
[16:22:59] <brennen>	 and sure enough.
[16:24:57] <brennen>	 my guess is that this is just because you're not a member of repos/releng/blubber.
[16:25:25] <brennen>	 (or any of its parent groups.)
[16:25:47] <brennen>	 that sure is a profoundly annoying limitation for the fork-and-merge-request workflow.
[16:27:15] <sbassett>	 Hey releng - interesting container infoleak issue here: https://phabricator.wikimedia.org/T343138.  I subbed your team and added thcipriani and brennen.
[16:32:26] <isaranto>	 I don't know what the policy is but all wmf engineers could be part of a group which can automatically be added to all repos as developer. Just thinking out loud here :)
[16:40:43] <dancy>	 brennen: My guess is that it's because dduvall is not a member of https://gitlab.wikimedia.org/isaranto/blubber
[16:42:34] <dancy>	 sbassett: thanks!
[17:22:34] <dduvall>	 dancy: i don't think that's it. isaranto is trying to edit an MR under repos/releng/blubber so wouldn't that be about his access to one of those groups in the hierarchy?
[17:23:10] <dduvall>	 seems like we should be granting everyone at least Reporter access
[17:24:41] <dancy>	 ah, yeah, that makes sense.  
[17:24:48] <dduvall>	 instance wide, or at least to `/repos`
[17:26:24] <dduvall>	 still seems a bit broken the way it's currently behaving, however. if someone is allowed to create an MR under a project in the first place, shouldn't they be able to assign a reviewer?
[17:26:43] <dduvall>	 i suppose the alternative would be to have bots handle that
[17:35:54] <brennen>	 i have a feeling that dropping everyone into /repos would have side effects we'd rather avoid.
[17:36:28] <brennen>	 ...although maybe there's a default level of access for active users we could tweak?
[17:50:28] <dduvall>	 is there? i can't seem to find one
[17:50:35] <dancy>	 I don't think so.
[18:14:07] <dduvall>	 isaranto: just released a new blubber (v0.19.0) that includes your change. add `# syntax=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.19.0` to the first line of your `.pipeline/blubber.yaml` to give it a whirl
[18:15:16] <isaranto>	 dduvall: awesome! I'll try it out first thing in the morning. Thanks a lot!
[18:16:34] <dduvall>	 np. thanks for the feature!
[18:28:56] <thcipriani>	 hrm, have to be a reporter to even see the list of merge requests https://docs.gitlab.com/ee/user/permissions.html it would be nice to be able to configure that somewhere. Seems like something I'm able to do without any permissions in GitHub land, even.
[18:30:33] <thcipriani>	 seems like there are sometimes two sides to permissions in the gitlab world (1) user role (2) repo settings re:what a user role is allowed to do
[18:30:56] <thcipriani>	 and then some of these things hard-set are instance-wide
[18:36:18] <dduvall>	 thcipriani: i believe role is a relation of member, so to affect the role of a user in relation to a group/project, there has to be a membership of some sort
[18:40:11] <thcipriani>	 right, a role is a relation of a member to a project. And in some cases a roles's permissions within a project are set on the project-level and in some cases on the instance-level. Like what role can merge to a protected branch is set on the project level vto be either maintainer or developer s. what role can delete a project is set on an instance level as owner
[18:41:37] <thcipriani>	 so, yeah, in order to view merge requests or assign a reviewer , according to the docs, you need to be a reporter on the project
[18:42:57] <thcipriani>	 the places to tweak that could be on the instance level: i.e., what role is allowed to list merge requests by default; at the project level what role is allowed to list merge requests; or maybe it's just hard-coded
[18:44:29] <thcipriani>	 anyway: you're right we'd need to add folks as *something* maybe we have leeway in the specific role
[18:45:29] <thcipriani>	 and maybe we can tweak the specific role set in 𝑛 places where 𝑛 is between 0 and 3
[18:48:47] <thcipriani>	 anyway, mostly this is ranting about having to give people a relation to a project for them to even see a list of merge requests: that seems like a weird decision.
[20:19:08] <bd808>	 Some of y'all may enjoy Yuvi's most recently published tool: https://werecomputersamistake.com/
[20:36:31] <dduvall>	 very nice
[20:45:13] <dancy>	 haha
[20:46:54] <brennen>	 hearty endorsement.
[20:48:12] <thcipriani>	 I read that as "we're computer's mistake" and was expecting some sci-fi
[20:57:02] <bd808>	 My brain keeps seeing were-computers-ami-stake too. I have AWS brain worms!
[20:59:58] <wikibugs>	 (03PS1) 10Ahmon Dancy: Review access change [dumps] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/943593
[21:02:41] <wikibugs>	 (03Abandoned) 10Ahmon Dancy: Review access change [dumps] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/943593 (owner: 10Ahmon Dancy)
[22:15:36] <icinga-wm>	 PROBLEM - Check systemd state on doc2002 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-host-data-sync.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[23:13:08] <icinga-wm>	 RECOVERY - Check systemd state on doc2002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[23:35:20] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog 📥), 10Data Engineering and Event Platform Team, 10Data-Platform-SRE: Migrate analytics/datahub pipeline to GitLab - https://phabricator.wikimedia.org/T341194 (10thcipriani) >>! In T341194#9052367, @thcipriani wrote: >>>! In T341194#...