[07:21:24] does anyone have any recommendation regarding matrix clients and Debian? [07:24:08] <_joe_> not atm, I'll take a look this week. If you find anything, please let us know [07:37:46] Konversation's efforts to gain Matrix support seem to have come to a halt, but Spectral and Quaternion are packaged in Debian. Haven't tested them myself, but planning to do so in the next weeks [08:01:26] huh. i see the slack integration for irccloud is going to be disabled (not that i've used it) [08:05:35] sounds like it [08:05:52] I use it, so I hope it's not disabled too soon [08:19:45] <_joe_> well AIUI the *real* channels will be on matrix, correct? [08:19:54] <_joe_> and slack will only receive bridged messages [08:25:22] that's my understanding as well [08:46:14] There are a few icinga alerts that showed up over the weekend, cp5002, mw1344, heze, search-alerts grafana, dbstore1004 [08:53:15] XioNoX: cp5002 depooled, thanks for the heads up [09:42:26] hey volans, jynus wants to involve you in a python style war ;) [09:42:58] I don't care what is being used, I just don't want to be in the middle- you can fight and I will chose the winner [09:43:17] as in, I will go with whoever comes on top [09:43:42] I remember another linter enforcing ' IIRC [09:43:52] lol [09:44:25] T211750 for context [09:44:25] T211750: Introduce Python code formatters usage - https://phabricator.wikimedia.org/T211750 [09:44:38] the only think I ask is to have a single formatting everywhere related [09:46:31] BTW, I only worry about string delimiters, as those are big patches [09:46:42] the others I don't care, whatever works for you [09:49:03] volans: the CR in question is https://gerrit.wikimedia.org/r/c/operations/software/wmfmariadbpy/+/621753 [09:49:35] I can have a look shortly [09:49:46] i don't have a real opinion on black. i just want an opinionated formatter so i can stop wasting brain cycles on this. [09:50:24] if ricardo plans to do the same commits, I don't mind, I flagged it becaues I didn't see it on cumin [09:51:31] the line size I think is also smaller than before [09:51:37] the whole discussion about linters is in the task I've outlined before [09:51:49] volans: i've read it [09:52:06] we kinda all agree it's better to have one, noone has shown enthusiasm for black :D [09:52:10] that's the tl;dr [09:52:32] i don't care about enthusiasm :P [09:52:41] i didn't like gofmt when i started using go, either [09:52:49] but it's far far better to have a standard [09:52:55] indeed [09:56:50] IIRC we also lost steam on having black for puppet, which is quite an uphill battle, however for new repositories and smaller code bases I think we should just go for it [09:57:30] (see what I did there?) [09:57:59] ✨ [10:01:27] <_joe_> kormat: just go with black [10:01:48] <_joe_> I am using it in all new python projects/files at save, like gofmt [10:01:56] <_joe_> I don't love it, but who cares about it? [10:02:01] exactly [10:02:14] <_joe_> you have my stamp of approval [10:02:22] <_joe_> kormat: volans cares :D [10:02:31] who? :) [10:02:33] <_joe_> we don't. Hence, we win [10:02:39] ah haha [10:02:41] one of the main problem was how to reliably setup it so that on pre-commit would locally format everything [10:03:01] volans: i have CI enforcing it in my patchset, so i don't care as much about that. [10:03:03] and then add a CI job that ensures that running black produces no diff [10:03:23] <_joe_> volans: black can validate code [10:03:25] kormat: it's painful to contibute if we don't add some automatic way to have it [10:03:39] <_joe_> volans: every editor can run a linter on save [10:03:58] sure but per-project is all manual to setup [10:04:01] for everyone of us [10:04:25] something more automagic would be better IMHO [10:04:37] anyway I've already lost that battle, so not blocking anyone here [10:06:29] <_joe_> more automagic like CI autodetecting a project is python and automatically running tox? [10:06:50] <_joe_> I've asked for something like that to releng for... years? [10:07:10] <_joe_> it's not a good reason to stop adoption of a standard formatter. [10:07:48] no, I meant to apply it locally before git review [10:08:16] <_joe_> that's way less convenient than configuring your editor to autoformat on save [10:08:21] <_joe_> IMHO [10:08:49] volans: my CR also runs it as part of `tox`. so if you're doing local tests before sending for review (which everyone does, right? :), you'll catch it [10:11:27] what about adding an option tox env that apply black making the changes locally too? (maybe with commit --amend, dunno) [10:11:32] *optional tox env [10:12:07] also see the effort from lego.ktm to standardize the whole thing [10:13:30] re: `blackify`? sure, i'd be happy to add that. [10:58:14] fyi i did some changes re black and the puppet repo but just notiuced the where not related to T211750, just updated so they should all be on that phab task now [10:58:15] T211750: Introduce Python code formatters usage - https://phabricator.wikimedia.org/T211750 [10:59:30] jbond42: that reminds me that we have also a bunch of external 3rd party files in the puppet repo that most likely should be excluded from this, I don't recall if the argument was already disccussed/resolved [11:01:41] volans: i think i had to add something for that when i updated the python pep8 checks [11:02:39] there is this convension [11:02:39] # skip files copied from upstream [11:02:39] next if file.end_with?('.original.py') [11:02:58] which delt with some files cloud wanted to geep in the repo as exact copies of upstream [11:03:46] jbond42: ack, I guess we should skipe those from black too [11:03:54] so yes could be more that work with pep8 but not black [11:03:58] yes [11:04:31] while since i looked at the code but i think the stuff i added uses the same function to collect python files for both pep8/flake and black [11:06:02] the biggest issue with that patch set and what ultimatly blocked it is we need to apply black8 to all the current files and accept that i.e. something like this https://gerrit.wikimedia.org/r/c/operations/puppet/+/554825/8. which looks like a big scary change as it toches a lot of files. the counter argument being if we plan to trust black we should start now [11:07:01] * jbond42 is happy to refresh the changes if i can get reviewers [12:54:12] huh. i just realised there's a stronger possible review result than "-2". you can click 'abandon' on someone else's change :) [12:55:05] or you could delete gerrit entirely [12:55:22] the middle ground: delete their gerrit account ;) [17:01:37] _joe_: we added recently https://phabricator.wikimedia.org/rOPUP737f0ad75dc8de98c468aaa84206edaa9f62a71c for obvious OOMs [17:01:56] but maybe in your case you would want to track the prometheus aggregated value [17:02:08] or slab allocations, if they are on prometheus [17:02:17] difficult to get right as you said [17:07:31] jynus: it's complicated because the thing to look for isn't the number of kernel-reported slab bytes, it's that the number of unreclaimable slabs was growing without bound [17:07:40] (and in our case, growing super-linearly) [17:10:13] but that did have an impact on non-free pages, didn't it? [17:14:25] yes, it did [17:14:54] so ofc not for debugging, but it could help for "early detection" [18:41:46] anyone around who's willing to admit to knowledge about the compiler-update-facts script? [18:42:15] it's failing for me with output ending in https://www.irccloud.com/pastebin/8jLvhtdj/ [18:47:40] rzl: I think it's an ssh issue [18:47:55] can you ssh to compiler1001.puppet-diffs.eqiad.wmflabs ? [18:48:20] ... I cannot. thanks, should have thought of that [18:48:23] looking at horizon now [18:48:44] rzl: could be either permissions or jumphost issue [18:49:08] nod [18:49:51] yeah I'm just not in the puppet-diffs project [18:50:48] oh [18:57:17] mutante: do you mind adding reuven on puppet-diffs? [18:57:24] I do not have superpowers there [18:57:42] I can add you, what's your shell name [18:57:50] rzl [18:57:51] thanks! [18:57:58] thank you apergos! [18:58:21] you are added as a regular member, not an admin [18:58:24] should do for now [18:58:35] 👍 much obliged [19:00:48] oh, i see i was too slow :) [19:03:08] working now, thanks all [19:03:16] sweet [23:00:19] DC-Ops is still part of SRE, right? the separation of "Data Center Operations" and "Site Reliability Engineering" at https://wikimediafoundation.org/role/staff-contractors/ would suggest otherwise