[17:41:24] If I want to do blocks from inside a maintenance script.. What's going to be the best way without duplicating a loooad of MW code? [17:41:29] Internal call to the API? [17:41:44] (which itself, looks to be a lot of duplicated code from SpecialBlock) [17:43:51] honest answer Reedy is that the business logic on the MW side needs refactoring into a controller that is used by the API and the special page [17:44:02] bd808: "no shit" [17:44:03] :D [17:44:19] * bd808 refrains from linking the wm.o page where he was going to run a team doing this 5 years ago [17:44:29] While I'm not looking to fix that, I don't want to make the problem worse by duplicating it further [17:51:05] bd808: See also EditPage.php. ;-) [17:52:48] James_F: for sure. The number of things that are duped between specialpages and api actions is repulsive really. I mean there is tech debt everywhere and that's just going to happen, but we have fundamentally bad legacy code reuse issues. [17:53:23] Yeah. It makes me quite sad when I try to explain MW to newbie devs. [17:53:39] <_joe_> blocks? [17:53:54] _joe_: account blocking [17:54:00] <_joe_> oh I see [17:54:13] * _joe_ runs away screaming [17:54:42] * James_F grins. [17:55:00] <_joe_> I came around seeking for some guidance on metrics though. Context is https://phabricator.wikimedia.org/T240685 [17:55:12] <_joe_> now I want to see how metrics are emitted from MediaWiki [18:13:31] EditPage is nasty [18:13:56] SpecialBlock basically just creates a DatabaseBlock object, calls all the setters on it and calls insert() [18:14:06] nothing that would be complicated to reproduce [18:14:42] that plus logging [18:15:26] Just seems stupid copy pasting that much code [18:15:59] https://github.com/wikimedia/mediawiki/blob/master/includes/api/ApiBlock.php#L50-L175 [18:16:04] Granted, might not need it all... [18:16:34] ideally teams that work with a specific area of code and do major refactorings on it would also move business logic into controllers, but it does not seem to happen when that team is not the core team [18:17:15] AbuseFilter also issues blocks btw if you want to look at a more programmatic example [18:17:24] _joe_: you may be looking for the things in includes/libs/stats [18:17:41] <_joe_> bd808: I was more looking at how that's used too [18:18:17] *nod* [18:18:24] https://github.com/wikimedia/mediawiki-extensions-AbuseFilter/blob/b28e9e8c1f5fdce6fdf3ea5af9739d56aedf2228/includes/AbuseFilterRunner.php#L1073-L1109 [18:19:18] that looks pretty reasonable [18:19:28] <_joe_> bd808: and clearly we have no standard on metrics naming or labeling [18:19:33] _joe_: I saw a bit of a comment from you about php shared nothing and I agree. At $DAYJOB-1 we used a sidecar type of process on each PHP host to catch metric info and then send it on to the right places. [18:19:36] no we don't [18:19:40] or log channels [18:19:46] <_joe_> so yeah [18:19:49] an enterprising person might move the log part into a dedicated Block method, SpecialBlock probably has the exact same code [18:19:49] or how to actually use structured logging [18:21:55] <_joe_> bd808: yeah so, prometheus works well when you use consistent naming and actual labels [18:21:56] Reedy, tgr: ApiBlock is a really bad example, because it calls into SpecialBlock... There's a "BlockManager" service now I see, maybe that's a good place for some of the backend code to live. [18:22:11] <_joe_> and right now translating what we have into anything consistent or rational seems impossible [18:22:38] On the plus side, I made DannyS712 do things the right way in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/563701 [18:22:49] _joe_: *nod* This is one of the things I anticipated when folks decided that Prometheus was the new hotness and graphite should die [18:23:02] it can be done, but its not going to be easy [18:23:06] <_joe_> bd808: there is a good reason for that [18:23:21] graphite doesn't scale well [18:23:27] <_joe_> it's more or less the differnce between grep and sql, too [18:23:33] <_joe_> in terms of searchability of metrics [18:24:14] <_joe_> well a crippled grep, given how graphite operates [18:25:00] Prometheus is neat, but it closes a lot of doors. Like not having a graphing api that produces images is a big change to work around [18:25:48] I expect that hundreds have been built by now by folks migrating from graphite [18:26:23] Or I'm thinking wrong and only green field projects are really using Prometheus [18:26:54] A ton of the CNCF ecosystem seems to presume infinite hardware, time, and no legacy migrations [18:27:37] <_joe_> bd808: almost everything but mediawiki works with prometheus for us now [18:27:52] <_joe_> and the graphing api for prometheus is grafana :P [18:28:31] grafana is a graph tool, but not an API. And MediaWiki is kind of a big thing for us I think? ;) [18:29:17] and "us" is scoped to prod SREs too there I think because we have not made the jump away from diamond in Cloud VPS yet [18:29:49] we have prometheus in the tools and deployment-prep projects but not the other ~170 prjects [18:32:11] <_joe_> "us" is scoped to all prod excluding MW. All services have their metrics collected by prometheus AIUI [18:33:06] <_joe_> and I don't think MediaWiki produces 1/10th of the total metrics we collect overall [18:33:22] <_joe_> if it wasn't important, I wouldn't care asking :P [18:35:17] Yeah, sorry if it sounds like I'm arguing against you or moving to Prometheus. I only mean to ramble about some of the migration issues and empathize over buried hard work in the long tail.