[10:18:05] can someone answer https://issue-tracker.miraheze.org/T12565#251370? out of my depth with this one [14:12:56] (just for personal reference, db available size before DBs dropped): db151 - 470G, db161 - 343G, db171 - 592G, db181 - 611G [14:13:12] Wow, I didn't realize we had so much DB space on the "new" infra [14:20:18] I wonder if we should optimise them [14:38:19] How much is used [14:39:12] from 30-60% depending on the db [14:39:24] but that much available space is huge, I remember the days when it was in the low double digits [14:39:25] Dang [14:39:33] That don’t sound fun [14:39:40] oh yes, fun times [14:39:54] then we had to compress revisions [14:40:33] and then deleting and dropping wikis was actually needed regularly as otherwise there wouldn't be enough space [14:40:45] the worst was when me and Paladox had to move DBs manually around to balance DBs [14:40:56] Yikes [14:41:08] How is the cluster of a new wiki decided [14:41:14] Random? [14:41:49] yaeh [14:41:58] it didn't use to! [14:42:05] Wha [14:42:09] but it'd be nice if it could be random but also based on how much space each one has [14:42:15] Yeah [14:42:22] wiki clusters weren't even in managewiki, they were in Database.php [14:42:32] The horror [14:43:03] @pixldev look at this heh - https://github.com/miraheze/mw-config/blob/25352ab9fca9b108bd3bfd8f05730656653fce80/Database.php#L5 [14:43:14] so most wikis were on a single cluster but the ones that weren't had to be specified there manually [14:45:26] That’s terrifying [14:45:43] yeah... we take so many things for granted now but to be a mw-admin back in the day was something else [14:46:05] manual extension enabling, manual cluster switches, fully manually imports and the list goes on [14:52:36] Anyways what’s new with infra [14:52:44] Saw smt about more mw servers [14:53:26] nini, let the bed bugs bite [14:53:42] I doubled our mw server count [14:53:46] No [14:53:53] so we should be running super fast now [14:53:58] give it a try if you haven't [14:53:59] Vroom [14:54:54] not unless they haven't been added to the load balancer on Cloudflare [14:55:14] they have 😉 [15:11:04] Why do 2 pools and not just add them to the original pool [15:11:27] there's a limit of 10 endpoints [15:11:27] You need to add it to WikiTide.org too [15:12:17] Ah [15:12:20] Config looks fine [15:12:33] I will update WikiTide.org [15:12:39] Which isn't on cf yet [15:12:49] But I have done the config on that mostly ready [15:14:52] I did that [15:15:19] @agentisai I also turned on adaptive routing cause that will prevent downtime if all the old but none of the new servers are down and a user is mid request [15:17:18] @agentisai did you create new mwtask too? [15:17:36] I will fix the deploy puppet's bug so it doesn't deploy from every task to all servers [15:17:48] And turn off the notify for non task [15:17:52] When back from holiday [15:20:59] @agentisai when back from my holiday, I might make it a pool by cloud servers [15:21:14] I think that's more logical than the current [15:46:19] @agentisai I see we have a mwtask on every cloud now [15:47:00] Oh great, thank you! [15:47:16] that does make a lot of sense, actually [15:47:39] yep, hopefully this will help alleviate some load issues with mwtask* servers [15:48:04] recently, it seems there's been a lot of transcoding jobs which end up taking over mwtask* [15:48:33] I looked and I think we just need to remove is_canary from one of the two current ones [15:48:38] Tbh [15:48:43] We could leave 2 [15:48:53] You haven't turned it on on the new ones [15:48:57] Which is good [15:49:11] 2 isn't too bad so one is prepped [15:49:22] We should be able to disable config auto deploy on one though [15:49:25] yeah, one is there for backup [15:49:33] I agree [15:49:45] That should be easy in puppet [15:49:54] If no one does it before next Tuesday then I will [15:50:14] Maybe dedicated one task to transcode as a priority [15:50:54] can we buy an AMD GPU for one of the cloud servers? [15:51:11] we do have the option to, actually [15:51:11] We could use it for hardware accelerated transcoding via VA-API [15:51:48] Fiberstate ran a promotion where if you prepaid for a year, they'd add in a 4090 [15:52:11] A GPU would be good [15:52:29] if the next fundraiser goes well, maybe that can be looked in to [15:52:36] that'd be interesting, heh [16:17:57] [1/2] (moved from general) Is there something with the cache or stuff? [16:17:58] [2/2] My wikis (lhmnwiki/cgwiki) are randomly showing English instead of Vietnamese [16:19:18] https://cdn.discordapp.com/attachments/1006789349498699827/1282737104769519707/L.png?ex=66e07185&is=66df2005&hm=db69ac5f34fa31f758db0b4643b6453c72768dafc2af4f5bdb3d0b2f09891366& [17:25:07] Weird [17:25:16] Could be a cache issue [17:26:07] @agentisai phab seems to be broken [17:26:36] what's the matter? [17:26:46] No styles [17:26:50] It's plain text [17:27:35] loads with styles for me [17:27:43] does the console say anything for you? [17:28:01] Mobile [17:28:28] It's back now [17:28:34] Dumb technology [18:42:24] @agentisai I'm getting not resolved for http://grafana.wikitide.net/ [19:05:59] Works for me [19:56:40] I actually had that too this morning [19:56:49] it worked after but it refused to resolve at the beginning