[08:27:18] <_joe_> and here is me hoping we'd get rid of wikireplicas as they exist today [08:32:24] ? [11:00:34] <_joe_> fabfur: go look at the architecture of wikireplicas, it will make things clearer :) [11:01:09] <_joe_> the TLDR is: we're mostly using mysql replication to do what should be an ETL job [11:01:55] <_joe_> but if you want the full picture, ask marostegui or Amir1 :) [12:15:14] FYI, I’ve requested an emergency backport in #wikimedia-operations, and per the previous discussion in the security channel I’ll go ahead with it soon unless someone objects [12:15:36] (the TL;DR was that I need a deployer, such as myself, and SRE should be aware, so posting here for awareness :)) [12:40:38] * Lucas_WMDE done deploying [14:24:53] dunno if y'all saw this, but it looks like the OSS Puppet fork just dropped https://thenewstack.io/openvox-the-community-driven-fork-of-puppet-has-arrived/ [14:25:37] there was an interesting convo some days ago [14:25:47] don't remember on which channel [14:30:00] Has anyone tried it yet? [14:36:25] install it to beta, quick :) [14:41:37] ...before there's a CVE that we have to sign a EULA to fix ;P [14:51:05] err whaat? [14:54:59] https://www.puppet.com/blog/open-source-puppet-updates-2025 "We will release hardened Puppet releases to a new location and will slow down the frequency of commits of source code to public repositories. " [14:55:46] inflatador: ansible time? [14:56:27] I thought Riccardo was writing a new configuration management systems for us. Oregano. [14:56:42] ;P [14:58:52] containerization time [14:59:48] <_joe_> ^^ [15:00:10] marostegui I hope so. Ansible can do everything puppet, spicerack, cumin etc can do and its popularity is somewhere up there with docker and k8s. Surely that merit at least an evaluation? [15:05:43] * kamila_ actually prefers puppet to ansible and thinks it's not just stockholm syndrome (but that's what someone with stockholm syndrome would say) [15:07:44] I hope the Puppet fork works out, but I don't see much future for config mgmt-only software [15:07:57] * Emperor steps away from the can of worms [15:08:33] gta_herewegoagain.gif [15:09:09] Is Ansible not "config mgmt-only software" ? [15:09:45] it's more of a task runner than it is config management [15:09:45] it's "config management but actually running scripts and not a dsl, so kitchen sink it is" [15:10:27] yeah I've worked with it before, both of those descriptions are correct [15:10:28] Nah, it can ad-hoc cmds like cumin, or operations stuff like spicerack. Like you could do `ansible my_hosts -m systemd -a "name=nginx state=restarted` [15:10:43] and yeah, it's a blunt instrument compared to puppet [15:11:13] puppet is a series of mistakes bolted onto ruby, and ansible is a series of mistakes bolted onto ssh [15:11:29] x) [15:11:39] ^^^^^ [15:11:41] I am a big fan of ansible right up until scaling to the point that choosing it was an utter catastrophe [15:11:44] https://bash.toolforge.org/quip/AVfTAUmefIH_7EDsriqu [15:12:11] Main reason I'm such a pain in the ass about this is that I've spent many years supporting weird, also-ran software (Openstack with Xenserver computer, OpenVZ-based DBaaS) etc [15:12:18] doing any kind of data manipulation or computation in Ansible DSL I found to be a massive pain when I used it [15:12:22] bash cdanis quote! [15:12:42] Plus I know what happens when the private equity draculas take over your nice open-source company [15:13:02] !bash puppet is a series of mistakes bolted onto ruby, and ansible is a series of mistakes bolted onto ssh [15:13:05] kamila_: Stored quip at https://bash.toolforge.org/quip/9q7svJQBvg159pQrNwZA [15:14:33] I mean, it sucks, I 100% agree...it's just kinda the only thing out there [15:14:44] Oh come on that's not true [15:14:46] There's chef [15:14:48] * claime runs away [15:15:14] general-purpose computing was actually a bad idea, as good as it has been for my employment [15:15:34] yeah let's make a mediawiki asic [15:15:54] cdanis: two for one, that's another one for bash [15:16:06] soon you will be doing a hat-trick [15:16:34] Coming down from the trees was a bad idea ;P [15:17:24] "In the beginning the Universe was created. This had made many people very angry and has been widely regarded as a bad move." -- Douglas Adams [15:17:46] :D [15:18:57] some would say agriculture [15:19:19] Becomnig bipeds/ [15:19:36] fix fingers pls [15:22:47] * Emperor reaches for their surgical kit [15:28:54] <_joe_> If we consider this an inflection point for puppet and we decided "sweet, let's replace it with another configuration management system" instead of taking the chance to get freed from thinking about individual hosts and/or containerize everything, we would be really foolish [15:29:44] +100 [15:29:57] <_joe_> in addition, I've used ansible and IMHO comparing it to cumin/spicerack *for the specific work spicerack and cumin do* is like saying both a Citroen Ami and a Porsche 911 are both cars. [15:30:28] <_joe_> and it's nto a slight on ansible, it has a very different use case from the two tools we've created. [15:31:00] SaltStack to the rescue! (who said Stockholm syndrome before?) [15:33:02] <_joe_> fabfur: we used salt in the past [15:33:08] <_joe_> I will censor myself. [15:33:21] Now hypertension made you stop? [15:48:32] fabfur: don't scratch old wounds [15:48:48] I had these wounds too! :) [15:50:50] my friend used to work for saltstack. I wouldn't wish that on anyone ;) [15:52:04] now works for vmware? [15:52:30] No, left that sinking ship awhile ago (thankfully) [15:52:33] ouch [16:03:34] As far as containerization, that is 100% the right move. But we have software that runs as root on every single host and executes remote code thousands of times per day. The vendor said they're going to slow-walk (or stop) providing updates. [16:04:41] remote code? [16:04:42] <_joe_> ok. [16:12:44] I think openvox will be fine short term, long term I worry if the small community of developers will be able to support such a large codebase [16:16:16] I hope so. Again, sorry I am such a PiTA about this. Years dealing w/ppl who thought nothing of owning/maintaining an Openstack fork in perpetuity so they wouldn't have to migrate off XenServer. You can imagine how that turned out ;( [16:18:02] I think it is a good question that should inform our longer term strategy around config management for hosts. [16:21:58] inflatador: I think there's a lot of awareness of the problem, and it's not like Puppet is beloved. I think the thing to do is to start making steps towards k8s or other containerized stuff as we are able. [16:34:04] cdanis 100%. I'm sad because I love Nomad, but the k8s ecosystem is the future [16:59:39] Turns out, there's a Floppotron3 now... https://www.youtube.com/watch?v=AtRVTYMA87k [17:52:42] using sretest1001 for some testing, holler if need it