[05:49:34] Starting x3 switchover [05:49:43] x1 codfw done [06:05:20] x3 switchover done [06:06:08] cezmunsta: nice!!! thank you [08:58:55] <_joe_> marostegui: I need to create a table on the hiddenparma db; can I self-serve? [08:59:13] _joe_: sure, you can connect from cumin - if you need help ping me [08:59:44] <_joe_> np [09:03:29] out of curiosity, where does that db live? and does it have backups? [09:11:05] I am answering myself, I guess that's requestctl and it is backed up already [09:14:11] actually, it doesn't have backups [09:15:15] jynus: https://phabricator.wikimedia.org/T411111 [09:16:40] as far as I can see I was not pinged there in any way [09:16:56] and I was on vacations at that time [09:19:12] I will add it and do a full misc audit [09:27:59] this is the patch: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1297634 [09:46:11] deploying new grants to m2 backup sources [10:07:55] es* read only backups are taking less time than expected codfw->eqiad only 8 hours and eqiad->codfw only 18h [11:55:01] FYI DBAs: https://wikitech.wikimedia.org/w/index.php?title=MariaDB%2Fmisc&diff=2423655&oldid=2367783 [11:58:42] also https://wikitech.wikimedia.org/w/index.php?title=MariaDB%2Fmisc&diff=2423657&oldid=2423656 [11:59:04] I don't expect my edits to be kept as is, feel free to polish, refine, etc. [11:59:40] ack :D [12:27:38] "Please click all letters matching the specified number" ... /me SEGV [12:28:14] XDDD [12:28:54] hehe [12:32:44] Let's see what AI makes of that statement :D [12:33:23] Well, it recognised that it was a CAPTCHA at least [12:37:23] It says everyting makes sense until questioned about good instructional design: "It's the kind of phrasing that emerges when a developer writes UI text based on the internal data model" :D [13:14:10] @marostegui https://phabricator.wikimedia.org/T427535 could be reopened for db1224 [13:14:38] meeting please reopen yeah [13:30:03] Hey I am back [13:30:07] Yeah, let's reopen that task [13:33:13] I did reopen it, I can chase it tomorrow but I'm not sure what else we can do on that host, all the firmwares were updated [BTW I'm in TOIL today] [13:34:16] Dont worry, I will take car eof it [13:34:23] Enjoy your time off [13:56:37] switched es3 codfw master [15:08:32] FYI to DBAs, zabe set commons to read new for file/filerevision instead of image/oldimage for a bit and reverted it (intentionally). We found a couple of issues in the couple of minutes but letting you know [16:12:11] o/ we are setting up cumin2003 with trixie, and so far puppet fails for wmfdb-admin not present in wikimedia-trixie [16:12:30] judging from the debian changelog we copied bullseye version to bookworm, so we may do the same for trixie [16:12:45] (assuming https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/wmfdb/+/refs/heads/master/debian/changelog is the right target) [16:12:45] elukey: it is being tracked at https://phabricator.wikimedia.org/T427900 [16:13:05] lovely, I'll add my comment there [16:13:36] elukey: Thanks - it will be handled by federico.3 but he's out tomorrow because of oncall comp, but I will include it in our team meeting notes for monday if that's ok [16:13:55] elukey: If you want to give it a go before monday, feel free to try! [16:14:15] marostegui: yeah that is completely fine, not urgent at all [16:15:07] thanks!