[08:21:24] Any reason not to start db2207 after its BIOS update on Friday? It is still depooled according to dbctl - it already ~3d or so behind in replication, so maybe seeing if it catches up will be a good test for it [08:25:14] @cezmunsta yes MariaDB can be started and maybe kept replicating for a good bit before repooling [08:27:21] @cezmunsta in general MariaDB is intentionally configured not to start by itself on a reboot because we want to do it manually after checking the host is healthy [08:28:56] It is now replicating again [08:52:29] federico3: when doing the pc/ms rolling restart, any reason that "if one.status != "todo" and two.status != "todo":" skips, rather than attending to the one that needs it? [08:55:27] there's a comment in the code: we expect to either find both host to be already done or both are todo. If they are "asymmetrical" something unexpected happened and it's safer (for now) to manually investigate why [08:56:24] in theory we could ask confirmation to continue interactively but I don't think it happens often [08:56:30] It is unclear from the MR why it was changed from "or"to "and" [08:56:57] It is the case for ms3 [08:57:25] So, what was "or" doing that wasn't desired? [08:59:02] https://gitlab.wikimedia.org/repos/data_persistence/dbtools/scripts/-/merge_requests/11/diffs#bb63fd84ea8da7025fa18b6c4334b91d4bd671e7_229_269 [08:59:54] .. and then in the scenario where it is skipped, doesn't that leave it all to be done manually anyway? [09:00:45] ah yes, it was updated. The initial version was more strict, then relaxed [09:01:24] OK, but it is still unclear why skipping is desirable given that something still needs to be done [09:01:47] because first it was better to take teh safest option [09:02:09] What is not safe?? [09:02:19] ... [09:03:31] It would seem safe to remove the one that is not todo from the list and continue to depool and manage the host that needs it [09:04:02] then once the script ran on a bunch of hosts without surprises I switched to and, so the check for "asymmetrical" condition is not there anymore [09:04:09] ... well, not quite in that order as the list is used for depooling and upgrades [09:04:16] keep in mind that we process hosts in pairs in this script [09:04:39] Yes, I have covered that in my comments [09:04:40] so usually they are processed together [09:05:13] so we wanted to start with taking 2 hosts and the time, both of them in todo, update both of them