[08:00:51] db2199 finally finished rebuilding, restarting replication [09:16:25] FIRING: SystemdUnitFailed: swift_ring_manager.service on ms-fe2009:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:21:37] [that's a swift-dispersion-report failure ; on re-run s-d-r completes OK] [10:16:25] RESOLVED: SystemdUnitFailed: swift_ring_manager.service on ms-fe2009:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [10:43:12] marostegui: re:T386242 all backup sources are upgraded and rebuild, I can give you a list if you want [10:43:12] T386242: Upgrade and rebuild s6 - https://phabricator.wikimedia.org/T386242 [10:43:27] jynus: ah excellent, no worries I will mark them [10:43:28] thank you! [10:44:36] marostegui: https://phabricator.wikimedia.org/P73657 [10:44:48] thanks! [10:47:00] same for backup1-* dbs, they are all upgraded [10:56:54] Regarding T385645#10582106 , to clarify, this is a question, not even a suggestion. [10:56:54] T385645: Drop event_variant column from echo_event - https://phabricator.wikimedia.org/T385645 [14:54:52] volans: where can I import the type returned by dbctl.instance.get from? Also how do I get the fqdn or the Instance and RemoteHosts from it? [15:02:42] FQDN: str(instance.host.hosts) returns the string representation of the NodeSet, given that in the case of Instance you have just one host it returns the FQDN of that host [15:05:00] for the first question: from conftool.extensions.dbconfig.entities import Instance [15:05:03] federico3: ^^^ [15:06:33] the object describes itself as conftool.loader.Dbconfig_instance [15:07:26] do we have 2 different types named Instance in conftool.extensions.dbconfig.entities VS spicerack.mysql ? [15:08:25] conftool.loader.Dbconfig_instance is due to the way that conftool loads the entities, for more details you have to ask its maintainers ;) [15:09:38] yes, Instance it's quite a common name, no surprises it's used in multiple project/repos [15:23:26] err... [15:24:06] I have a dbconfig "Instance" and I need to get a spicerack.mysql.Instance [15:27:35] spicerack.mysql().get_dbs(f"{dbctl_instance.name}.*").list_hosts_instances()[0] # limited to single instances [15:30:01] ok [16:47:35] hi data-persistence folks, as part of the ongoing effort to migrate to liberica we need to switch all low-traffic services to IPIP and maglev. It's time to migrate apus. I've submitted the CRs to gerrit https://gerrit.wikimedia.org/r/q/topic:%22T387290%22 (T387290), It would be great if somebody could review the CRs and coordinate with me to migrate apus. Thanks <3 [16:47:36] T387290: Migrate apus LB VIPs to IPIP encapsulation - https://phabricator.wikimedia.org/T387290 [17:26:24] sigh, another "swift did what you told it" ticket T387340 [17:26:25] T387340: Unable to restore File:Blason_famille_fr_de-Lichy_(2).svg - https://phabricator.wikimedia.org/T387340 [17:30:41] Emperor: do you have the list of tickets like this handy somewhere? [17:36:15] https://phabricator.wikimedia.org/project/board/131/ [17:36:20] Backlog size: 558 tickets [17:39:02] I can go back through my own log and note the ones I've wasted^Wspent time on; I sometimes think about adding a "not swift's fault" column to the workboard, but that feels passive-aggressive, even for me :)