[07:29:33] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10136469 (10ABran-WMF) T374512 done. all remaining hosts are either non prod critical or depoolable [07:47:43] 10Mail, 06Infrastructure-Foundations, 07User-notice-archive: Notifications stop after bot edits until page is manually viewed or watchlist is marked as read - https://phabricator.wikimedia.org/T374404#10136488 (10Aklapper) [07:47:46] 10Mail, 06Infrastructure-Foundations, 07User-notice-archive: Stop sending change notification email if edit is done by a bot - https://phabricator.wikimedia.org/T356984#10136489 (10Aklapper) [10:33:14] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Move db2209 uplink from asw-c5-codfw to lsw1-c5-codfw - https://phabricator.wikimedia.org/T374523 (10cmooney) 03NEW p:05Triage→03Medium [10:46:32] 10netops, 06Infrastructure-Foundations, 06SRE, 13Patch-For-Review: Routed Ganeti: Add support for VM QoS marking - https://phabricator.wikimedia.org/T374392#10136850 (10cmooney) 05Open→03Resolved Patch merged, working as expected: ` cmooney@ganeti2033:~$ cat /etc/nftables/postrouting/05_trust-vm-ds... [10:47:30] 10netops, 06DBA, 06DC-Ops, 06Infrastructure-Foundations, and 2 others: Move db2209 uplink from asw-c5-codfw to lsw1-c5-codfw - https://phabricator.wikimedia.org/T374523#10136856 (10ABran-WMF) I'll get to T374425 to get to T374421 and unblock this T374523 [10:48:18] 10netops, 06DBA, 06DC-Ops, 06Infrastructure-Foundations, and 2 others: Move db2209 uplink from asw-c5-codfw to lsw1-c5-codfw - https://phabricator.wikimedia.org/T374523#10136865 (10cmooney) >>! In T374523#10136856, @ABran-WMF wrote: > I'll get to T374425 to get to T374421 and unblock this T374523 Thanks! [10:48:31] 10netops, 06DBA, 06DC-Ops, 06Infrastructure-Foundations, and 2 others: Move db2209 uplink from asw-c5-codfw to lsw1-c5-codfw - https://phabricator.wikimedia.org/T374523#10136866 (10cmooney) [10:48:32] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Migrate codfw servers in rows C & D from legacy ASW to LSW - https://phabricator.wikimedia.org/T370630#10136867 (10cmooney) [11:43:00] 10SRE-tools, 06collaboration-services, 06Infrastructure-Foundations, 10Puppet-Core, and 4 others: Migrate roles to puppet7 - https://phabricator.wikimedia.org/T349619#10137027 (10MoritzMuehlenhoff) [12:26:43] 10netops, 06collaboration-services, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: Migrate servers in codfw racks D1 & D2 from asw to lsw - https://phabricator.wikimedia.org/T373102#10137107 (10jcrespo) I will want to stop ms backups at codfw for backup2011 before it happens. No big deal if I don't... [12:26:47] 10netops, 06collaboration-services, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: Migrate servers in codfw racks D7 & D8 from asw to lsw - https://phabricator.wikimedia.org/T373105#10137114 (10jcrespo) I will want to stop ms backups at codfw for backup2007 before it happens. No big deal if I don't... [13:22:29] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Test prototype fundraising pybal replacement based on haproxy + anycast-healthchecker. - https://phabricator.wikimedia.org/T373942#10137275 (10cmooney) >>! In T373942#10134407, @Jgreen wrote: > Hey @cmooney, we have two servers online in frack-c... [13:48:39] cdanis: thanks for the patience, aux migrated to PSS :) Lemme know if you see anything weird [13:48:47] elukey: thanks for doing the work! <3 [13:49:24] 10netops, 06collaboration-services, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: Migrate servers in codfw racks C4 & C5 from asw to lsw - https://phabricator.wikimedia.org/T373097#10137421 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=5ff7d01a-40d8-4196-9008-7bf9b79ea4e8)... [14:06:29] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, and 2 others: (2) new singlemode fiber patches from dmarc to routers for IX ports - https://phabricator.wikimedia.org/T373376#10137512 (10cmooney) Still chasing Equinix to get this sorted, back-and-forth now with them for almost 2 weeks without... [14:08:46] cdanis: FYI https://github.com/jaegertracing/helm-charts/pull/600 [14:09:54] ty! [14:09:57] https://github.com/jaegertracing/helm-charts/pull/541 was mine [15:45:55] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10137884 (10ABran-WMF) depoolable hosts have been depooled https://phabricator.wikimedia.org/P68929 [16:07:44] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10137966 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=6257a49b-1ea6-4675-9944-c5d85eb38288) set by cmooney@c... [16:21:08] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10138029 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=6458a64c-9bf9-4b09-a6e1-82f1e6f72fc3) set by cmooney@c... [16:28:46] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10138056 (10cmooney) All hosts successfully moved and responding to ping again. [16:36:16] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Migrate servers in codfw racks C6 & C7 from asw to lsw - https://phabricator.wikimedia.org/T373101#10138100 (10ABran-WMF) hosts are repooling