[00:03:27] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10030299 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1248.eqiad.wmnet with OS bullseye... [00:03:45] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10030300 (10Jclark-ctr) [07:58:13] 06serviceops, 10Parsoid (Tracking), 13Patch-For-Review: Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10030731 (10akosiaris) [10:27:39] 06serviceops, 10Parsoid (Tracking), 13Patch-For-Review: Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10031156 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by akosiaris@cumin1002 for host parse1001.eqiad.wmnet with OS bullseye [10:27:47] 06serviceops, 10Parsoid (Tracking), 13Patch-For-Review: Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10031158 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by akosiaris@cumin1002 for host parse2001.codfw.wmnet with OS bullseye [11:01:44] 06serviceops, 10Parsoid (Tracking): Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10031357 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by akosiaris@cumin1002 for host parse1001.eqiad.wmnet with OS bullseye completed: - parse1001 (**PASS**) - Downtimed on Icin... [11:06:16] 06serviceops, 10Parsoid (Tracking): Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10031388 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by akosiaris@cumin1002 for host parse2001.codfw.wmnet with OS bullseye completed: - parse2001 (**PASS**) - Downtimed on Icin... [11:35:58] 06serviceops, 10Parsoid (Tracking): Cleanup parsoid-php service - https://phabricator.wikimedia.org/T359387#10031481 (10akosiaris) [11:52:32] 06serviceops, 06Infrastructure-Foundations, 13Patch-For-Review: Cleanup old Docker images running Debian Stretch/Jessie/Buster - https://phabricator.wikimedia.org/T367427#10031528 (10elukey) [11:53:02] 06serviceops, 06Infrastructure-Foundations, 13Patch-For-Review, 07Security: Upgrade K8s docker images running in Wikimedia production on Buster to either Bullseye or Bookworm - https://phabricator.wikimedia.org/T368366#10031545 (10elukey) From docker report (k8s images) set to work only with Bullseye+ imag... [12:54:53] akosiaris: we're you adding BGP for parse2001 ? [12:58:26] BGP is established to that host on both CRs in codfw now [12:59:03] thanks for the heads up on the lvs thing, why the puppetdb import script modified things so they were wrong I'll need to troubleshoot once the dust has settled on the netbox upgrade [13:11:46] topranks: both parse2001 and parse1001 [13:11:58] if you 've commited already for both, thanks! [13:12:04] otherwise I can take over one of the 2 pairs [13:12:14] akosiaris: no I only did for parse2001 in codfw - eqiad should not have been affected [13:12:28] so run homer for cr*eqiad if you've not already [13:12:43] however homer is not working right now - so bear with us I guess ? [13:12:54] or if it's urgent I can add them to the k8s bgp group for you manually [13:13:45] no urgency, I 'll just wait it out [13:13:59] and then do eqiad. Thanks! [13:14:12] ok, thanks! [13:27:35] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10031866 (10Jclark-ctr) wikikube-worker1250 1129. # 2 wikikube-worker1251 1131. # 1 wikikube-worker1252 1132. # 5 wikikube-worker1253 1133. # 4 w... [14:46:07] 06serviceops, 10Data-Platform-SRE (2024.07.29 - 2024.08.16), 13Patch-For-Review: Create a helm chart for the cloudnativepg postgresql operator - https://phabricator.wikimedia.org/T364797#10032187 (10bking) [14:46:45] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: hw troubleshooting: CPU 2 machine check error detected for rdb1014.eqiad.wmnet - https://phabricator.wikimedia.org/T370633#10032189 (10Jclark-ctr) Confirmed: Service Request 195103368 was successfully submitted. [16:50:23] 06serviceops, 06collaboration-services, 06SRE, 10Release-Engineering-Team (Radar): replace production buster deployment servers - https://phabricator.wikimedia.org/T364656#10032916 (10thcipriani) [18:23:52] 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: MVP: Privately server wiktech via mw-on-k8s - https://phabricator.wikimedia.org/T371537 (10jijiki) 03NEW [19:10:44] 06serviceops, 10LPL Technical Support, 06SRE, 10Wikimedia-Site-requests, 13Patch-For-Review: Change $wgMaxArticleSize limit from byte-based to character-based - https://phabricator.wikimedia.org/T275319#10033367 (10Fuzzy) And yet another law – [[ https://he.wikisource.org/wiki/פקודת_הרוקחים | The Israeli... [21:02:16] 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: MVP: Privately server wiktech via mw-on-k8s - https://phabricator.wikimedia.org/T371537#10033727 (10bd808) The config thing that most needs to be changed to use the multiversion images is `/etc/mediawiki/WikitechPrivateSettings.php` where a number of secre...