[01:47:01] PROBLEM - MariaDB sustained replica lag on pc2009 is CRITICAL: 2.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2009&var-port=9104 [01:54:05] RECOVERY - MariaDB sustained replica lag on pc2009 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=pc2009&var-port=9104 [03:42:30] 10DBA, 10Performance-Team, 10Wikimedia-Rdbms, 10MW-1.36-notes, 10Performance-Team-publish: Mysterious replication lag observed by MW in Codfw - https://phabricator.wikimedia.org/T248481 (10Krinkle) [03:43:08] 10DBA, 10Performance-Team, 10Wikimedia-Rdbms, 10MW-1.36-notes, 10Performance-Team-publish: Mysterious replication lag observed by MW in Codfw - https://phabricator.wikimedia.org/T248481 (10Krinkle) 05Open→03Resolved [19:14:29] beta cluster mariadb currently has the socket in /tmp/mysql.sock. is there any reason to keep it there and not on the default (/run/mysql/mysqld.sock)? [19:53:08] Majavah: nope [19:53:34] we migrated it from /tmp to /run a couple of years ago [19:54:22] ⚓ T148507 Move socket away from /tmp on mariadb, haproxy and proxysql (https://phabricator.wikimedia.org/T148507) [19:54:29] oh wow that was...5 years ago [19:54:33] time flies [19:55:25] ah but we actually finished it on 2018 so yeah, 3 years ago "only" [19:56:19] that task is private [19:59:25] marostegui: if down/read-only-time is not a concern, would changing that be as simple as changing it on the config file in puppet repo, running puppet and restarting mariadb? or is there something else to be adjusted too? I'm not seeing any uses in ops/puppet at least [20:01:58] I don't think there's anything else using it. changing the config and restarting MySQL should be it. I would suggest though, maybe avoiding doing this on a weekend, as support would be limited if something doesn't go as planned [20:02:46] yeah, not doing that on a weekend, but given I already am touching the config file in T110115 I don't see a reason not to do that [20:02:46] T110115: Possible to run writes (e.g. UPDATE) on Beta Cluster replica - https://phabricator.wikimedia.org/T110115