[08:46:49] db writes are very spiky, specially on s3, since 5am [08:47:15] https://grafana.wikimedia.org/goto/QO5Gwi3HR?orgId=1 [08:55:03] I think it is some kind of import into ptwikimedia [08:57:07] or some insert into cache into ptwikinews [08:59:20] which would point to updateSpecialPages.php [08:59:59] and it would make sense to hit s3 harder, as it has more wikis [09:00:08] so probably nothing to worry [09:02:14] this is useful to see spikes in activity: https://grafana.wikimedia.org/d/GpL5R8CGz/mysql-query-rate [11:40:13] stopping the circular replication now. Fingers crossed [11:48:49] Done with no issues [11:48:55] DB maint may resume [12:02:36] 👏👏👏 [12:21:45] 🎉 [12:44:30] I will re set test-s4 read only based on comments on this channel [12:48:58] T397746 confirms it was cleaned up/not used [12:48:59] T397746: cleanup db2230 - https://phabricator.wikimedia.org/T397746 [13:05:02] apologies if already discussed, but I wonder if there is any maintenance that has to be prioritized while eqiad is depooled? [13:08:04] Amir1: ^^^ the remaining schema changes? [13:09:04] We need to switchover x3 and a couple other sections [13:09:15] I can try when I get to a laptop [13:14:39] not intending to rush people, just to help/plan if needed before 2nd October [13:18:34] It's already in my todo list. Don't worry about it. I even prioritized which sections etc. [13:21:40] remember not to overload yourself 🙏