[02:11:37] <AaronSchulz>	 marostegui: do you know how proxysql on wasat uses random enwiki servers? I saw that in a puppet commit message but can't find configuration for that.
[06:14:08] <wikibugs>	 10DBA, 10Operations, 10ops-codfw: Degraded RAID on db2044 - https://phabricator.wikimedia.org/T181779#3818983 (10Marostegui) 05Open>03Resolved All good! Thank you Papaul!  ``` root@db2044:~# hpssacli controller all show config  Smart Array P420i in Slot 0 (Embedded)    (sn: 0014380264FFFB0)      Port Nam...
[06:21:20] <marostegui>	 AaronSchulz: As far as I know it is not configured yet as it is pending some answers
[06:21:42] <marostegui>	 AaronSchulz: I would have expected the list of servers to connect to to be in the proxysql database, but it is empty
[06:22:00] <marostegui>	 So I assume it is not configured (T175672#3778177)
[06:22:00] <stashbot>	 T175672: Make apache/maintenance hosts TLS connections to mariadb work - https://phabricator.wikimedia.org/T175672
[06:23:56] <marostegui>	 mysql:root@localhost [(none)]> SELECT * FROM monitor.mysql_server_connect_log ORDER BY time_start_us DESC LIMIT 10;
[06:23:59] <marostegui>	 Empty set (0.00 sec)
[07:16:27] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3819130 (10Marostegui)
[07:22:01] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3819136 (10Marostegui) db1098:3316 and 3317 starting to get pooled.  Pending things:  ** Fully pool db1098:3316 and db1098:3317 (to be finished today 7th) *** Finished compressing tables on db1099:3311 a...
[07:35:39] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1068 - https://phabricator.wikimedia.org/T182288#3819160 (10Marostegui) p:05Triage>03High @Cmjohnson can we get this replaced as soon as we can? This is s4 master.
[08:44:45] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3819251 (10Marostegui)
[10:15:10] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3819411 (10Marostegui)
[11:32:25] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3819660 (10Marostegui)
[15:29:34] <wikibugs>	 10DBA: convert dbstore1001 to multi-instance + InnoDB compressed by importing db shards to it - https://phabricator.wikimedia.org/T159430#3820292 (10Marostegui)
[15:32:25] <wikibugs>	 10DBA, 10Analytics, 10Patch-For-Review: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1532172 (10Marostegui) Hi!  What's the status of this?
[15:34:55] <wikibugs>	 10DBA, 10Analytics, 10Patch-For-Review: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1532183 (10elukey) >>! In T108850#3820300, @Marostegui wrote: > Hi! >  > What's the status of this?  Going to be done before EOQ :)
[15:35:22] <wikibugs>	 10DBA, 10Analytics, 10Patch-For-Review: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#3820313 (10Marostegui) Awesome! Thanks! :-)
[16:01:37] <wikibugs>	 10DBA, 10Analytics, 10Patch-For-Review, 10User-Elukey: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#3820420 (10elukey)
[16:23:28] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3820561 (10Marostegui)
[16:45:26] <Amir1>	 marostegui: hey, the wb_changes tables is getting smaller in wikidatawiki
[16:45:36] <marostegui>	 is it? :-)
[16:45:45] <Amir1>	 in two days it will become several times smaller
[16:45:57] <marostegui>	 really????
[16:45:59] <marostegui>	 <3 <3
[16:46:12] <Amir1>	 yes, check the difference of the last rows comparing to old rows
[16:46:14] <marostegui>	 we have to do a schema change server per server, so we can optimize it along with it
[16:46:28] <Amir1>	 this is one of things I've been working on in the past couple of weeks
[16:46:42] <marostegui>	 actually, I will start doing it on the new s8 hosts
[16:46:47] <marostegui>	 as they are having no traffic
[16:46:52] <Amir1>	 that'd be great
[16:47:01] <marostegui>	 if I start next week, is that enough?
[16:47:08] <Amir1>	 it would be great if you start next week
[16:47:11] <marostegui>	 coolio
[16:47:15] <marostegui>	 let me create a task for it
[16:47:17] <Amir1>	 so the whole table become smaller 
[16:47:22] <Amir1>	 thanks :)
[16:47:32] <marostegui>	 it is around 500G now compressed :|
[16:47:39] <Amir1>	 Wow
[16:49:40] <marostegui>	 Oh wait!
[16:49:42] <marostegui>	 wb_changes!
[16:49:47] <marostegui>	 I thought of wb_terms!
[16:50:49] <marostegui>	 nah, it is 3.3G, I will not create a task for it, just optimize it when depooling the servers for the other schema change
[16:54:58] <Amir1>	 marostegui: okay, for wb_terms I think partitioning it would not be bad
[16:56:16] <marostegui>	 Yeah, it wouldn't be a bad idea, but we'd need to see how it would perform and all that jazz
[16:56:24] <marostegui>	 the usual checks with partitioning...
[16:56:36] <Amir1>	 specially since we don't use it for search anymore, partitioning based on term_full_entity_id will increase performance drastically 
[16:57:08] <marostegui>	 yeah, its size is getting out of hand
[23:55:13] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1068 - https://phabricator.wikimedia.org/T182288#3819156 (10Cmjohnson) disk is replaced and is rebuilding  Enclosure Device ID: 32 Slot Number: 0 Drive's position: DiskGroup: 0, Span: 0, Arm: 0 Enclosure position: 1 Device Id: 0 WWN: 5000C50005643B18 Seque...