[09:02:10] 10DBA, 10Operations, 10Patch-For-Review: correctable memory errors db1068 (commons primary master database) - https://phabricator.wikimedia.org/T213664 (10Marostegui) 05Open→03Resolved And back again: `RECOVERY - EDAC syslog messages on db1068 is OK: (C)4 ge (W)2 ge 1` As Jaime said: T213664#4924636 thi... [09:07:09] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10Patch-For-Review, 10User-Ladsgroup: Drop change_tag.ct_tag column in production - https://phabricator.wikimedia.org/T210713 (10Marostegui) [09:13:49] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10Patch-For-Review, 10User-Ladsgroup: Drop change_tag.ct_tag column in production - https://phabricator.wikimedia.org/T210713 (10Marostegui) [09:14:51] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10Patch-For-Review, 10User-Ladsgroup: Drop change_tag.ct_tag column in production - https://phabricator.wikimedia.org/T210713 (10Marostegui) s7 eqiad progress [] labsdb1011 [] labsdb1010 [] labsdb1009 [] dbstore1003 [] dbstore1002 [] db1125 [] db1116... [09:15:02] 10Blocked-on-schema-change, 10MediaWiki-Change-tagging, 10Patch-For-Review, 10User-Ladsgroup: Drop change_tag.ct_tag column in production - https://phabricator.wikimedia.org/T210713 (10Marostegui) [09:34:10] I am going to grant the nagios account grants for socket on localhost [09:34:17] for db1111 [09:34:19] cool [09:34:33] it may be the only host missing [09:34:40] well, of the core ones [09:34:45] or misc [09:34:47] yeah, maybe [09:34:49] there may be others [09:34:57] which means we could do some cleanup of accounts [09:35:04] I will write it on my TODO [09:35:58] https://phabricator.wikimedia.org/T146149#4465432 [09:36:07] in case you want to write it there [09:38:04] MariaDB read only test-s4 [09:38:11] Version 10.1.37-MariaDB, Uptime 1870006s, read_only: False, 20.29 QPS, connection latency: 0.003287s, query latency: 0.000969s [09:38:16] ^that is for db1111 [09:38:18] \o/ [09:38:25] but it won't alert, but we may notice? [09:38:51] yeah, it will show up on icinga [09:38:56] but notifications are disabled, so no noise [09:39:08] ^addshore [09:39:16] \o [09:39:47] no action needed, just FYI [09:48:09] read_only: True on db1112 is detected [10:03:30] 10DBA, 10CheckUser: Provide a strategy for testing the performance of queries needed to show the list of user-agents for each IP - https://phabricator.wikimedia.org/T212092 (10jcrespo) So for the second query, the problems are similar-- they are "GROUP BYs without WHERE*" (* or the WHERE clause is too open), w... [11:52:21] 10DBA, 10AbuseFilter, 10Stewards-and-global-tools, 10Patch-For-Review, 10User-Daimona: Addition of last hit date to Special:AbuseFilter table - https://phabricator.wikimedia.org/T93564 (10Daimona) I tried to implement the query above in IndexPager, but I cannot get it to work... I pushed a [[https://gerr... [12:02:44] 10DBA, 10AbuseFilter, 10Stewards-and-global-tools, 10Patch-For-Review, 10User-Daimona: Addition of last hit date to Special:AbuseFilter table - https://phabricator.wikimedia.org/T93564 (10jcrespo) yeah, whether mediawiki sql abstraction supports it or not is a different beast, you should ask a mediawiki... [12:23:00] well, im currently migrating the whole of wb_terms to a super normalized version on db1111 to see what it looks like [12:23:08] <3 [12:23:19] working with 1.7 billion rows is a bit of a pain :P [12:23:37] welcome to the club! [12:24:04] Think I might have normalized a tiny bit too much, might alter it slightly and try something else once this first set of migration queries actually runs [12:24:33] https://usercontent.irccloud-cdn.com/file/YxiSPBZp/image.png [12:24:48] found a pretty and free (but not open) db designer thingy too [12:25:08] which one? [12:25:19] https://app.sqldbm.com [12:25:32] love the dark theme too [12:31:19] so db1111 can be used on any way you want, but my advice would be to use it for validation/performance [12:31:49] once it works on an empty schema [12:32:02] ack! I created this schema initially with a subset of 1 million rows, testing it out, now want to throw all of the data in there and test it out some more [12:33:18] again, you can do as you want, just a suggestion [16:40:20] mariadb 10.1.38 is out apparently [16:42:37] quite some fixes to mariadbbackup https://mariadb.com/kb/en/library/mariadb-10138-changelog/ [16:45:37] they said they were going to release it with an upgraded connector [16:46:09] note many changes are for galera [16:46:44] the mariabackup changes are intersting, though [17:01:46] marostegui: you didn't retry any backup, did you? [17:02:22] (no errors this week) [17:09:33] nope [17:34:09] 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1073 - https://phabricator.wikimedia.org/T215050 (10Marostegui) 05Open→03Resolved Thanks @Cmjohnson for replacing disk #6! ` 17:31 <+icinga-wm> RECOVERY - MegaRAID on db1073 is OK: OK: optimal, 1 logical, 2 physical, WriteBack policy `