[05:26:40] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s4 - https://phabricator.wikimedia.org/T166206#3322089 (10Marostegui) db1053 is done: ``` root@neodymium:/home/marostegui# for i in `cat s4_tables`; do echo $i; mysql --skip-ssl -hdb1053 commo... [05:27:00] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Convert unique keys into primary keys for some wiki tables on s4 - https://phabricator.wikimedia.org/T166206#3322090 (10Marostegui) [05:53:52] 10DBA: Convert unique keys into primary keys for some wiki tables on s2 - https://phabricator.wikimedia.org/T166205#3322144 (10Marostegui) codfw master is done and replicated downstream [09:49:14] I saw some phabs about trusty hosts not being able to be reimaged into jessie? what's the symptom? [09:49:28] I'm asking because ms-be2014 isn't even trying to pxeboot, despite my instructions [09:49:49] https://phabricator.wikimedia.org/T166683 -> this? [09:50:09] yeah [09:50:19] doesn't seem the same symptoms though [09:50:34] https://phabricator.wikimedia.org/T166683#3310820 [09:50:35] check that [09:50:56] j*nus saw that host installed correctly without the upgrade [09:50:59] so... [09:51:36] ugh, ok thanks marostegui I'll try poking at it some more [09:52:22] let us know if you find something [09:56:03] heh I see the dhcp/tftp requests on install2002 but then nothing happens and falls back to hdd boot [09:57:59] ah becase I'm stupid, that's why [09:59:21] nevermind, PEBCAK [11:04:31] 10DBA, 10Operations, 10Wikimedia-Site-requests: Rename user "Mlpearc" to "FlightTime" on Central Auth - https://phabricator.wikimedia.org/T166028#3322892 (10MarcoAurelio) [11:06:05] 10DBA, 10Operations, 10Wikimedia-Site-requests: Rename user "Mlpearc" to "FlightTime" on Central Auth - https://phabricator.wikimedia.org/T166028#3283276 (10MarcoAurelio) @jcrespo and @Marostegui for your opinion. [12:06:07] 10DBA, 10Cognate, 10Wikidata, 10User-Addshore: Cognate DB review - https://phabricator.wikimedia.org/T148988#3323227 (10Tobi_WMDE_SW) [12:26:44] 10DBA, 10Operations, 10Wikimedia-Site-requests: Rename user "Mlpearc" to "FlightTime" on Central Auth - https://phabricator.wikimedia.org/T166028#3323325 (10Marostegui) @MarcoAurelio thanks for the heads up!. We will make sure we have no schema maintenance scheduled for when you decide to run it. Ping any of... [12:46:54] 10DBA, 10Operations, 10Wikimedia-Site-requests: Rename user "Mlpearc" to "FlightTime" on Central Auth - https://phabricator.wikimedia.org/T166028#3323381 (10jcrespo) Same recommendations apply than T167031#3315249 plus the additional of not making both at the same time. [15:08:48] 10DBA: DROP OAI-related tables - https://phabricator.wikimedia.org/T139342#3324155 (10Marostegui) [15:55:06] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: x1 master db1031: Faulty BBU - https://phabricator.wikimedia.org/T166108#3324426 (10jcrespo) {F8399947} [17:23:44] Is the labsdb replag for s4 (Commons) a known issue? It is up to 59h [17:24:14] maybe [17:24:32] check sal, maybe manuel has a running alter [17:24:35] where did you see it? [17:24:44] https://tools.wmflabs.org/replag/ [17:25:28] looks like there are some alters happening in SAL -- https://tools.wmflabs.org/sal/production?p=0&q=s4&d= [17:25:43] 06:20 Deploy alter table s4 - on labsdb1001 - T166206 [17:25:44] T166206: Convert unique keys into primary keys for some wiki tables on s4 - https://phabricator.wikimedia.org/T166206 [17:25:52] that is probably it [17:27:27] *nod* looks like s4 fixes have been popular recently. :) [17:27:30] "alter table pagelinks add primary key" [17:27:33] is running [17:27:46] in an ideal world, we depool that server during maintenance [17:27:47] I let the floks in #wikimedia-cloud who were asking know [17:27:50] that is not possible [17:27:52] *folks [17:27:54] on labsdb1001 [17:28:03] but it will be the normal thing on labsdb-web [17:28:09] (new labs) [17:28:22] people can use for now labsdb1003 [17:29:28] I think I've seen your instructions on that but I don't remember where they are [17:30:28] there is lot of maintenance coming [17:30:30] BTW [17:30:34] to mediawiki [17:30:47] and I think that will break many tools [17:30:54] did you saw the ticket? [17:31:58] Ah, you did: https://phabricator.wikimedia.org/T166732#3306771 https://phabricator.wikimedia.org/T166798 [17:33:42] yeah. its going to melt the world for a while. [17:49:31] would it be doable to turn on query logging for a while to audit T166798 in labs? [17:49:31] T166798: Decide whether back-compat views for upcoming major schema changes will be provided in the Labs replicas - https://phabricator.wikimedia.org/T166798 [17:51:21] that was a drive by comment -- I have to go now! [18:43:08] query logging? [18:43:31] that is normally quite impossible, specially on labs due to the traffic [18:43:39] what do you need? [18:43:54] if you need to know how tables are used, we already have that [18:44:11] on the performance_schema [19:18:08] 10DBA, 10MediaWiki-Change-tagging, 10Schema-change: change_tag table needs redesign - https://phabricator.wikimedia.org/T164167#3224201 (10Catrope) >>! In T164167#3224664, @TTO wrote: >> Getting statistics of hitcounts is impossible on big tables. > https://gerrit.wikimedia.org/r/334337/ tries to improve thi... [20:23:19] 10DBA, 10Operations, 10Wikimedia-Site-requests: Rename user "Mlpearc" to "FlightTime" on Central Auth - https://phabricator.wikimedia.org/T166028#3330283 (10MarcoAurelio) I have time to do it now if it is a good moment @jcrespo / @Marostegui.