[05:20:06] 10DBA, 10Data-Services, 10Security-Team: gblrename log_type missing on replicas - https://phabricator.wikimedia.org/T178752#3702504 (10bd808) The `gblrename` type was not in [[https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/role/templates/labs/db/views/maintain-views.yaml... [06:08:01] 10DBA, 10Cloud-Services: Multiple concurrent long running queries from s51434 overloading labsdb1003/labsdb1009 - https://phabricator.wikimedia.org/T133705#3702537 (10Marostegui) These are no longer on the processlist - I guess they finished if they were not killed by anyone (not me) [07:59:33] 10DBA, 10Cloud-Services: Multiple concurrent long running queries from s51434 overloading labsdb1003/labsdb1009 - https://phabricator.wikimedia.org/T133705#3702735 (10Magnus) 05Open>03Resolved [08:07:54] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3702739 (10Marostegui) a:03Marostegui [08:13:54] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3702744 (10Marostegui) I have renamed this table on the following servers s5: wikidatawiki: db1092 ``` root@db1092[wi... [08:15:08] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3664286 (10Marostegui) [10:04:52] 10DBA, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Chinese-Sites, 10Patch-For-Review: Enable description usage tracking on further wikis - https://phabricator.wikimedia.org/T178515#3702928 (10hoo) I just moved the deploy from 13:00–14:00 UTC today to 12:00–13:00 UTC, as that fits my schedule bet... [10:05:23] 10DBA, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Chinese-Sites, 10Patch-For-Review: Enable description usage tracking on further wikis - https://phabricator.wikimedia.org/T178515#3702929 (10Marostegui) Sounds good, thanks for the heads up! [12:18:46] 10DBA, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Chinese-Sites: Enable description usage tracking on further wikis - https://phabricator.wikimedia.org/T178515#3703074 (10hoo) 05Open>03Resolved This has been deployed. Yet there are no description usages :/ [13:29:05] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: db1082 storage crashed - https://phabricator.wikimedia.org/T178460#3693095 (10Cmjohnson) @marostegui the HP tech will be at the data center today to swap the controller. Is the server depooled? [13:52:29] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: db1082 storage crashed - https://phabricator.wikimedia.org/T178460#3703239 (10Marostegui) @Cmjohnson db1082 is now off, feel free to power it off once the replacement has been done. Thank you! [14:59:19] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3703411 (10Marostegui) [15:50:17] hello! BBU battery state unknown for db1046 [15:50:33] hey [15:50:44] where did you see that? [15:51:07] https://phabricator.wikimedia.org/T166141 you might want to try that [15:51:39] actually [15:51:41] to be more specific [15:51:49] I wanted to try sudo megacli -AdpBbuCmd -BbuLearn -aALL [15:51:53] yeah [15:51:54] that is it [15:51:57] super :) [15:52:07] you might need to run it twice [15:52:08] so [15:52:12] run it, leave it like 30-1h [15:52:19] and if it doesn't work, let's do it again [15:52:27] ahahahah [15:52:28] okok [15:52:46] yeah…those BBUs... [15:53:01] marostegui: do you have some time this week to work on the puppet change to refactor the analytics replica/slave code? [15:53:15] maybe it would be good to downtime the host till tomorrow and add that task number as a comment? [15:53:22] elukey: i will try to make time for it :) [15:53:52] all right let me know if you'll have one, I don't want to harass you this week :) [15:57:04] I am sure volans will also be happy to give you a -1 [15:57:05] XDDD [15:57:44] rotfl [15:59:51] It is not funny anymore for Riccardo to -1 me, too easy :P [16:00:58] maybe a -2 this time? [17:03:39] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: db1082 storage crashed - https://phabricator.wikimedia.org/T178460#3703916 (10Cmjohnson) The controller has been replaced and the server has been powered on. @marostegui please resolve task when you're comfortable with the new controller. [18:31:19] 10DBA, 10Operations, 10ops-eqiad, 10Patch-For-Review: db1082 storage crashed - https://phabricator.wikimedia.org/T178460#3704198 (10Marostegui) Thanks @Cmjohnson - I have started MySQL and will leave it running over night. If all goes well, I will close this ticket tomorrow. The crashes are not so frequen...