[07:29:12] 10DBA, 10Operations, 10ops-eqiad: db1100 crashed - https://phabricator.wikimedia.org/T175973#3609605 (10jcrespo) [07:49:36] 10DBA, 10Operations: Lost access to x1-analytics-slave - https://phabricator.wikimedia.org/T175970#3609786 (10jcrespo) [07:53:31] 10DBA, 10Operations: Lost access to x1-analytics-slave - https://phabricator.wikimedia.org/T175970#3609796 (10elukey) After a chat with @jcrespo we decided to point the x1-analytics-slave domain to dbstore1002 (analytics-slave) - https://gerrit.wikimedia.org/r/#/c/378211 db1029 is a production host and it wou... [08:07:58] 10DBA, 10Operations, 10ops-eqiad: db1100 crashed - https://phabricator.wikimedia.org/T175973#3609801 (10jcrespo) @Cmjohnson @robh I assume there is not much left to do here at dc/provider level except keeping a record of the crash and complain if it repeats? This is one of the latest models bought. [08:13:09] 10DBA, 10Operations, 10ops-eqiad: db1100 crashed - https://phabricator.wikimedia.org/T175973#3609810 (10jcrespo) However, it is a lot of coincidence that it crashes just hours after peing pooling and having some load: https://gerrit.wikimedia.org/r/378003 (it has been idle for weeks before). I would like to... [08:16:09] 10DBA, 10Operations, 10ops-eqiad: db1100 crashed - https://phabricator.wikimedia.org/T175973#3609813 (10jcrespo) p:05Triage>03Low Low after being depooled. [08:41:55] 10DBA, 10Community-Tech-Sprint: Issue with maintenance script: SELECTing revisions with high rev_id is painfully slow - https://phabricator.wikimedia.org/T175962#3609201 (10IKhitron) I'm not a developer, so my question can be stupid, but still: you need to collect the revisions for particular IP ranges. Why di... [09:07:36] 10DBA, 10Community-Tech-Sprint: Issue with maintenance script: SELECTing revisions with high rev_id is painfully slow - https://phabricator.wikimedia.org/T175962#3609201 (10jcrespo) @IKhitron revision_userindex is a made up view that it is cloud wikireplicas only- production has very similar content, but has d... [09:13:19] 10DBA, 10Community-Tech-Sprint: Issue with maintenance script: SELECTing revisions with high rev_id is painfully slow - https://phabricator.wikimedia.org/T175962#3609898 (10IKhitron) Didn't understand almost a word. But if what you said was that revision_userindex has not all the needed data, you can get at le... [10:02:51] 10DBA, 10Community-Tech-Sprint: Issue with maintenance script: SELECTing revisions with high rev_id is painfully slow - https://phabricator.wikimedia.org/T175962#3610029 (10jcrespo) @MusikAnimal So this is a couple of things- MySQL here may be confusing, but it is not the one at fault (it is executing the quer... [11:17:48] 10DBA: Global rename for RadioFan - https://phabricator.wikimedia.org/T175946#3608784 (10jcrespo) Which wikis was he mainly editing? We have some ongoing performance issues on wikidata and commons, and I would like to delay it if it impacts those. [16:41:17] 10DBA, 10Operations, 10Phabricator, 10ops-eqiad: Decommission db1048 (was Move m3 slave to db1059) - https://phabricator.wikimedia.org/T175679#3610984 (10mmodell) @jcrespo: ok whenever works for you I'll try to be available. [17:45:36] 10DBA: Global rename for RadioFan - https://phabricator.wikimedia.org/T175946#3611110 (10Ajraddatz) It's all on enwiki. [18:20:58] 10DBA: Global rename for RadioFan - https://phabricator.wikimedia.org/T175946#3611194 (10jcrespo) If it is enwiki, then this can go at any time someone is around in case of a problem. I propose you 15h UTC on Monday as admins and developers of both Europe and Americas should be around (and me in particular), but... [18:24:40] 10DBA, 10Operations: Lost access to x1-analytics-slave - https://phabricator.wikimedia.org/T175970#3611209 (10Etonkovidova) @elukey Thanks! I can access I need an access to `echo_` tables and I do not see them replicated there (or the acccess is not allowed?). [18:32:41] 10DBA, 10Operations: Lost access to x1-analytics-slave - https://phabricator.wikimedia.org/T175970#3611225 (10jcrespo) @Etonkovidova Can you provide more information about how you plan to use that data? It was not initially included on analytics data because it was both more difficult technologically and not n... [19:21:22] 10DBA, 10Operations: Lost access to x1-analytics-slave - https://phabricator.wikimedia.org/T175970#3611260 (10Etonkovidova) @jcrespo Echo notifications is a project that Global Collaboration team is responsible for (I am a QA for that team). From time to time, following users' complaints or requests, or our ow... [20:45:19] 10DBA: Global rename for RadioFan - https://phabricator.wikimedia.org/T175946#3611320 (10Ajraddatz) I'll probably be asleep then! But I'll be around sometime between 1600 and 1700 UTC on Monday, and I'll go on IRC and ask in -ops when I am on.