[07:44:41] 10DBA, 10Cloud-Services, 10Patch-For-Review, 10User-Urbanecm: Prepare and check storage layer for id_internalwikimedia - https://phabricator.wikimedia.org/T196748#4269866 (10Urbanecm) [07:45:35] 10DBA, 10Cloud-Services, 10Patch-For-Review, 10User-Urbanecm: Prepare and check storage layer for id_internalwikimedia - https://phabricator.wikimedia.org/T196748#4267274 (10Urbanecm) Finally, they decided to use id-internal.wikimedia.org, so @Marostegui, when amending the filters, please use this new dbna... [09:14:35] 10DBA, 10Cloud-Services, 10Patch-For-Review, 10User-Urbanecm: Prepare and check storage layer for id_internalwikimedia - https://phabricator.wikimedia.org/T196748#4269919 (10Marostegui) Thanks for the heads up. I will create the filter for `id_internalwikimedia` and restart all sanitarium on Monday. [09:28:22] 10DBA, 10MediaWiki-extensions-UserMerge: UserMerge using SELECT img_name FROM image WHERE img_user = ... but img_user doesn't have an index - https://phabricator.wikimedia.org/T105395#4269925 (10Marostegui) 05Open>03Resolved Agreed with Jaime. Closing this for now. It has been a while since this was opene... [10:42:09] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269936 (10Marostegui) [10:45:28] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269948 (10Marostegui) p:05Triage>03High [10:45:35] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269949 (10greg) Related: The Gerrit upgrade included a migration that created many new git refs. Those are replicated to Phabricator and thus it also had to ingest/index them. [10:46:22] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269950 (10Marostegui) Ah right! I'm from my phone and cannot check what the writes are. Any ETA for that to be finished? [10:47:06] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269951 (10greg) Not sure, @mmodell ? @demon ? [10:53:04] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269954 (10Marostegui) Codfw is lagging behind as it cannot cope with the amount of writes. Not a big deal as it is not used, but it is an indicative of how massive it is. It w... [11:04:37] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269955 (10Paladox) It will be a long while as phab has to parse all the new commits (notedb) we should probaly try to ignore refs/changes/**/meta in phabricator. [11:17:47] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269958 (10Marostegui) >>! In T196840#4269955, @Paladox wrote: > It will be a long while as phab has to parse all the new commits (notedb) we should probaly try to ignore refs/... [11:18:57] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4269961 (10Paladox) Yep days. [11:50:25] 10DBA, 10Wiki-Loves-Monuments-Database: mysqldump is timing out preventing all tables from being included in the dump - https://phabricator.wikimedia.org/T138517#4269990 (10Multichill) 05Resolved>03Open tools.heritage@tools-bastion-02:~/logs$ date zo jun 10 11:44:09 UTC 2018 tools.heritage@tools-bastion-02... [12:06:06] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4270016 (10demon) We haven't needed to replicate any refs other than heads and tags since we brought gitiles online... Disable them. Now. And prune them from Phab while we're... [12:47:14] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4270059 (10Paladox) But phabricator changed it's behaviour and now clones refs/**. So to fix this we need regex to not clone refs/changes/**/meta. [21:58:25] 10DBA, 10Gerrit, 10Operations, 10Phabricator: Massive increase of writes in m3 section - https://phabricator.wikimedia.org/T196840#4270673 (10Paladox)