[06:13:32] <wikibugs>	 10DBA, 10Patch-For-Review: Support multi-instance on core hosts - https://phabricator.wikimedia.org/T178359#3826938 (10Marostegui)
[07:03:45] <wikibugs>	 10DBA, 10Data-Services, 10Goal, 10Patch-For-Review, 10cloud-services-team (FY2017-18): Migrate all users to new Wiki Replica cluster and decommission old hardware - https://phabricator.wikimedia.org/T142807#3826999 (10bd808)
[07:03:48] <wikibugs>	 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Identify tools hosting databases on labsdb100[13] and notify maintainers - https://phabricator.wikimedia.org/T175096#3826997 (10bd808) 05Open>03Resolved a:03bd808
[07:06:17] <wikibugs>	 10DBA, 10Data-Services: Consider granting `CREATE TEMPORARY TABLES` to labsdbuser - https://phabricator.wikimedia.org/T179628#3827000 (10bd808) 05Open>03declined Declined per @jcrespo's explanation in T179628#3788031.
[07:06:19] <wikibugs>	 10DBA, 10Data-Services, 10Goal, 10Patch-For-Review, 10cloud-services-team (FY2017-18): Migrate all users to new Wiki Replica cluster and decommission old hardware - https://phabricator.wikimedia.org/T142807#3827002 (10bd808)
[07:23:47] <wikibugs>	 10DBA, 10Operations: Decommission db1034 - https://phabricator.wikimedia.org/T182556#3827026 (10Marostegui) p:05Triage>03Normal
[08:08:52] <wikibugs>	 10DBA: convert dbstore1001 to multi-instance + InnoDB compressed by importing db shards to it - https://phabricator.wikimedia.org/T159430#3827135 (10jcrespo) Well, as we saw with dbstore2001, all shards fit into it, but they cannot replicate. Should we go for larger servers and leave dbstore1001 as temporary dum...
[08:11:47] <wikibugs>	 10DBA: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450#3827137 (10jcrespo) I think this is a generalized problem, and we should detect it globally, get a clean up list (but on a separate ticket with lower precedence).
[08:12:56] <wikibugs>	 10DBA: convert dbstore1001 to multi-instance + InnoDB compressed by importing db shards to it - https://phabricator.wikimedia.org/T159430#3827138 (10Marostegui) >>! In T159430#3827135, @jcrespo wrote: > Well, as we saw with dbstore2001, all shards fit into it, but they cannot replicate. Should we go for larger s...
[08:16:18] <marostegui>	 can you take care of T182450? if not, I will do it myself tomorrow :-)
[08:16:18] <stashbot>	 T182450: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450
[08:16:30] <jynus>	 sure
[08:16:39] <marostegui>	 don't want to mess up with your plans though
[08:16:47] <marostegui>	 if you prefer, I can do it tomorrow
[08:16:49] <jynus>	 as soon as I finish looking at emails
[08:17:07] <jynus>	 do you want me to reimport pagelinks on dbstore1002, too?
[08:17:08] <marostegui>	 awesome!
[08:17:18] <marostegui>	 if you could, that'd be nice
[08:17:41] <marostegui>	 it broke twice during the holidays (1 for dewiki and 1 for wikidatawiki)
[08:17:47] <jynus>	 1h ongoing and only 50 mails to go
[08:17:56] <marostegui>	 hahaha
[08:18:01] <marostegui>	 good luck
[08:19:02] <marostegui>	 http://i1.wp.com/sarahwhat.com/wp-content/uploads/2015/01/giphy-2.gif?resize=600%2C327
[08:52:47] <wikibugs>	 10DBA: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450#3827169 (10jcrespo) a:03jcrespo
[09:29:44] <wikibugs>	 10DBA, 10Data-Services, 10Patch-For-Review: LabsDB infrastructure pending work - https://phabricator.wikimedia.org/T153058#3827203 (10jcrespo)
[09:29:46] <wikibugs>	 10DBA: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450#3827201 (10jcrespo) 05Open>03Resolved ``` $ du -hs sqldata* 649G    sqldata.s2 895G    sqldata.s4 539G    sqldata.s6 739G    sqldata.s7 $ df -h Filesystem             Size  Used Avail Use% Mounted...
[09:31:00] <wikibugs>	 10DBA: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450#3823554 (10Marostegui) ``` root@db1102:~# df -hT /srv Filesystem            Type  Size  Used Avail Use% Mounted on /dev/mapper/tank-data xfs   3.6T  2.8T  856G  77% /srv ```
[09:31:47] <wikibugs>	 10DBA: Clean up old binlogs from db1102 (sanitarium multi-instance) - https://phabricator.wikimedia.org/T182450#3827206 (10jcrespo) Yes, sorry, wrong grep.
[09:37:20] <wikibugs>	 10DBA: Identify and delete non-useful files on database hosts taking space unnecesarily - https://phabricator.wikimedia.org/T182563#3827227 (10jcrespo)
[09:37:27] <wikibugs>	 10DBA: Identify and delete non-useful files on database hosts taking space unnecesarily - https://phabricator.wikimedia.org/T182563#3827239 (10jcrespo) p:05Triage>03Low
[09:39:48] <jynus>	 did you file a task for the dbstore1002 issues or only phastes?
[09:40:05] <jynus>	 I don't need a task, just in case I am not finding it
[09:40:20] <marostegui>	 No no, I didn't
[09:40:24] <jynus>	 ok
[09:40:38] <jynus>	 you know phab search couldn't be trusted
[09:40:46] <marostegui>	 haha yeah
[09:41:39] <jynus>	 we should upgrade dbstore1002 soon-ish
[09:41:49] <jynus>	 it is one of the last hosts to still be on ubuntu
[09:41:55] <marostegui>	 oh and 10.0.22
[10:28:00] <jynus>	 while dbstore1002 is ongoing
[10:28:18] <jynus>	 I know there is other priorities, but I want to checkout 8.0rc
[10:32:44] <marostegui>	 sure!
[10:33:09] <marostegui>	 where will you test it?
[10:33:25] <jynus>	 I only want to compile it and test it locally
[10:33:40] <jynus>	 I do not intend to upload it as it is a release candidate
[10:34:00] <marostegui>	 yeah, not sure if you wanted to leave a slave just replicating or something
[10:34:19] <jynus>	 not now, we'll see
[10:54:09] <marostegui>	 something happened to the DBA board on phabricator or is it just me?
[10:54:15] <marostegui>	 check the "in progress" column
[10:54:35] <marostegui>	 lots of tickers got in there all of a sudden
[10:54:46] <marostegui>	 right
[13:40:32] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: Decommission old coredb machines (<=db1050) - https://phabricator.wikimedia.org/T134476#3827867 (10Marostegui)
[13:40:36] <wikibugs>	 10DBA, 10Analytics-Kanban, 10Operations, 10Patch-For-Review, 10User-Elukey: Decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3827865 (10Marostegui) 05Open>03Resolved As per our chat, closing this. Thanks for all the hard work you've put to make this happen!
[15:39:46] <marostegui>	 Did you see what I added to our etherpad regarding the goal?
[15:40:21] <jynus>	 yes
[15:40:33] <mark>	 so is the migration to codfw just to work around budget shortage?
[15:40:57] <jynus>	 no, we need that to work on dbstore in all cases
[15:41:05] <marostegui>	 mark: we are talking about a different etherpad, the one we use for ourselves
[15:41:12] <mark>	 ok
[15:41:19] <marostegui>	 just sent it to you
[15:42:33] <mark>	 ok :)
[15:42:34] <mark>	 big goal
[15:48:50] <mark>	 so besides today's meeting, I think we'll talk about it on thursday a bit also?
[15:49:00] <marostegui>	 yeah, makes sense
[15:49:18] <mark>	 cool
[16:00:05] <wikibugs>	 10DBA, 10Patch-For-Review: Checksum data on s7 - https://phabricator.wikimedia.org/T163190#3828339 (10Marostegui) p:05Triage>03Normal a:03Marostegui
[16:07:28] <wikibugs>	 10DBA, 10Structured-Data-Commons, 10Wikidata, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018): Deploy MCR storage layer - https://phabricator.wikimedia.org/T174044#3828388 (10thiemowmde) p:05Triage>03Normal
[19:15:00] <wikibugs>	 10DBA, 10Phabricator, 10Security: Improve privilege separation for phabricator's config files and mysql credentials - https://phabricator.wikimedia.org/T146055#3829050 (10mmodell) Upon further reflection, I suspect that it won't work to sandbox the repository operations to just those databases - operations o...