[01:40:00] 10DBA, 06Labs, 10Labs-Infrastructure, 06Operations: Enable access to Wikipedia Tulu (tcywiki) on labs replicas - https://phabricator.wikimedia.org/T142223#2546632 (10AlexMonk-WMF) Note that this can be done by any member of the 'ops' group in puppet, it does not need to wait for my maintain-replicas rewrit... [07:14:09] 10DBA, 06Operations, 06Performance-Team, 10Traffic, and 2 others: Apache <=> mariadb SSL/TLS for cross-datacenter writes - https://phabricator.wikimedia.org/T134809#2546887 (10jcrespo) [07:28:13] 10DBA, 06Labs, 10Labs-Infrastructure: labsdb* has no automatic failover solution - https://phabricator.wikimedia.org/T141097#2546914 (10jcrespo) [07:28:16] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Labs databases rearchitecture (tracking) - https://phabricator.wikimedia.org/T140788#2546915 (10jcrespo) [07:28:19] 10DBA, 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Setup and provision labsdb1009, labsdb1010 and labsdb1011 - https://phabricator.wikimedia.org/T140452#2546913 (10jcrespo) [07:31:35] 10DBA, 06Labs, 10Labs-Infrastructure: Decommission labsdb1001 and labsdb1003 - https://phabricator.wikimedia.org/T142807#2546917 (10jcrespo) [07:31:55] 10DBA, 06Labs, 10Labs-Infrastructure: Decommission labsdb1001 and labsdb1003 - https://phabricator.wikimedia.org/T142807#2546932 (10jcrespo) [07:31:58] 10DBA, 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Setup and provision labsdb1009, labsdb1010 and labsdb1011 - https://phabricator.wikimedia.org/T140452#2546933 (10jcrespo) [07:33:51] 10DBA, 06Labs, 10Labs-Infrastructure: labsdb* has no automatic failover solution - https://phabricator.wikimedia.org/T141097#2546936 (10jcrespo) @coren, We cannot wait more on this; @mark has specifically asked me to unblock this so we can go on with T142807. [07:35:45] 07Blocked-on-schema-change, 10DBA, 10ArchCom-RfC, 10Wikimedia-Site-requests, 06Wikisource: Schema change for page content language - https://phabricator.wikimedia.org/T69223#2546940 (10Nemo_bis) > if it is on core, it has to be applied to all wikis This appears to be a novel policy, and as such should b... [08:22:10] 07Blocked-on-schema-change, 10DBA, 10ArchCom-RfC, 10Wikimedia-Site-requests, 06Wikisource: Schema change for page content language - https://phabricator.wikimedia.org/T69223#2547054 (10jcrespo) @Nemo_bis check this page: https://wikitech.wikimedia.org/wiki/Deployments Do you see when it says group0 "1.... [08:38:14] 10DBA, 06Labs, 10Tool-Labs: Replication seems to be halted for multiple databases - https://phabricator.wikimedia.org/T142310#2547071 (10jcrespo) 05Open>03Resolved a:03jcrespo We identified the tool that was probably causing the memory issues- there was a single tool that was not latency-critical; taki... [08:45:28] 10DBA, 10Architecture, 06Operations: Architecture decision to solve the need larger serves (for better capacity and consolidation) vs. more, smaller servers (for high availability) - https://phabricator.wikimedia.org/T124681#2547076 (10jcrespo) 05Open>03Resolved a:03jcrespo I think naturally this has l... [08:46:23] 10DBA, 06Labs, 10Labs-Infrastructure: Decommission labsdb1001 and labsdb1003 - https://phabricator.wikimedia.org/T142807#2547080 (10jcrespo) p:05Triage>03High [08:59:33] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#604724 (10TTO) I'd suggest also to delete the AFT log entries from the `logging` table, as they don't display properly on the wikis, they no longer convey any useful information, and all the data... [09:03:11] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547121 (10jcrespo) TTO please do, although that is more of a "content handling" issue, not a schema schange, so that can be done by anyone with a maintenance script. With this I mean it is less... [09:48:30] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547234 (10TTO) >>! In T59185#2547121, @jcrespo wrote: > TTO please do, although that is more of a "content handling" issue, not a schema schange, so that can be done by anyone with a maintenance... [09:53:09] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547239 (10jcrespo) Will deleting this tables has to be blocked on that, or can I proceed as intended? [09:58:54] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547266 (10TTO) >>! In T59185#2547239, @jcrespo wrote: > Will deleting these tables has to be blocked on that, or can I proceed as intended? No, not blocked. Go ahead :) [10:04:54] 10DBA, 10MediaWiki-extensions-WikimediaMessages: Expunge old AFTv5 log entries on WMF wikis - https://phabricator.wikimedia.org/T115303#2547270 (10jcrespo) Happy to help, specially if you need a handy backup first. [10:47:35] 10DBA, 06Labs, 10Labs-Infrastructure: labsdb* has no automatic failover solution - https://phabricator.wikimedia.org/T141097#2547381 (10mark) @jcrespo, that's not what I said. :) It sounds like this is not an easy decision that we can make without consequences and without gathering more information first. I... [12:01:21] 10DBA, 06Operations: script & docs to rename wiki databases - https://phabricator.wikimedia.org/T83609#2547574 (10Krenair) [12:01:24] 10DBA, 06Operations: script & docs to rename wiki databases - https://phabricator.wikimedia.org/T83609#916322 (10Krenair) Thanks. [13:05:15] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547747 (10jcrespo) This already had a very visible impact: https://grafana.wikimedia.org/dashboard/db/server-board?panelId=17&fullscreen&from=1470991083476&to=1471006652380&var-server=db1057&va... [13:16:16] 10DBA, 07Tracking: Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking) - https://phabricator.wikimedia.org/T54921#2547770 (10jcrespo) [13:16:18] 10DBA: Archive and drop all article feedback related tables from all wikis - https://phabricator.wikimedia.org/T59185#2547767 (10jcrespo) 05Open>03Resolved a:03jcrespo I have dropped the following ones: ``` | aft_article_answer | | aft_article_answer_text | |... [13:22:05] 10DBA, 07Tracking: Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking) - https://phabricator.wikimedia.org/T54921#2547780 (10jcrespo) [13:51:01] 10DBA, 07Tracking: Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking) - https://phabricator.wikimedia.org/T54921#2547858 (10jcrespo) To clarify my previous comment, if mediawiki code has it, but we use a different storage method/config, those tables are not unused (because the... [13:58:11] <_joe_> jynus: can we extend the grant for the puppet database to nitrogen temporarily? [13:58:26] <_joe_> nevermind, scratch that [13:58:28] temporarrily? [13:58:45] <_joe_> sorry, no need in fact [13:58:57] because of the proxy, it should not be needed [13:59:12] although if it is permanenet, it should be puppetized