[00:10:17] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2621619 (10Dereckson) [00:11:02] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2054159 (10Dereckson) [00:12:24] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2621627 (10Dereckson) [05:55:15] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2622297 (10Marostegui) I will open a bug report to tokudb today and paste here the link to it [05:58:02] manuel, if you are not too busy, I would like you to help me with something relatively important [05:59:00] jynus: of course [06:00:12] there is this task: https://phabricator.wikimedia.org/T143874 [06:00:35] I need to figure out how much disks to buy [06:01:04] lets see [06:01:16] you should be already aware of the tokudb => innodb compression I intend [06:01:57] Yeah, and after seeing the issues yesterday, I am even more all forward forit [06:02:27] even with tokudb we are running out of disk space on dbstores (I had to migrate on backup service away, and do not want to do that) [06:02:57] so as the servers are still under warranty (and we really do not need power), I proposed to only buy disks [06:03:53] 5-10 TB * RAID level * server * datacenter of ssds are not cheap [06:04:12] so maybe you could do some calculation in parallel [06:04:40] and then discuss how we all reached to those numbers [06:04:44] sure thing [06:04:52] Any dbstore works as an example? [06:04:54] some restrictions [06:05:06] ~support 3 years of growth [06:05:08] like, which dbstore can I use to exatrapolate? [06:05:37] well, for now dbstore1001 and dbstore2001 [06:05:57] okay [06:06:07] but we have some tests on innodb compressed, as we will get different numbers [06:06:12] than regular innodb [06:06:14] and tokudb [06:06:37] so db1073 is innodb compressed for the top 10 tables [06:06:41] (enwiki only) [06:06:54] I was going to ask yeah if you had some examples of compressed vs non compressed [06:07:04] and you knew I wanted to do some more tests on s4 [06:07:11] but didn't get to it [06:07:30] yeah [06:07:32] but I was asked to do a best effort aproximation now [06:07:40] and later tune it [06:07:47] Sure [06:08:13] so I hoped that if 2 people do it, we would have less possiblity of error [06:08:17] So this would need to give us around 3 years, support, right? [06:08:38] yes [06:08:43] right [06:09:23] make your own calculations, and ping me when you have something [06:09:28] I will do the same now [06:10:19] obviouly, things like this will be very helpful: https://grafana-admin.wikimedia.org/dashboard/db/server-board?panelId=17&fullscreen [06:14:42] indeed, I was looking for graphs [06:14:59] It will take me a bit longer than you to come up with a plan, but I will ping you once I get something decent :) [08:54:58] 10Blocked-on-schema-change, 10DBA, 10ArchCom-RfC, 10Wikimedia-Site-requests, and 2 others: Schema change for page content language - https://phabricator.wikimedia.org/T69223#2622513 (10Halibutt) Hmmmm, as a code-illiterate I just read the entire discussion (having been referred here by Nikerabbit from T145... [09:43:15] 10DBA, 03Community-Tech-Sprint: Create a maintenance script for populating the local_user_id and global_user_id fields in the centralauth localuser table - https://phabricator.wikimedia.org/T142503#2622602 (10Niharika) Moving this to Ready and working on {T142507} first per Legoktm's code review: > I would sug... [12:31:13] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2622875 (10Marostegui) I have opened the bug report: https://bugs.launchpad.net/percona-server/+bug/1621852 [12:31:29] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2622876 (10Marostegui) a:03Marostegui [13:44:47] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2623005 (10Marostegui) @Jdforrester-WMF looks like ImageMetricsCorsSupport_11686678 was created yesterday. Can you check from your side... [17:33:59] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2623622 (10Jdforrester-WMF) The reference to this was dropped from the code in https://gerrit.wikimedia.org/r/#/c/230982/ which was mer... [18:18:06] 10DBA, 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 13Patch-For-Review, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2623856 (10dduvall) [18:18:11] 10DBA, 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 13Patch-For-Review, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2409864 (10dduvall) We're on for 9/15 @ 1500-1700 UTC and I've sent... [18:36:31] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2623910 (10Tgr) All four records seem to be from the same user (geolocated to US, looking at mobile jawiki, using IE 9), over the span... [18:53:19] 10DBA, 03Community-Tech-Sprint, 13Patch-For-Review, 07Schema-change: Add local_user_id and global_user_id to localuser table in centralauth database - https://phabricator.wikimedia.org/T141951#2623982 (10kaldari) I'm going to move this to Done for Community Tech (since the SQL patch is merged), but leave t... [18:53:41] 10DBA, 03Community-Tech-Sprint, 13Patch-For-Review, 07Schema-change: Add local_user_id and global_user_id to localuser table in centralauth database - https://phabricator.wikimedia.org/T141951#2517589 (10kaldari) a:05Niharika>03jcrespo [18:54:32] 10DBA, 03Community-Tech-Sprint, 13Patch-For-Review, 07Schema-change: Add local_user_id and global_user_id to localuser table in centralauth database - https://phabricator.wikimedia.org/T141951#2517589 (10kaldari) [18:54:53] 10DBA, 03Community-Tech-Sprint, 13Patch-For-Review, 07Schema-change: Add local_user_id and global_user_id fields to localuser table in centralauth database - https://phabricator.wikimedia.org/T141951#2517589 (10kaldari) [21:11:34] 10Blocked-on-schema-change, 10DBA, 10Wikimedia-Site-requests, 06Wikisource, and 2 others: Schema change for page content language - https://phabricator.wikimedia.org/T69223#2624590 (10RobLa-WMF) This was approved by #ArchCom following an IRC meeting we held a couple of weeks ago (E263) and following the su... [22:09:35] 10DBA, 10MediaWiki-Maintenance-scripts, 06Operations, 06Release-Engineering-Team, and 2 others: Add section for long-running tasks on the Deployment page (specially for database maintenance) - https://phabricator.wikimedia.org/T144661#2624661 (10greg) >>! In T144661#2618330, @jcrespo wrote: >> It's kind of... [22:22:52] 10DBA, 10MediaWiki-Revision-deletion, 13Patch-For-Review: Revision Deletion page won't load for too many (10) revisions - https://phabricator.wikimedia.org/T104313#2624667 (10Catrope) >>! In T104313#2197813, @Catrope wrote: >>>! In T104313#2196828, @Catrope wrote: >> If I add `FORCE INDEX(PRIMARY)` for the r... [22:39:59] 10DBA, 10MediaWiki-Maintenance-scripts, 06Operations, 06Release-Engineering-Team, and 2 others: Add section for long-running tasks on the Deployment page (specially for database maintenance) - https://phabricator.wikimedia.org/T144661#2624715 (10bd808) >>! In T144661#2624661, @greg wrote: > So, here's a pr...