[01:51:13] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2618119 (10Dereckson) @jcrespo Do you prefer to create a new labswiki_flow database for... [01:53:48] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2054159 (10Catrope) >>! In T127792#2618119, @Dereckson wrote: > @jcrespo Do you prefer t... [01:56:58] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2618125 (10Dereckson) [01:57:53] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2054159 (10Dereckson) [02:38:22] 10DBA, 06Operations, 10Traffic, 06WMF-Legal, and 2 others: dbtree loads third party resources (from jquery.com and google.com) - https://phabricator.wikimedia.org/T96499#2618187 (10Dereckson) [02:39:50] 10DBA, 06Operations, 10Traffic, 06WMF-Legal, and 2 others: dbtree loads third party resources (from jquery.com and google.com) - https://phabricator.wikimedia.org/T96499#1218427 (10Dereckson) [06:17:55] 10DBA, 10MediaWiki-Maintenance-scripts, 06Operations, 06Release-Engineering-Team, and 2 others: Add section for long-running tasks on the Deployment page (specially for database maintenance) - https://phabricator.wikimedia.org/T144661#2618330 (10jcrespo) > Also, re the inclusion criteria part: if we list t... [06:31:41] 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2618336 (10jcrespo) >>! In T127792#2618122, @Catrope wrote: >>>! In T127792#2618119, @De... [07:27:10] 10Blocked-on-schema-change, 10DBA, 10ArchCom-RfC, 10Wikimedia-Site-requests, and 2 others: Schema change for page content language - https://phabricator.wikimedia.org/T69223#2618358 (10Nikerabbit) [08:19:25] so marostegui, you are working on T141407, right? [08:19:26] T141407: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407 [08:19:45] jynus: Yes I am [08:20:05] oh, remember to own it (I always forget) [08:20:26] jynus: Good point, will do it right now [08:20:27] just ask for help if you see any issue [08:20:46] jynus: thanks - will do [08:20:53] (and I mean it!) [08:21:47] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2618426 (10Marostegui) a:03Marostegui [08:21:48] also log it on -operations when you drop tables- that is the main channel for coordionation [08:22:21] jynus: Sounds good, I will do it now [08:32:52] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2618451 (10Marostegui) Hello, We have executed the following in db1046, db1047 and dbstore1002 ``` set sql_log_bin=0 ; drop table Im... [09:36:22] 10DBA, 10Analytics-EventLogging, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2618625 (10Marostegui) On Friday I will check again if the tables have been recreated and get back to you @Jdforrester-WMF [12:24:57] 10DBA: hitcounter and _counter tables are on the cluster but were deleted/unsused? - https://phabricator.wikimedia.org/T132837#2618932 (10Marostegui) a:03Marostegui [12:38:28] 10DBA, 06Operations, 05Prometheus-metrics-monitoring: Create a script to regenerate prometheus mysqld exporter listing that works with puppetdb - https://phabricator.wikimedia.org/T145072#2619017 (10jcrespo) [13:44:04] 10DBA, 06Labs, 10Labs-Infrastructure: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2619183 (10Marostegui) [14:15:42] 10DBA: hitcounter and _counter tables are on the cluster but were deleted/unsused? - https://phabricator.wikimedia.org/T132837#2619239 (10Marostegui) Hello, @demon can you review what I want to do to see if it looks good to you? First, I would like to focus on hitcounter first, so we can minimize risks, so t... [14:17:23] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2619249 (10jcrespo) This happened much more often before the latest upgrade (10.0.16, I think?). Since the upgrade to 10.0.22, this hadn't happened for months (6 m... [14:25:37] 10DBA, 10ChangeProp, 10MediaWiki-API, 06Operations, and 3 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2619266 (10mobrovac) [14:34:13] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2619285 (10Anomie) The query is simple enough: ``` SELECT tl_from,tl_namespace AS `bl_namespace`,tl_title AS `bl_title`,page_id,page_title,page_namespac... [14:38:36] hello [14:38:57] I'd like to know if https://github.com/wikimedia/operations-software-redactatron/blob/master/scripts/cols.txt is still the standard used to redact data from labsdb [14:39:23] we're loading mediawiki data into hadoop and we're not making it public yet, but would like to keep that option for later [14:47:40] ping jynus or marostegui [14:48:58] uh, difficult topic [14:49:11] let me answer in private [14:49:21] maybe a bit offtopic here [15:06:41] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2619417 (10Volans) Just to add some more info, in this specific `templatelinks ` table there are 4.8M rows with those conditions: ``` MariaDB PRODUCTIO... [15:10:34] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2619422 (10jcrespo) The problem with the straight join (or hints in general) is that you fix things now and break it elsewhere on other wikis, or for an... [17:31:24] 10DBA, 03Community-Tech-Sprint, 13Patch-For-Review, 07Schema-change: Add local_user_id and global_user_id to localuser table in centralauth database - https://phabricator.wikimedia.org/T141951#2619823 (10Niharika) Follow up work in {T142503} [17:47:56] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2619896 (10jcrespo) >>! In T145079#2619417, @Volans wrote: > Just to add some more info, in this specific `templatelinks ` table there are 4.8M rows wit... [18:13:12] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2620064 (10jcrespo) It happened again. I am going to kill the server again, then convert the table to innodb, as that seemed to do the trick before. ``` Id:... [18:26:26] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2620173 (10Volans) @jcrespo out of curiosity and learning, is this related to the fact that `templatelinks` is partitioned? Also if the partition key is... [18:45:16] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2620315 (10Marostegui) Same query again - that is interesting. When I hit the MariaDB bug in the past, it used to be _always_ the same kind of query (a LOAD DATA i... [18:49:23] 10DBA, 06Labs, 10Labs-Infrastructure, 07Upstream: mysqld process hang in db1069 - S2 mysql instance - https://phabricator.wikimedia.org/T145077#2620335 (10jcrespo) BTW, it indeed worked, which I think almost confirms it is a Toku issue (again). [19:37:14] 10DBA, 10ChangeProp, 10MediaWiki-API, 10MediaWiki-Database, and 4 others: Investigate slow transcludedin query - https://phabricator.wikimedia.org/T145079#2620594 (10Anomie) >>! In T145079#2620173, @Volans wrote: > @jcrespo out of curiosity and learning, is this related to the fact that `templatelinks` is...