[00:01:20] 06Data-Engineering, 07Sustainability: Move some analytics jobs to day time in Virginia - https://phabricator.wikimedia.org/T384166#11458455 (10Ladsgroup) FWIW. Almost all of the power consumption in analytics infra (understandably) is coming from an-worker nodes. There is some large consumption coming from pre... [06:13:57] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06Data-Persistence, 05MW-1.46-notes (1.46.0-wmf.7; 2025-12-16), and 2 others: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11458691 (10Marostegui) Changing the tags - pleas... [07:49:21] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11458860 (10JAllemandou) In the meatime I'm going to reenable the indexation job with more resources, allowing it not to... [09:50:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11459188 (10cmooney) >>! In T412443#11458860, @JAllemandou wrote: > Either sampling more, or keeping one week of data fo... [10:23:45] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11459310 (10ayounsi) >>! In T412443#11459188, @cmooney wrote: >>>! In T412443#11458860, @JAllemandou wrote: >> Either sa... [10:57:27] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06Data-Persistence, 05MW-1.46-notes (1.46.0-wmf.7; 2025-12-16), and 2 others: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11459428 (10kostajh) Moving back to "Ready" so th... [11:00:21] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add sic_updated_timestamp column and associated indexes to the cusi_case table on WMF wikis - https://phabricator.wikimedia.org/T412687 (10Dreamy_Jazz) 03NEW [11:02:14] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06Data-Persistence, 05MW-1.46-notes (1.46.0-wmf.7; 2025-12-16), and 2 others: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11459453 (10Dreamy_Jazz) [11:05:03] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add sic_updated_timestamp column and associated indexes to the cusi_case table on WMF wikis - https://phabricator.wikimedia.org/T412687#11459466 (10Marostegui) 05Open→03Resolved a:03Marostegui This has been done ` [11:03:32] marostegui@cumin... [11:05:21] 06Data-Engineering, 10Dumps-Generation, 06Wikibase Reuse Team, 10Wikidata, and 2 others: No Wikidata dumps for Week 40 of 2025 (recurring issue) - https://phabricator.wikimedia.org/T406429#11459469 (10Jakob_WMDE) Hi @Hannah_Bast, we found that it's only the `latest-*` links that are broken. The latest... [11:05:29] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add sic_updated_timestamp column and associated indexes to the cusi_case table on WMF wikis - https://phabricator.wikimedia.org/T412687#11459470 (10Dreamy_Jazz) Thanks! [11:09:11] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06Data-Persistence, 05MW-1.46-notes (1.46.0-wmf.7; 2025-12-16), and 2 others: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11459496 (10Dreamy_Jazz) 05Open→03Resolved [11:13:58] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11459538 (10Marostegui) [11:14:43] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11459542 (10Marostegui) [11:19:31] 06Data-Engineering, 10Dumps-Generation, 06Wikibase Reuse Team, 10Wikidata, and 2 others: No Wikidata dumps for Week 40 of 2025 (recurring issue) - https://phabricator.wikimedia.org/T406429#11459550 (10BTullis) Yes, I see. There are lots of symlinks that are printing to the wrong location, so do not wor... [12:04:39] 06Data-Engineering, 10Dumps-Generation, 06Wikibase Reuse Team, 10Wikidata, and 2 others: No Wikidata dumps for Week 40 of 2025 (recurring issue) - https://phabricator.wikimedia.org/T406429#11459775 (10Hannah_Bast) @BTullis @Jakob_WMDE Thank you for the workaround. I wasn't aware that these were always... [12:15:28] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11459819 (10Marostegui) [13:33:48] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11460057 (10JAllemandou) I've been spending time on this, here's my latest finding: By reducing retention to 30 days, we... [13:48:28] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11460139 (10cmooney) >>! In T412443#11460057, @JAllemandou wrote: > I'll keep the conf as is (30 days retention), and wi... [14:00:14] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11460243 (10Marostegui) [14:06:25] !log Test Kitchen mw-user experiment (poll 24816) - adds: growthexperiments-revise-tone; removes: none; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [14:06:28] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:01:06] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11460592 (10Marostegui) [15:01:19] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11460594 (10Marostegui) [15:37:33] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops, 13Patch-For-Review: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11460806 (10ayounsi) p:05Triage→03Medium [16:09:05] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06MW-Interfaces-Team, 10Event-Platform: mediawiki.page_change.v1 event stream - Investigate mistmatched meta.dt and dt (and rev_dt) fields - https://phabricator.wikimedia.org/T409105#11460975 (10Ottomata) Nice @JMonton-WMF! > If we create a page_c... [16:29:33] 06Data-Engineering, 10CheckUser, 06DBA, 07Essential-Work, and 4 others: Drop the cupe_private column in the cu_private_event table - https://phabricator.wikimedia.org/T409710#11461093 (10Dreamy_Jazz) Tagging with #DBA for awareness (we should have a #schema-change-in-production ticket soon) [16:29:56] 06Data-Engineering, 10CheckUser, 06DBA, 07Essential-Work, and 4 others: Drop the cupe_private column in the cu_private_event table - https://phabricator.wikimedia.org/T409710#11461096 (10Dreamy_Jazz) [16:39:43] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Infrastructure-Foundations, 10netops, 13Patch-For-Review: Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11461141 (10JAllemandou) a:03JAllemandou [16:40:09] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Investigate and ensure Druid snapshot retention is limited to 3 - https://phabricator.wikimedia.org/T412322#11461147 (10JAllemandou) a:03JAllemandou [16:55:51] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.11.07 - 2025.11.28): Airflow-main scheduler loop sometimes slows down markedly - https://phabricator.wikimedia.org/T412003#11461198 (10JAllemandou) [17:37:58] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10Dumps-Generation: Update dump mirror rsync allowlist to reflect new IP address for Scatter - https://phabricator.wikimedia.org/T409006#11461361 (10xcollazo) 05In progress→03Resolved [18:04:27] !log created HDFS folder `/wmf/data/wme`, owned by the analytics-platform-eng user. [18:04:29] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:15:35] 06Data-Engineering, 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list November 2025 - https://phabricator.wikimedia.org/T410768#11461737 (10GFontenelle_WMF) 05Open→03Resolved [20:18:48] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Fix reconcile bug where user_id is not being populated correctly. - https://phabricator.wikimedia.org/T411803#11461753 (10xcollazo) Ok, lets recap: * The reconcile changes are now in production, and for daily reconcile, it has run successfully for two...