[01:47:23] 06Data-Engineering, 06Data-Engineering-Radar, 06Commons, 06Data-Persistence, and 5 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#11649214 (10Zabe) [05:37:48] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Update imagelinks primary key on wmf production - https://phabricator.wikimedia.org/T415786#11649445 (10Marostegui) [08:21:34] 06Data-Engineering, 06Data-Platform-SRE, 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11649666 (10brouberol) The way airflow egress works is by assigning [external services](https://wikitech.wikimedia.org/wiki/Kub... [09:17:34] (03PS5) 10Tarrow: Split active_user_changes.sql into user/temp account versions and run both [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243177 (https://phabricator.wikimedia.org/T416680) (owner: 10Andrew McAllister (WMDE)) [09:25:34] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11649823 (10Vgutierrez) a:03Vgutierrez [09:30:45] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11649836 (10Vgutierrez) waiting for mcollins approval, I've pinged them on Slack cause I've failed to find their phabricator user so far [09:41:55] (03CR) 10Tarrow: [C:04-1] Add license and contributing guide + update readme (034 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 (owner: 10Andrew McAllister (WMDE)) [09:55:48] (03CR) 10Tarrow: [C:04-1] Add license and contributing guide + update readme (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 (owner: 10Andrew McAllister (WMDE)) [10:36:25] (03CR) 10Thiemo Kreuz (WMDE): Add license and contributing guide + update readme (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 (owner: 10Andrew McAllister (WMDE)) [10:53:47] (03PS1) 10Andrew McAllister (WMDE): Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 [10:57:18] (03PS6) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [11:06:35] (03CR) 10Thiemo Kreuz (WMDE): [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [11:07:12] (03CR) 10Lucas Werkmeister (WMDE): "I’m not happy with this SQL style :D why is there so much horizontal and vertical whitespace, even inside a single clause like `JOIN table" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243177 (https://phabricator.wikimedia.org/T416680) (owner: 10Andrew McAllister (WMDE)) [11:11:25] (03CR) 10Lucas Werkmeister (WMDE): Add contributing guide and update readme (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 (owner: 10Andrew McAllister (WMDE)) [11:18:33] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [11:20:58] (03CR) 10Lucas Werkmeister (WMDE): Add contributing guide and update readme (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 (owner: 10Andrew McAllister (WMDE)) [11:23:48] (03PS2) 10Andrew McAllister (WMDE): Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 [11:35:44] (03CR) 10Hoo man: [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [11:40:27] (03CR) 10Lucas Werkmeister (WMDE): Add BSD 3-Clause License to the repo backdated to first commit year (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [11:40:30] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [11:53:15] (03CR) 10Rosalie Perside (WMDE): [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [12:03:32] (03PS7) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [12:05:07] (03PS8) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [12:06:49] (03PS9) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [12:20:51] (03CR) 10Silvan Heintze: [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [12:25:31] (03PS6) 10Andrew McAllister (WMDE): T416680 Split active_user_changes.sql into user/temp account versions and run both Bug: T416680 [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243177 (https://phabricator.wikimedia.org/T416680) [12:59:02] (03CR) 10Ladsgroup: [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [12:59:07] 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 10Page Content Service, and 6 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11650528 (10Jdforrester-WMF) @Krinkle: The only outstanding work here is for MW Engineering (Restbase); if you're dec... [13:00:44] 06Data-Engineering, 10ChangeProp, 10EventStreams, 10Recommendation-API, and 3 others: Migrate node-based services in production to node12 - https://phabricator.wikimedia.org/T290750#11650541 (10Jdforrester-WMF) Note to self: Same logic for Restbase applies here pending the outcome of T349118#11650527. [13:08:58] (03PS10) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [13:16:52] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11650583 (10Vgutierrez) got mcollins approval via Slack, we need #data-engineering approval now (that's @Milimetric / @Ottomata) [13:23:05] (03CR) 10Zabe: [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [13:27:56] (03PS11) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [13:30:02] (03PS12) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [13:30:05] 06Data-Engineering, 06Data-Platform-SRE: Reduce noise from HdfsRpcQueueLength alert - https://phabricator.wikimedia.org/T418152#11650651 (10Gehel) p:05Triage→03High [13:31:00] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11650652 (10Gehel) [13:31:10] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11650654 (10Gehel) p:05Triage→03High [13:55:54] (03PS13) 10Andrew McAllister (WMDE): Add contributing guide and update readme [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243181 [13:56:04] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11650740 (10Milimetric) approved! Welcome to moar data [14:04:03] (03CR) 10Hashar: [C:03+1] "My "contributions" were merely:" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [14:29:11] 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 06MediaWiki-Engineering, and 7 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11650838 (10Krinkle) [14:35:17] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11650875 (10mikez-WMF) Ah thank you for doing that! I was also confused why I couldn't find her in Phabricator and was going to ask in our 1:1 later today. I appreciate it! [14:45:03] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 13Patch-For-Review, 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11650927 (10awight) @brouberol That's amazing, thank you. I'll wait for the ch... [14:57:02] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-private-users for maxbinderWMF - https://phabricator.wikimedia.org/T417655#11650968 (10MatthewVernon) OK, I've tagged #data-engineering, since I think this is their ballpark now. Hopefully they can help :) [14:58:32] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 13Patch-For-Review, 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11650974 (10brouberol) I have merged the patch. It will be taken into account i... [15:00:49] 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 06MediaWiki-Engineering, and 7 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11650999 (10Krinkle) >>! In T349118#11650527, @Jdforrester-WMF wrote: > @Krinkle: The only outstanding work here is... [15:02:37] 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 06MediaWiki-Engineering, and 6 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11651007 (10Krinkle) 05Open→03Resolved p:05Triage→03Medium [15:06:58] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-private-users for maxbinderWMF - https://phabricator.wikimedia.org/T417655#11651035 (10Aklapper) @MBinder_WMF: Please feel also free to [link your LDAP account to your Phabricator account](https://phabricator.wikimedia.org/s... [15:07:31] (03CR) 10Umherirrender: [C:03+1] Add BSD 3-Clause License to the repo backdated to first commit year [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1243749 (owner: 10Andrew McAllister (WMDE)) [15:10:46] 06Data-Engineering, 10ChangeProp, 10EventStreams, 10Recommendation-API, and 3 others: Migrate node-based services in production to node12 - https://phabricator.wikimedia.org/T290750#11651054 (10Krinkle) 05Open→03Resolved [15:12:37] 06Data-Engineering, 10ChangeProp, 10EventStreams, 06MediaWiki-Engineering, and 15 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#11651060 (10Krinkle) [15:13:23] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 13Patch-For-Review, 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11651069 (10awight) I think it works! ` nc -X connect -x url-downloader.eqiad.... [15:13:33] 06Data-Engineering, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 13Patch-For-Review, 03WMDE-TechWish-Sprint-2026-02-17-Beautiful-Beetroots: Airflow devenv (WMDE) cannot see webproxy - https://phabricator.wikimedia.org/T417633#11651070 (10awight) 05Open→03Resolved a:03awight [15:20:08] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-private-users for maxbinderWMF - https://phabricator.wikimedia.org/T417655#11651112 (10Ottomata) A failure while logging in is more related to CAS (?right?), which IIUC is using LDAP for authorization. Is Max in either the... [15:27:19] 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 06MediaWiki-Engineering, and 6 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11651153 (10Jdforrester-WMF) >>! In T349118#11650975, @Krinkle wrote: >>>! In T349118#11650527, @Jdforrester-WMF... [15:28:55] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 07OKR-Work (WE1 FY2025-26): WE1.5.3 Productize Data for Monthly Active Moderator Actions - https://phabricator.wikimedia.org/T410940#11651155 (10AKhatun_WMF) After requirements gathering with Research and other teams ([[ https://docs.google.com/document... [15:37:28] 06Data-Engineering, 10Data-Platform, 06Moderator-Tools-Team, 10PersonalDashboard, 06Product-Analytics (Kanban): Personal Dashboard Instrumentation Superset Dashboard - https://phabricator.wikimedia.org/T412137#11651197 (10Samwalton9-WMF) [15:37:41] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 07Epic: Roll instrument out to 100% of enwiki - https://phabricator.wikimedia.org/T418385 (10Milimetric) 03NEW [15:42:38] 06Data-Engineering, 10Dumps-Generation: Data missing from en.wiktionary.org February 2026 "MediaWiki Content File Exports" compared to "XML Database dump" - https://phabricator.wikimedia.org/T417596#11651229 (10APizzata-WMF) > I will test the new algorithm to see if we catch this case After testing it appears... [16:25:05] FIRING: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [16:30:05] FIRING: [3x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [16:35:05] FIRING: [3x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [16:40:05] RESOLVED: [3x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [17:04:11] 06Data-Engineering, 06Data-Engineering-Radar, 06Machine-Learning-Team, 10Event-Platform, 13Patch-For-Review: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11651593 (10Ottomata) [17:04:24] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE: Reduce noise from HdfsRpcQueueLength alert - https://phabricator.wikimedia.org/T418152#11651595 (10Ottomata) [17:05:58] 06Data-Engineering, 06Movement-Insights: Improve referrer tracking/classification using `utm_source` URL parameter - https://phabricator.wikimedia.org/T408185#11651611 (10Ottomata) p:05Medium→03Low [17:06:49] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Update imagelinks primary key on wmf production - https://phabricator.wikimedia.org/T415786#11651623 (10Ottomata) [17:09:28] 06Data-Engineering, 06DBA, 10MediaWiki-Special-pages, 06serviceops-radar, 13Patch-For-Review: Move MediaWiki QueryPages computation to Hadoop - https://phabricator.wikimedia.org/T309738#11651635 (10Ottomata) [17:10:40] 06Data-Engineering, 06Data-Engineering-Icebox, 06DBA: Move Mostcategories computation to Hadoop - https://phabricator.wikimedia.org/T413362#11651637 (10Ottomata) [17:14:06] 06Data-Engineering, 10Dumps-Generation: Some wikimedia 20260101 dump files missing - https://phabricator.wikimedia.org/T413767#11651651 (10Ahoelzl) @xcollazo any next steps needed on DE side? Is this resolved? [17:16:05] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Data Pipelines, 06Data-Platform-SRE (2026-02-13 - 2026-03-06), 07Essential-Work: Airflow dynamic task mapping logs mix up when, on rerun, an id is mapped to a different map_index_template - https://phabricator.wikimedia.org/T408802#11651656 (10Otto... [17:17:47] 06Data-Engineering, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services: Drop support for cl_to, cl_collation and il_to from wikireplicas - https://phabricator.wikimedia.org/T417492#11651659 (10Ottomata) [17:18:54] 06Data-Engineering, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services: Drop support for cl_to, cl_collation and il_to from wikireplicas - https://phabricator.wikimedia.org/T417492#11651676 (10Ottomata) @Milimetric @Snwachukwu just double checking: we don't sqoop categorylink related tables from a... [17:19:00] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11651679 (10Ahoelzl) Approved. [17:19:13] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Superset for mikez - https://phabricator.wikimedia.org/T418098#11651681 (10Ahoelzl) [17:20:39] 06Data-Engineering, 10Dumps-Generation: Data missing from en.wiktionary.org February 2026 "MediaWiki Content File Exports" compared to "XML Database dump" - https://phabricator.wikimedia.org/T417596#11651692 (10Ottomata) [[ https://wikimedia.slack.com/archives/C05RHK7PS6Q/p1771859228331369 | Semi-relevant slac... [17:20:52] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 06Research: MediaWiki content history dataset issues - https://phabricator.wikimedia.org/T415311#11651694 (10Ahoelzl) Related https://phabricator.wikimedia.org/T417596 [17:21:00] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Dumps-Generation: Data missing from en.wiktionary.org February 2026 "MediaWiki Content File Exports" compared to "XML Database dump" - https://phabricator.wikimedia.org/T417596#11651697 (10Ottomata) [17:25:52] 06Data-Engineering, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services: Drop support for cl_to, cl_collation and il_to from wikireplicas - https://phabricator.wikimedia.org/T417492#11651715 (10Milimetric) @Ottomata we do: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/f12678b3a... [17:29:28] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 06MW-Interfaces-Team, 06Traffic, 06MediaWiki-Platform-Team (Radar), and 2 others: haproxy: capture x-wmf-* headers in webrequest data set - https://phabricator.wikimedia.org/T417864#11651727 (10Ottomata) [17:29:53] 06Data-Engineering, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-platform-eng-admins for milimetric - https://phabricator.wikimedia.org/T417906#11651729 (10Ottomata) [17:30:05] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-platform-eng-admins for milimetric - https://phabricator.wikimedia.org/T417906#11651733 (10Ottomata) [17:33:54] 06Data-Engineering, 06Test Kitchen, 10Wikidata, 10Wikidata Analytics: Add rcshowwikidata property to the existing PrefUpdate instrumentation for wmf_raw.mediawiki_user_properties - https://phabricator.wikimedia.org/T418246#11651755 (10Ottomata) @Milimetric please help groom this. Who owns PrefUpdate and wh... [17:34:04] 06Data-Engineering, 06Data-Engineering-Radar, 06Test Kitchen, 10Wikidata, 10Wikidata Analytics: Add rcshowwikidata property to the existing PrefUpdate instrumentation for wmf_raw.mediawiki_user_properties - https://phabricator.wikimedia.org/T418246#11651758 (10Ottomata) [17:36:33] 06Data-Engineering: spark-sql warns about mismatching table schema for event.EditAttemptStep - https://phabricator.wikimedia.org/T418065#11651774 (10Ottomata) 05Open→03Resolved a:03Ottomata It looks like Aleks fixed this for this table. Marking as resolved, feel free to reopen if incorrect. [17:38:00] 06Data-Engineering, 10Data Pipelines: Refine: Use Spark SQL instead of Hive JDBC - https://phabricator.wikimedia.org/T209453#11651784 (10Ottomata) [17:40:40] 06Data-Engineering, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services: Drop support for cl_to, cl_collation and il_to from wikireplicas - https://phabricator.wikimedia.org/T417492#11651801 (10Ottomata) Hah, sorry! I guess I meant: "the cloud replicas we access for analytics-hadoop ingestion" [17:41:28] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-platform-eng-admins for milimetric - https://phabricator.wikimedia.org/T417906#11651807 (10Ottomata) Approved! [17:41:38] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-platform-eng-admins for milimetric - https://phabricator.wikimedia.org/T417906#11651810 (10Ottomata) > maybe all analytics-admins should have admin in all airflow admin groups... [17:50:00] 06Data-Engineering, 10Dumps-Generation: Some wikimedia 20260101 dump files missing - https://phabricator.wikimedia.org/T413767#11651840 (10xcollazo) > What would be helpful - to look at some less local syncing status page/log closer to the servers, if available. Other than the `dumpstatus.json` mentioned on T... [17:55:41] 06Data-Engineering, 10Dumps-Generation: wikidatawiki fails dumps of the wbt_* tables, also lagging on XML Dumps - https://phabricator.wikimedia.org/T396125#11651856 (10xcollazo) Considering there has been no asks from community for these tables AFAIK, and this has been ongoing for 1+ year, I think we should de... [18:06:18] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 13Patch-For-Review: Adapt Sqoop for imagelinks schema changes - https://phabricator.wikimedia.org/T416481#11651886 (10xcollazo) > But there is a diff in row counts I would expect the numbers to shift a bit considering that you did a manual sqoop that w... [18:10:50] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 13Patch-For-Review: Adapt Sqoop for imagelinks schema changes - https://phabricator.wikimedia.org/T416481#11651899 (10xcollazo) Can we also please report on the other two tests from T416481#11636745? [18:11:59] 06Data-Engineering, 06MW-Interfaces-Team, 10RESTBase-API, 06ServiceOps new, 10ServiceOps-SharedInfra: AQS Wikimedia REST API - new API version - https://phabricator.wikimedia.org/T407863#11651904 (10Ottomata) a:05Ottomata→03None [18:16:20] (03CR) 10Snwachukwu: [V:03+2 C:03+2] "Testing done. I'll go ahead to merge." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1239200 (https://phabricator.wikimedia.org/T416481) (owner: 10Snwachukwu) [18:21:31] 06Data-Engineering, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services: Drop support for cl_to, cl_collation and il_to from wikireplicas - https://phabricator.wikimedia.org/T417492#11651954 (10Ottomata) Ah: This is covered by {T416481}. Moving to DE radar. [18:25:21] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Inconsistent page title styles in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T410405#11651978 (10xcollazo) Ok this is how it all went: `wmf_content.mediawiki_content_history_v1`... [18:29:27] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Inconsistent page title styles in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T410405#11651983 (10xcollazo) Rerunning @Isaac's repro from description: ` spark.sql(""" SELECT... [18:33:15] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Inconsistent page title styles in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T410405#11652005 (10xcollazo) @Isaac, could you do a check on your side to see whether we can close? [18:34:36] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content: Missing/inconsistent page_redirect_target field for redirects in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T400632#11652007 (10xcollazo) Over at T410405 we did an UPDATE to all our content tables so... [18:47:14] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content: Missing/inconsistent page_redirect_target field for redirects in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T400632#11652061 (10xcollazo) Now we still need to do something to reconcile pages that are... [19:00:51] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Inconsistent page title styles in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T410405#11652075 (10Isaac) @xcollazo thanks! I looked through the queries + outputs and that's exactl... [19:30:36] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 13Patch-For-Review: Adapt Sqoop for imagelinks schema changes - https://phabricator.wikimedia.org/T416481#11652153 (10Snwachukwu) Sure @xcollazo . NB: All test was done using `snapshot=2026-01` First, I compared the count of production wmf_raw.mediawi... [19:40:17] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 13Patch-For-Review: Adapt Sqoop for imagelinks schema changes - https://phabricator.wikimedia.org/T416481#11652190 (10Snwachukwu) With regards to > A manual check of a few specific page_titles. Make sure they appear on both versions. I'm not sure at... [19:48:58] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 13Patch-For-Review: Adapt Sqoop for imagelinks schema changes - https://phabricator.wikimedia.org/T416481#11652200 (10xcollazo) > I'm not sure at what point you think would be best to check, Perhaps the point where we were loosing rows before would be... [20:04:47] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Fix mediawiki event enrichment to work with newest version of Blubber - https://phabricator.wikimedia.org/T406872#11652226 (10Ottomata) [20:04:48] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 07Essential-Work, 10Event-Platform, 13Patch-For-Review: Upgrade mediawiki-event-enrichment jobs to Flink 1.20.2 and Java 17 - https://phabricator.wikimedia.org/T408918#11652225 (10Ottomata) [20:07:22] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 06Content-Transform-Team, 06Reader Growth Team, 06Wikipedia-Android-App-Backlog, and 3 others: Add page_id and namespace to X-Analytics header in Mobile App requests (2025 remake) - https://phabricator.wikimedia.org/T409358#11652228 (10Ottomata) @Jg... [20:12:30] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Fix mediawiki event enrichment to work with newest version of Blubber - https://phabricator.wikimedia.org/T406872#11652231 (10Ottomata) [20:15:17] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Fix mediawiki event enrichment to work with newest version of Blubber - https://phabricator.wikimedia.org/T406872#11652233 (10Ottomata) [20:19:33] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Fix mediawiki event enrichment to work with newest version of Blubber - https://phabricator.wikimedia.org/T406872#11652241 (10Ottomata) Developer experience folks are helping in [[ https://wikimedia.slack.com/arch... [23:16:30] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Refactor pingback reports pipelines using dbt - https://phabricator.wikimedia.org/T418190#11652662 (10amastilovic) >>! In T418190#11644291, @GGoncalves-WMF wrote: > Nice! Just wondering, do we know who uses that output CSV and where? > > I'm asking for m... [23:17:00] 06Data-Engineering: Refactor pingback analytics pipeline - https://phabricator.wikimedia.org/T415283#11652664 (10amastilovic) a:03amastilovic [23:17:30] 06Data-Engineering: Refactor pingback analytics pipeline - https://phabricator.wikimedia.org/T415283#11652665 (10amastilovic) [23:17:31] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Refactor pingback reports pipelines using dbt - https://phabricator.wikimedia.org/T418190#11652666 (10amastilovic) [23:17:44] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Refactor pingback analytics pipeline - https://phabricator.wikimedia.org/T415283#11652667 (10amastilovic)