[00:56:15] 06Data-Engineering, 06Research, 13Patch-For-Review: A dataset sensor should work indepent of airflow instance - https://phabricator.wikimedia.org/T386973#10577687 (10amastilovic) @xcollazo I commented on the MR but I wanted to leave the same comment here - I agree with your rationale. If specific customizati... [01:24:03] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10577713 (10amastilovic) NOTE: For the test k8s deployment, let's use a feature branch called [[ https://gitlab.wikimedia.org/repos/data-... [08:41:39] 06Data-Engineering, 06Research, 13Patch-For-Review: A dataset sensor should work indepent of airflow instance - https://phabricator.wikimedia.org/T386973#10578097 (10brouberol) @amastilovic we could leverage your `k8s_proof_url` function to rewrite the URL I think! [10:04:32] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Drop event_variant column from echo_event - https://phabricator.wikimedia.org/T385645#10578338 (10Marostegui) x1 moved back to RBR until T385645#10525207 is fixed! [11:05:01] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work, 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10578498 (10brouberol) airflow-test-k8s should be ready to run analytics DAGs. [11:07:23] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Drop event_variant column from echo_event - https://phabricator.wikimedia.org/T385645#10578511 (10Ladsgroup) 05Stalled→03Open The code should have reached production now. [12:47:57] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE-Mediawiki-Content: Figure root cause of silent failures when computing metrics for mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T387033#10578695 (10xcollazo) After further inspection, the error appears to be a skein application OO... [12:55:16] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Drop event_variant column from echo_event - https://phabricator.wikimedia.org/T385645#10578714 (10Marostegui) db1179 has been altered. I am leaving it for a few more hours to make sure it is good. [14:01:49] 10Data-Engineering (Q3 2024 January 1st - March 31th): [HAProxy migration] Fix HAProxy `uri_host` and `accept_language` differences with VarnishKafka - https://phabricator.wikimedia.org/T386354#10578949 (10JAllemandou) The same type of result happens when looking at different times (2025-0216T15:00 and 2025-02-2... [14:07:47] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 06MediaWiki-Platform-Team, and 3 others: Drop module_deps table in WMF prod - https://phabricator.wikimedia.org/T385997#10578957 (10Ladsgroup) Dropped it in all of s5, If all goes well, I will drop it everywhere tomorrow. [15:29:08] 06Data-Engineering, 10Commons-Impact-Metrics-Requests: Request for Images_from_Wiki_Loves_Africa_2024 - https://phabricator.wikimedia.org/T381352#10579370 (10Dominicbm) >>! In T381352#10577170, @Anthere wrote: > I finally tested today. Works pretty well. I have not tested all options... but several of thos... [16:57:03] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE-Mediawiki-Content, 10Image-Suggestions, 10Section-Level-Image-Suggestions, 06Structured-Data-Backlog: [SPIKE] Check the Wikimedia content history dataset - https://phabricator.wikimedia.org/T385787#10579818 (10mfossati) Discussion thread about... [18:32:35] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Research: A dataset sensor should work indepent of airflow instance - https://phabricator.wikimedia.org/T386973#10580155 (10Ottomata) [18:59:55] 06Data-Engineering, 10AQS2.0, 10Commons-Impact-Metrics, 13Patch-For-Review: Get more data to populate CIM time series datasets for the AQS local test environment - https://phabricator.wikimedia.org/T372989#10580274 (10EChukwukere-WMF) @xcollazo I believe the data is ready int the cassandra test environment... [19:30:29] 14Data-Engineering (Q2 2024 October 1st - December 31th): Remove stale GDI Equity Landscape jobs. - https://phabricator.wikimedia.org/T369649#10580353 (10xcollazo) Note: today I noticed old DagRuns were still part of our Analytics Airflow instance. I deleted them, so now they are not part of the Airflow dat... [21:06:07] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10580653 (10Ottomata) (This comment is a half rubber-ducking, half seeking validation and... [21:06:16] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Research: A dataset sensor should work indepent of airflow instance - https://phabricator.wikimedia.org/T386973#10580657 (10amastilovic) > we could leverage your k8s_proof_url function to rewrite the URL I think! @brouberol I didn't see this comment so... [22:06:49] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 10Charts (Sprint 17), 07Schema-change-in-production: Deploy patch-gjlw_namespace_text.sql on x1.commonswiki for JsonConfig - https://phabricator.wikimedia.org/T385917#10580894 (10Ladsgroup) >>! In T385917#10572161, @bvibber wrote: > @Ladsgroup I'm reas... [23:03:39] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10580999 (10Ahoelzl) Although the initial analysis goal has been redefined here https://phabricator.wikimedia.org/T383175#10481404 (due to lack of availab...