[02:05:37] 06Data-Engineering, 10Community-Tech (Sea Lion Squad): SQLite/PostgreSQL native upsert - https://phabricator.wikimedia.org/T392786 (10tstarling) 03NEW [04:58:08] 06Data-Engineering, 10Community-Tech (Sea Lion Squad): SQLite/PostgreSQL native upsert - https://phabricator.wikimedia.org/T392786#10771467 (10tstarling) [05:06:45] 06Data-Engineering, 10Community-Tech (Sea Lion Squad): SQLite/PostgreSQL native upsert - https://phabricator.wikimedia.org/T392786#10771472 (10tstarling) I wrote [[https://www.postgresql.org/message-id/a31a76b6-b176-47fe-8778-9dfece231341%40wikimedia.org|this pgsql-general post]]. [08:48:09] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10771752 (10FCeratto-WMF) [08:50:12] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Kubernetes Resource Optimization - https://phabricator.wikimedia.org/T392668#10771756 (10Antoine_Quhen) [08:51:06] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Kubernetes Resource Optimization - https://phabricator.wikimedia.org/T392668#10771757 (10Antoine_Quhen) 05Open→03In progress [08:51:20] 10Data-Engineering (Q4 2025 April 1st - June 30th), 13Patch-For-Review: [Refine Refactoring] Refine jobs should be scheduled by Airflow: deployment - https://phabricator.wikimedia.org/T369845#10771760 (10Antoine_Quhen) 05Open→03In progress [08:52:47] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Kubernetes Resource Optimization - https://phabricator.wikimedia.org/T392668#10771763 (10Antoine_Quhen) a:03Antoine_Quhen [09:09:49] (03CR) 10Filippo Giunchedi: [C:03+1] "Untested but LGTM, see also I7bb8e758b4 to add statsd_exporter" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [09:27:15] (03PS1) 10Joal: Remove wbt tables from sqoop utility [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139419 (https://phabricator.wikimedia.org/T391006) [09:27:34] 10Data-Engineering (Q4 2025 April 1st - June 30th), 07Essential-Work, 13Patch-For-Review: Remove sqoop code for wikibase term storage - https://phabricator.wikimedia.org/T391006#10771864 (10JAllemandou) a:03JAllemandou [10:33:49] (03CR) 10Lucas Werkmeister (WMDE): Add Prometheus stats push (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [10:42:54] (03PS17) 10Hasan Akgün (WMDE): Add Prometheus stats push [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) [10:43:02] (03CR) 10Hasan Akgün (WMDE): Add Prometheus stats push (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [10:57:25] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "LGTM; I’m not sure if we want to merge it immediately or wait for the Puppet change…" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [11:45:28] (03PS1) 10Joal: Prepare sqoop and CIM queries for mediawiki image table migration [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 (owner: 10Mforns) [11:45:28] (03CR) 10Joal: "You also need the create-table scripts for those new tables 😊. And a nit about mapper-weight." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 (owner: 10Mforns) [12:43:44] PROBLEM - Webrequests Varnishkafka log producer on cp5026 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:44:40] PROBLEM - Webrequests Varnishkafka log producer on cp5029 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:44:40] PROBLEM - Webrequests Varnishkafka log producer on cp5028 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:44:42] PROBLEM - Webrequests Varnishkafka log producer on cp5032 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:44:43] PROBLEM - Webrequests Varnishkafka log producer on cp5031 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:44:43] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06MediaWiki-Platform-Team, 10Event-Platform, 13Patch-For-Review, 05SUL3: Some events in mediawiki.page_change.v1 refers to auth.wikimedia.org in meta.uri and meta.domain - https://phabricator.wikimedia.org/T388825#10772391 (10Tgr) [12:45:42] PROBLEM - Webrequests Varnishkafka log producer on cp5025 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:50:25] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10772395 (10Ladsgroup) We could now do masters (except s1/s4/s8) [12:51:42] RECOVERY - Webrequests Varnishkafka log producer on cp5028 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [12:53:54] 06Data-Engineering, 13Patch-For-Review: Upgrade Refinery Jobs to Spark 3 - https://phabricator.wikimedia.org/T291386#10772414 (10Ottomata) 05Open→03Resolved a:03Ottomata I think this task is long done! Please reopen if I am incorrect. [12:57:32] (03CR) 10Filippo Giunchedi: "statsd-exporter is in place on stat1011" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [12:57:56] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Handle Late-Arrived Events - https://phabricator.wikimedia.org/T370665#10772428 (10Ottomata) > • In parallel, we can progressively override meta.dt in EventGate to reduce the number of late events to an acceptable margin. (if ok wi... [13:00:51] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Handle Late-Arrived Events - https://phabricator.wikimedia.org/T370665#10772453 (10Ottomata) > → I don’t think this job should send alert emails for every error or late event — only if the global level of late events exceeds a cert... [13:04:20] RECOVERY - Webrequests Varnishkafka log producer on cp5025 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [13:04:20] RECOVERY - Webrequests Varnishkafka log producer on cp5026 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [13:05:20] RECOVERY - Webrequests Varnishkafka log producer on cp5029 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [13:07:18] RECOVERY - Webrequests Varnishkafka log producer on cp5032 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [13:07:20] RECOVERY - Webrequests Varnishkafka log producer on cp5031 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka [13:16:37] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "Hm, I don’t think it’s working properly yet… this command just hung until I pressed Ctrl+C:" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:17:45] 06Data-Engineering, 06Data-Engineering-Radar, 10Structured-Data-Backlog (Current Work): [L] Track commons deletion requests - https://phabricator.wikimedia.org/T370898#10772545 (10Cparle) 05Open→03Resolved This is done, and the data is currently being gathered and stored in hive in `mediawiki_upload_... [13:19:28] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Experimentation Lab: FY 24-25 SDS 2.4.9 CDN Synthetic Beacon: EventGate & Varnish: update to receive events from beacon event v2 - https://phabricator.wikimedia.org/T391959#10772554 (10Ottomata) > experiment field is an optional property in the base schema... [13:21:38] (03PS2) 10Mforns: Prepare sqoop and CIM queries for mediawiki image table migration [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 [13:22:38] (03CR) 10Mforns: "Done" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 (owner: 10Mforns) [13:25:16] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "Okay, looking at `WikimediaStatsd.php`, I think we need to add `-w 1` to the `nc` command – it’s UDP, there’s no connection to close, `nc`" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:25:31] (03CR) 10Filippo Giunchedi: "I'm not a dogstatsd expert, not sure about the format, though the exporter is working properly:" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:32:10] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "Ok, then either I failed in “manually executing” the PHP code and incorrectly assembled the line, or the PHP code is buggy 😅 thanks!" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:35:09] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "I think I found the bug. I separated the label and value with `=`, but it should be `:`." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:35:37] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Add Prometheus stats push [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:36:08] (03Merged) 10jenkins-bot: Add Prometheus stats push [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:37:52] (03PS1) 10Lucas Werkmeister (WMDE): Add Prometheus stats push [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139471 (https://phabricator.wikimedia.org/T389344) [13:38:02] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] "let’s try it :o" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139471 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [13:38:39] (03Merged) 10jenkins-bot: Add Prometheus stats push [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139471 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [13:40:21] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] "Dammit, I +2ed and completely forgot about the missing `-w 1`. Let’s add that in a separate change then." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:44:17] (03PS1) 10Hasan Akgün (WMDE): Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139473 (https://phabricator.wikimedia.org/T389344) [13:47:15] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139473 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:47:41] (03Merged) 10jenkins-bot: Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139473 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:50:10] 06Data-Engineering, 06Data-Engineering-Radar, 10BDC-Implementation, 06Data-Platform-SRE, 07Epic: [MinIO] Investigate load-balancing approaches to eliminate SPOF - https://phabricator.wikimedia.org/T392249#10772688 (10Jgreen) [14:02:09] (03PS1) 10Lucas Werkmeister (WMDE): Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139480 (https://phabricator.wikimedia.org/T389344) [14:02:20] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139480 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [14:02:51] (03Merged) 10jenkins-bot: Add -w 1 to current nc command to close the connection after sending stats [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139480 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [14:43:17] 06Data-Engineering: Refine to Hive with Airflow – Switch Over day scripts - https://phabricator.wikimedia.org/T392696#10772875 (10Antoine_Quhen) [14:56:17] (03CR) 10Joal: [C:03+1] "LGTM :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 (owner: 10Mforns) [15:19:39] (03PS1) 10Lucas Werkmeister (WMDE): Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139499 (https://phabricator.wikimedia.org/T389344) [15:27:31] (03CR) 10Lucas Werkmeister (WMDE): "(`load.php` is this repo’s version of an “autoloader”; new classes need to be added there manually.)" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139499 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:27:57] (03CR) 10Hasan Akgün (WMDE): [C:03+2] Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139499 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:28:24] (03Merged) 10jenkins-bot: Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139499 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:29:00] (03PS1) 10Lucas Werkmeister (WMDE): Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139504 (https://phabricator.wikimedia.org/T389344) [15:29:06] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139504 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:29:35] (03Merged) 10jenkins-bot: Add WikimediaStatsdExporter to load.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139504 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:34:18] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#10773143 (10Ottomata) [15:34:45] (03PS1) 10Lucas Werkmeister (WMDE): Don’t use str_ends_with() with yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139505 (https://phabricator.wikimedia.org/T389344) [15:35:43] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#10773176 (10Ottomata) @Antoine_Quhen @tchin here is a patch: https://gitlab.wikimedia.org/repos/data-engineering/eventgate... [15:39:14] (03PS2) 10Lucas Werkmeister (WMDE): Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139505 (https://phabricator.wikimedia.org/T389344) [15:45:32] (03CR) 10Hasan Akgün (WMDE): [C:03+2] Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139505 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:45:59] (03Merged) 10jenkins-bot: Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139505 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:46:05] (03PS1) 10Lucas Werkmeister (WMDE): Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139506 (https://phabricator.wikimedia.org/T389344) [15:46:12] (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139506 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:46:39] (03Merged) 10jenkins-bot: Don’t use str_ends_with() yet [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1139506 (https://phabricator.wikimedia.org/T389344) (owner: 10Lucas Werkmeister (WMDE)) [15:47:05] 06Data-Engineering, 06Data-Engineering-Radar, 10BDC-Implementation, 06Data-Platform-SRE, 07Epic: [MinIO] Investigate load-balancing approaches to eliminate SPOF - https://phabricator.wikimedia.org/T392249#10773231 (10Jgreen) Bikeshedding with Dallas we came up with this scheme to investigate further: - h... [16:13:19] 10Data-Engineering (Q4 2025 April 1st - June 30th): Refine to Hive with Airflow – Handle Late-Arrived Events - https://phabricator.wikimedia.org/T370665#10773358 (10JAllemandou) a:03JAllemandou [16:15:16] 10Data-Engineering (Q4 2025 April 1st - June 30th): Migrate Gobblin job repository to GitLab - https://phabricator.wikimedia.org/T390247#10773363 (10amastilovic) [16:20:38] 10Data-Engineering (Q4 2025 April 1st - June 30th): NEW BUG REPORT significantly increased edit revert rate for 2025-03 edits; Android, iOS, Mobile Web, Other - https://phabricator.wikimedia.org/T391708#10773391 (10gmodena) a:05mforns→03Ahoelzl [16:25:03] 10Data-Engineering (Q4 2025 April 1st - June 30th), 13Patch-For-Review: Migrate Gobblin to Airflow - https://phabricator.wikimedia.org/T390249#10773422 (10amastilovic) 05Resolved→03In progress [16:26:13] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Create Airflow pipeline to produce wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T391283#10773426 (10xcollazo) 96 24GB executors works best indeed, with runtimes under 2 hours. One more thing I want to... [16:39:13] 06Data-Engineering, 06Data-Engineering-Radar, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432#10773517 (10Krinkle) [16:53:44] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#10773554 (10Ottomata) Just discussed with @JAllemandou. He pointed out that we can't actually always override `meta.dt` in... [16:56:46] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#10773562 (10Ottomata) [17:04:44] 06Data-Engineering, 10DPE-Mediawiki-Content, 10GitLab (Upstream pit of despair 🕳️): Investigate artifact mismatch error when running mw_content_merge_events_to_mw_content_history_daily - https://phabricator.wikimedia.org/T391123#10773587 (10xcollazo) [17:05:22] 06Data-Engineering, 06Data-Engineering-Radar, 10DPE-Mediawiki-Content, 10GitLab (Upstream pit of despair 🕳️): Investigate artifact mismatch error when running mw_content_merge_events_to_mw_content_history_daily - https://phabricator.wikimedia.org/T391123#10773589 (10xcollazo) [17:40:01] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#10773688 (10Ottomata) Updated the patch! [19:04:55] 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Engineering-Wikistats, 07Russian-Sites: "Active editors by country" doesn't display numbers for Belarus, Kazakhstan, Russia - https://phabricator.wikimedia.org/T333716#10774063 (10Iniquity) Any progress here? [19:54:29] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Experimentation Lab, 13Patch-For-Review: FY 24-25 SDS 2.4.9 CDN Synthetic Beacon: EventGate & Varnish: update to receive events from beacon event v2 - https://phabricator.wikimedia.org/T391959#10774172 (10dr0ptp4kt) >>! In T391959#10769333, @tchin wrote:... [21:07:15] 06Data-Engineering, 06Data-Engineering-Radar, 07Epic: Data Platform Data Incidents - https://phabricator.wikimedia.org/T378559#10774555 (10Ahoelzl) [21:40:19] 10Data-Engineering (Q4 2025 April 1st - June 30th): Enable Spark data lineage for all Airflow instances - https://phabricator.wikimedia.org/T386862#10774668 (10Ahoelzl) [22:59:19] 06Data-Engineering, 10Community-Tech (Sea Lion Squad), 07PostgreSQL, 07SQLite: SQLite/PostgreSQL native upsert - https://phabricator.wikimedia.org/T392786#10774910 (10tstarling) [23:34:11] 07Analytics-Data-Problem, 06Data-Engineering: Strange statistics for language variants for languages other than zh and sr - https://phabricator.wikimedia.org/T392624#10774932 (10nshahquinn-wmf)