[00:17:42] (03PS4) 10Zabe: querypage: Add UnusedCategories.hql [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1268292 (https://phabricator.wikimedia.org/T422448) [02:04:50] (03PS6) 10Snwachukwu: mediarequest_hourly: use file/filetypes as media_classification ground truth [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1279651 (https://phabricator.wikimedia.org/T421743) [02:16:31] RESOLVED: MediawikiPageHtmlFeatureCountsChangeEnrichHighKafkaConsumerLag: ... [02:16:31] High Kafka consumer lag for mw_page_html_feature_counts_change_enrich in eqiad - https://wikitech.wikimedia.org/wiki/MediaWiki_Event_Enrichment/HTML_Enrichment#Alerting - https://grafana.wikimedia.org/d/K9x0c4aVk/flink-app?orgId=1&var-datasource=eqiad%20prometheus/k8s-dse&var-namespace=mw-page-html-feature-counts-change-enrich&var-helm_release=production&var-operator_name=All&var-flink_job_name=mw_page_html_feature_counts_change_enrich - . [02:16:31] https://alerts.wikimedia.org/?q=alertname%3DMediawikiPageHtmlFeatureCountsChangeEnrichHighKafkaConsumerLag [04:30:06] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Edit type Enrichment: Update documentation - https://phabricator.wikimedia.org/T424225#11877672 (10AKhatun_WMF) [04:40:36] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Research-engineering, 06Research-Freezer, 10Event-Platform, 13Patch-For-Review: Productionized Edit Types - https://phabricator.wikimedia.org/T351225#11877678 (10AKhatun_WMF) [08:58:42] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066 (10Dreamy_Jazz) 03NEW [08:59:07] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066#11878296 (10Dreamy_Jazz) [08:59:23] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066#11878297 (10Dreamy_Jazz) [09:03:03] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066#11878316 (10Dreamy_Jazz) [09:56:57] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop SecurePoll tables from closed wikis - https://phabricator.wikimedia.org/T423128#11878504 (10Dreamy_Jazz) [10:12:16] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Accelerate sqoop landing for MediaWiki History private tables - https://phabricator.wikimedia.org/T424355#11878567 (10APizzata-WMF) All the sqoop commands are executed by the service `refinery-sqoop-whole-mediawiki.service` that runs the bin `/usr/local/bin/... [10:47:17] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop SecurePoll tables from closed wikis - https://phabricator.wikimedia.org/T423128#11878734 (10Novem_Linguae) > However, these wikis never had local elections because no closed wikis have securepoll_log which means no ele... [10:53:18] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop SecurePoll tables from closed wikis - https://phabricator.wikimedia.org/T423128#11878747 (10Dreamy_Jazz) Ran the above query and I got `0` for all closed wikis, so should be safe to proceed [11:41:05] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), and 2 others: [Epic] Replace Archiva with Gitlab artifact repositories - https://phabricator.wikimedia.org/T367315#11878868 (10atsuko) [11:42:46] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), and 2 others: [Epic] Replace Archiva with Gitlab artifact repositories - https://phabricator.wikimedia.org/T367315#11878874 (10atsuko) [11:43:56] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), and 2 others: [Epic] Replace Archiva with Gitlab artifact repositories - https://phabricator.wikimedia.org/T367315#11878890 (10atsuko) [12:57:39] 06Data-Engineering, 07Sustainability: Move some analytics jobs to day time in Virginia - https://phabricator.wikimedia.org/T384166#11879216 (10mforns) >Perhaps we should figure what are our most compute intensive jobs (i.e. biggest bang for the buck) and go from there? +1 [14:30:03] !log Test Kitchen mw-user experiment (poll 183187) - adds: none; removes: logged-in-retention-round1; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [14:30:04] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:44:35] 06Data-Engineering, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 07Epic, 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#11879570 (10BTullis) I have built a debian package containing the spark 3... [14:59:39] 06Data-Engineering, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 07Epic, 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#11879621 (10BTullis) Oh, now this is not so good. ` btullis@an-test-clien... [15:17:29] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill video play request data for the past 3 months - https://phabricator.wikimedia.org/T425121 (10Snwachukwu) 03NEW [15:18:51] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10AQS2.0: Introduce a new AQS endpoint to expose video plays - https://phabricator.wikimedia.org/T415202#11879695 (10Snwachukwu) @New_York-air . I have created {T425121} to handle the backfilling task. [15:44:00] 06Data-Engineering, 07Sustainability: Move some analytics jobs to day time in Virginia - https://phabricator.wikimedia.org/T384166#11879819 (10Ahoelzl) p:05Triage→03Medium [16:22:04] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Create an MVP data product of API requests - https://phabricator.wikimedia.org/T419522#11879932 (10KCVelaga_WMF) a:03KCVelaga_WMF As per discussions, I will be taking this up with DPE support (from @amastilovic). [16:24:38] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Ingest JSON access logs for dumps.wikimedia.org in the data lake - https://phabricator.wikimedia.org/T425128 (10Ahoelzl) 03NEW [16:27:49] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Ingest JSON access logs for dumps.wikimedia.org in the data lake - https://phabricator.wikimedia.org/T425128#11879973 (10Ahoelzl) Duplicate: https://phabricator.wikimedia.org/T403159 ? [17:58:58] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix failing dbt task `product-analytics.upload_bandwidth_hourly` - https://phabricator.wikimedia.org/T425135 (10amastilovic) 03NEW [17:59:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix failing dbt task `product-analytics.upload_bandwidth_hourly` - https://phabricator.wikimedia.org/T425135#11880147 (10amastilovic) From the YARN logs: ` 17:18:05 1 of 2 START sql incremental model wmf_traffic.stg_webrequest_upload_bandwidth . [RUN] 26/0... [18:38:48] FIRING: EventgateLatency: Elevated latency for POST events on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLatency [19:03:48] RESOLVED: EventgateLatency: Elevated latency for POST events on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLatency [19:35:48] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11880494 (10mforns) [19:48:22] 06Data-Engineering, 06Data-Engineering-Radar, 10Data Pipelines, 10Wikibase (3rd party installations), and 2 others: Wikimedia Hackathon 2026: Pipeline for Moving Data Between Wikibase Instances and Wikidata - https://phabricator.wikimedia.org/T423333#11880504 (10Superraptor123) Progress so far in GitHub is...