[06:25:30] 06Data-Engineering, 10CampaignEvents, 06DBA, 06Connection-Team (Connection-Current-Sprint), 07Schema-change-in-production: Apply ce_address cleanup schema changes in production (x1) - https://phabricator.wikimedia.org/T409101#11361899 (10Marostegui) 05Open→03Resolved Done in the master with repli... [06:30:31] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity, and 3 others: Add the sis_trigger_id and sis_trigger_type columns to the cusi_signal table on WMF wikis - https://phabricator.wikimedia.org/T409733#11361908 (10Marostegui) p:05Triage→03Medium [06:35:51] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity, and 3 others: Add the sis_trigger_id and sis_trigger_type columns to the cusi_signal table on WMF wikis - https://phabricator.wikimedia.org/T409733#11361913 (10Marostegui) 05Open→03Resolved This is done:... [06:42:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11361917 (10Marostegui) [06:43:07] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11361918 (10Marostegui) [11:24:43] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11362605 (10JMonton-WMF) Here is an example of how to do the [[ https://github.com/wikimedia/analytics-refinery/blob/master/hql/projectview/hourly/aggregate_pageview_to_proje... [11:34:09] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11362648 (10JMonton-WMF) Also, if we want to consider it, the [[ https://docs.getdbt.com/reference/resource-configs/trino-configs#deleteinsert-strategy | Trino adapter has th... [11:42:59] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11362663 (10JMonton-WMF) There is also another [[ https://docs.getdbt.com/docs/build/incremental-microbatch | strategy for micro batches ]] that allow for retries and backfil... [12:07:00] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11362716 (10JMonton-WMF) I'm trying to give different examples, so we can consider later if they are useful or not. [[ https://gitlab.wikimedia.org/repos/data-engineering/d... [12:16:13] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11362738 (10JAllemandou) This `pre_hook` solution is great :) If we can build our own materialization defining such a hook, it would be perfect. The only concern I see here i... [13:29:29] 06Data-Engineering, 06Data-Persistence, 06DBA, 10MediaWiki-Core-Revision-backend, and 6 others: Rethink rev_sha1 field - https://phabricator.wikimedia.org/T389026#11362881 (10ops-monitoring-bot) Cookbook cookbooks.sre.wikireplicas.update-views run by ladsgroup: Started updating wiki replica views [13:34:56] 06Data-Engineering, 06Data-Persistence, 06DBA, 10MediaWiki-Core-Revision-backend, and 6 others: Rethink rev_sha1 field - https://phabricator.wikimedia.org/T389026#11362893 (10ops-monitoring-bot) Cookbook cookbooks.sre.wikireplicas.update-views started by ladsgroup completed: - an-redacteddb1001.eqiad.wmnet... [13:43:25] 06Data-Engineering, 06Data-Persistence, 06DBA, 10MediaWiki-Core-Revision-backend, and 6 others: Rethink rev_sha1 field - https://phabricator.wikimedia.org/T389026#11362923 (10ops-monitoring-bot) Cookbook cookbooks.sre.wikireplicas.update-views run by ladsgroup: Started updating wiki replica views [13:49:48] 06Data-Engineering, 06Data-Persistence, 06DBA, 10MediaWiki-Core-Revision-backend, and 6 others: Rethink rev_sha1 field - https://phabricator.wikimedia.org/T389026#11362951 (10ops-monitoring-bot) Cookbook cookbooks.sre.wikireplicas.update-views started by ladsgroup completed: - an-redacteddb1001.eqiad.wmnet... [15:21:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11363448 (10JMonton-WMF) Good points! I might need a bit of your help to work on them. I believe we could create a `dt` field in the SELECT clause using `year/month/day/hour`... [16:08:12] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11363562 (10Marostegui) [16:21:47] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Iceberg Merge strategies with dbt - https://phabricator.wikimedia.org/T409099#11363639 (10JAllemandou) Very cool ! Let's schedule some time to work together on the Iceberg time partitioning. [16:29:28] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11363704 (10Marostegui) [17:10:16] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): mediawiki_history_dumps failure - https://phabricator.wikimedia.org/T409470#11363834 (10JAllemandou) This problem is due to the job still using the old jar version after the patches we made changing the schema. I'll provide a patch and deploy it today. [17:18:17] (03PS1) 10Joal: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 [17:19:39] (03PS2) 10Joal: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 [17:21:42] (03CR) 10Snwachukwu: [C:03+2] Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:22:25] (03CR) 10Snwachukwu: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:22:50] (03CR) 10Snwachukwu: "You added application.log file" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:23:51] (03CR) 10Snwachukwu: [C:04-1] Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:23:57] (03CR) 10Snwachukwu: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:27:36] (03CR) 10Mforns: "Agree! changelog LGTM." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:33:30] (03CR) 10CI reject: [V:04-1] Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [17:47:12] (03PS3) 10Joal: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 [17:47:58] (03CR) 10Joal: [C:03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [18:03:44] (03Merged) 10jenkins-bot: Update changelog.md to release v0.3.8 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1203870 (owner: 10Joal) [18:42:27] 06Data-Engineering, 06Research: Request for Hourly Pageview Data for multiple articles– July 18 to September 8, 2025 - https://phabricator.wikimedia.org/T409676#11364355 (10Madocmadofmadog) Hi @Aklapper, do you have an update on the possible status for access for the hourly views? Best, Madocmadofmadog [18:54:32] 06Data-Engineering, 06Research: Request for Hourly Pageview Data for multiple articles– July 18 to September 8, 2025 - https://phabricator.wikimedia.org/T409676#11364417 (10JAllemandou) Hi folks, you can fin pageview hourly dumps here: https://dumps.wikimedia.org/other/pageview_complete It'll be a lot to downl... [19:00:01] Starting build #53 for job analytics-refinery-maven-release [19:13:47] 06Data-Engineering-Radar, 06Experimentation Lab, 07Essential-Work, 10Event-Platform: More strictly validate X-Experiment-Enrollments-Header - https://phabricator.wikimedia.org/T401198#11364484 (10dr0ptp4kt) {T409106} is more prescriptive about improvement to current state handling and error logging. Now, w... [19:14:09] Project analytics-refinery-maven-release build #53: 04FAILURE in 14 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/53/ [19:49:59] Starting build #54 for job analytics-refinery-maven-release [19:50:41] 06Data-Engineering, 06Data-Platform-SRE, 10superset.wikimedia.org: Thanos / Prometheus metrics names with uppercase characters not accessible from Superset SQL Lab or Presto CLI - https://phabricator.wikimedia.org/T409874 (10dr0ptp4kt) 03NEW [19:53:36] 06Data-Engineering, 06Data-Platform-SRE, 10superset.wikimedia.org: Thanos / Prometheus metrics names with uppercase characters not accessible from Superset SQL Lab or Presto CLI - https://phabricator.wikimedia.org/T409874#11364588 (10dr0ptp4kt) [20:06:45] Project analytics-refinery-maven-release build #54: 04STILL FAILING in 16 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/54/ [20:49:32] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11364771 (10Eevans) [23:08:42] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Discovery-Search, 10Wikidata, 10Wikidata-Query-Service, and 2 others: [EPIC] Upgrade flink jobs to java 17 - https://phabricator.wikimedia.org/T404340#11364925 (10tchin) [23:54:19] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Revive data engineering alert metrics dashboard - https://phabricator.wikimedia.org/T399518#11364939 (10Ahoelzl) data-engineering-alerts-monitor bot email is available. However programmatic access through the Gmail API is only poss...