[08:56:43] 06Data-Engineering, 06Data-Platform-SRE: canary events (codfw partition) are not populated for dumps_v1 since 2026-11-11 hour=20 - https://phabricator.wikimedia.org/T444577#12547522 (10KCVelaga_WMF) p:05Triage→03High [09:02:23] Starting build #43 for job analytics-refinery-update-jars [09:04:31] 06Data-Engineering, 06Data-Platform-SRE: canary events (codfw partition) are not populated for dumps_v1 since 2026-11-11 hour=20 - https://phabricator.wikimedia.org/T444577#12547544 (10KCVelaga_WMF) Might be larger issue than dumps, canary events DAG has been failing since that hour. [09:04:35] Project analytics-refinery-update-jars build #43: 04FAILURE in 2 min 11 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/43/ [09:40:21] 06Data-Engineering, 06Wikidata Platform Team: workflow_utils: Support for automatic Changelog generation from git history - https://phabricator.wikimedia.org/T444588 (10trueg) 03NEW [09:40:31] 06Data-Engineering, 06Wikidata Platform Team: workflow_utils: Support for automatic Changelog generation from git history - https://phabricator.wikimedia.org/T444588#12547859 (10trueg) 05Open→03In progress p:05Triage→03Low [09:54:07] 14Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Commons, 06Data-Persistence, 06DBA, and 3 others: FY2025-26 WE 6.4.1: Move links tables of commons to a dedicated cluster - https://phabricator.wikimedia.org/T398709#12547916 (10Wargo) [11:15:05] 06Data-Engineering, 06Data-Platform-SRE: canary events (codfw partition) are not populated for dumps_v1 since 2026-11-11 hour=20 - https://phabricator.wikimedia.org/T444577#12548322 (10BTullis) We are seeing the following error from the Aiflow job. ` [2026-11-11, 20:15:21 UTC] {subprocess.py:106} INFO - Error:... [11:24:09] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06Data-Platform-SRE, 10Test Kitchen, 07Essential-Work: Get schema browser in GrowthBook to work - https://phabricator.wikimedia.org/T440403#12548347 (10APizzata-WMF) Hello everyone! Yes, `apizzata_wmf_psi.stg_bad_faith_edit_reverts_weekly` belon... [15:13:40] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): data lake: server admin log (sal) - https://phabricator.wikimedia.org/T435417#12549392 (10Antoine_Quhen) Thanks @Addshore for the cross-check and the diagram. What "the bot escapes it" means: Before saving to the wiki, Stashbot rewrites the message (... [15:38:04] (03PS1) 10Milimetric: Sqoop wbc_entity_usage from x1 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) [15:38:23] (03CR) 10Milimetric: "haven't tested yet" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) (owner: 10Milimetric) [16:19:26] 14Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10MediaWiki-extensions-Wikibase-Client, 10Wikidata: Move the wbc_entity_usage table onto a dedicated DB shard - https://phabricator.wikimedia.org/T176273#12549770 (10xcollazo) [16:19:28] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10MediaWiki-extensions-Wikibase-Client, 10Wikidata, 10Wikidata Integration in Wikimedia projects, 13Patch-For-Review: Data Lake - sqoop of wikidata_entity_usage needs to adapt to move to x1 Ma... - https://phabricator.wikimedia.org/T431477#12549768 [16:20:04] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10MediaWiki-extensions-Wikibase-Client, 10Wikidata, 10Wikidata Integration in Wikimedia projects, 13Patch-For-Review: Data Lake - sqoop of wikidata_entity_usage needs to adapt to move to x1 Ma... - https://phabricator.wikimedia.org/T431477#12549772 [16:23:02] (03CR) 10Xcollazo: [C:03+1] "LGTM, but I have no experience in this area." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) (owner: 10Milimetric) [16:52:53] (03CR) 10A-pizzata: Sqoop wbc_entity_usage from x1 (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) (owner: 10Milimetric) [16:55:12] (03CR) 10Milimetric: Sqoop wbc_entity_usage from x1 (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) (owner: 10Milimetric) [17:04:40] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1369088 (https://phabricator.wikimedia.org/T431477) (owner: 10Milimetric) [17:40:48] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10EditCheck, 10VisualEditor, 10VisualEditor Suggestion Mode, and 2 others: Emit revision edit_suggestions_counts change as an event data product. - https://phabricator.wikimedia.org/T444212#12550066 (10Ottomata) Okay! @AKhatun_WMF stuff is read... [17:42:07] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10EditCheck, 10VisualEditor, 10VisualEditor Suggestion Mode, and 2 others: Emit revision edit_suggestions_counts change as an event data product. - https://phabricator.wikimedia.org/T444212#12550072 (10Ottomata) [18:34:02] 14Analytics, 06Data-Engineering, 10Data-Engineering-Wikistats: Grand Sheik El Hajj Menelik Allah Alraheem Ali - https://phabricator.wikimedia.org/T444638 (10ElHajjMenelikAllahAlraheemAli) 03NEW [18:37:53] (03CR) 10Xcollazo: [C:03+2] hdfs: Add hdfs_usage_rollup_v1 and hdfs_capacity_v1 tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1363408 (https://phabricator.wikimedia.org/T438056) (owner: 10Xcollazo) [18:37:58] (03CR) 10Xcollazo: [V:03+2 C:03+2] hdfs: Add hdfs_usage_rollup_v1 and hdfs_capacity_v1 tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1363408 (https://phabricator.wikimedia.org/T438056) (owner: 10Xcollazo) [18:51:19] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Dumps-Generation, 13Patch-For-Review: mediawiki-content-pipelines: Gate the File Exports on the ingest of the monthly reconcile - https://phabricator.wikimedia.org/T442415#12550366 (10xcollazo) 05In progress→03Resolved [18:53:56] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Dumps-Generation, 13Patch-For-Review: Data missing from en.wiktionary.org "MediaWiki Content File Exports" compared to "XML Database dump" - https://phabricator.wikimedia.org/T417596#12550372 (10xcollazo) 05In progress→03Resolved [18:56:48] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: Zombie Airflow DagRun try resulted in an Incremental MWH failure - https://phabricator.wikimedia.org/T443787#12550379 (10xcollazo) 05In progress→03Resolved [21:40:40] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): KAPOW initial backfill - https://phabricator.wikimedia.org/T442233#12551109 (10mforns) [21:45:08] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Event-Platform: Seeing frequent 504 timeouts from the event service lose api-request events, erroring with "Invalid event service response body" - https://phabricator.wikimedia.org/T440247#12551122 (10tchin) Looping back on this, went to take a lo... [21:46:44] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): KAPOW initial backfill - https://phabricator.wikimedia.org/T442233#12551129 (10mforns)