[00:17:13] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to data and Superset for Daria-WMDE (Daria Ammalainen (WMDE)) - https://phabricator.wikimedia.org/T420716#11752206 (10KFrancis) Hi all, the NDA is complete. Thanks! [00:17:34] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to superset for alice.moutinho - https://phabricator.wikimedia.org/T420751#11752207 (10KFrancis) Hi all, the NDA is complete. Thanks! [00:18:20] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for keren.ramirezWMDE - https://phabricator.wikimedia.org/T420896#11752209 (10KFrancis) Hi all, the NDA is complete. Thanks! [00:45:57] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 06Editing-team, and 2 others: EditPage save hooks pass an entire `EditPage` object - https://phabricator.wikimedia.org/T251588#11752242 (10SomeRandomDeveloper) Regarding the edit constraint system, I think something like this would be a good... [02:35:55] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to data and Superset for Daria-WMDE (Daria Ammalainen (WMDE)) - https://phabricator.wikimedia.org/T420716#11752333 (10Scott_French) [02:36:11] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to data and Superset for Daria-WMDE (Daria Ammalainen (WMDE)) - https://phabricator.wikimedia.org/T420716#11752334 (10Scott_French) Great, thank you! I should be able to wrap this up tomorrow. [02:36:58] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Superset for keren.ramirezWMDE - https://phabricator.wikimedia.org/T420896#11752335 (10Scott_French) [02:37:08] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to superset for alice.moutinho - https://phabricator.wikimedia.org/T420751#11752336 (10Scott_French) [02:39:22] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to superset for alice.moutinho - https://phabricator.wikimedia.org/T420751#11752337 (10Scott_French) [03:37:15] FIRING: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [03:42:15] RESOLVED: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [07:58:02] 06Data-Engineering, 06Data-Engineering-Radar, 10Data Pipelines, 10Pageviews-Anomaly, and 3 others: Analyze possible bot traffic for frwiki article Cookie (informatique) - https://phabricator.wikimedia.org/T313114#11752544 (10hashar) [09:44:00] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform: Update HTML pipeline schema - rendering_content_change - https://phabricator.wikimedia.org/T421341 (10JMonton-WMF) 03NEW [10:05:52] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10MediaWiki-extensions-WikimediaEvents, and 3 others: Could not hoist data into experiment.subject_id for event - https://phabricator.wikimedia.org/T421152#11752836 (10Sgs) [10:42:25] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 06Growth-Team, 10Image-Suggestions: Add an Image: filtering by suggestion "kind" or "confidence" - https://phabricator.wikimedia.org/T368987#11752952 (10APizzata-WMF) ALIS run correctly and here the results: ` spark.sql( """ SELECT ta... [11:00:11] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10MediaWiki-extensions-WikimediaEvents, and 4 others: Could not hoist data into experiment.subject_id for event - https://phabricator.wikimedia.org/T421152#11753017 (10Manvikesarwani09) I have uploaded a patchset . can you review my patchset. [11:35:01] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity, 07Schema-change-in-production: Drop cusi_case, cusi_signal, and cusi_user tables from wikis where they are unused - https://phabricator.wikimedia.org/T421353 (10Dreamy_Jazz) 03NEW [11:36:22] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity, 07Schema-change-in-production: Drop cusi_case, cusi_signal, and cusi_user tables from wikis where they are unused - https://phabricator.wikimedia.org/T421353#11753125 (10Dreamy_Jazz) [11:59:02] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to data and Superset for Daria-WMDE (Daria Ammalainen (WMDE)) - https://phabricator.wikimedia.org/T420716#11753179 (10hnowlan) 05Open→03In progress Merged, thank you @Scott_French! [12:01:25] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Superset for keren.ramirezWMDE - https://phabricator.wikimedia.org/T420896#11753187 (10hnowlan) 05Open→03In progress Merged, thank you Scott! [12:05:26] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to superset for alice.moutinho - https://phabricator.wikimedia.org/T420751#11753193 (10hnowlan) 05Open→03In progress Merged, thank you Scott! [14:20:16] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform: Fix PyFlink log levels - https://phabricator.wikimedia.org/T419997#11753964 (10JMonton-WMF) Here we have another example where the logs seem to be produced in multiple lines, rather than JSON, making Logstash to count them as individua... [14:30:15] !log Test Kitchen edge-unique experiments (poll 29587) - adds: none; removes: synth-test-new-external-path; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [14:30:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:29:54] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to data and Superset for Daria-WMDE (Daria Ammalainen (WMDE)) - https://phabricator.wikimedia.org/T420716#11754466 (10Scott_French) 05In progress→03Resolved a:03Scott_French @Daria-WMDE - Your access sho... [15:30:03] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to superset for alice.moutinho - https://phabricator.wikimedia.org/T420751#11754469 (10Scott_French) 05In progress→03Resolved a:03Scott_French @Alice.moutinho - Your access should be live now. Please reo... [15:30:10] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to Superset for keren.ramirezWMDE - https://phabricator.wikimedia.org/T420896#11754472 (10Scott_French) 05In progress→03Resolved a:03Scott_French @kera_wmde - Your access should be live now. Please reope... [15:44:26] 06Data-Engineering, 06Data-Engineering-Radar, 10FR-Tech-Analytics, 06Data-Platform-SRE (2026-03-06 - 2026-03-27), 13Patch-For-Review: Create FR Tech Airflow instance - https://phabricator.wikimedia.org/T417213#11754567 (10brouberol) ` btullis@seaborgium:~$ ldapadd -f airflow-fr-tech-ops.ldif -D "cn=admin... [15:44:58] 06Data-Engineering, 06Data-Engineering-Radar, 10FR-Tech-Analytics, 06Data-Platform-SRE (2026-03-06 - 2026-03-27), 13Patch-For-Review: Create FR Tech Airflow instance - https://phabricator.wikimedia.org/T417213#11754573 (10brouberol) {F73766678} All good and deployed! [15:45:08] 06Data-Engineering, 06Data-Engineering-Radar, 10FR-Tech-Analytics, 06Data-Platform-SRE (2026-03-06 - 2026-03-27), 13Patch-For-Review: Create FR Tech Airflow instance - https://phabricator.wikimedia.org/T417213#11754576 (10brouberol) 05In progress→03Resolved [16:06:57] 06Data-Engineering, 10Event-Platform: mediawiki.page_change.v1 event - add a page type field - https://phabricator.wikimedia.org/T409462#11754682 (10Ottomata) @JAllemandou I think this is semi-relevant to the discussion we were just having about [[ https://docs.google.com/spreadsheets/d/1ug6litAUY300nZVE5lNHma... [16:13:42] 06Data-Engineering, 10Event-Platform: mediawiki.page_change.v1 event - add a page type field - https://phabricator.wikimedia.org/T409462#11754747 (10JAllemandou) This is indeed very relevant @Ottomata . If we could have this info in the event it'd be very useful. [16:37:59] 06Data-Engineering, 06Discovery-Search, 06serviceops-radar, 10Event-Platform: [Event Platform] Store Flink HA metadata in Zookeeper - https://phabricator.wikimedia.org/T331283#11754842 (10bking) @Ottomata [[ https://wikimedia.slack.com/archives/C055QGPTC69/p1774283000849439?thread_ts=1774275213.488269&cid=... [17:00:06] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10Event-Platform: Profile SimpleEditType to identify inefficiencies in mwedittype - https://phabricator.wikimedia.org/T421412 (10AKhatun_WMF) 03NEW [17:00:54] 06Data-Engineering, 10Event-Platform: Profile SimpleEditType to identify inefficiencies in mwedittype - https://phabricator.wikimedia.org/T421412#11754911 (10AKhatun_WMF) [17:11:41] 06Data-Engineering, 06Data-Engineering-Radar, 10FR-Tech-Analytics, 06Data-Platform-SRE (2026-03-06 - 2026-03-27), 13Patch-For-Review: Create FR Tech Airflow instance - https://phabricator.wikimedia.org/T417213#11754936 (10IAckerman-WMF) Awesome! [17:17:41] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): The revision_seconds_to_identity_revert field in wmf.mediawiki_history has sometimes negative values - https://phabricator.wikimedia.org/T419267#11754948 (10mforns) @Ahoelzl @GGoncalves-WMF @CMyrick-WMF reached out to us, interested in this task, since sh... [17:29:36] 06Data-Engineering, 06Discovery-Search, 06serviceops-radar, 10Event-Platform: [Event Platform] Store Flink HA metadata in Zookeeper - https://phabricator.wikimedia.org/T331283#11754959 (10Ottomata) > the operator doesn't know about ZK; it only checks the K8s-native HA Hm, perhaps! I'm not sure though. II... [17:54:42] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): The revision_seconds_to_identity_revert field in wmf.mediawiki_history has sometimes negative values - https://phabricator.wikimedia.org/T419267#11755034 (10Tchanders) This will be a dependency for PSI's work too, since we're planning to base some importa... [18:25:07] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 10AQS2.0: Consider updating our heuristics for media type classification in AQS / wikistats - https://phabricator.wikimedia.org/T419882#11755251 (10Snwachukwu) I ran some investigation on case container "ogg" and and here are my finding for a full-month... [19:28:54] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Schedule three new monthly DBT models for Movement Insights - https://phabricator.wikimedia.org/T420069#11755584 (10amastilovic) Update: I've finally figured out the reason this was failing in Airflow - the skein driver was running out of memory and silen... [19:29:27] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Schedule three new monthly DBT models for Movement Insights - https://phabricator.wikimedia.org/T420069#11755640 (10amastilovic) 05Open→03Resolved [19:33:06] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Move all currently scheduled DBT DAGs to the `dbt_scheduled` Airflow DAGs - https://phabricator.wikimedia.org/T421434 (10amastilovic) 03NEW [19:33:43] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th): Move all currently scheduled DBT DAGs to the `dbt_scheduled` Airflow DAGs - https://phabricator.wikimedia.org/T421434#11755675 (10amastilovic) [21:31:17] 06Data-Engineering (Q3 FY25/26 January 1st - March 31th), 07OKR-Work (WE1 FY2025-26): Schedule three new monthly DBT models for Movement Insights - https://phabricator.wikimedia.org/T420069#11756149 (10ldelench_wmf) [22:21:48] FIRING: EventgateLatency: Elevated latency for POST events on eventgate-analytics-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-analytics-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLatency [22:26:48] RESOLVED: EventgateLatency: Elevated latency for POST events on eventgate-analytics-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-analytics-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLatency [22:34:27] 06Data-Engineering, 06Data-Engineering-Radar, 10MediaWiki-extensions-EventLogging, 07Essential-Work, 06Test Kitchen (Experiment Platform Sprint 21): Migrate "WikiLambda API" instrument to use the Test Kitchen SDK - https://phabricator.wikimedia.org/T415254#11756329 (10Sfaci) a:03Sfaci