[08:57:53] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10574212 (10brouberol) ` brouberol@cephosd1001:~$ sudo radosgw-admin user create --uid=airflow-main --display-name="airflow-main" {... [08:58:43] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10574215 (10brouberol) ` brouberol@stat1008:~$ s3cmd --access_key=$access_key --secret_key=$secret_key --host=rgw.eqiad.dpe.anycast.wmn... [08:59:38] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10574216 (10brouberol) ` brouberol@cephosd1001:~$ sudo radosgw-admin user create --uid=postgresql-airflow-main --display-name="postgres... [08:59:46] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10574217 (10brouberol) ` brouberol@stat1008:~$ s3cmd --access_key=$access_key --secret_key=$secret_key --host=rgw.eqiad.dpe.anycast.wmn... [09:04:51] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10574219 (10brouberol) ` brouberol@krb1001:~$ sudo kadmin.local addprinc -randkey analytics/airflow.discovery.wmnet@WIKIMEDIA brouberol... [09:48:36] (03CR) 10Aqu: [C:04-1] "Looks good, but I think there is a typo." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1117633 (https://phabricator.wikimedia.org/T384650) (owner: 10Conniecc1) [11:45:05] \quit [11:45:23] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 06MediaWiki-Platform-Team, and 3 others: Drop module_deps table in WMF prod - https://phabricator.wikimedia.org/T385997#10574865 (10Ladsgroup) a:03Ladsgroup Dropped it in cebwiki. Tomorrow I will drop it in all of s5. [14:10:23] 06Data-Engineering, 06Research, 13Patch-For-Review: A dataset sensor should work indepent of airflow instance - https://phabricator.wikimedia.org/T386973#10575165 (10xcollazo) >>! In T386973#10572403, @CodeReviewBot wrote: > amastilovic opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/... [15:42:25] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10575516 (10brouberol) `airflow-main` is deployed! ` brouberol@deploy2002:~$ kube_env airflow-main-deploy dse-k8s-eqiad brouberol@depl... [16:12:57] 10Data-Engineering (Q3 2024 January 1st - March 31th), 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10575693 (10brouberol) The UI is accessible on my end, I've been able to log-in and run a DAG: https://airflow.wikimedia.org/dags/purge... [17:05:02] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10575902 (10Gehel) [17:05:13] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 10Discovery-Search (2025.02.10 - 2025.02.28): Upload an image with flink-k8s-operator version that supports flink 1.20 - https://phabricator.wikimedia.org/T377137#10575903 (10Gehel) [18:15:36] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Data Pipelines, 10Observability-Metrics, 07Essential-Work, and 2 others: Disable Data Platform Engineering generated graphite metrics and dashboards - https://phabricator.wikimedia.org/T372855#10576204 (10Ahoelzl) [18:17:03] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work, 10Event-Platform: Gobblin-wmf Gitlab migration and maintenance - https://phabricator.wikimedia.org/T370368#10576226 (10Ahoelzl) [18:17:20] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work, 13Patch-For-Review: Migrate analytics Airflow DAGs to k8s Airflow deployment - https://phabricator.wikimedia.org/T386282#10576230 (10Ahoelzl) [18:17:42] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE-Mediawiki-Content: List out all migration candidates for mediawiki_content_history - https://phabricator.wikimedia.org/T386757#10576234 (10Ahoelzl) [18:17:58] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Update eventstreams Grafana Dashboards to fix GC metrics - https://phabricator.wikimedia.org/T386709#10576235 (10Ahoelzl) [18:18:16] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: [Data Quality] Implement wiki completeness check for MediaWiki History - https://phabricator.wikimedia.org/T365203#10576242 (10Ahoelzl) [18:21:55] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE HAProxy Migration, 13Patch-For-Review: [HAProxy migration] HAProxy and VarnishKafka should produce compatible datasets - https://phabricator.wikimedia.org/T382571#10576273 (10Ahoelzl) [18:22:01] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE HAProxy Migration: [HAProxy migration] Validate `pageview` and `unique_devices` generated from `webrequest_frontend` - https://phabricator.wikimedia.org/T386343#10576274 (10Ahoelzl) [19:11:45] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 10Charts (Sprint 17), 07Schema-change-in-production: Deploy patch-gjlw_namespace_text.sql on x1.commonswiki for JsonConfig - https://phabricator.wikimedia.org/T385917#10576549 (10CCiufo-WMF) [19:31:00] 06Data-Engineering, 06Data-Engineering-Radar, 10Dumps-Generation, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), and 3 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10576615 (10Gehel) [19:58:15] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [20:00:54] !log Doing a refinery deployment (refinery only, not refinery-source). T384531. [20:00:57] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:00:57] T384531: WikiLambda metrics: Repair the allowlist for mediawiki_product_metrics_wikifunctions_ui; backfill missing data as needed - https://phabricator.wikimedia.org/T384531 [20:34:06] !log Deployed refinery using scap, then deployed onto hdfs [20:34:08] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:45:35] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Update eventstreams Grafana Dashboards to fix GC metrics - https://phabricator.wikimedia.org/T386709#10576940 (10tchin) There doesn't seem to be a `service` label on these GC metrics so I used `kubernetes_namespace` instead. GC Rate Aver... [20:55:22] (03PS2) 10Conniecc1: Add the Incident reporting System interaction schema to the allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1117633 (https://phabricator.wikimedia.org/T384650) [21:30:13] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Growth-Structured-Tasks, 06Growth-Team, 10Image-Suggestions, and 7 others: wmf.wikidata_item_page_link and wmf.wikidata_entity snapshots stuck at 2025-01-20 - https://phabricator.wikimedia.org/T386255#10577063 (10xcollazo) >>! In T386255#10568434, @x... [21:58:13] 06Data-Engineering, 10Commons-Impact-Metrics-Requests: Request for Images_from_Wiki_Loves_Africa_2024 - https://phabricator.wikimedia.org/T381352#10577170 (10Anthere) I finally tested today. Works pretty well. I have not tested all options... but several of those of interest to me and that was ok ! Will ex... [22:38:56] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Abstract Wikipedia team, 10function-evaluator, 10function-orchestrator, 07Essential-Work: WF service logging seems to be partially missing - https://phabricator.wikimedia.org/T386972#10577353 (10Ahoelzl) [22:39:09] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10577354 (10Ahoelzl) [22:39:25] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work, 10Event-Platform, 13Patch-For-Review: Upgrade eventgate-wikimedia to node20 - https://phabricator.wikimedia.org/T383814#10577355 (10Ahoelzl)