[01:12:29] 06Data-Engineering: [SPIKE] Learn and document how to use Flink-CDC from MediaWiki MariaDB locally - https://phabricator.wikimedia.org/T373144#10147259 (10Ottomata) @NoZeroDay check this out! Paimon 0.9 has been released, and it has an [[ https://paimon.apache.org/releases/release-0.9/#universal-format | Iceber... [01:24:04] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Observability-Metrics, 10Event-Platform, and 3 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10147260 (10Ottomata) Oh, it's POST? With a POST body? or is the data just urlencoded query pa... [06:15:19] 06Data-Engineering, 10CampaignEvents, 10EntitySchema, 10JsonConfig, and 14 others: Add namespace descriptions for Special:NamespaceInfo in WMF-deployed extensions - https://phabricator.wikimedia.org/T373070#10147315 (10Pppery) [08:10:58] 10Data-Engineering (Q1 2024 July 1st - September 30th): Handle Late-Arrived Events from Gobblin into Airflow triggered Refine - https://phabricator.wikimedia.org/T370665#10147571 (10JAllemandou) I ran an analysis over the `/wmf/data/raw/event` folder (event-platform events not yet refined) on Friday 13th of Sept... [08:17:07] 06Data-Engineering, 10Temporary accounts: Prepare EventBus for temp accounts - https://phabricator.wikimedia.org/T374811 (10kostajh) 03NEW [08:19:23] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 10Temporary accounts: Prepare EventLogging for temp accounts - https://phabricator.wikimedia.org/T374812#10147620 (10kostajh) [08:36:18] 10Data-Engineering (Q1 2024 July 1st - September 30th): Handle Late-Arrived Events from Gobblin into Airflow triggered Refine - https://phabricator.wikimedia.org/T370665#10147666 (10dcausse) >>! In T370665#10147571, @JAllemandou wrote: > I think the next steps of this analysis are to investigate why we have late... [08:51:12] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10147729 (10brouberol) A new `airflow-dags` including the library has been published: https://gitlab.wikimedia.org/api/v4/project... [08:54:28] (03PS1) 10Gerrit maintenance bot: Add kge.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073157 (https://phabricator.wikimedia.org/T374815) [08:57:09] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10147813 (10brouberol) ` brouberol@an-test-client1002:~$ wget https://gitlab.wikimedia.org/api/v4/projects/93/packages/generic/ai... [08:57:13] 06Data-Engineering, 10Temporary accounts (Blockers to minor pilot wiki deployment): Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#10147817 (10kostajh) >>! In T347510#10014913, @kostajh wrote: >>>! In T347510#9949406, @lbowmaker wrote:... [09:08:54] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10147865 (10brouberol) Everything seems to be working fine {F57514438} [09:50:25] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Observability-Metrics, 10Event-Platform, and 3 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10148065 (10phuedx) >>! In T355837#10147260, @Ottomata wrote: > Oh, it's POST? With a POST bod... [10:01:42] 06Data-Engineering, 10Temporary accounts, 10Event-Platform: Prepare EventBus for temp accounts - https://phabricator.wikimedia.org/T374811#10148122 (10gmodena) [10:51:39] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10148391 (10brouberol) ` root@apt1002:~# wget https://gitlab.wikimedia.org/api/v4/projects/93/packages/generic/airflow/2.9.3/airf... [12:17:19] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Observability-Metrics, 10Event-Platform, and 3 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10148629 (10colewhite) TIL labels can be extracted from the metric name using `label_replace()`... [13:15:34] hola folks - I have something I don't understand about connecting to our clouddb instance [13:15:57] btullis or brouberol, would any of you have a second for me please? [13:16:32] ask away, we'll see if I can find out [13:17:02] Same here. [13:17:16] I used to be able to connect to clouddb1021.eqiad.wmnet, but now I'm receiving: ERROR 2005 (HY000): Unknown MySQL server host 'clouddb1021.eqiad.wmnet' [13:17:33] Using the CLI mysql [13:17:33] That host has been replaced with an-redacteddb1001.eqiad.wmnet [13:17:40] Ahhhhhh! [13:17:52] When has that happenned? [13:18:25] It was probably while you were away. The name change caused no end of kerfuffle :-) [13:18:48] T365453 [13:18:50] T365453: Bring an-redacteddb1001 into service to replace clouddb1021 - https://phabricator.wikimedia.org/T365453 [13:18:51] of course, it got changed in the code but I forgot to pull the latest version - My bad [13:19:00] Thanks so much btullis for the pointers :) [13:19:15] You're very welcome. [13:28:13] (03PS1) 10Joal: Add 6 new projects to the sqoop list [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073214 (https://phabricator.wikimedia.org/T368788) [13:31:00] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.06 - 2024.09.27), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10148865 (10brouberol) [13:31:38] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform, 13Patch-For-Review: 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10148872 (10JAllemandou) a:03JAllemandou [13:32:45] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform, 13Patch-For-Review: 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10148863 (10JAllemandou) Patch sent (see above). One wiki was already added to the list (`btmwiki`), and another was not yet pres... [13:33:12] (03CR) 10Joal: [V:03+2 C:03+2] "Merging" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1072576 (https://phabricator.wikimedia.org/T374644) (owner: 10Gerrit maintenance bot) [13:33:32] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.06 - 2024.09.27), 13Patch-For-Review: Retry package added needs the types-retry 0.9.9.4 typing stub - https://phabricator.wikimedia.org/T374396#10148858 (10brouberol) 05Open→03Resolved All airflow instances have been... [13:34:20] (03CR) 10Joal: [V:03+2 C:03+2] Add kge.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073157 (https://phabricator.wikimedia.org/T374815) (owner: 10Gerrit maintenance bot) [13:35:32] (03Abandoned) 10Joal: Add bdr.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1059474 (https://phabricator.wikimedia.org/T371760) (owner: 10Gerrit maintenance bot) [13:37:11] (03Abandoned) 10Joal: Add cs.wikivoyage to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1056553 (https://phabricator.wikimedia.org/T370913) (owner: 10Gerrit maintenance bot) [14:16:45] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Flink job to enrich reconciliation events - https://phabricator.wikimedia.org/T368787#10149038 (10Ahoelzl) [14:37:55] 06Data-Engineering, 10Data-Services, 06Trust and Safety Product Team: Hide the value of gb_address column in public replicas if gb_autoblock_parent_id is not null - https://phabricator.wikimedia.org/T371486#10149274 (10Dreamy_Jazz) [14:40:48] 06Data-Engineering, 10Data-Services, 06Trust and Safety Product Team, 10Temporary accounts (Blockers to minor pilot wiki deployment): Hide the value of gb_address column in public replicas if gb_autoblock_parent_id is not null - https://phabricator.wikimedia.org/T371486#10149292 (10Dreamy_Jazz) [14:42:16] 06Data-Engineering, 10Data-Services, 10GlobalBlocking, 06Stewards-and-global-tools, and 2 others: Hide the value of gbw_address in the global_block_whitelist table if the associated gb_id has gb_autoblock_parent_id as not null - https://phabricator.wikimedia.org/T374855#10149306 (10Dreamy_Jazz) [14:45:44] 06Data-Engineering, 10Data-Services, 06Trust and Safety Product Team, 10Temporary accounts (Blockers to minor pilot wiki deployment), 10Trust and Safety Product Sprint (Sprint Beatboxing (Sept 16-27)): Hide the value of gb_address column in public replic... - https://phabricator.wikimedia.org/T371486#10149316 [18:30:53] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Dumps 2.0 (Kanban Board), 10Event-Platform, 10MW-1.43-notes (1.43.0-wmf.22; 2024-09-10), 13Patch-For-Review: [Event Platform] Instrument EventBus with prometheus MW Statslib - https://phabricator.wikimedia.org/T363587#10150491 (10gmodena) >>! In T... [19:06:35] 06Data-Engineering, 06Data Products, 10Dumps 2.0, 10Event-Platform: Consider renaming columns and/or table to abide by the data modeling guidelines - https://phabricator.wikimedia.org/T366542#10150586 (10xcollazo) >>! In T366542#10142079, @xcollazo wrote: > On a meeting yeterday with the #dumps_2.0 team, w... [20:02:57] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Observability-Metrics, 10Event-Platform, and 3 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10150765 (10Krinkle) No, it actually matters in the opposite direction. We have scalable, simpl... [22:30:02] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Dumps 2.0 (Kanban Board): Problem deploying - missing airflow_client dependency - https://phabricator.wikimedia.org/T372014#10151055 (10amastilovic) This was resolved some time ago when SRE released updated Airflow Debian package. [22:46:10] (03CR) 10Milimetric: [C:04-1] "I'd like to block merging this until we discuss the custom data fields." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064438 (https://phabricator.wikimedia.org/T372987) (owner: 10David Martin) [22:47:26] (03CR) 10Milimetric: [V:03+2 C:03+2] "This patch functionally removes fields, so I'll merge it. But I have the same concerns with the custom data fields as in the sister patch" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064435 (https://phabricator.wikimedia.org/T372986) (owner: 10David Martin)