[08:48:58] 06Data-Engineering, 06MW-Interfaces-Team, 10Event-Platform: Make DomainEvents serializable - https://phabricator.wikimedia.org/T379936#10325563 (10daniel) p:05Triage→03Low [08:51:14] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10325572 (10daniel) p:05Triage→03High [08:51:22] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10325574 (10daniel) p:05High→03Medium [08:52:07] 06Data-Engineering, 06MW-Interfaces-Team, 10Event-Platform: Explore a mechanism for publishing domain events to an event bus - https://phabricator.wikimedia.org/T379935#10325576 (10daniel) p:05Triage→03Medium [08:54:09] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06MW-Interfaces-Team, 13Patch-For-Review: Explore mechanism for publishing domain events - https://phabricator.wikimedia.org/T378933#10325582 (10daniel) p:05Triage→03Medium [11:28:41] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10326135 (10BTullis) a:03BTullis [11:40:41] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10326165 (10BTullis) Let's target version 3.5.3. I'll start making... [11:48:41] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Upgrade pyspark within the Airflow containers to use pyspark version 3.5.3 - https://phabricator.wikimedia.org/T380033 (10BTullis) 03NEW [11:56:01] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10326252 (10BTullis) [11:56:41] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Create Airflow containers with pyspark version 3.5.3 - https://phabricator.wikimedia.org/T380033#10326256 (10BTullis) [11:59:31] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Create Spark docker images for version 3.5.3 - https://phabricator.wikimedia.org/T380035 (10BTullis) 03NEW [12:23:02] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Create Spark docker images for version 3.5.3 - https://phabricator.wikimedia.org/T380035#10326373 (10BTullis) [12:24:52] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Create Spark docker images for version 3.5.3 - https://phabricator.wikimedia.org/T380035#10326399 (10BTullis) [12:25:04] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data-Platform-SRE (2024.11.09 - 2024.11.29): Create Spark docker images for version 3.5.3 - https://phabricator.wikimedia.org/T380035#10326402 (10BTullis) a:03BTullis [12:54:57] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.x and Hive to version 3.1.x - https://phabricator.wikimedia.org/T379385#10326538 (10JAllemandou) Something not to forget about is that this upgrade is also about upgrading to Java 11! [12:57:11] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.x and Hive to version 3.1.x - https://phabricator.wikimedia.org/T379385#10326571 (10brouberol) @JAllemandou do we //have// to upgrade to JDK11 at the same time, or can we phase in the client JDK upgrades after we upgrade Hadoop it... [15:36:38] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10327116 (10daniel) > What are the Sink and Source interfaces for? The reason for having separate int... [15:40:00] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10327138 (10daniel) > This would allow you to wire Subscriber classes with a ListenerRegistry, instea... [15:45:23] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10327174 (10daniel) > I wonder if a channel concept in the interfaces will be useful. We can default... [16:03:54] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 10Metrics Platform: Remove deprecated $logger parameter from EventLogging::submit() - https://phabricator.wikimedia.org/T376410#10327311 (10Milimetric) p:05Triage→03Medium [16:03:55] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 10Metrics Platform: Remove deprecated $logger parameter from EventLogging::submit() - https://phabricator.wikimedia.org/T376410#10327312 (10Milimetric) [18:15:38] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 10Event-Platform, and 2 others: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10327898 (10Ottomata) > In the end, ListenerProvider::getListeners() and Subscriber::registerListener... [21:08:06] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.x and Hive to version 3.1.x - https://phabricator.wikimedia.org/T379385#10328367 (10BTullis) >>! In T379385#10326538, @JAllemandou wrote: > Something not to forget about is that this upgrade is also about upgrading to Java 11! OK...