[05:48:02] milimetric: phuedx: FYI https://people.wikimedia.org/~krinkle/T403507-head-pixel.html, tags in the head are invalid html, and causes some weird but subtle side-effects in which metadata is no longer perceived by HTML parsers and browsers as in the , but forced into the [05:48:25] compare to https://people.wikimedia.org/~krinkle/T403507-head-pixel-fixed.html which is identical, except for the img tag [07:04:02] brett: the main cluster is managed by ServiceOps, Data Platform deals with the jumbo cluster [10:03:00] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.11.07 - 2025.11.28): airflow-dags shorten build/test time - https://phabricator.wikimedia.org/T406874#11373617 (10brouberol) 05In progress→03Resolved [10:07:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.11.07 - 2025.11.28): airflow-dags shorten build/test time - https://phabricator.wikimedia.org/T406874#11373643 (10brouberol) {F70211882} The red bar is when we deployed [10:31:49] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Event-Platform: Tests depending on EventBus fail when EventStreamConfig is not loaded - https://phabricator.wikimedia.org/T410122 (10hashar) 03NEW [11:44:48] 06Data-Engineering, 06Traffic, 07Essential-Work, 10Experimentation Lab Radar: Cookie % has been rejected because it is foreign and does not have the "Partitioned" attribute - https://phabricator.wikimedia.org/T375256#11374053 (10Back_ache) {F70214945} added screenshot for clarity and to show this is still... [14:53:07] 06Data-Engineering, 06Machine-Learning-Team, 06serviceops: Enable ChangeProp to consume mediawiki.page_content_change.v1 - https://phabricator.wikimedia.org/T409469#11374466 (10JMonton-WMF) > @Ottomata, what work is required to produce mediawiki.page_content_change.v1 to Kafka main? I'm expecting just some h... [15:19:53] 06Data-Engineering, 06Machine-Learning-Team, 06serviceops: Enable ChangeProp to consume mediawiki.page_content_change.v1 - https://phabricator.wikimedia.org/T409469#11374579 (10dcausse) If pushing to kafka-main you might need to increase broker's `message.max.bytes` see T344688. [15:24:34] 06Data-Engineering, 06Machine-Learning-Team, 06serviceops: Enable ChangeProp to consume mediawiki.page_content_change.v1 - https://phabricator.wikimedia.org/T409469#11374597 (10tchin) Would we also need to explicitly create the topics in main? Is auto topic creation enabled there? [15:57:01] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Java-Scala-Standardization, 06Discovery-Search (2025.10.20 - 2025.11.07), 07Essential-Work: Ignore MacOS .DS_Store in parent pom - https://phabricator.wikimedia.org/T407514#11374716 (10pfischer) Done, 1.98 is released (externally and internally) [16:08:25] (03PS30) 10Aleksandar Mastilovic: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [16:08:45] (03CR) 10Aleksandar Mastilovic: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [16:52:20] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019#11374944 (10xcollazo) Following T321736#8353296 we added a 60GB volume to keep all the docker images: ` ssh -J xcollazo@bastion.wmcloud.org xcollazo@gitlab-docker-... [17:16:21] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019#11375012 (10xcollazo) Even though applying the `docker` puppet class had failed before (T410083#11372512), applying it now (via configs in horizon) was succeessful... [17:29:35] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019#11375029 (10xcollazo) Similar to T321736#8357399: ` $ sudo cat /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 connection_max_age = "15m0s" shut... [17:33:43] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019#11375058 (10xcollazo) (Deleted old `conda-analytics` packages from https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/packages) [17:35:28] 06Data-Engineering, 06Machine-Learning-Team, 06serviceops: Enable ChangeProp to consume mediawiki.page_content_change.v1 - https://phabricator.wikimedia.org/T409469#11375060 (10dcausse) >>! In T409469#11374597, @tchin wrote: > Would we also need to explicitly create the topics in main? Is auto topic creation... [18:00:09] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 2 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11375122 (10bking) 05Open→03Invalid a:05RKemper→03bking [18:03:06] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 2 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11375132 (10MoritzMuehlenhoff) This is this task closed as invalid? This has broken (and still breaks)... [18:12:28] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 2 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11375142 (10bking) @Muehlenhoff Apologies, I was taking the ticket for myself to add the above patch.... [18:12:49] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 2 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11375143 (10bking) 05Invalid→03In progress [19:21:49] (03CR) 10Xcollazo: [C:03+1] "LGTM, with the suggested comment on the CREATE statements." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [19:37:24] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Global Editor Metrics - Data Pipeline - https://phabricator.wikimedia.org/T405039#11375362 (10amastilovic) OK so I've now officially backfilled the `wmf_contributors.` and `wmf_reader... [20:07:37] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019#11375434 (10xcollazo) Currently stock on what seems to be a proxying issue: Failure example: https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-... [20:08:32] 06Data-Engineering, 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375437 (10xcollazo) 05Open→03In progress p:05Triage→03Medium [20:08:45] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375440 (10xcollazo) [20:13:01] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375466 (10xcollazo) From cloudvps instance itself we can reach the offending hosts: ` $ hostname -f gitlab-docker-runner-v2.analytic... [20:13:46] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375477 (10xcollazo) a:03xcollazo [20:22:53] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375501 (10xcollazo) Ok I can repro directly on by creating a manual container from the docker image: ` xcollazo@gitlab-docker-runner... [20:34:58] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07Essential-Work: Upgrade gitlab-docker-runner to latest debian version - https://phabricator.wikimedia.org/T408019#11375553 (10xcollazo) Tried setting a proxy via https://wikitech.wikimedia.org/wiki/HTTP_proxy, but no luck either. [21:01:55] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): SDS 1.3.6 SPUR bot detection analysis - https://phabricator.wikimedia.org/T407103#11375644 (10Antoine_Quhen) [21:04:45] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): SDS 1.3.6 SPUR bot detection analysis - https://phabricator.wikimedia.org/T407103#11375647 (10Antoine_Quhen) Waiting for client side signal for more Spur.us dataset evaluations. In the meantime we may study using only the some proxy providers.