[00:37:43] 06Data-Engineering, 06Growth-Team, 06MediaWiki-Engineering, 10MediaWiki-extensions-WikimediaEvents, and 7 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10417760 (10Krinkle) A few updates: * The infra is now ready for use and available on all wikis a... [00:42:24] 06Data-Engineering, 06Growth-Team, 06MediaWiki-Engineering, 10MediaWiki-extensions-WikimediaEvents, and 7 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10417771 (10Krinkle) 05Open→03Resolved Example of first migrant: {T382549}. [07:44:03] 10Data-Engineering (Q2 2024 October 1st - December 31th): Implement a data retention policy for webrequest_frontend datasets - https://phabricator.wikimedia.org/T379024#10418069 (10gmodena) [07:54:41] 10Data-Engineering (Q2 2024 October 1st - December 31th): Haproxy kafka and varnishkafka produce compatible datasets - https://phabricator.wikimedia.org/T382571 (10gmodena) 03NEW [10:39:35] 06Data-Engineering, 10Structured-Data-Backlog (Current Work): [L] Track commons deletion requests - https://phabricator.wikimedia.org/T370898#10418330 (10Cparle) Hi @amastilovic ... I'm not sure we need anything deployed to HDFS, I've already created the database we're writing to (`mediawiki_upload_tracking`,... [11:05:42] !log restarted hadoop-mapreduce-historyserver.service on an-master1003 for T382575 [11:05:44] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:05:45] T382575: MapReduce history server is repeatedly crashing - https://phabricator.wikimedia.org/T382575 [12:52:42] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 06Experimentation Lab, 06Movement-Insights, 13Patch-For-Review: Backfill and recalculate unique devices data from July 2024 to present - https://phabricator.wikimedia.org/T378852#10418557 (10Ahoelzl) 1st stage backfill completed f... [12:53:30] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 06Experimentation Lab, 06Movement-Insights, 13Patch-For-Review: Backfill and recalculate unique devices data from July 2024 to present - https://phabricator.wikimedia.org/T378852#10418562 (10Ahoelzl) [12:54:10] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Movement-Insights: Temporarily Extend Retention Window for webrequest tables - https://phabricator.wikimedia.org/T375943#10418571 (10Ahoelzl) [12:54:30] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Movement-Insights: Temporarily Extend Retention Window for webrequest tables - https://phabricator.wikimedia.org/T375943#10418572 (10Ahoelzl) Reverted back to 90 days. [13:27:29] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Research, 10Data-Platform-SRE (2024.11.30 - 2024.12.20), 03Discovery-Search (Current work): Low available space on Hadoop / HDFS - https://phabricator.wikimedia.org/T381707#10418666 (10Ahoelzl) Reclaiming additionally retained webrequest data got... [14:04:05] !log roll-restarting hadoop nameservers to pick up https://gerrit.wikimedia.org/r/c/operations/puppet/+/1105893 [14:04:06] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:41:36] !log Started the namenode service on an-master1003 after crash [14:41:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:59:49] 06Data-Engineering, 10Structured-Data-Backlog (Current Work): [L] Track commons deletion requests - https://phabricator.wikimedia.org/T370898#10418836 (10mfossati) Review done: https://gitlab.wikimedia.org/repos/structured-data/upload-tracking/-/merge_requests/3 Moving back to ready for dev [15:02:09] 06Data-Engineering, 10Wikidata, 03Discovery-Search (Current work), 10Event-Platform, 13Patch-For-Review: Configure https://stream.wikimedia.org to expose rdf-streaming-updater.mutation - https://phabricator.wikimedia.org/T374921#10418844 (10dcausse) a:03dcausse [15:39:02] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 10Dumps-Generation: 20241201 wikidatawiki xml dump not progressing - https://phabricator.wikimedia.org/T382084#10418924 (10xcollazo) Looks like last logs from `wikidatawiki` `2024-12-01` run are from `2024-12-12`: ` xcollazo@snapshot1... [15:51:00] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 10Dumps-Generation: 20241201 wikidatawiki xml dump not progressing - https://phabricator.wikimedia.org/T382084#10418942 (10xcollazo) Let's try and finish it. `snapshot1011` will soon be busy with the `2024-12-20` partial run, so let's... [16:48:09] !log Deploying latest analytics Airflow instance DAGs. T377852. [16:48:12] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:48:12] T377852: Tune Reconciliation mechanism to do historic runs (all revisions, all wikis) - https://phabricator.wikimedia.org/T377852 [19:46:01] !log failed over the hadoop namenode services from an-master1004 to an-master1003. [19:46:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:46:44] !log restarted the hadoop-hdfs-namenode service on an-master1004 to pick up the new settins as well. [19:46:46] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log