[00:21:56] 06Data-Engineering, 06Product-Analytics, 10Wmfdata-Python: Wmfdata-Python triggers a Pandas warning during mariadb.run and hive.run - https://phabricator.wikimedia.org/T324135#10270794 (10nshahquinn-wmf) p:05Triage→03Medium [01:15:19] 06Data-Engineering, 10Wmfdata-Python: Wmfdata does not specify tabulate as a dependency - https://phabricator.wikimedia.org/T378430 (10nshahquinn-wmf) 03NEW p:05Triage→03Medium [01:15:35] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python: Wmfdata does not specify tabulate as a dependency - https://phabricator.wikimedia.org/T378430#10270857 (10nshahquinn-wmf) [01:35:14] 06Data-Engineering, 10Wmfdata-Python: Wmfdata's Kerberos check should not fail if there's a valid keytab - https://phabricator.wikimedia.org/T378432 (10nshahquinn-wmf) 03NEW p:05Triage→03Low [09:28:37] 06Data-Engineering, 10Data-Engineering-Wikistats, 10Data Pipelines, 10LPL Technical Support, and 2 others: Merge ks-Arab and ks-Deva to ks - https://phabricator.wikimedia.org/T314476#10271392 (10MaryMunyoki) [09:39:51] (03CR) 10Gmodena: gobblin: add webrequest_frontend pull job. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [09:54:44] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Movement-Insights: Temporarily Extend Retention Window for webrequest tables - https://phabricator.wikimedia.org/T375943#10271479 (10Aklapper) @Ahoelzl / @Jallemandou: This open task has `Unbreak Now!` priority (["needs to be fixed immediately, settin... [10:12:21] FYI, an-worker1165 is down since ~ 4 hours [10:12:40] looking at the SEL there's a broken DIMM module which took it down [10:14:23] created https://phabricator.wikimedia.org/T378454 to have DC ops look at it, but this might need additional steps on the Hadoop level until the hardware is fixed? [10:42:04] (03CR) 10Joal: [C:03+1] gobblin: add webrequest_frontend pull job. (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [11:58:30] (03PS1) 10Gerrit maintenance bot: Add tcy.wiktionary to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1084071 (https://phabricator.wikimedia.org/T378463) [11:58:53] (03PS1) 10Gerrit maintenance bot: Add tcy.wikisource to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1084074 (https://phabricator.wikimedia.org/T378471) [14:22:41] (03CR) 10Gmodena: gobblin: add webrequest_frontend pull job. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [14:32:46] joal: o/ [14:32:58] I noticed https://grafana.wikimedia.org/d/000000585/hadoop?orgId=1&var-hadoop_cluster=analytics-hadoop&viewPanel=54&from=now-24h&to=now while checking alerts [14:33:15] known? Looks like something is bombarding hdfs [14:34:05] Hi elukey [14:34:51] we're receiving alerts and I've been checking with xcollazo. I think it's the dumps2 jobs (so many files to play with ...) [14:35:19] I also monitor other possibly failing jobs, as HDFS is slower [14:35:36] anyhow, thanks for the ping on this elukey [14:35:55] ahhh okok :) [16:09:46] 06Data-Engineering, 10Event-Platform, 10Web Team Essential Work 2024 (Migrate to new Event Platform), 10Web-Team-Backlog (FY2024-25 Q2 Sprint 2): Deprecate use of desktop- and mobilewebuiactions in Event Platform - https://phabricator.wikimedia.org/T368678#10273551 (10ovasileva) 05Open→03Resolved [16:30:55] (03PS4) 10Snwachukwu: Add Query for cu_log table in sqoop. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082515 (https://phabricator.wikimedia.org/T364398) [16:37:26] (03PS3) 10Mforns: Shift is_redirect_to_pageview upstream to webrequest [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1078733 (https://phabricator.wikimedia.org/T375527) (owner: 10Milimetric) [16:38:56] (03CR) 10Mforns: [V:03+2] Shift is_redirect_to_pageview upstream to webrequest (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1078733 (https://phabricator.wikimedia.org/T375527) (owner: 10Milimetric) [17:36:51] (03PS2) 10Mforns: Modify the automated traffic detection pipeline to include redirects [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082543 (https://phabricator.wikimedia.org/T375527) [17:58:47] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform-SRE, 06SRE, 13Patch-For-Review: Streamline Data Platform access approvals for WMF staff - https://phabricator.wikimedia.org/T370424#10274131 (10Ottomata) Merged! [18:03:50] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Movement-Insights: Temporarily Extend Retention Window for webrequest tables - https://phabricator.wikimedia.org/T375943#10274171 (10Mayakp.wiki) p:05Unbreak!→03High I changed the priority to High since the task of extending the retention window w... [18:15:42] (03PS4) 10Mforns: Shift is_redirect_to_pageview upstream to webrequest [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1078733 (https://phabricator.wikimedia.org/T375527) (owner: 10Milimetric) [18:16:31] (03CR) 10Mforns: [V:03+2] Shift is_redirect_to_pageview upstream to webrequest [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1078733 (https://phabricator.wikimedia.org/T375527) (owner: 10Milimetric) [18:41:40] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform-SRE, 06SRE: Streamline Data Platform access approvals for WMF staff - https://phabricator.wikimedia.org/T370424#10274381 (10Ottomata) Also updated docs here: https://wikitech.wikimedia.org/w/index.php?title=SRE%2FClinic_Duty%2FAccess_r... [18:49:58] (03PS1) 10Snwachukwu: Add create table statement file for wmf_raw.cu_log table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1084208 (https://phabricator.wikimedia.org/T364398) [18:58:13] 06Data-Engineering, 06Machine-Learning-Team, 10Event-Platform: Create new mediawiki.page_links_change stream based on fragment/mediawiki/state/change/page - https://phabricator.wikimedia.org/T331399#10274440 (10Ottomata) [19:05:19] (03CR) 10Snwachukwu: [C:03+2] Add Query for cu_log table in sqoop. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082515 (https://phabricator.wikimedia.org/T364398) (owner: 10Snwachukwu) [19:05:54] (03CR) 10Snwachukwu: [V:03+2 C:03+2] Add Query for cu_log table in sqoop. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082515 (https://phabricator.wikimedia.org/T364398) (owner: 10Snwachukwu) [19:18:39] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python, 10GitLab (Project Migration): Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10274495 (10xcollazo) > Blank the GitLab repo, leaving only a readme pointing to the new repository and metadata.py. Did you mean... [19:22:56] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python, 10GitLab (Project Migration): Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10274508 (10xcollazo) >>! In T304544#10248852, @nshahquinn-wmf wrote: > @fkaelin is interested in co-hosting a Wmfdata work session... [19:30:47] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10274532 (10Ladsgroup) That log doesn't look complete, can you send the one for "T367781.log" somewhere? [19:32:44] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10274534 (10Ladsgroup) Without looking at the logs, db2155 is dump replica which probably means it failed to drain it si... [20:41:18] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python, 10GitLab (Project Migration): Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10274709 (10Dzahn) (regardless of _WHEN_ you actually do that) Suggesting you go to https://gitlab.wikimedia.org/projects/new and fr... [20:46:20] 06Data-Engineering, 10Event-Platform, 10MW-1.43-notes (1.43.0-wmf.27; 2024-10-15), 13Patch-For-Review, and 2 others: Delete redundant mobile- and desktopwebuiactions event in WikimediaEvents - https://phabricator.wikimedia.org/T376065#10274727 (10Jdlrobson) [20:46:31] 06Data-Engineering, 10Event-Platform, 10Web Team Essential Work 2024 (Migrate to new Event Platform), 10Web-Team-Backlog (FY2024-25 Q2 Sprint 2): Deprecate use of desktop- and mobilewebuiactions in Event Platform - https://phabricator.wikimedia.org/T368678#10274732 (10Jdlrobson) [21:57:05] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python, 10GitLab (Project Migration): Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10274955 (10nshahquinn-wmf) [22:20:09] 06Data-Engineering, 06Movement-Insights, 10Wmfdata-Python, 10GitLab (Project Migration): Move Wmfdata-Python from Github to Gitlab - https://phabricator.wikimedia.org/T304544#10275027 (10nshahquinn-wmf) >>! In T304544#10274495, @xcollazo wrote: >> Blank the GitLab repo, leaving only a readme pointing to t... [23:30:04] 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo, 10Tool-Pageviews, 10Tool-wikistatistics2-0: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549 (10MusikAnimal) 03NEW [23:31:38] 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo, 10Tool-Pageviews, 10Tool-wikistatistics2-0: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275187 (10MusikAnimal) I've tagged the components that this app may potentially be part of. If any maintainers o... [23:34:19] 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo, 10Tool-Pageviews, 10Tool-wikistatistics2-0: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275188 (10MusikAnimal) [23:37:42] 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo, 10Tool-Pageviews, 10Tool-wikistatistics2-0: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275191 (10MusikAnimal)