[04:19:23] RECOVERY - Check unit status of monitor_refine_event_sanitized_analytics_delayed on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_event_sanitized_analytics_delayed https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [04:32:11] PROBLEM - Check unit status of monitor_refine_event_sanitized_analytics_delayed on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_event_sanitized_analytics_delayed https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [06:53:09] !log drop an-airflow1001's old airflow logs to fix root partition almost filled up [06:53:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:08:32] * elukey leaves https://ai.facebook.com/blog/using-ai-for-database-queries-on-any-unstructured-data-set/ for joal [07:41:15] * joal thanks elukey :) [07:41:21] o/ [07:45:34] joal: Really trivial question, but I've read that count(*) can be more expensive than e.g. count(1), and wondering if it makes any difference in Hive. [07:47:00] Hi awight - AFAIK there is no difference - the had been differences when systems weren't optimizing queries, and with count(*) you'd have read every row/field - Nowaday any querying system should be smart enough to run count(*) == count(1) :) [07:47:14] * awight mops brow [07:47:30] This is great news, thanks for the confirmation! [07:47:32] By habit I always write COUNT(1), but it's just a habit :) [07:48:43] Honestly not a bad habit, count(*) is a confusing syntax in its own right. It's not actually counting non-null columns, or anything of the sort... [07:50:09] apparently `count(colname)` *is* counting non-nulls in that specific column. [07:51:10] Interesting thread awight - https://stackoverflow.com/questions/2710621/count-vs-count1-vs-countpk-which-is-better [07:53:35] It's a good point about left joins, clearly count is gonna cause pain in that case. [07:53:54] awight: it depends what you wish to count! [07:54:21] awight: just joking - I use count(1) only when counting all - otherwise I count(column) [07:57:25] (03PS9) 10Awight: Aggregate the number of times template data was available in VE's template dialog [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713650 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [07:57:52] (03CR) 10Awight: "PS 9: drop length parameter from substr" [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713650 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [07:58:29] (03PS4) 10Awight: Track multi-part transclusion stats for VE template dialog [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/714065 (https://phabricator.wikimedia.org/T272589) [08:00:23] (03PS5) 10Awight: Aggregate the number of times template data was available in TemplateWizard [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713647 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [08:00:43] (03CR) 10Awight: "PS 5: omit length parameter to substr" [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713647 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [08:03:46] 10Analytics, 10Analytics-Kanban, 10Prod-Kubernetes, 10SRE, and 3 others: Move eventgate services to use TLS only - https://phabricator.wikimedia.org/T255871 (10JMeybohm) >>! In T255871#7261361, @Ottomata wrote: > I think that will do it. helm template looks good locally. > > @JMeybohm is it ok that I mov... [08:04:53] (03CR) 10Awight: [V: 03+2 C: 03+2] Aggregate the number of times template data was available in VE's template dialog [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713650 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [08:05:08] (03CR) 10Awight: [V: 03+2] Track multi-part transclusion stats for VE template dialog [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/714065 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [08:05:26] (03CR) 10Awight: [V: 03+2 C: 03+2] Aggregate the number of times template data was available in TemplateWizard [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713647 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [08:07:21] (03CR) 10Kosta Harlan: [C: 03+2] Add analytics/mediawiki/mentor_dashboard/visit [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714099 (https://phabricator.wikimedia.org/T289369) (owner: 10Urbanecm) [08:08:19] (03Merged) 10jenkins-bot: Add analytics/mediawiki/mentor_dashboard/visit [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714099 (https://phabricator.wikimedia.org/T289369) (owner: 10Urbanecm) [08:09:40] (03Abandoned) 10WMDE-Fisch: Collect metrics of all wikis [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/655886 (https://phabricator.wikimedia.org/T271894) (owner: 10WMDE-Fisch) [09:09:19] (03CR) 10Awight: "Spotted a mistake." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/713647 (https://phabricator.wikimedia.org/T272589) (owner: 10Andrew-WMDE) [09:23:47] 10Analytics, 10Event-Platform, 10SRE, 10serviceops, 10Patch-For-Review: DRY kafka broker declaration in helmfiles - https://phabricator.wikimedia.org/T253058 (10JMeybohm) [10:04:14] joal: I was playing around and found that the `count(tablename.*)` syntax breaks hive, > SemanticException TOK_ALLCOLREF is not supported in current context [10:04:46] awight: wow, fun! [10:04:58] Not a problem for me, but it makes the thread suggestions suspect ;-) [10:05:08] riiiiiight! [10:19:20] (03CR) 10Awight: [V: 03+1] "Manually tested." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715459 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [10:30:13] (03CR) 10WMDE-Fisch: [C: 03+2] "Looks good. Seems to make sense." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715459 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [10:30:28] (03CR) 10Awight: [V: 03+2] Check whether any event of the day includes TemplateData [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715459 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [10:56:31] (03CR) 10Awight: "This change is ready for review." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715465 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [11:00:09] (03CR) 10Awight: [V: 03+2 C: 03+2] "Trivial, self-merging." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715465 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [11:00:44] (Hopefully that doesn't trigger the curse) [11:04:39] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+2] Nudge start date to trigger a tiny backfill [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715465 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [11:11:32] 10Analytics, 10FR-Tech-Analytics, 10Privacy Engineering: event.WikipediaPortal referer modification - https://phabricator.wikimedia.org/T279952 (10mforns) I will bring up this task today at our grooming meeting, so we prioritize it. [11:37:41] (03CR) 10Awight: "I... this didn't make it into the earlier patch. It's needed to prevent a syntax error." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715485 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [11:43:36] (03CR) 10Mforns: [C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/706605 (https://phabricator.wikimedia.org/T280649) (owner: 10Joal) [11:43:47] (03CR) 10Awight: [V: 03+2 C: 03+2] Missed from commit [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715485 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [12:30:08] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+2] Missed from commit [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/715485 (https://phabricator.wikimedia.org/T272589) (owner: 10Awight) [13:40:26] 10Analytics, 10 Data-Engineering, 10Growth-Team, 10Metrics-Platform, and 4 others: Migrated Server-side EventLogging events recording http.client_ip as 127.0.0.1 - https://phabricator.wikimedia.org/T288853 (10Ottomata) Hello! > I believe that the top-level http and meta properties are in a sense "owned" b... [14:29:08] hi elukey and ottomata (and a-team in general ;). Just FYI, The autocomplete was not working for me (or being veeeery slow) in the newpyter notebooks. I just found that is known issue, with a setting called jedi. Here is how to solve it: tinyurl.com/adkavae8 and there is also a permanent solution here https://tinyurl.com/56398zsm [14:30:19] huh, ok! dsaez could you file a bug? i'm just getting back from vaca and have about 3 billion emails and slacks to get through :) [14:31:04] ooh, welcome back, sure I'll create a phab ticket, nothing urgent. [14:31:33] ty :) [14:46:54] hello ottomata :) [14:47:52] HELLOOOooooooOooO [14:48:58] heyyy! [14:53:10] Hi ottomata :) [15:40:30] 10Analytics, 10Metrics-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014 (10jlinehan) [16:51:38] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Add 6 worker nodes to the HDFS Namenode config of the Analytics Hadoop cluster - https://phabricator.wikimedia.org/T275767 (10Ottomata) (♡ヮ♡) [17:00:53] (03PS3) 10Shay Nowick: Creating android_setting_action schema Bug: T285779 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714871 (https://phabricator.wikimedia.org/T285779) [17:00:55] (03PS1) 10Shay Nowick: Create android_user_state schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715562 [17:00:57] (03PS1) 10Shay Nowick: Create mobile-apps/android_user_state 1.0.0 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715563 [17:02:09] (03CR) 10jerkins-bot: [V: 04-1] Creating android_setting_action schema Bug: T285779 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714871 (https://phabricator.wikimedia.org/T285779) (owner: 10Shay Nowick) [17:02:11] (03PS2) 10Shay Nowick: Create android_user_state schema Bug: T285779 Change-Id: Iac1e505db912758cc02541b218d8095d0dc7ef4b [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715562 (https://phabricator.wikimedia.org/T285779) [17:06:05] 10Analytics-Radar, 10Product-Analytics, 10Growth-Team (Current Sprint), 10MW-1.37-notes (1.37.0-wmf.18; 2021-08-09), 10Patch-For-Review: Add geolocation information to Growth schemas - https://phabricator.wikimedia.org/T287121 (10Ottomata) I guess that would be {T290014}? I'll comment over there. [17:08:02] (03PS2) 10Shay Nowick: Create mobile-apps/android_user_state 1.0.0 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715563 [17:08:14] 10Analytics, 10Metrics-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014 (10Ottomata) Responding to @tgr's question from {T287121}: > @Krinkle pointed out on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/713526 that collecting I... [17:26:11] mforns: I have matching numbers between hive-queries and cassandra loaded data :) [17:26:24] mforns: If ok for you, let's merge my patch tomorrow :) [17:29:47] mforns: I have updated the etherpad train - I'll do the restart if you wish :) [17:30:09] mforns: finally, let's synchronize tomorrow on your QA scripts :) [17:56:09] 10Analytics, 10Metrics-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014 (10Krinkle) >>! In T290014#7319476, @Ottomata wrote: > This wouldn't be automated though, instrumentation code would do it. Would that work? Depends on the definition of "in... [18:26:40] 10Analytics-Radar, 10Dumps-Generation: xmldatadumps dumpstatus.json files only readable by root - https://phabricator.wikimedia.org/T287989 (10Ottomata) Thank you all! [18:35:01] 10Analytics, 10Metrics-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014 (10Ottomata) Oh, it's a cookie? ` GeoIP=US:NY:Brooklyn:40.70:-73.97:v4; ` https://wikitech.wikimedia.org/wiki/Geolocation I guess we'd want a similar field for cookie val... [18:36:50] 10Analytics, 10Metrics-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014 (10Ottomata) > Depends on the definition of "instrumentation code". To me that means client-side JS calling mw.eventLog or PHP code. Yes, whatever constructs the event. `... [18:57:45] (03CR) 10Ottomata: "Interesting. So this removes the postinstall step, so the git-add hook will not be automatically installed. I guess you are suggesting t" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714875 (owner: 10Krinkle) [18:58:48] joal: \\\o/// [18:58:54] yes, we can merge tomorrow [18:59:04] ok, on scripts tomorrow [19:20:55] PROBLEM - Check unit status of produce_canary_events on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [19:27:41] 10Analytics, 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible: 502, connect failed for intake-analytics.wikimedia.beta.wmflabs.org - https://phabricator.wikimedia.org/T289029 (10Urbanecm) So, I tried to just reboot that host, and...the URI sounds to work again now? [19:28:45] 10Analytics, 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Urbanecm: 502, connect failed for intake-analytics.wikimedia.beta.wmflabs.org - https://phabricator.wikimedia.org/T289029 (10Urbanecm) 05Open→03Resolved a:03Urbanecm Boldly closing this, eventlogging_HomepageVisit events... [19:28:55] this one was simple [19:33:32] (03PS1) 10Ottomata: Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 [19:35:08] (03CR) 10Gergő Tisza: [C: 03+1] Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 (owner: 10Ottomata) [19:36:40] (03CR) 10Urbanecm: [C: 03+1] Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 (owner: 10Ottomata) [19:57:29] (03CR) 10Mholloway: [C: 03+1] Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 (owner: 10Ottomata) [20:01:38] (03CR) 10Ottomata: [C: 03+2] Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 (owner: 10Ottomata) [20:01:40] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Review access change [schemas/event/secondary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/715520 (owner: 10Ottomata) [20:04:09] RECOVERY - Check unit status of produce_canary_events on an-launcher1002 is OK: OK: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [20:42:36] 10Analytics, 10Analytics-Kanban, 10Prod-Kubernetes, 10SRE, and 2 others: Move eventgate services to use TLS only - https://phabricator.wikimedia.org/T255871 (10Ottomata) @JMeybohm, I merged that and am trying to apply for eventgate-logging-external staging. Diff looks good: ` 20:23:31 [@deploy1002:/srv/... [21:07:40] (03PS3) 10Krinkle: build: Document simpler alternative contribution flow [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714875 [21:10:47] (03CR) 10Bearloga: [C: 04-2] Creating android_setting_action schema Bug: T285779 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714871 (https://phabricator.wikimedia.org/T285779) (owner: 10Shay Nowick) [21:10:58] (03CR) 10Bearloga: [C: 04-2] Create android_user_state schema Bug: T285779 Change-Id: Iac1e505db912758cc02541b218d8095d0dc7ef4b [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715562 (https://phabricator.wikimedia.org/T285779) (owner: 10Shay Nowick) [21:11:04] (03CR) 10Bearloga: [C: 04-2] Create mobile-apps/android_user_state 1.0.0 [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/715563 (owner: 10Shay Nowick) [21:11:33] (03CR) 10Krinkle: build: Document simpler alternative contribution flow (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/714875 (owner: 10Krinkle) [21:12:11] 10Analytics, 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Urbanecm: 502, connect failed for intake-analytics.wikimedia.beta.wmflabs.org - https://phabricator.wikimedia.org/T289029 (10Ottomata) Hm, I can't reproduce anymore. https://intake-analytics.wikimedia.beta.wmflabs.org/_info [21:12:34] 10Analytics, 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Urbanecm: 502, connect failed for intake-analytics.wikimedia.beta.wmflabs.org - https://phabricator.wikimedia.org/T289029 (10Ottomata) Oh, oops, commented before seeing your comments. OoooK! [21:12:53] ottomata: fortunately, restart sometimes fix things! :-) [21:22:06] :) [21:40:57] (03Abandoned) 10Andrew Bogott: Make a 'tests' dir and move our one test file there [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/710603 (https://phabricator.wikimedia.org/T210359) (owner: 10Andrew Bogott) [21:41:25] (03PS1) 10Andrew Bogott: Make a 'tests' dir and move our one test file there [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715607 (https://phabricator.wikimedia.org/T210359) [21:41:31] (03PS1) 10Andrew Bogott: Added tests for output.py [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715608 [22:12:45] 10Analytics, 10SecTeam-Processed, 10Security: Trusted-Contributors have +2 over schemas/event/secondary - https://phabricator.wikimedia.org/T282832 (10Urbanecm) [22:12:50] 10Analytics, 10SecTeam-Processed, 10Security: Trusted-Contributors have +2 over schemas/event/secondary - https://phabricator.wikimedia.org/T282832 (10Urbanecm) (taks published, as there is no PII -- the access is already restricted) [22:50:33] (03PS1) 10Andrew Bogott: app.py: remove redundant main definition [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715619 [22:50:36] (03PS1) 10Andrew Bogott: app.py: process with black in anticipation of a huge reformat [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715620 [22:50:41] (03PS1) 10Andrew Bogott: app.py: move flask creation into create_app() [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715621 [22:50:55] (03PS1) 10Andrew Bogott: Add an explicit yaml loader to suppress a deprecation notice [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715622 [22:59:22] (03CR) 10Bstorm: Add an explicit yaml loader to suppress a deprecation notice (031 comment) [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715622 (owner: 10Andrew Bogott) [23:03:59] (03PS2) 10Andrew Bogott: Use yaml.safe_load suppress deprecation notices [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715622 [23:04:32] (03CR) 10Andrew Bogott: Use yaml.safe_load suppress deprecation notices (031 comment) [analytics/quarry/web] (buster) - 10https://gerrit.wikimedia.org/r/715622 (owner: 10Andrew Bogott) [23:58:25] (03CR) 10Bstorm: "Is this one dead now?" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/710604 (owner: 10Andrew Bogott)