[00:26:24] 10Analytics-EventLogging, 10Analytics-Radar, 10Front-end-Standards-Group, 10MediaWiki-extensions-WikimediaEvents, and 2 others: Provide a reusable getEditCountBucket function for analytics purposes - https://phabricator.wikimedia.org/T210106 (10Jdlrobson) Just wanted to note that this came up again in T250... [03:03:11] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10Samwilson) I think it'd be fine to use TemplateWizard logging as a guinea pig. I don't think anyone's do... [05:48:20] hello team, today I need to change my tires so I'll join a little bit later in the morning :) [07:03:57] back! [07:26:13] 10Analytics, 10Analytics-Cluster, 10Operations: Segfault for systemd-sysusers.service on stat1007 - https://phabricator.wikimedia.org/T256098 (10elukey) [07:28:33] 10Analytics, 10Analytics-Cluster, 10Operations: Segfault for systemd-sysusers.service on stat1007 - https://phabricator.wikimedia.org/T256098 (10elukey) [07:33:23] 10Analytics, 10Analytics-Cluster, 10Operations: Segfault for systemd-sysusers.service on stat1007 - https://phabricator.wikimedia.org/T256098 (10elukey) One note is: ` elukey@stat1007:~$ apt-cache policy libsystemd0 libsystemd0: Installed: 241-5~bpo9+1 Candidate: 241-5~bpo9+1 Version table: *** 241-5... [07:37:11] 10Analytics, 10Analytics-Cluster, 10Operations: Segfault for systemd-sysusers.service on stat1007 - https://phabricator.wikimedia.org/T256098 (10elukey) [07:48:26] 10Analytics, 10Analytics-Cluster, 10Operations: Segfault for systemd-sysusers.service on stat1007 - https://phabricator.wikimedia.org/T256098 (10MoritzMuehlenhoff) We could try to track this down whether a specific user definitions makes it crash, by first removing individual files from /usr/lib/sysusers.d (... [08:47:21] 10Analytics, 10Analytics-Kanban: Spike, see how easy/hard is to scoop all tables from Eventlogging log database - https://phabricator.wikimedia.org/T250709 (10elukey) @Nuria @Milimetric do I have the green light to wipe db1108 and reimage it to Buster? I'll then apply the new config to become our backup db node. [09:23:17] 10Analytics, 10Analytics-Cluster: Co-locate Presto with Hadoop worker nodes - https://phabricator.wikimedia.org/T256108 (10elukey) [09:23:31] thanks elukey --^ :) [09:26:14] ah! so no bonjour until I do something that interest joal, good to know! [09:26:18] :P [09:26:21] :-P [09:26:24] huhu [09:26:44] elukey: I've been to the dentist this morning, and I'm not a normal human being anymore [09:26:52] (not sure I actually ever was ...)P [09:27:12] And now that I'm officially late - Bonjour elukey :) [09:28:05] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Create anaconda .deb package with stacked conda user envs - https://phabricator.wikimedia.org/T251006 (10elukey) [09:28:54] joal: :D [09:31:16] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Make anomaly detection correctly handle holes in time-series - https://phabricator.wikimedia.org/T251542 (10JAllemandou) I should have checked better - there are 2 jar versions in that job and I used the wrong one - the bump has already been made (second t... [09:32:54] (03Abandoned) 10Joal: Bump dataquality refinery-job jar version [analytics/refinery] - 10https://gerrit.wikimedia.org/r/607094 (https://phabricator.wikimedia.org/T251542) (owner: 10Joal) [09:56:51] (03PS7) 10Joal: Add pageview_actor_hourly table and oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/606127 (https://phabricator.wikimedia.org/T255467) [09:57:32] (03PS4) 10Joal: Update unique-devices jobs to use pageview_actor_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/606233 (https://phabricator.wikimedia.org/T250744) [09:58:19] (03PS6) 10Joal: Update clickstream and interlanguage jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/606449 [09:58:39] (03PS4) 10Joal: Aggregate pageview_hourly from pageview_actor_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/607096 (https://phabricator.wikimedia.org/T256049) [09:58:57] REBAAAAAAAAASE ! --^ [10:27:15] * elukey lunch! [11:34:46] just done https://wikitech.wikimedia.org/wiki/Yubikey-SSH [11:35:25] very nice, the yubikey can be used to store a RSA key pair (protected with pass) [11:35:35] nice!! [11:35:59] I have the yubikey 5, it is relatively cheap for what offers [11:36:09] (2FA, ssh, etc..) [11:36:31] to be fair it is gpg that does the heavylifting, buuut the key is really nice [12:02:14] 10Analytics-Radar, 10Operations, 10SRE-Access-Requests, 10Patch-For-Review, 10Product-Analytics (Kanban): Creation of a new POSIX group and system user for the Product Analytics team - https://phabricator.wikimedia.org/T255039 (10elukey) @mpopov I had a chat with Moritz, I'll take care of amending the co... [12:02:29] 10Analytics-Radar, 10Operations, 10SRE-Access-Requests, 10Patch-For-Review, and 2 others: Creation of a new POSIX group and system user for the Product Analytics team - https://phabricator.wikimedia.org/T255039 (10elukey) a:05mpopov→03elukey [12:08:15] 10Analytics-Data-Quality, 10QuickSurveys, 10WMDE-Technical-Wishes-Team, 10MW-1.35-notes (1.35.0-wmf.37; 2020-06-16), and 2 others: Remove Do Not Track support for QuickSurveys - https://phabricator.wikimedia.org/T254224 (10Lena_WMDE) 05Open→03Resolved a:03Lena_WMDE [13:07:07] ottomata: o/ [13:07:13] hello! [13:07:43] for some reason I managed to lock my account on archiva-new.wikimedia.org, I am trying to remove the "mirrored" repo but archiva doesn't collaborate.. when you have a min, do you mind to unblock me? [13:08:12] (and for some reason it thinks I tried to loging 26 times in a row failing) [13:09:20] ah I also deleted the section for stat100x host access on wikitech, saw the ping this morning :) [13:10:57] (going afk for a bit, coffee) [13:44:51] elukey: sorry forgot to look at IRC after your ping [13:44:56] ok i can log into archiav-new [13:44:59] what do you want me to do? [13:47:13] ottomata: for some reason I am unblocked, I thought it was you but probably archiva does it only temporarily [13:47:17] so all good :D [13:47:51] I am trying now to create the "mirrored" repo group, deleting the "mirrored" repo first [13:47:57] but archiva is sloooow [13:48:16] and I suspect that it does OOM half way through [13:49:24] oof [14:05:56] with 2G of heap (instead of 512) I get to Cannot delete repository /var/lib/archiva/repositories/mirrored [14:06:02] * elukey cries in a corner [14:06:37] but then I refreshed, and it is gone [14:10:28] all right I was able to create a new "mirrored" repository as repository group [14:10:35] but now we'd need to test if it works or not [14:10:54] (03CR) 10Mforns: [C: 03+1] "LGTM! +1 If it's tested using update_reports.py, feel free to merge on my side (if Jenniferwang agrees)." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/606734 (https://phabricator.wikimedia.org/T247417) (owner: 10Nuria) [14:21:15] crazy [14:23:23] archiva smells like half-abandoned, 3.0 IIRC was in the making but was never released [14:23:49] ahahhaha I don't believe it [14:23:56] I just went to the website to double check [14:23:59] "19th June 2020: The new Apache Archiva release version 2.2.5 is ready for download" [14:24:35] haha nice! [14:24:48] barely abandoned! [14:25:32] we could upgrade before making the switch between archiva and archiva-new [14:25:43] it is a bug-fix release, shouldn't change much in theory [14:26:51] * elukey interview time [14:29:58] oh hm, milimetric very late but we should probably have joal in this meeting if we could! [14:30:09] he's got more info esp on data and dump generation pipeline stuff [14:30:23] joal: if you are there we are meeting right now with pro api folks about html dumpo [14:30:30] inviting you just in case you can make it! [14:36:10] (03CR) 10Mforns: [C: 03+1] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/606752 (https://phabricator.wikimedia.org/T255779) (owner: 10Joal) [14:56:20] 10Analytics, 10Better Use Of Data, 10Product-Analytics: Bug: 'Include Time' option in table visualization produces "0NaN-NaN-NaN NaN:NaN:NaN" - https://phabricator.wikimedia.org/T256136 (10mpopov) [15:08:01] 10Analytics, 10Analytics-Kanban: Spike, see how easy/hard is to scoop all tables from Eventlogging log database - https://phabricator.wikimedia.org/T250709 (10Nuria) Green light @elukey [15:17:29] 10Analytics: Reload Hive2Druid datasources from already indexed data instead of raw data - https://phabricator.wikimedia.org/T232852 (10Nuria) p:05Triage→03Low [15:45:31] 10Analytics: Measure Community Backlog. - https://phabricator.wikimedia.org/T155497 (10Nuria) 05Open→03Declined [15:46:23] 10Analytics: Set a timeout for regex parsing in the Eventlogging processors - https://phabricator.wikimedia.org/T200760 (10Nuria) p:05Triage→03Low [15:47:14] 10Analytics: Set a timeout for regex parsing in the Eventlogging processors - https://phabricator.wikimedia.org/T200760 (10Nuria) 05Open→03Declined [15:47:18] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Eventlogging's processors stopped working - https://phabricator.wikimedia.org/T200630 (10Nuria) [15:47:40] 10Analytics: Estimate how long a new Dashiki Layout for Qualtrics Survey data would take - https://phabricator.wikimedia.org/T184627 (10Nuria) 05Open→03Declined [15:48:25] 10Analytics: Review parent task for any potential pageview definition improvements - https://phabricator.wikimedia.org/T156656 (10Nuria) 05Open→03Resolved [15:48:31] 10Analytics-Kanban, 10Analytics-Radar, 10Datasets-General-or-Unknown, 10Patch-For-Review, and 2 others: Pageview dumps incorrectly formatted, need to escape special characters - https://phabricator.wikimedia.org/T144100 (10Nuria) [15:49:07] 10Analytics: Script that synchronizes EL purging white-list with schema talk pages - https://phabricator.wikimedia.org/T170019 (10Nuria) 05Open→03Declined [15:51:10] mforns: nice airflow writeup! ::) [15:51:31] 10Analytics, 10Analytics-Dashiki: Dashiki, Unique Devices and Pageview data breakdown doesn't work if any of the items are not available for the project - https://phabricator.wikimedia.org/T136125 (10Nuria) 05Open→03Declined [15:52:25] 10Analytics, 10Analytics-Dashiki: Better menu for metrics - https://phabricator.wikimedia.org/T136126 (10Nuria) 05Open→03Declined [15:56:39] ottomata: thanks :] [16:01:53] mforns, ottomata standuuuup [16:02:03] mforns ottomata [16:02:10] standddduppp [16:04:47] team, have to restart my computer [16:15:14] (03CR) 10Nuria: [C: 03+2] Add a corrected bzip2 codec for spark [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/603590 (https://phabricator.wikimedia.org/T243241) (owner: 10Joal) [16:16:01] (03CR) 10Nuria: [C: 03+2] Correct bug in webrequest host normalization [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/606752 (https://phabricator.wikimedia.org/T255779) (owner: 10Joal) [16:20:22] (03PS1) 10MNeisler: Add the new VisualEditorFeatureUse fields to eventlogging whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/607309 (https://phabricator.wikimedia.org/T256048) [17:00:07] (03CR) 10Nuria: Add the new VisualEditorFeatureUse fields to eventlogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/607309 (https://phabricator.wikimedia.org/T256048) (owner: 10MNeisler) [17:36:08] * elukey off! [17:36:09] o/ [17:36:19] bye elukey [18:35:43] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10Ottomata) ok, thanks! [19:16:56] 10Analytics, 10Analytics-EventLogging, 10Better Use Of Data, 10Event-Platform, and 3 others: EventLogging MEP Upgrade Phase 2 (Sampling) - https://phabricator.wikimedia.org/T234594 (10mpopov) [19:24:35] 10Analytics, 10Better Use Of Data, 10Event-Platform: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10mpopov) [19:32:35] (03CR) 10Mforns: "LGTM overall!" (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/597740 (https://phabricator.wikimedia.org/T252857) (owner: 10Fdans) [19:33:38] 10Analytics, 10Better Use Of Data, 10Event-Platform: EventStreamConfig should generate and provide the stream cc map - https://phabricator.wikimedia.org/T256169 (10mpopov) [19:34:28] 10Analytics, 10Better Use Of Data, 10Event-Platform: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10mpopov) [19:38:46] 10Analytics-Radar, 10AbuseFilter, 10Cognate, 10ConfirmEdit (CAPTCHA extension), and 28 others: Replace PageContent(Insert|Save)Complete hooks - https://phabricator.wikimedia.org/T250566 (10DannyS712) [19:42:25] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10mpopov) [19:57:16] 10Analytics, 10Better Use Of Data, 10Event-Platform: EventStreamConfig should generate and provide the stream cc map - https://phabricator.wikimedia.org/T256169 (10mpopov) [20:07:31] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10Nuria) @mpopov Seeing that recent instrumentation like {T250282} is still not using MEP I think we should focus on adoption rather that a... [20:28:53] 10Analytics, 10Product-Analytics: Data missing in event_prefupdate in Druid - https://phabricator.wikimedia.org/T256178 (10nettrom_WMF) [21:14:00] o/ does anyone know if similar changes are being made to this (T254646) in HDFS etc.? for instance, `wmf.geoeditors_blacklist_country`? [21:14:01] T254646: Reconsidering how we name things - https://phabricator.wikimedia.org/T254646 [21:22:58] (03PS1) 10Conniecc1: Product team want to add a editors monthly table in Druid available to use in Superset/Turnilo. Since the monthly aggregations are compiled from editors_daily dataset, we want to add following dimensions to editors_daily dataset. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/607361 (https://phabricator.wikimedia.org/T256050) [23:02:03] Just as a side note: somebody is using ~88Gb RAM on our new stat1008. Just sayin', I am currently working to optimize an analytical system that used to utilize ~30Gb to work in less than 20Gb.