[00:54:29] (03CR) 10Nuria: [C: 03+2] whitelist: Add new ge.wikimedia.org [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552138 (https://phabricator.wikimedia.org/T236389) (owner: 10MarcoAurelio) [00:54:33] (03CR) 10Nuria: [V: 03+2 C: 03+2] whitelist: Add new ge.wikimedia.org [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552138 (https://phabricator.wikimedia.org/T236389) (owner: 10MarcoAurelio) [01:23:49] 10Analytics, 10Product-Analytics (Kanban): Make an Analytics Data Lake table to provide meta info about wikis - https://phabricator.wikimedia.org/T184576 (10Neil_P._Quinn_WMF) 05Open→03Resolved Now that I've added the wiki names to the dataset, along with some other fixes, ([commit 817dc0d](https://github.... [02:15:38] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Shorten the time it takes to move files from hadoop to dump hosts by Kerberizing/hadooping the dump hosts - https://phabricator.wikimedia.org/T234229 (10Bstorm) @elukey Sounds fine. As long as the software is in the MB range, it shouldn't... [06:12:54] 10Analytics, 10Analytics-Kanban: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10Marostegui) [06:12:56] 10Analytics-EventLogging, 10Analytics-Kanban: Sunset MySQL data store for eventlogging - https://phabricator.wikimedia.org/T159170 (10Marostegui) [06:59:38] !log restart hdfs-cleaner on an-coord1001 [06:59:40] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:08:08] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10elukey) @mepps Thanks for reaching out! A couple of comments: * Kafka and eventlogging will keep working as expected, the only part that will have some downtime is the data import to... [07:08:21] joal: --^ [07:08:24] :( [07:08:26] bonjour! [07:09:19] RECOVERY - Check the last execution of hdfs-cleaner on an-coord1001 is OK: OK: Status of the systemd unit hdfs-cleaner https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [07:25:57] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Shorten the time it takes to move files from hadoop to dump hosts by Kerberizing/hadooping the dump hosts - https://phabricator.wikimedia.org/T234229 (10elukey) @Bstorm thanks a lot! Yes the Analytics refinery is a scap repo so it will be... [07:35:35] wow joal I just seen your spark snippet for the cassandra logs parsing, niceeeee [08:03:41] elukey: Bonjour! [08:04:25] elukey: analyzing cluster logs on the cluster is the kind of loop I like ;) [08:51:22] joal: hello! [08:51:47] Hi djellel :) [08:52:54] joal: where do I find the pages-meta-history dumps on hdfs? or, is there a hive table? [08:55:03] djellel: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Content/XMLDumps/Mediawiki_wikitext_history [08:56:07] djellel: I think that page contains the inforamtion you need - If not, please ping me so that I update it :) [08:59:41] joal: that's perfect! i have code to process the zipped dump files. this should work as well. [08:59:57] djellel: on the clustrer? [09:01:39] joal: MR code. It's not deployed on the cluster. [09:02:06] djellel: I'm interested if you can point me to the code :) [09:04:30] joal: it's not shared yet, but sure I'll send you a pointer when it is. [09:05:15] ack djellel :) The part I'm interested in is how is XML processed [09:19:22] interesting elukey - All spark kernels from swap in yarn seem to have gone - Have we restarted the thing? [09:19:49] joal: the thing? [09:19:57] jupyterhub [09:20:18] elukey: --^ [09:20:19] not that I know, plus it shouldn't delete any file when restarting [09:21:07] ah wait let me re-parse your question [09:21:40] so in yarn's ui there is no spark or related kernel running from swap anymore? [09:22:14] ah yes just seen it [09:22:31] sorry I didn't read it correctly the first time, my bad [09:22:43] elukey: most of them are gone, while yesterday there still was a lot [09:24:30] joal: do you remember one? We could check specific systemd units on notebooks [09:25:17] I don't elukey - I just noticed this morning that scheduler was a lot emptier this mornong [09:25:45] elukey: Maybe people have gone and stopped their notebooks! [09:28:48] no idea [09:29:07] ok elukey - let's see if anyone pings on this :) [09:29:29] joal: Nov 20 22:08:17 notebook1003 jupyterhub[9176]: OSError: [Errno 12] Cannot allocate memory [09:29:39] Ahh :) [09:29:42] Here we go [09:29:50] This explains that [09:29:52] somebody was hammerring the host :( [09:33:13] elukey: Just seen https://phabricator.wikimedia.org/T212824 [09:35:03] elukey: I wonder how we should go for that --^ [09:36:42] joal: that is a long running task, I started to work on it a while ago but got stuck due to other priorities.. need to spend a bit more time on it and see how users can be throttled [09:37:22] elukey: In the meantime, we (I) should take some time reaching users and explaining I guess [09:39:35] joal: I think that we should prioritize the task for next Q, ideally all users should have a pre-defined limit.. My idea is to have a default for everybody like "you cannot consume more than X% or cpu and ram, independently from how idle the host is" [09:39:59] yes let's do it after kerberos and the db replica [09:40:10] sure [09:42:30] joal: it uses regex, with as a record delimiter [09:43:25] djellel: the code contains a hadoop input-format I assume [09:44:43] djellel: thanks for looking - I'll have a look myself when code is released [09:45:06] joal configuration.set("textinputformat.record.delimiter", ""); [09:45:43] Ah :) [09:45:54] makes sense [09:46:43] djellel: downside of the approach: you loose page-information at revision level if I don't mistake [09:48:50] joal: Maybe can try it as it. Are the bz2 dumps kept on hdfs? I couldn't find this info yet (still looking..) [09:50:23] djellel: data is here: /wmf/data/raw/mediawiki/dumps/pages_meta_history [09:50:33] cool! [10:49:06] o/ [10:49:25] joal: might you have a few mins at some point today to guide me through some things I'm trying to do in a notebook? [10:50:19] addshore: Hi! I surely will :) I have a meeting outside now, and will be back in ~1h - Can I ping you early afternoon? [10:50:36] sounds good! [10:50:41] * addshore is also in a meeting currently [11:21:12] meeting got canceled, I'm back! [11:21:16] addshore: when you want :) [11:21:38] :D just finishing off mine, then also have mydaily, so in ~ 30 mins? :) [11:22:30] 10Analytics, 10Analytics-Kanban: Set up automatic deletion for netflow datasource in Druid - https://phabricator.wikimedia.org/T229674 (10mforns) This is deployed and working :D https://tinyurl.com/wfuyybs [11:28:20] (03CR) 10Mforns: [V: 03+2 C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/550536 (https://phabricator.wikimedia.org/T234229) (owner: 10Joal) [11:30:40] (03CR) 10Mforns: [V: 03+2 C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/549861 (https://phabricator.wikimedia.org/T237271) (owner: 10Joal) [11:31:20] elukey, is it OK to merge this already? https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/550945/ [11:31:24] and deploy? [11:31:31] or joal ^ :] [11:31:59] mforns: nope - not yet :) [11:32:19] joal, ok ok [11:45:06] (03PS2) 10Mforns: Make hdfs-cleaner resilient to in-flight file deletion [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/552128 (https://phabricator.wikimedia.org/T238304) [11:45:51] joal: just shoving some breakfast into my mouth! [11:46:09] addshore: don't forget to chew :) [11:46:19] (03CR) 10Mforns: "Ottomata, thanks for the review!" (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/552128 (https://phabricator.wikimedia.org/T238304) (owner: 10Mforns) [11:46:20] hehe [11:52:35] joal: ready if you are! [11:52:58] let's go addshore :) https://meet.google.com/rxb-bjxn-nip [12:54:27] any idea why mvn is prepending "/srv" to my source files on stat1007 when I compile? [13:01:04] (03PS1) 10Elukey: WIP - Superset 0.35.1 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/552238 [13:08:52] joal: so I poked around a bit more, added some more UAs to the throttle list for the query service :) And found out there are 2.5 queries a second just doing the same simple lookups for a single value from a single entitiy over and over again [13:09:00] and, now i really love notebooks [13:10:00] that's great addshore :) [13:12:15] * elukey afk for a couple of hours [13:49:37] (03CR) 10Mforns: [V: 03+2 C: 03+2] "Self-merging, because I wrote this change together with joal (in pair) and all comments have been resolved (I believe), so that I can carr" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/552128 (https://phabricator.wikimedia.org/T238304) (owner: 10Mforns) [13:55:05] (03PS1) 10Mforns: Update changelog.md for v0.0.107 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/552256 [13:55:24] (03CR) 10Mforns: [V: 03+2 C: 03+2] Update changelog.md for v0.0.107 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/552256 (owner: 10Mforns) [14:00:00] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Modern Event Platform: Schema Registry: Implementation - https://phabricator.wikimedia.org/T206789 (10jlinehan) >>! In T206789#5679422, @Ottomata wrote: > We will need to do some schema namespacing within the repos a... [14:10:57] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Modern Event Platform: Schema Registry: Implementation - https://phabricator.wikimedia.org/T206789 (10jlinehan) If we can't come to any other consensus, I think I'd compromise on `primary/secondary`. It's neutral eno... [14:15:43] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Modern Event Platform: Schema Registry: Implementation - https://phabricator.wikimedia.org/T206789 (10Ottomata) I'm good with primary / secondary [15:10:13] !log deployed refinery-source v0.0.107 [15:10:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:11:10] (03CR) 10Mforns: [V: 03+2] Update cassandra jar in related jobs for logging [analytics/refinery] - 10https://gerrit.wikimedia.org/r/551893 (https://phabricator.wikimedia.org/T236698) (owner: 10Joal) [15:23:13] (03PS1) 10Mforns: Bump up refinery jar version to 107 for affected jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552279 [15:24:48] (03CR) 10Mforns: [V: 03+2 C: 03+2] "Self-merging for deployment train" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552279 (owner: 10Mforns) [15:28:26] can elukey or ottomata please merge this jar version bump up? ^ thanks!!! [15:33:00] mforns: I am back, checking [15:34:07] mforns: what should I merge?? [15:34:22] elukey, https://gerrit.wikimedia.org/r/552279 [15:34:41] elukey, wait.. [15:35:13] you already merged no? [15:35:19] elukey, sorry, it's this one: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/552280/ [15:35:29] my baddd [15:38:23] thank youuuu :D [15:38:35] :) [15:41:31] mforns: did you check yesterday if the sanitization systemd timer was ok on an-coord1001? [15:45:58] elukey, the netflow one? [15:46:47] yep [15:49:09] elukey, today I checked in turnilo that the data was being sanitized at the correct time range, and right now checked the timers in an-coord1001 and the hourly one disappeared, which is cool! and the daily one is there, as expected! [15:49:15] all seems good! [15:49:56] mforns: yep I meant from the systemd units point of view [15:50:07] it was a good idea for an excercise to practice with those [15:50:09] !log deployed refinery (with v0.0.107) [15:50:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:50:31] I see [15:52:31] if you want and when you have time :) [15:52:37] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10EYener) To clarify the expected downtime: Is it 45 minutes or 3 to 4 hours? On the Fundraising side, as @mepps mentioned, 12/2 at 15 or 16 UTC is the launch of a large fundraising eff... [15:56:26] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10elukey) >>! In T238560#5681740, @EYener wrote: > To clarify the expected downtime: Is it 45 minutes or 3 to 4 hours? Can be 3/4 hours, it really depends on the issues that we'll find... [15:57:53] joal: --^ [15:58:47] 10Analytics, 10Analytics-Kanban: Make hdfs-cleaner resilient to in-flight files deletion - https://phabricator.wikimedia.org/T238304 (10mforns) a:03mforns [16:29:58] (03PS2) 10Elukey: WIP - Superset 0.35.1 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/552238 [16:42:45] deployed superset 0.35.1 in staging, I just noticed that we need presto deps for a dashboard (didn't include it) but everything seems working! [16:45:34] Hey folks. I'm trying to do a `git review -R` from stat1007 and in just hangs. I've got https_proxy set and I can confirm that I can wget some pages behind https. Do you know of another reason why git review -R might just hang? [16:48:15] halfak: o/ in theory there should be global git configs to use a http proxy (/etc/gitconfig IIRC) but I am not sure if git push goes via https as well.. if not, it will be blocked by the great analytics firewall [16:48:50] I can do git push to other repos. But not git review. :\ [16:49:04] I'm using the https endpoint for the gerrit repo. [16:50:31] ah wait sorry -R, for some reason my brain thought about git review [16:50:33] lemme check [16:51:20] (simple git review) [16:51:58] mmm so it should be only to avoid rebase? [16:52:24] Oh yeah. Standard practice from the docs. But not important. [16:52:30] if it hangs it means that it is trying to create a tcp conn to gerrit that is not whitelisted by the firewall [16:52:36] git review also hangs. [16:52:38] so it simply hangs until a timeout occurs [16:52:46] and if you do git push refs/for/etc.. ? [16:52:46] I think that's right. [16:53:29] Hmm. I usually don't push directly to gerrit :| Not sure I want to do that. [16:53:35] But I can push directly to github [16:54:01] git push origin HEAD:refs/for/master is the equivalent of git review [16:54:13] Oh! I see. Let me try. [16:54:29] because I am pretty sure that it uses a port not whitelisted [16:54:36] Aha! That worked [16:55:43] iiiiinteresting [16:55:49] so that uses https [16:55:56] git review does not [16:56:04] Aha! Interesting. [16:56:13] this is my speculation [16:56:14] Yeah, I'd like to use https from the stat machines for lots of reasons. [16:56:39] Oh I see. Just speculation at this point. [16:56:56] yeah :) [16:58:51] * halfak considers just using this work-around. [16:59:08] What do you think? Is this a Bad Idea(TM)? [16:59:13] elukey, ^ [17:03:22] halfak: no problem from my side :) [17:03:26] ping ottomata [17:03:34] as long as your credentials are kept safe [17:04:16] \o/ Cool. Thanks for your help elukey. [17:08:58] (03PS3) 10Elukey: WIP - Superset 0.35.1 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/552238 [17:10:12] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10Ejegg) During the whole first week of the banner campaign last year we were raising between one and three million dollars a day, so it would be really stressful to lose our ability to... [17:16:07] 10Analytics, 10Analytics-Kanban: Rerun pingback reports to categorize software versions correctly. - https://phabricator.wikimedia.org/T238389 (10mforns) @CCicalese_WMF I think all re-runs are done! I checked the dashboard and couldn't find any missing data point. Please do check as well! I will move this to d... [17:29:33] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10Nuria) Let's clarify some things here: none of the data in hadoop is realtime, pageviews and eventlogging are delayed between 3-4 hours from real time events. So an issue with CentralN... [17:31:26] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10Nuria) Data in Kafka is Real time however. Turnilo, Jupyter work with data in hadoop, the FR kafka puller works with data in kafka. [17:41:00] 10Analytics, 10Analytics-Kanban: Rerun pingback reports to categorize software versions correctly. - https://phabricator.wikimedia.org/T238389 (10CCicalese_WMF) @mforns Thank you! I think the reports supporting https://pingback.wmflabs.org/#php-version still need to be re-run, though. The drilldown reports are... [17:47:40] 10Analytics: Output schema with mediawiki_history snapshots dumps - https://phabricator.wikimedia.org/T238668 (10fdans) p:05Triage→03Normal [17:48:34] 10Analytics, 10Release Pipeline, 10Patch-For-Review, 10Release-Engineering-Team (Pipeline), 10Services (watching): Migrate EventStreams to k8s deployment pipeline - https://phabricator.wikimedia.org/T238658 (10fdans) p:05Triage→03Normal [17:50:41] 10Analytics, 10Analytics-Kanban: Rerun pingback reports to categorize software versions correctly. - https://phabricator.wikimedia.org/T238389 (10mforns) @CCicalese_WMF Oh! Yes, actually nothing's wrong, but RU is still re-runing... I see now that the php_drilldown for v1.34 is still calculating. After that th... [17:52:03] 10Analytics, 10Multi-Content-Revisions (Tech Debt): Adapt mediawiki history for MCR - https://phabricator.wikimedia.org/T238615 (10fdans) p:05Triage→03Normal [17:52:23] 10Analytics: HDFS space usage steadily increased over the past month - https://phabricator.wikimedia.org/T238648 (10fdans) 05Open→03Resolved a:03fdans [17:55:50] 10Analytics: Wikistats2 time related bugs - https://phabricator.wikimedia.org/T231248 (10mforns) a:03fdans [17:55:53] 10Analytics: Wikistats2 time related bugs - https://phabricator.wikimedia.org/T231248 (10mforns) p:05Triage→03High [18:01:19] 10Analytics, 10Analytics-EventLogging: Update pingback reports to use heartbeat pings to filter data - https://phabricator.wikimedia.org/T236178 (10mforns) @CCicalese_WMF We won't be able to dedicate time to this this quarter. We will schedule this tentatively for next quarter. [18:02:44] 10Analytics, 10Analytics-EventLogging: Update pingback reports to use heartbeat pings to filter data - https://phabricator.wikimedia.org/T236178 (10mforns) p:05Triage→03Normal [18:05:24] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10observability: Validation error for invalid value type should include property name - https://phabricator.wikimedia.org/T116719 (10mforns) a:03Ottomata [18:05:31] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10observability: Validation error for invalid value type should include property name - https://phabricator.wikimedia.org/T116719 (10mforns) p:05High→03Triage [18:05:33] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10observability: Validation error for invalid value type should include property name - https://phabricator.wikimedia.org/T116719 (10mforns) p:05Triage→03High [18:07:06] 10Analytics, 10Analytics-Wikistats: Active Editors metric per project family - https://phabricator.wikimedia.org/T188265 (10mforns) p:05High→03Normal [18:07:23] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Create report for "articles with most contributors" in Wikistats2 - https://phabricator.wikimedia.org/T204965 (10mforns) p:05High→03Normal [18:12:22] 10Analytics, 10Analytics-EventLogging: Update pingback reports to use heartbeat pings to filter data - https://phabricator.wikimedia.org/T236178 (10CCicalese_WMF) Thanks for letting me know! [18:13:33] 10Analytics, 10Growth-Team, 10Product-Analytics: Revisions missing from mediawiki_revision_create - https://phabricator.wikimedia.org/T215001 (10mforns) @nettrom_WMF Hi, can you provide the query you used to get that data, so we can more easily determine where the problem is? Thanks! [18:13:40] 10Analytics, 10Growth-Team, 10Product-Analytics: Revisions missing from mediawiki_revision_create - https://phabricator.wikimedia.org/T215001 (10mforns) p:05High→03Triage [18:13:43] 10Analytics, 10Growth-Team, 10Product-Analytics: Revisions missing from mediawiki_revision_create - https://phabricator.wikimedia.org/T215001 (10mforns) p:05Triage→03High [18:14:19] 10Analytics, 10Operations, 10Patch-For-Review, 10User-Elukey: Archival of home directories on servers with very large homes - https://phabricator.wikimedia.org/T215171 (10mforns) p:05High→03Normal [18:14:53] 10Analytics: Update grouped-wiki files for sqoop - https://phabricator.wikimedia.org/T219326 (10mforns) p:05High→03Low [18:16:01] 10Analytics: Include user group expiry events in mediawiki history reconstruction - https://phabricator.wikimedia.org/T225249 (10mforns) p:05High→03Normal [18:17:11] 10Analytics: Rename event_sanitized to event_longterm - https://phabricator.wikimedia.org/T225751 (10mforns) p:05High→03Normal [18:20:08] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10mforns) p:05High→03Triage [18:20:11] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10mforns) p:05Triage→03High [18:21:51] 10Analytics, 10Analytics-Data-Quality: Set entropy alarm in editors per country per wiki - https://phabricator.wikimedia.org/T227809 (10mforns) p:05High→03Normal [18:22:38] 10Analytics, 10Analytics-Kanban, 10Tool-Pageviews: Add referrer to mediarequests dataset to inform about project - https://phabricator.wikimedia.org/T228151 (10mforns) [18:22:50] 10Analytics, 10Analytics-Kanban, 10Tool-Pageviews: Add referrer to mediarequests dataset to inform about project - https://phabricator.wikimedia.org/T228151 (10mforns) 05Open→03Resolved a:03mforns [18:22:53] 10Analytics, 10Analytics-Kanban, 10Tool-Pageviews: Load media requests data into cassandra - https://phabricator.wikimedia.org/T228149 (10mforns) [18:25:49] 10Analytics, 10Product-Analytics: Add page protection status to MediaWiki history tables - https://phabricator.wikimedia.org/T230044 (10mforns) It is difficult for us to add more features to the MediaWiki history, because any addition makes the already complex codebase a little bit more complex. We'll reduce t... [18:25:52] 10Analytics, 10Product-Analytics: Add page protection status to MediaWiki history tables - https://phabricator.wikimedia.org/T230044 (10mforns) p:05High→03Low [18:26:02] 10Analytics, 10Analytics-Kanban: Set up automatic deletion for netflow datasource in Druid - https://phabricator.wikimedia.org/T229674 (10Nuria) 05Open→03Resolved [18:26:20] 10Analytics, 10Pageviews-API: Adding top counts for wiki projects (ex: WikiProject:Medicine) to pageview API - https://phabricator.wikimedia.org/T141010 (10mforns) p:05Normal→03Low [18:35:39] joal nuria, just updated the job, will keep an eye on it tomorrow morning for failures [18:35:49] <3 fdans :) [18:45:56] 10Analytics-Kanban, 10Better Use Of Data, 10Event-Platform, 10Operations, and 8 others: Set up eventgate-logging-external in production - https://phabricator.wikimedia.org/T236386 (10Ottomata) [18:50:01] 10Analytics-Kanban, 10Better Use Of Data, 10Event-Platform, 10Operations, and 8 others: Set up eventgate-logging-external in production - https://phabricator.wikimedia.org/T236386 (10Ottomata) Ok thanks for the help today @akosiaris and @Joe, HTTPS via envoyproxy is finally working! I will be off tomorrow... [19:01:45] mforns: I have triple checked various places and how data is structured: I actually think easiest would be to add the data to geoeditors-daily :( [19:02:15] mforns: this is the way we went for network-origin, and I therefore assume we should do the same for bots [19:02:35] ok... [19:02:41] +1! [19:03:26] mforns: now the question is: we have 2 types of bot (by-name or by-group) [19:03:44] mforns: Should we add both? Only group (well defined)? [19:04:28] mforns: as this dataset (geoeditors) is bound to be a basis for other analysis (I think the wmcs case is one of many), maybe having both makes sense? [19:11:49] mforns: also, looks like pageview still have unexpectedvalues :( [19:13:04] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10Mayakp.wiki) > You should have an email with the tmp pass! @Mayakp.wiki please check in your spam folder if you don't find the email, I have tested my script to create identities only with @wik... [19:13:26] joal : do you have any notebook where you parse the xml dumps in the cluster? [19:14:00] nuria: xmlparsing is done in an oozie job, then text is in avro and can be read using a hive table [19:14:02] joal, the last unexpected pageview alarm I received was before the restart [19:14:31] mforns: no restart involved in pageview alerts as they use the 'current' folder [19:14:46] mforns: time for last alert should be deploy time [19:15:12] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10Iflorez) Hi there! here's my request for Kerberos credentials for Hadoop access on stat100X and notebook100X. My username is iflorez. I am a contractor and I have a signed NDA on file. LDAP us... [19:17:31] 10Analytics, 10Analytics-Kanban: Add bot edits to geoeditors-daily - https://phabricator.wikimedia.org/T238855 (10JAllemandou) [19:18:21] 10Analytics, 10Analytics-Kanban: Add bot edits to geoeditors-daily - https://phabricator.wikimedia.org/T238855 (10JAllemandou) [19:18:34] nuria: I don't have a notebook using the data, but can provide examples easily [19:20:48] nuria: also, I create the task above to fix the diff we have for wmcs edits - let me know if you think it's not the correct way to go [19:21:56] joal: nice, makes total sense. We should probably do that sooner rather than later [19:22:19] 10Analytics, 10Analytics-Kanban: Add bot edits to geoeditors-daily - https://phabricator.wikimedia.org/T238855 (10JAllemandou) [19:23:36] 10Analytics, 10Analytics-Kanban: Add bot edits to geoeditors-daily - https://phabricator.wikimedia.org/T238855 (10JAllemandou) [19:23:40] nuria: I just updated with a question on which bots to add and how... [19:23:48] nuria: need some examples on wikitext parsing? [19:23:55] joal: sure [19:24:12] joal: i would like to add those to docs in wikitech [19:24:28] Makes sense [19:25:30] nuria: Let me create a task about updating current state of text-processing-on-cluster stuff (update docs, add an ooie job for page_meta_current, add an ooie job for siteinfo parsing) [19:31:13] 10Analytics: Update wikitext-processing on hadoop various aspects - https://phabricator.wikimedia.org/T238858 (10JAllemandou) [19:31:22] joal: does the cluster have the dumps with all content revisions? [19:31:32] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10elukey) >>! In T237605#5682350, @Iflorez wrote: > Hi there! here's my request for Kerberos credentials for Hadoop access on stat100X and notebook100X. My username is iflorez. I am a contractor a... [19:31:54] nuria: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Content/XMLDumps/Mediawiki_wikitext_history [19:32:08] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10elukey) >>! In T237605#5682338, @Mayakp.wiki wrote: > >> You should have an email with the tmp pass! @Mayakp.wiki please check in your spam folder if you don't find the email, I have tested my... [19:32:44] * elukey off! [19:37:00] nuria: I just updated the page I pasted above, adding a section on row data (the real XMLdumps) [19:37:36] going for diner team - later! [19:37:53] joal: ok! [19:38:22] joal: will try to do something with this in notebooks [19:43:51] nuria: tips - using a small wiki (that data is huge) 'simplewiki' is an example - and use greedy regexes (pages can be long...) [19:49:48] joal, I'm a bit confused, there are 5 new projects missing from the pageview whitelist [19:49:50] is that normal? [19:50:14] field_name unexpected_value _c2 [19:50:15] project gcr.wikipedia 41 [19:50:15] project ge.wikimedia 363 [19:50:15] project min.wiktionary 1103 [19:50:15] project shy.wiktionary 93 [19:50:15] project szy.wikipedia 358 [19:51:27] joal: missing on table but they are on file? [19:51:31] sorry mforns [19:51:45] mforns: teh projects are missing on table but they are on file? [19:52:08] nuria, the projects are in the pageview_unexpected_values table [19:52:16] I assume they are not in the whitelist [19:55:05] mforns: min.wiktionary is not [19:55:47] 10Analytics, 10Analytics-Kanban: Request for a large request data set for caching research and tuning - https://phabricator.wikimedia.org/T225538 (10lexnasser) @Danielsberger Checking in again. I have documented the tentative info about the **upload** dataset on Wikitech [[ https://wikitech.wikimedia.org/wi... [19:56:14] mforns: ya, neither of those are [19:56:21] yea [19:56:30] strange all those happened within a day [19:58:16] mforns: i bet you they created the domains just recently [19:58:27] mforns: they are all "reserved domains" but do not hold content [19:58:52] I see [19:59:06] mforns: not all , just lied [19:59:22] mforns: let's add them all to the pageview whitelist though, right? [19:59:24] the one that is already in was added today [19:59:38] sure, creating patch now [20:00:21] (03PS1) 10Mforns: Add missing projects to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552327 [20:01:26] nuria, can you please review? ^ [20:02:25] mforns: looking [20:03:29] (03CR) 10Nuria: [C: 03+2] Add missing projects to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552327 (owner: 10Mforns) [20:03:31] (03CR) 10Nuria: [V: 03+2 C: 03+2] Add missing projects to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/552327 (owner: 10Mforns) [20:03:59] thaaankss :] [20:07:06] !log deploying refinery to add pageview whitelist changes and stop alerts [20:07:08] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:07:32] 10Analytics, 10Analytics-Kanban: Doubts and questions about Kerberos and Hadoop - https://phabricator.wikimedia.org/T238560 (10AndyRussG) Hi, all, >>! In T238560#5682048, @Nuria wrote: > could you give some examples of issues you expect to see and troubleshoot (maybe some tickets from the past?)? There are i... [20:11:56] 10Analytics, 10Analytics-Kanban: Request for a large request data set for caching research and tuning - https://phabricator.wikimedia.org/T225538 (10Danielsberger) @lexnasser this is great, thank you! I really like the idea of a dedicated wikitech page. I can contribute some parsing and processing scripts in t... [20:14:09] 10Analytics, 10Analytics-Kanban: Create kerberos principals for users - https://phabricator.wikimedia.org/T237605 (10EYener) I would also like to request Kerberos credentials for stat100x and notebook100x machines. My username is eyener. Thank you!