[00:04:30] mforns: may be worth reading - http://oldfashionedsoftware.com/2008/09/27/tail-recursion-basics-in-scala/ [00:04:59] madhuvishy, thx reading [00:08:14] madhuvishy, so if we keep the ok-rdd in the function scope until the recursive call returns, we are making the code prone to call-stack errors [00:09:20] Yeah, if we pass everything we need back, we can tail call optimize [00:09:33] aha [00:10:07] I was actually trying an iterative method, I think this will make checkpointing and unpersisting easier [03:32:58] Analytics-Backlog: Remove queue of batches from EL code - https://phabricator.wikimedia.org/T121151#1896949 (Nuria) [03:40:51] Analytics-Backlog, Wikipedia-iOS-App-Product-Backlog, Zero, hardware-requests, and 3 others: Request one server to suport piwik analytics - https://phabricator.wikimedia.org/T116312#1896972 (Nuria) @Dzahn: the machine we are requesting will receive traffic from annual report website via js beacon.... [06:19:10] Analytics-Backlog: Add instruction text next to the input fields in the Program Global Metrics Report {kudu} - https://phabricator.wikimedia.org/T121899#1897098 (Milimetric) p:Triage>High [06:19:37] Analytics-Backlog, Analytics-Wikimetrics: Central repository of global metrics reports {kudu} - https://phabricator.wikimedia.org/T121286#1897099 (Milimetric) p:Triage>High [06:27:51] Analytics-Backlog, Wikipedia-iOS-App-Product-Backlog, Zero, hardware-requests, and 3 others: Request one server to suport piwik analytics - https://phabricator.wikimedia.org/T116312#1897101 (Dzahn) @Nuria thanks, yes, i added a DNS change to add piwik.wm.org https://gerrit.wikimedia.org/r/#/c/260525/ [08:28:09] Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1897162 (jcrespo) MobileWikiAppShareAFact_12588711 [11:05:51] (PS1) Addshore: Rename ref script to general metrics script [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260550 [11:06:06] (CR) Addshore: [C: 2 V: 2] Rename ref script to general metrics script [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260550 (owner: Addshore) [11:07:08] (PS1) Addshore: Remove old unused bulk sparql stuff [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260551 [11:07:20] (CR) Addshore: [C: 2 V: 2] Remove old unused bulk sparql stuff [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260551 (owner: Addshore) [11:07:58] hi a-team! [11:26:28] (PS1) Addshore: Add showcase items script [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260553 [11:26:56] (PS2) Addshore: Add showcase items script [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260553 (https://phabricator.wikimedia.org/T119069) [11:27:05] (CR) Addshore: [C: 2 V: 2] Add showcase items script [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260553 (https://phabricator.wikimedia.org/T119069) (owner: Addshore) [11:57:09] Hi mforns :) [11:57:18] hi joal! how are you feeling? [11:57:27] mwarf :) [11:57:30] mmm [11:58:07] will you take a rest today? [11:58:17] I'll try to help at least a bit [11:58:28] do you want us to read somesprak [11:58:30] ? [11:58:37] sure! batcave? [11:58:41] OMW ! [12:16:09] Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1897569 (jcrespo) Maintenance on db1047 has been done. [12:24:44] (PS1) Addshore: Fix metrics.json location [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260554 [12:27:07] (PS1) Addshore: Fix showcase api result path [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260555 [12:27:31] (CR) Addshore: [C: 2] Fix metrics.json location [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260554 (owner: Addshore) [12:27:35] (CR) Addshore: [C: 2] Fix showcase api result path [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260555 (owner: Addshore) [12:29:14] (PS2) Addshore: Fix showcase api result path [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260555 [12:29:25] (CR) Addshore: [V: 2] Fix showcase api result path [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260555 (owner: Addshore) [12:29:32] #spamspamspam [12:47:37] (PS2) Addshore: Fix metrics.json location [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260554 [12:47:47] (CR) Addshore: [V: 2] Fix metrics.json location [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260554 (owner: Addshore) [12:50:54] (PS1) Addshore: Use json api for showcase items [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260556 [12:51:27] (CR) Addshore: [C: 2 V: 2] Use json api for showcase items [analytics/limn-wikidata-data] - https://gerrit.wikimedia.org/r/260556 (owner: Addshore) [12:57:12] joal, yt? [12:57:15] yup [12:57:41] wasup mforns ? [12:57:46] hey, I was thinking that it may be convenient if the case class can treat both country and country_code as the same dimension [12:58:00] and when transformed into a row, handle that transparently [12:58:13] why not :) [12:58:13] you think that is possible? [12:58:28] hm, I don't know yet, but I'll keep thast in mind ! [12:58:33] ok :] [12:59:25] for the moment I'm juggling with rdd and dataframes [13:00:38] mmmmmm, joal do you think country and country_code are always congruent? meaning each country value has 1 and only 1 country_code value? [13:00:48] mforns: it should [13:00:52] ok [13:01:12] mforns: milietric has written a function to convert code to country name, so we should assume it is a bijection :) [13:01:29] cool, cool word :] [13:11:07] Analytics-Tech-community-metrics, DevRel-December-2015: Empty "subject" and "creator" fields for mailing list thread on mls.html - https://phabricator.wikimedia.org/T116284#1898045 (Dicortazar) After having a look at the data, it's clear that nowadays that info was updated, but we were able to reproduce t... [13:40:32] Analytics-Backlog, Analytics-General-or-Unknown, WMDE-Analytics-Engineering, Wikidata, Story: [Story] Statistics for Special:EntityData usage - https://phabricator.wikimedia.org/T64874#1898228 (Addshore) I have reduced the above query to only show what we want to look at: ``` SELECT CONCAT... [15:01:17] Analytics-Tech-community-metrics, DevRel-January-2016: demographics.html: "Tickets participants" has "184 attracted" data for 1yearLcanasdiaz [15:55:09] kafka1001 and kafka1002 are showing this: [15:55:12] Check that eventlogging-service-eventbus is running [15:55:20] PROCS CRITICAL: 0 processes with command name '/srv/deployment/eventlogging/eventbus/bin/eventlogging-service' [15:56:06] ! [15:56:08] looking :) [15:56:17] :) [15:59:33] hola [16:09:52] a-team: will not attend standup today as i have the manager's meeting [16:10:05] ok nuria [16:10:13] a-team: have sent e-scrum [16:13:35] Analytics-Backlog, Wikipedia-iOS-App-Product-Backlog, Zero, hardware-requests, and 3 others: Request one server to suport piwik analytics - https://phabricator.wikimedia.org/T116312#1898439 (Nuria) @Dzahn: thank you. Let us know when we have a box so we can deploy . [16:29:41] ottomata: can you take a look at this blacklisting: https://gerrit.wikimedia.org/r/#/c/260595/? thank you [16:30:43] nuria: blacklisting will keep events from being produced to eventlogging-valid-mixed topic [16:31:20] so, unless you manually had a consumer that knew how to backfill them from the eventlogging_XXYY topic [16:31:29] the events will not just automatically flow back into mysql when you turn the blacklist off [16:32:12] ottomata: ah true, now, is there a better way to stop producing to this one table? [16:33:02] ottomata: *i think* it should be ok to loose some events though, if we can avoid it it will be better of course [16:33:46] nuria: the events will all still be produced to the specific topic [16:33:53] hm [16:33:59] ottomata: so then, we can spin [16:34:17] a consumer that reads that topic and puts them into valid-mixed, right? [16:34:17] ja you could backfill by running a consumer from that topic into valid-mixed, or into mysql, either way. [16:34:24] buuut, you'd have to know the offset from where to start [16:34:39] which isn't obvious [16:34:43] hmm [16:34:44] ottomata: to be uber precise yes [16:34:52] ottomata: but aproximate is ok [16:35:05] you'd have to find the last event in mysql, then try to find out what the offset of that event is in the specific topic [16:35:10] it can be done [16:35:16] nuria: or [16:35:21] ottomata: aham [16:35:30] you could just start a consumer proc now that consumes that topic to a file [16:35:34] and then backfill from that [16:37:00] ottomata: ah yes. [16:37:11] that would probably be easier [16:37:21] as long as you don't plan to have it on for a long time, and as long as the data isn't too big [16:37:32] so ja, you could black list like this, and then backfill manually [16:37:48] ottomata: ok, let me see if i can figure out how to change the configuration on EL box to do that. [16:40:29] nuria: [16:40:32] just do it manually [16:40:37] as yourself [16:40:41] run a consumer in a screen [16:40:59] ottomata: i was going to use this config but change it : [16:41:05] https://www.irccloud.com/pastebin/7caNkMb8/ [16:41:06] eventlogging-consumer 'kafka:///...?topic=eventlogging_XXXXX' file:///home/nuria/blabla.out [16:41:16] k [16:41:17] yup! [16:41:20] that'll do [16:41:31] ottomata: k, rather use file so i have it for reference, give me a sec [16:41:40] nuria: change identity in that too [16:41:49] identity=nuria-backfill [16:41:51] or whatever [16:41:52] doesn't matter [16:42:23] ok [16:42:41] and topic is just the table right? no schema version [16:45:09] ottomata: do we have the list of topics somewhere? [16:45:32] nuria, topic is eventlogging_ [16:45:46] ottomata: same case? [16:46:10] nuria, you could do [16:46:11] kafkacat -L -b kafka1012 | grep topic [16:46:13] from stat1002 [16:46:15] nuria, yes [16:46:57] ahhhh, yes i forgot that from 1002 yyou can access these [16:47:25] you would i think i remember from testing avro like 1000 times [16:52:03] ottomata: ok, done, stream of events looks kind of big for my homedir [16:52:17] ottomata: maybe there is a better partition where i can put it? [16:53:09] ottomata: maybe /srv? [16:53:21] ottomata: but it works so change can be merged [16:55:52] sure [16:56:02] ottomata: I imagine i have to be sudo to write to srv [16:56:09] not on stat1002 :) [16:56:11] ottomata: letme try [16:56:27] ottomata: ahem .. i was doing this on eventlog1001 [16:56:45] oh nuria on stat1002 its /a [16:56:49] ja you can use stat1002 [16:56:55] it isn't setup.py installed there [16:56:57] so you have to do [16:57:10] export PYTHONPATH=/srv/deployment/eventlogging/eventlogging [17:02:46] Analytics-Kanban, Patch-For-Review: Change analytics kafka cluster JMX metrics to be prefixed with cluster name and change alerts and dashboards [5 pts] - https://phabricator.wikimedia.org/T121643#1898527 (Ottomata) [17:03:18] Analytics-Kanban, EventBus: LVS/PyBal for eventbus - https://phabricator.wikimedia.org/T122217#1898530 (Ottomata) NEW a:Ottomata [17:05:28] ottomata: in 1002 things do not work due to [17:05:31] https://www.irccloud.com/pastebin/nKZCHLTp/ [17:05:59] Analytics-EventLogging, Analytics-Kanban, EventBus, Patch-For-Review: Puppetize eventlogging-service - https://phabricator.wikimedia.org/T118780#1898545 (Ottomata) Open>Resolved [17:06:01] Analytics-Kanban, Discovery, EventBus, MediaWiki-General-or-Unknown, and 7 others: EventBus MVP - https://phabricator.wikimedia.org/T114443#1898546 (Ottomata) [17:06:21] Analytics-Kanban, EventBus, Patch-For-Review: Refactor kafka puppet roles with hiera for more generic use than analytics [8 pts] - https://phabricator.wikimedia.org/T120957#1898548 (Ottomata) [17:06:24] Analytics-Kanban, EventBus, Patch-For-Review: Make analytics kafka puppet uses use new role::kafka::analytics::* classes [5 pts] - https://phabricator.wikimedia.org/T121659#1898547 (Ottomata) Open>Resolved [17:06:28] Analytics-Kanban, operations, Patch-For-Review: Move misc/udp2log.pp to a module [3 pts] - https://phabricator.wikimedia.org/T122058#1898550 (Ottomata) Open>Resolved [17:06:57] nuria: HUHhhH! that shouldn't happen [17:06:59] looking [17:07:14] ottomata: you can repro with [17:07:16] ottomata: /usr/bin/python /srv/deployment/eventlogging/eventlogging/bin/eventlogging-consumer @/home/nuria/MobileWikiAppShareAFact-consumer [17:10:41] yurik: around? [17:10:47] OHH we have seen this before..>.>... [17:11:04] this is some python module conflict bug [17:12:03] nuria: try export DJANGO_SETTINGS_MODULE='GRR' [17:12:08] export DJANGO_SETTINGS_MODULE='THIS IS A BUG' [17:12:12] ottomata: jajaja [17:12:15] sure [17:13:52] ottomata: working now [17:14:14] ottomata: we can merge blacklisting at your convenience [17:17:53] ottomata: wait, file doesn't appear [17:19:56] ottomata: no , it no works [17:19:59] https://www.irccloud.com/pastebin/cXQAeFBs/ [17:23:22] hmm, strange, that's not the zk uri you gave it [17:23:35] ottomata: to repro [17:23:54] "/usr/bin/python /srv/deployment/eventlogging/eventlogging/bin/eventlogging-consumer @/home/nuria/MobileWikiAppShareAFact/MobileWikiAppShareAFact-consumer" [17:26:14] nuria [17:26:15] missing ? [17:26:17] before topic [17:26:24] also, you don't need blacklist in there [17:26:30] cat /tmp/otto-consumer [17:30:07] ottomata: ah, sorry, cut & paste error from the other machine [17:30:48] ottomata: working now, that stream is huge though [17:30:59] ottomata: 3M in secs [17:32:13] aye hm [17:32:15] well nuria [17:32:21] HMMM [17:32:22] nuria [17:32:26] yes [17:32:31] since mysql handles the duplicates fine [17:32:41] we could just consume a message from the end of that topic now [17:32:43] and note the offset [17:32:47] madhuvishy, yep [17:32:50] then, blacklist [17:33:01] then, when we are ready to start producing to valid-mixed again [17:33:10] HMmMMmm [17:33:11] ah [17:33:11] hm [17:33:13] no [17:33:18] we can't produce to valid0mixed again [17:33:19] but, ok [17:33:50] yeah we can unblacklist, and then run a consumer process that consumes directly from the topic, starting at our noted offset, and produces to mysql [17:34:18] we'd just have to stop it once that process catches back up to wherever we unblacklisted [17:34:26] MhhhH, file is easier... [17:34:32] how long are you going to have it off nuria? [17:34:40] ottomata: ya but even easier, do not backfil [17:34:53] ottomata: jaime was saying that about 1 day [17:35:00] ottomata: which is going to be gigs [17:35:03] and gigs [17:35:31] ottomata: i say we do away with data unless they tell us otherwise on e-mail thread [17:35:52] nuria: 996G avail on /a on stat1002 [17:36:14] ottomata: k, will go for that and that will limit our backfilling capacity [17:36:19] k [17:36:41] madhuvishy, still around? [17:37:16] oh joal, I also saw that some fields are interchanged [17:37:19] yurik: hey, yes. On phone now - commuting to office [17:38:03] madhuvishy, ah, ok )) let me know when ready [17:38:15] the indexes of the original rows are not like in: https://wikitech.wikimedia.org/wiki/Analytics/Data/Pageview_hourly [17:38:22] Will be there in 10 minutes ish. [17:39:16] nuria: where are you writing data? [17:39:38] ah found it [17:39:52] nuria: go ahead and start your process [17:39:54] I will merge [17:41:01] mforns: ?? [17:41:20] ottomata: process started [17:41:32] joal, it seems that the fields in the Rows are not in the order specified in wikitech... is it possible? [17:41:57] mforns: sounds weird :( [17:42:03] nuria: you will have to restart eventlogging (or just that consumer) on eventlog1001 [17:42:10] oh wait [17:42:11] running puppet... [17:44:12] joal, found the problem. My BAAD [17:44:21] ? [17:44:39] my fault, wrong indexes [17:44:39] Please let me know mforns, could also be bad myslef: ) [17:44:49] oh of course [17:45:20] joal, check the indexes for year, month, day and hour at the end of rowToRowMap [17:48:56] joal, need to run an errand, will be back in a while [17:49:17] if you're not there any more, I'll see you tomorrow morning [17:50:43] nuria, I need to run an errand, I'm preparing for tomorrow's travel [17:50:54] mforns_away: k [17:50:55] I won't be able to make it to tasking [17:51:06] yurik: here [17:51:14] madhuvishy, yeppi ) [17:51:21] nuria, I'll be back in a while [17:51:59] nuria: you may restart eventlogging on eventlog1001 [17:52:49] yurik: ok - so you'd like to record graph usage stats in graphite - correct? [17:52:59] madhuvishy, yes [17:53:30] madhuvishy, we have built this -- http://searchdata.wmflabs.org/maps/ [17:53:36] for maps [17:53:52] but that system is too centrally controlled, and I would like that same data to be available via grafana [17:54:02] and for graphs too [17:54:17] yurik: okay - because of the way webrequest data is imported - we cannot do real time stats at the moment - for the restbase stuff, i have a spark job scheduled every hour that looks through an hours data, counts requests to /api/rest something, and send it to graphite [17:54:20] ottomata: restarted [17:55:21] madhuvishy, there are two components to it -- the varnish log analysis (just like what you do for restbase), and frontend event logging (not sure how that is done to be honest, i think its a query to the events db) [17:55:30] getting the first part would be great [17:56:53] yurik: ah alright - i think best way to do the varnish log stuff is to use spark - just to be consistent with our other jobs. i guess we can schedule an R script - would just be harder for us to maintain [17:57:15] madhuvishy, i would much rather we don't use R :)) [17:57:24] sql is usually powerful enough )) [17:58:01] where would i start? [17:58:17] yurik: can you file a task tagging Analytics - and mentioning what request patterns to match [17:58:24] sure [18:00:08] yurik: for Spark - we would need to use Scala, can also do python - but most other refinery jobs we maintain use Scala at the moment. I know search has some spark python code - so I'd probably check with joal and ottomata first. [18:00:29] we have tasking meeting now, so I can ask [18:00:36] * yurik knows python... not scala though [18:02:41] joal: we are in batcave [18:10:10] Analytics, Graph, Graphoid: Add usage to Grafana - https://phabricator.wikimedia.org/T122226#1898719 (Yurik) NEW [18:10:14] madhuvishy, https://phabricator.wikimedia.org/T122226 [18:11:15] Analytics, Graph, Graphoid: Add usage to Grafana - https://phabricator.wikimedia.org/T122226#1898730 (Yurik) [18:17:25] Analytics-Kanban, Analytics-Wikimetrics, Puppet: Cleanup Wikimetrics puppet module so it can run puppet continuously without own puppetmaster {dove} [? pts] - https://phabricator.wikimedia.org/T101763#1898743 (Nuria) The goal of this task is to do enough changes so we can deploy wikimetrics with ease,... [18:32:30] Analytics-Kanban, Analytics-Wikimetrics, Puppet: Cleanup Wikimetrics puppet module so it can run puppet continuously without own puppetmaster {dove} [? pts] - https://phabricator.wikimedia.org/T101763#1898780 (Nuria) Things to do: Move alembic and pip install, db creation, any server initialization, a... [18:32:37] Analytics-Kanban, Analytics-Wikimetrics, Puppet: Cleanup Wikimetrics puppet module so it can run puppet continuously without own puppetmaster {dove} [21 pts] - https://phabricator.wikimedia.org/T101763#1898781 (Nuria) [18:33:11] Analytics-Kanban, Analytics-Wikimetrics, Puppet: Use fabric to deploy wikimetrics {dove} - https://phabricator.wikimedia.org/T122228#1898784 (madhuvishy) NEW a:madhuvishy [18:41:46] Analytics-Kanban, Reading-Admin: Visualization of Browser data to substitute current reports on wikistats - https://phabricator.wikimedia.org/T118329#1898811 (Milimetric) just adding this here: https://www.netmarketshare.com/browser-market-share.aspx?qprid=2&qpcustomd=0 [18:43:17] yurik: thanks [18:45:02] madhuvishy: you there? [18:45:26] dr0ptp4kt: yeah [18:45:36] madhuvishy: can you spare a couple minutes for a video chat? [18:46:19] dr0ptp4kt: aah, i'm in tasking meeting - and was gonna go to lightning talks after. Can we do it at 12? if not i can ping when this meeting is done [18:48:47] madhuvishy: i just set a meeting. thanks! [18:51:39] thanks [19:00:32] Analytics-Kanban, Reading-Admin: Visualization of Browser data to substitute current reports on wikistats - https://phabricator.wikimedia.org/T118329#1898851 (Nuria) * tabular layout on dashiki (similar to http://reportcard.wmflabs.org ) (design a simple to edit config data schema) [8 pts] * make a graph... [19:19:12] Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1898880 (jcrespo) ``` MariaDB EVENTLOGGING m4 localhost log > ALTER TABLE MobileWikiAppShareAFact_12588711 ENGINE=TokuDB;... [19:22:55] Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1898885 (jcrespo) It would take 12-24 hours to convert the most problematic one, Edit_13457736. Do you want to do a batch... [19:43:52] ottomata|lunch: maintenance is done, whenever you can: https://gerrit.wikimedia.org/r/#/c/260628/ [20:12:05] nuria: done, puppet has run, you can restart eventlogging [20:27:46] ottomata: ok, done [20:27:52] that was pretty painless [20:29:06] cool! [20:29:09] backfilling ok? [20:34:20] milimetric: hiya! should I merge this now? [20:34:23] you ok to watch and check? [20:34:24] https://gerrit.wikimedia.org/r/#/c/257696/ [20:36:26] ottomata: i did not do any backfilling yet, sent an e-mail and i was thinking to do it if someone requests it [20:37:03] !log restarted event logging and removed table that was blacklisted for maintenance: https://gerrit.wikimedia.org/r/#/c/260628/ [20:39:08] oh ok [20:40:51] Yurik: did you filed the ticket? I do not think you need real time data, just usage data, correct? [20:41:24] nuria, https://phabricator.wikimedia.org/T122226 [20:42:10] Analytics, Analytics-Backlog, Graph, Graphoid: Add usage to Grafana - https://phabricator.wikimedia.org/T122226#1899202 (Nuria) [20:42:47] nuria, i don't really need the realtime data. I would be interested in getting user behavior data, like if the user clicks "play" button on the graph [20:42:56] Analytics, Analytics-Backlog, Graph, Graphoid: Add usage to Grafana - https://phabricator.wikimedia.org/T122226#1898719 (Nuria) >* "distinct users" count? (distinct ip+user agent is a good enough approximation I guess) This is not correct for the most part. [20:43:13] yurik: and you have instrumented that ? [20:43:22] nuria: no not yet [20:43:31] nuria, not yet - should be fairly easy though, right? [20:43:37] yurik: I see, that is data that is to come via EL [20:43:39] yurik: do you wanna set up a meeting do talk? [20:43:49] sure, even now if you want [20:43:58] nuria: no - i think they mostly come from webrequest [20:44:12] madhuvishy: the play button push? [20:44:19] yurik: hmmm too many meetings today - saturated [20:44:38] nuria: oh ya not that [20:44:54] but where is that? [20:44:54] madhuvishy: ah ok, plain requests yes. [20:45:01] nuria, https://github.com/wikimedia/mediawiki-extensions-Graph/blob/master/modules/graph-loader.js#L34 [20:45:26] madhuvishy: this falls into the bucket of "usage" of other systems like pageview api for example [20:45:28] nuria, madhuvishy - take a look -- https://www.mediawiki.org/wiki/Extension:Graph/Demo#Vega_2.0_Interactive_Examples [20:45:40] click the "play" button :) [20:45:49] nuria: webrequest already gets the api requests [20:46:06] we should use spark may be, count the things yurik wants to track [20:46:10] and send them to graphite [20:46:17] madhuvishy:I know. We have several tickets to this regard [20:46:23] Count usage of "x" [20:47:01] hmmm yes, this is more specific though - like what we have in the RestBaseRequests job [20:47:05] ideally we would have 1 job that does all the harvesting, not 1 per feature/api we wnat to track [20:47:09] *want [20:47:21] right [20:47:45] we can generalize the restbase job to do more than what it's doing [20:48:25] yurik: i see [20:48:58] madhuvishy: or split that in two, 1 harvest metrics to a table other publishes those to somewhere [20:49:16] yurik: yeah i've seen the interactive graphs :) was just at your talk [20:49:25] madhuvishy: but i understand now, thanks yurik [20:49:29] excelente ) [20:50:00] nuria: yeah that could be done too - i have been thinking about that more because it'd be good to have the data somewhere that's not the graphite backend [20:50:15] the current restbase job just sends to graphite though [20:50:18] also we need some entities so we are not dealing with regex [20:50:21] btw, we already collect the maps usage via hive queries, but it would probably be good to use grafana for that too [20:50:27] very similar to graphs [20:50:34] i have all the queries [20:50:54] yurik: cool, yeah we can flush out the different things to be tracked [20:51:13] btw, we do have specs for the different end points [20:51:13] nuria: but i'm guessing given scala, spark and oozie, we'd have to instrument it [20:51:20] this is sounding more like we should have an analytics grafana thing, no? [20:51:24] maybe the filter could even be made to consume those? [20:51:35] somehting for non ops related metrics [20:51:41] ottomata: yeah [20:51:47] one day! :) [20:52:13] ottomata: ya, we certainly do not want n queries for n systems we are tracking with regexes scattered all over ... [20:52:29] ottomata: i was wondering though where this code should go - currently the restbase job is in refinery - but din't search recently have some spark stuff in python? [20:52:46] madhuvishy: ideally we want spark in scala [20:52:51] yes [20:52:59] madhuvishy: to not increase maintenance costs [20:53:04] well, nuria if folks put these things in eventlogging, like, maps could send an event [20:53:08] and we maintain it in refinery-source? [20:53:12] I do have code that turns the path specs from a swagger spec into a regexp matcher [20:53:14] madhuvishy: maintaining several platforms is a lot of work [20:53:15] then anyone could send from kafka and into statsd pretty easily [20:53:20] if they had a spot to run a little thing [20:53:37] that is basically what ori's statsv thing does [20:53:40] its separate from eventlogging [20:53:44] gwicke: ahem... the least regexes the better [20:53:45] but, similar idea [20:53:56] nuria: what do you use for matching? [20:54:00] gwicke: ideally we would inspects headers not paths to get taht info [20:54:18] ideally it would just be a stat or event emitted from the app :) [20:54:19] nuria: we could put the path in a header, too ;) [20:54:27] ottomata: to track graph api requests? [20:54:38] more seriously, somewhere paths need to be mathed [20:54:39] i thoguht yurik was asking how many times someone clicked on a button [20:54:40] *matched [20:55:24] ottomata: oh yes - that's the more frontend-y part of the problem i think - we were gonna focus on the things we can get from webrequest first. [20:55:24] and ideally, we wouldn't need to manually maintain regexps to keep up with API specs [20:55:57] ottomata, yes, that's what I would be interested in knowning [20:56:30] maps will want to send events like "user is browsing", etc. I think we already have those as part of the eventlog stream. MaxSem will know better [20:56:51] what was the question? [20:57:08] MaxSem, remember you did something to track general maps usage (client side) [20:57:22] we added some code to WIWOSM, etc [20:57:38] we are discussing adding that code to see it in grafana [20:58:05] grafana is amazing - we all get to easily configure graphs [20:58:09] write an eventlogging subscriber that sends the stats you want to graphite [20:58:27] yurik: can't you add to your x -analytics headers in the extension something like 'graph pageview' [20:58:33] for example: https://github.com/wikimedia/operations-puppet/blob/production/modules/webperf/files/deprecate.py [20:58:40] yeah, was going to suggest that too [20:58:47] well [20:58:48] yurik: so instyaed of matching regexes we just look at headers [20:58:48] so many discussions going on at once :) [20:58:49] not with zmq :) [20:58:52] but ja [20:58:53] like that [20:58:58] madhuvishy: jajaj [20:59:01] hehe [20:59:08] i have a kafka one somewhere [20:59:16] do you have one that uses eventlogging code? [20:59:18] we will need path matching for cached API requests in the future [20:59:20] that would be easiest [20:59:42] for e in get_reader(kafka:///?topic=eventloggging_Maps...): [20:59:45] Analytics, Analytics-Backlog, Graph, Graphoid: Add usage to Grafana - https://phabricator.wikimedia.org/T122226#1899218 (Nuria) Rather than having to use regex can't you add to your x -analytics headers in the extension something like 'graph pageview' so those come tagged in and we just... [21:00:12] nuria, not really - the "interact" command loads standard resources from resourceloader [21:00:19] gwicke: everything should come tagged as to what it is pageview, preview, api request [21:00:22] i don't think i have a kafka one readymade [21:00:33] nuria: just tell all clients to do the matching right ;) [21:00:40] aye [21:00:44] i think it would be better to send a separate udp call to report usage [21:01:28] gwicke: no, there is no way you can do that right, you will run into conflicts and overcount eventually [21:01:39] gwicke: but that is teh futureeeee [21:01:41] *the [21:01:59]