[07:39:39] jbond42: i've opened https://phabricator.wikimedia.org/T256448 to track what we were discussing the other day, but i'm not excluding the possibility that i'll send a CR to implement it myself [08:01:35] kormat: awesome thanks [08:05:48] jbond42: wait - you have your own phab project? _fancy_ [08:05:53] how do you use it? [08:06:09] kormat: https://phabricator.wikimedia.org/project/view/4357/ [08:06:53] huuuh. i think i might want me one of those [08:07:57] i just created a task and taged Project-Admins https://phabricator.wikimedia.org/T236922 [08:17:58] cool, created one for myself. [08:43:15] +1 on personal phab tags, having a workboard works better for me than assignment [09:56:45] kormat: link your mediawiki account with phabricator, afaik they need it for security issue access [09:58:42] Majavah: done, thanks [13:29:00] elukey: XioNoX: something we should experiment with for netflow: https://github.com/allegro/turnilo/issues/634 [13:38:14] seems really nice! [13:38:33] we have a test instance cdanis on an-tool1005 (alongside with superset) [13:38:52] you have to create a ssh tunnel but then you can make any experiments with the config [13:39:29] (turnilo runs on 9091) [13:40:01] ack, ty [13:50:49] cdanis: that's awesome, thanks! [13:51:07] yeah, hopefully some day it becomes an actual UI [13:51:11] the other feature request that I had was to use standard prefixes, like megabytes/MB instead of "billion bytes" [13:51:42] but I look forward to being able to stop running things like `units 21.5e9bytes/minute gigabit/second` [13:51:46] yup, same here [13:51:56] you can do Gbps for the latter, fwiw [13:52:05] right [13:52:14] "units 15 billion bytes/hour Mbps" works [13:52:24] I just typed gigabit/second the first time and have been C-r'ing ever since :) [13:52:39] cdanis: can I nerdsnipe you for https://phabricator.wikimedia.org/T254332 [13:52:58] ahah I have this starred in my gmail [13:54:04] AS names is something I'm very interested in, as would be some sort of GeoIP information [13:54:31] and yeah I'm *starting* to unintentionally memorize exporter IPs, but I'd really rather not [13:55:58] I'm looking at something like traffic per region, which in many cases vary [13:56:01] *varies [13:56:15] I mean in the sense of "top ASNs we don't peer with in this region" [13:56:25] right [15:02:01] speaking of nerd snipe, a goal for next Q is to migrate netflow data to a more Eventgate-like layout https://phabricator.wikimedia.org/T248865 [15:02:40] the idea is to have a kafka topic for each datacenter (that corresponds 1:1 to each netflowX001 instance in theory) [15:03:01] that might make the 'regional' thing easier [15:03:11] (although it doesn't seem hard as-is; there's a small handful of exporter ips) [15:03:13] TIL `units` [15:03:20] skipping the HTTP validation part (pmacct will keep pushing directly to kafka) [15:03:21] oh XioNoX it's very good [15:04:12] with the caveat that short unit names can sometimes map to surprising things [15:04:16] the thing that I have to think about is how to migrate the existing data to the new layout, since atm we have custom puppet code to import data to hdfs, refine, push to druid, etc.. [15:04:21] kormat: conformability erorr [15:08:14] cdanis: hmm? [15:08:28] the region thing seems orthogonal to me [15:08:47] kormat: sorry, that's a misspelling of the output units gives when you ask it to do something that doesn't make sense [15:09:24] cdanis: ah :) i'm more familiar with getting truly bizarre results instead [15:09:53] because it turns out you divided megabytes by.. the planck constant or something. [15:10:02] seems we're talking about R regions containing N routers each forwarding to a pmacct instance (of which we have M), with each pmacct instance pushing to a kafka topic (of each we have K) [15:16:47] I don't know why we'd have multiple kafka topics (but indifferent myself :) [15:17:00] what I care about data-wise is the router->region mapping [15:17:13] really, "R" from the list above [15:48:58] paravoid: it is more a eventgate thing, since every event that gets validated by eventgate-analytics (in this case) is pushed to a topic with a prefix indicating the datacenter [15:49:35] since pmacct doesn't currenty support sending netflow data via HTTP POST, we thought to skip the eventgate initial validation and push directly to kafka [15:49:49] but it must follow what eventgate does (at high level) basically [15:52:30] the idea is to avoid keep using an ad-hoc setting for netflow, and just move it under the eventgate umbrella [15:53:41] ok [15:54:03] I would say that not being able to do POSTs and instead pushing to kafka directly is a feature :P