[10:17:46] <wikibugs>	 10Analytics-Clusters, 10Operations, 10vm-requests: Create a ganeti VM in eqiad: an-test-ui1001.eqiad.wmnet - https://phabricator.wikimedia.org/T266648 (10elukey) ` elukey@cumin1001:~$ sudo cookbook sre.ganeti.makevm eqiad_B an-test-ui1001.eqiad.wmnet --vcpus 2 --memory 4 --disk 20 --network analytics START -...
[10:30:28] <elukey>	 klausman: morning!
[10:30:40] <elukey>	 if you are feeling better today we could create a VM in ganeti
[10:30:45] <elukey>	 for the test druid node
[10:39:06] <klausman>	 Sounds good
[10:42:59] <elukey>	 https://wikitech.wikimedia.org/wiki/Ganeti#Create_a_test_VM is the starting point
[10:43:24] <elukey>	 luckily ipam in all the dcs (except codfw) is now managed by netbox
[10:43:40] <elukey>	 so no DNS changes needed upfront to allocate ips etc..
[10:43:52] <elukey>	 (we used to do changes manually before recently)
[10:46:41] <klausman>	 So the fields in the Creation ticket. What's the project name? Location is probably eq, service name "Druid"? Internal network, as for the size of the machine, I have no clue.
[10:47:45] <elukey>	 I think we can skip the project, eqiad is right, service name I thinl an-test-druid1001.eqiad.wmnet
[10:48:04] <elukey>	 network is Analytics VLAN (there is a option for it in the cookbook)
[10:48:18] <elukey>	 for the vm size, we'd need to think about it
[10:48:42] <elukey>	 in the old test cluster, we had an old hadoop worker as single-node test cluster
[10:48:55] <elukey>	 but it was relatively beefy compared to what a vm normally is
[10:49:35] <elukey>	 I think we can definitely settle for something like 4 vcores, 8G of ram, 50/100G disk
[10:49:48] <elukey>	 that is not a small vm
[10:49:55] <elukey>	 but not huge either :)
[10:50:19] <klausman>	 Ok, I just looked at an-druid1001, and its druid JVM has 73G RSS O.o
[10:50:22] <elukey>	 Druid has a on-disk cache so 100G might be good
[10:50:39] <elukey>	 yeah but we load a ton of data to those nodes
[10:51:03] <elukey>	 for the test cluster we just upload a little little subset of data
[10:51:09] <elukey>	 just to prove that the whole pipeline works
[10:51:09] <klausman>	 Ok, so 4c, 7G, 100G. Any other requirements?
[10:51:14] <elukey>	 8G 
[10:51:19] <klausman>	 typo :)
[10:51:22] <elukey>	 ack :)
[10:51:30] <elukey>	 and analytics vlan, other than that I'd say no
[10:51:38] <klausman>	 ok
[10:55:40] <elukey>	 klausman: let's rename the title with something more meaningful :D
[10:55:59] <klausman>	 Sorry, trying to grok the six miles of prod server lifecycle doc
[10:57:09] <elukey>	 :)
[10:57:32] <klausman>	 So I have to create the vm in netbox first?
[10:57:51] <elukey>	 nono we can just run the cookbook
[10:58:08] <elukey>	 it will allocate the ips etc.. and also commit the dns change
[10:58:14] <klausman>	 Ok.
[10:58:22] <elukey>	 (the step to generate the commit takes minutes, be patient)
[10:58:39] <elukey>	 to run the cookbooks, we can jump on cumin1001
[10:58:54] <elukey>	 and with sudo cookbook sre.ganeti.makevm -h you can see the options
[10:59:09] <elukey>	 the other thing to decide is what row to use, usually I pick the least crowded
[10:59:13] <elukey>	 there are steps on the doc
[10:59:58] <klausman>	 Hurm. there's no easy way to see which Ganeti node is on which row?
[11:00:41] <elukey>	 https://netbox.wikimedia.org/search/?q=ganeti10&obj_type=
[11:02:00] <klausman>	 Does this look sane:
[11:02:02] <klausman>	 sudo cookbook sre.ganeti.makevm --vcpus 4 --memory 8 --disk 100  --network analytics eqiad_D an-test-druid1001.eqiad.wmnet
[11:05:08] <elukey>	 checking
[11:05:34] <elukey>	 +1 looks good
[11:05:47] <elukey>	 remember to run it in tmux/screen
[11:05:51] <elukey>	 (veeery long)
[11:06:50] <klausman>	 aaand it once more does not see my tmux
[11:07:23] <elukey>	 I usually tmux then sudo, and it works fine
[11:08:01] <klausman>	 https://phabricator.wikimedia.org/P13100
[11:08:05] <klausman>	 It does not, for me.
[11:08:54] <elukey>	 very weird
[11:09:09] <klausman>	 FOr some reason, sudo unsets the TMUX var for me:
[11:09:13] <klausman>	 cumin1001 ~ $ sudo -i
[11:09:15] <klausman>	 root@cumin1001:~# echo $TMUX
[11:09:19] <klausman>	 root@cumin1001:~# 
[11:10:14] <klausman>	 And looking at cumin's sudoers/sudoers.d it's no surprise. Nothing in there preserves $TMUX
[11:10:49] <klausman>	 I mean, I can set it manually, of course (which I'll do now), but something is broken
[11:12:46] <elukey>	 so I can repro as well, but if I just tmux + sudo echo $TMUX it works
[11:12:57] <elukey>	 but we can open a task to infra for this
[11:13:06] <klausman>	 oh, you mean tmux sudo cookbook, all in one commandline?
[11:13:29] <elukey>	 nono first tmux, then sudo cookbook etc.. inside
[11:13:33] <elukey>	 this is what I do usually
[11:13:58] <klausman>	 well, that doesn't work for me (and never has)
[11:14:10] <elukey>	 no idea then
[11:14:34] <klausman>	 I mean, if you start tmux on cumin, then sudo -i (to just a shell), is TMUX set for you?
[11:16:06] <klausman>	 Ah! but `sudo echo $TMUX` has $TMUX evaluated in the current shell, not the sudo'd one
[11:17:06] <elukey>	 ack
[11:17:15] <elukey>	 let's wait a bit before attempting again
[11:17:32] <klausman>	 Well, the VM thing is running now, I set TMUX by hand
[11:17:40] <elukey>	 ah okok
[11:40:43] <elukey>	 klausman: when the cookbook is done, you should see at the end a mac address
[11:41:14] <elukey>	 what I do usually is pasting all output in the task https://phabricator.wikimedia.org/T266648#6587763
[11:45:13] <klausman>	 ack, will do
[11:49:39] <elukey>	 going afk for lunch, will check later :)
[11:55:48] <klausman>	 Aye, capitano
[12:47:34] * klausman lunch
[13:27:24] <elukey>	 ahahha thanks for the "capitano"
[13:37:49] <elukey>	 klausman: when you are done ping me so we can finish druid :)
[13:38:05] <klausman>	 I'll be back in 20m or so
[13:38:46] <elukey>	 yep even an hour, didn't mean to rush :)
[14:07:33] <wikibugs>	 10Analytics: Quick data exploration CLI - https://phabricator.wikimedia.org/T265765 (10CDanis) Visidata looks amazing, thanks!  I love this idea.  The biggest use case I have in mind right now is for ad-hoc analysis of Network Error Logging data (see T257527).  This is a case where we have no control over the cl...
[14:09:17] <wikibugs>	 10Analytics-Clusters, 10Analytics-Kanban, 10Operations: Switch Zookeeper to profile::java - https://phabricator.wikimedia.org/T264176 (10elukey) Change rolled out!
[14:25:11] <elukey>	 !log restart zookeeper on an-conf1001 for openjdk upgrades
[14:25:14] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[14:29:32] <wikibugs>	 (03PS2) 10Mforns: Add Refine transform function for Netflow data set [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332)
[14:33:03] <wikibugs>	 (03PS3) 10Mforns: Add Refine transform function for Netflow data set [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332)
[14:33:23] <wikibugs>	 (03CR) 10Mforns: Add Refine transform function for Netflow data set (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332) (owner: 10Mforns)
[14:40:06] <wikibugs>	 10Analytics-Clusters, 10Analytics-Kanban, 10Operations: Switch Zookeeper to profile::java - https://phabricator.wikimedia.org/T264176 (10MoritzMuehlenhoff) 05Open→03Resolved
[14:40:16] <klausman>	 elukey: Ready when you are
[14:47:27] <klausman>	 Made a patch for the DHCP entry. Do we also need a partman setup or will there be a useful default?
[14:50:35] <elukey>	 here I am
[14:51:01] <elukey>	 yep it also need partman
[14:51:14] <elukey>	 see https://gerrit.wikimedia.org/r/c/operations/puppet/+/637387
[14:52:56] <klausman>	 ack
[14:53:18] <klausman>	 I presume this'd work: an-test-druid*) echo partman/flat.cfg virtual.cfg ;; \
[14:53:45] <elukey>	 yep, you can also add it |an-test-druid* to the other ones
[14:53:47] <elukey>	 as you prefer
[14:54:37] <wikibugs>	 (03CR) 10Mforns: [V: 03+2] "@Joal, responding to your comments:" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332) (owner: 10Mforns)
[14:54:46] <klausman>	 Updated the change
[15:03:06] <elukey>	 yep looks good
[15:03:27] <elukey>	 one nit - I usually add an entry in site.pp as well to avoid an extra change, but not really important
[15:03:48] <elukey>	 klausman: so now puppet needs to run on the dhcp hosts before starting the vm, as described in the doc
[15:04:57] <klausman>	 Ack, already done that. Also running it for apt (partman)
[15:06:22] <klausman>	 Is role(druid::analytics::worker) the right role?
[15:10:04] <elukey>	 for the moment let's use role(insetup)
[15:11:22] <klausman>	 roge
[15:11:24] <klausman>	 +r
[15:13:13] <klausman>	 Role: https://gerrit.wikimedia.org/r/c/operations/puppet/+/637511
[15:14:55] <klausman>	 Will now start the instance and see to the install
[15:17:02] <elukey>	 ack
[15:18:10] <wikibugs>	 10Analytics, 10Analytics-Wikistats, 10I18n: [[Wikimedia:Wikistats-metrics-top-mediarequests-name/jam]] translation issue - https://phabricator.wikimedia.org/T266669 (10Aklapper)
[15:19:36] <klausman>	 How long does it usually take before the console shows something useful?
[15:21:22] <elukey>	 very soon usually
[15:21:26] <elukey>	 line a couple of mins
[15:22:19] <elukey>	 I checked with gnt-instance info and I see "State: configured to be up, actual state is up"
[15:22:38] <elukey>	 so in theory the start should just work
[15:23:42] <klausman>	 `ganeti1011 ~ $ sudo gnt-instance console an-test-druid1001.eqiad.wmnet` seems to just hang
[15:25:06] <elukey>	 weird it should be running
[15:26:12] <klausman>	 does the console work for you?
[15:27:12] <elukey>	 nope
[15:28:05] <elukey>	 ctrl+] seems also not working
[15:29:02] <elukey>	 klausman: fixed-address an-test-druid1001.eqiad.net;
[15:29:05] <elukey>	 spot the typo :)
[15:29:38] <klausman>	 oops. Not enough wm
[15:42:17] <klausman>	 A-ha, now it works
[15:45:10] <elukey>	 gooood
[15:47:11] <ottomata>	 cdanis: o/
[15:47:25] <ottomata>	 so, you want to get the NEL data into Hive instead of logstash?
[15:47:32] <ottomata>	 if so, and if we keep client_ip there, which we can
[15:47:35] <ottomata>	 you'll get auto geocoding
[15:47:52] <cdanis>	 I think so?  but I wanted to make sure it would still be vaguely realtime there
[15:48:07] <ottomata>	 it lags by a few hours
[15:48:10] <cdanis>	 ah
[15:48:12] <ottomata>	 it'd still be in kafka
[15:48:15] <ottomata>	 realtime
[15:48:15] <cdanis>	 hm :/
[15:48:51] <cdanis>	 I have to admit that I feel like I'm missing context on a lot of things: why client_ip is undesirable in logstash, and if a realtime-ish setup like we have for wmf_netflow is easily reproducible or maintainable 
[15:50:20] <ottomata>	 ah
[15:50:37] <ottomata>	 is netflow realtimeish?  don't remember, ah yes we are ingesting into druid for that?
[15:50:47] <ottomata>	 i'm pretty sure the realtime netflow stuff isn't going to have auto geocoding
[15:50:55] <ottomata>	 as for client_ip in logstash
[15:51:00] <ottomata>	 i don't really get it either
[15:51:25] <ottomata>	 but i guess timo feels that logstash data has a simpler attack vector than analytics cluster stuff, and he doesn't think we should keep client_ip there
[15:51:34] <ottomata>	 but, i guess  that can be discussed more
[15:51:49] <ottomata>	 i don't think keeping it there (for < 90 days) is really any different than what we do in Hadoop
[15:51:57] <ottomata>	 but ¯\_(ツ)_/¯
[15:52:16] <cdanis>	 yeah we're ingesting into druid for netflow
[15:52:22] <cdanis>	 yeah I don't either
[15:52:31] <cdanis>	 OTOH I'm not that happy with kibana anyway ofc 🙃
[15:53:08] <ottomata>	 ha me neither
[15:53:21] <ottomata>	 i think we could probably do the same for nel that we do for netflow...i think its a bit manual but it could be ok
[15:53:40] <ottomata>	 but, if you want the counry and ASN in there
[15:53:47] <ottomata>	 yeah we 'd have to get it from the headers and set it 
[15:53:51] <ottomata>	 in eventgate i guess
[15:53:52] <ottomata>	 hmmm
[15:54:11] <ottomata>	 man that schema based header population idea is a good one, it just is a bigger decision than a one off
[15:54:12] <ottomata>	 maybe..
[15:54:14] <ottomata>	 lemme think about it more and discuss
[15:54:19] <ottomata>	 maybe we can do that
[15:54:20] <ottomata>	 i mean
[15:54:33] <ottomata>	 i can always add a stupid 'if $schema == NEL' { ...}
[15:54:36] <ottomata>	 to eventgate-wikimedia
[15:54:36] <cdanis>	 haha yeah
[15:54:38] <ottomata>	 but that sucks
[15:54:41] <cdanis>	 I had considered that too
[15:54:51] <cdanis>	 we could even do the mmdb lookups there, but yes, that is not great
[15:55:03] <cdanis>	 (not that doing it in inline C in VCL in ERB is great either, but)
[15:55:06] <klausman>	 elukey: running first puppet pass (cert is done and signed)
[15:58:56] <elukey>	 nice!
[16:00:42] <ottomata>	 the right thing would be to do ths mmdb look\ups in a stream processor!
[16:02:28] <ottomata>	 fdans:  yoohoo
[16:03:01] <ottomata>	 andrewbogott: uh oh adding analytics-internal to cloud-announce was a mistake!
[16:03:04] <ottomata>	 can you remove that?
[16:03:30] <andrewbogott>	 ottomata: how so?  It hasn't been that noisy has it?
[16:03:56] <andrewbogott>	 + don't you have an unsubscribe link?
[16:04:52] <andrewbogott>	 anyway, I unsub'd you
[16:05:15] <ottomata>	 we'll have to sub individually
[16:05:34] <ottomata>	 otherwise dan will have to approve person who emails cloud-announce in analytics internal
[16:05:41] <ottomata>	 and anyone who replies to cloud-announce emails
[16:05:45] <andrewbogott>	 ah, I see, that sounds like a pain
[16:05:52] <ottomata>	 but, thank you!
[16:15:21] <wikibugs>	 10Analytics: Refine event pipeline at this time refines data in hourly partitions without knowing if the partition  is complete - https://phabricator.wikimedia.org/T252585 (10Ottomata)
[16:17:07] <wikibugs>	 10Analytics, 10Event-Platform: Enable canary events for all streams - https://phabricator.wikimedia.org/T266798 (10Ottomata)
[16:17:24] <wikibugs>	 10Analytics: Refine event pipeline at this time refines data in hourly partitions without knowing if the partition  is complete - https://phabricator.wikimedia.org/T252585 (10Ottomata)
[16:17:26] <wikibugs>	 10Analytics, 10Event-Platform: Enable canary events for all streams - https://phabricator.wikimedia.org/T266798 (10Ottomata)
[16:17:40] <wikibugs>	 10Analytics: Refine event pipeline at this time refines data in hourly partitions without knowing if the partition  is complete - https://phabricator.wikimedia.org/T252585 (10Ottomata) To call this task done, we should first complete {T266798}.
[16:17:54] <wikibugs>	 10Analytics, 10Event-Platform: Refine event pipeline at this time refines data in hourly partitions without knowing if the partition  is complete - https://phabricator.wikimedia.org/T252585 (10Ottomata)
[16:18:14] <wikibugs>	 10Analytics, 10Patch-For-Review: Undo any temporary changes made while running in codfw - https://phabricator.wikimedia.org/T261865 (10fdans)
[16:18:24] <klausman>	 elukey: post-install puppet runs complete (all changes converged, no mor changes happening)
[16:19:14] <elukey>	 \o/
[16:20:06] <elukey>	 yep I can login, very nice
[16:20:12] <elukey>	 I have a code change ready to deploy the role
[16:20:50] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Undo any temporary changes made while running in codfw - https://phabricator.wikimedia.org/T261865 (10Milimetric) 05Open→03Resolved
[16:21:13] <dsaez>	 Hi all
[16:21:16] <milimetric>	 hi dsaez 
[16:21:32] <wikibugs>	 10Analytics-Clusters, 10Analytics-Kanban: Move the stat1004-6-7 hosts to Debian Buster - https://phabricator.wikimedia.org/T255028 (10fdans) 05Open→03Resolved
[16:21:34] <wikibugs>	 10Analytics, 10Analytics-Kanban: Move the Analytics infrastructure to Debian Buster - https://phabricator.wikimedia.org/T234629 (10fdans)
[16:21:46] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Combine filters and splits on wikistats UI - https://phabricator.wikimedia.org/T249758 (10fdans) 05Open→03Resolved
[16:21:52] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Import page_props table to Hive - https://phabricator.wikimedia.org/T258047 (10fdans) 05Open→03Resolved
[16:21:58] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Refactor breakdowns so they allow more than one dimension to be active - https://phabricator.wikimedia.org/T255757 (10fdans) 05Open→03Resolved
[16:22:00] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Combine filters and splits on wikistats UI - https://phabricator.wikimedia.org/T249758 (10fdans)
[16:22:17] <milimetric>	 fdans: you should get bulk-edit task permissions from Andre so you can just multi-close everything in the Done column
[16:22:39] <fdans>	 milimetric: but we don't want to do all of them for now right?
[16:22:45] <fdans>	 there are a bunch that haven't been discussed
[16:22:47] <dsaez>	 Quick Question: There is a way to do cross-db queries in mysql in the current setup on the stat machines? For example I need to join results from wikidatawiki with EnWiki
[16:23:03] <milimetric>	 fdans: yeah, but that's temporary, eventually we'll be caught up
[16:23:13] <milimetric>	 dsaez: no
[16:23:36] <dsaez>	 thanks milimetric. I'll go with HIVE then. 
[16:23:40] <milimetric>	 (that's one of the reasons we import the data into hadoop and are trying to figure out a way to make a more real-time import)
[16:24:32] <milimetric>	 (the reason is because the databases are sharded over different servers, and especially the big ones like enwiki and wikidatawiki, those are like one per box or something)
[16:24:32] <dsaez>	 got it. 
[16:32:26] <wikibugs>	 10Analytics-Kanban, 10Triagers, 10acl*phabricator: Add fdans to triagers for batch task editing - https://phabricator.wikimedia.org/T266801 (10fdans)
[16:36:36] <wikibugs>	 10Analytics-Radar, 10Editing-team, 10MediaWiki-Page-editing, 10Platform Engineering, and 2 others: EditPage save hooks pass an entire `EditPage` object - https://phabricator.wikimedia.org/T251588 (10fdans) cc @Milimetric
[16:37:18] <wikibugs>	 10Analytics, 10Analytics-Kanban: Analytics Presto improvements - https://phabricator.wikimedia.org/T266639 (10fdans)
[16:45:37] <wikibugs>	 10Analytics-Clusters, 10Operations, 10vm-requests: Create a ganeti VM in eqiad: an-test-ui1001.eqiad.wmnet - https://phabricator.wikimedia.org/T266648 (10elukey) 05Open→03Resolved
[16:45:57] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Event-Platform: eventgate-analytics-external occasionally seems to fail lookups of dynamic stream config from MW EventtStreamConfig API - https://phabricator.wikimedia.org/T266573 (10fdans) p:05Triage→03High
[16:47:22] <wikibugs>	 10Analytics: Decide to move or not to PrestoSQL - https://phabricator.wikimedia.org/T266640 (10fdans) p:05Triage→03Medium
[16:47:30] <wikibugs>	 10Analytics: Test Alluxio as cache layer for Presto - https://phabricator.wikimedia.org/T266641 (10fdans) p:05Triage→03Medium
[16:51:05] <wikibugs>	 10Analytics-Radar, 10Operations, 10ops-eqiad: an-presto1004 down - https://phabricator.wikimedia.org/T253438 (10Cmjohnson) Spoke with Dell tech, Chris Bennet today.  The ball was dropped by Dell, nobody ordered the new part and our case was left open and not owned by anyone.   Today a new case for the backpl...
[16:51:20] <wikibugs>	 10Analytics, 10Wiktionary: Add editors per country data for Wiktionary projects - https://phabricator.wikimedia.org/T266643 (10fdans) p:05Triage→03Medium
[16:52:05] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10I18n: [[Wikimedia:Wikistats-metrics-top-mediarequests-name/jam]] translation issue - https://phabricator.wikimedia.org/T266669 (10fdans) p:05Triage→03High a:03fdans
[16:52:54] <wikibugs>	 10Analytics, 10Event-Platform: Enable canary events for all streams - https://phabricator.wikimedia.org/T266798 (10fdans) p:05Triage→03Medium
[17:14:55] * elukey afk for ~30 mins!
[18:20:41] <wikibugs>	 (03PS4) 10Mforns: Add Refine transform function for Netflow data set [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332)
[18:21:05] <elukey>	 are we going to deploy today?
[18:22:31] <wikibugs>	 (03CR) 10Mforns: [V: 03+2] "I added some unit tests." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332) (owner: 10Mforns)
[18:40:00] <elukey>	 fdans: (if you are busy don't worry) - are we deploying today?
[18:42:48] <wikibugs>	 (03CR) 10Neil P. Quinn-WMF: Oozie job for Wikipedia Preview stats (035 comments) [analytics/wmf-product/jobs] - 10https://gerrit.wikimedia.org/r/635578 (https://phabricator.wikimedia.org/T261953) (owner: 10Sbisson)
[18:43:24] <wikibugs>	 (03CR) 10Neil P. Quinn-WMF: Oozie job for Wikipedia Preview stats (032 comments) [analytics/wmf-product/jobs] - 10https://gerrit.wikimedia.org/r/635578 (https://phabricator.wikimedia.org/T261953) (owner: 10Sbisson)
[19:10:46] <iflorez>	 Hello A-team,
[19:10:46] <iflorez>	 I'm unable to load Jupyter Lab on stat6 since the upgrade earlier in the month. I wonder if you have tips or recommendations? The default version is installed and I just checked with Morten and his has been working without issue. To note, I am able to use Jupyter notebooks, just unable to load labs. 
[19:11:39] <fdans>	 elukey: since joseph is ooto let's push to next week I think
[19:13:42] <ottomata>	 iflorez:  hi, i can try and look into in a a few minutes, but first i hafta ask...does the new conda based jupyter work for you?
[19:13:46] <ottomata>	 or is this the old swap stuff?
[19:14:00] <ottomata>	 https://wikitech.wikimedia.org/wiki/Analytics/Systems/Jupyter#Newpyter
[19:26:04] * elukey afk!
[19:53:35] <iflorez>	 yup, I was using the old swap setup. I will try the new conda setup. The docs say that's only on stat8 at the moment?
[20:01:32] <wikibugs>	 (03PS5) 10Mforns: Add Refine transform function for Netflow data set [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332)
[20:11:06] <mforns>	 ottomata: does this make any sense? https://gerrit.wikimedia.org/r/c/operations/puppet/+/637559
[20:35:17] <iflorez>	 success! thank you @ottomata. 
[20:35:17] <iflorez>	 https://wikitech.wikimedia.org/wiki/Analytics/Systems/Jupyter#Newpyter on stat6 worked. 
[21:09:56] <ottomata>	 great! 
[21:10:10] <ottomata>	 iflorez:  that'll be the new and eventually only way to do things anyway :) SWAP will be deprecated eventually one day 
[21:10:19] <ottomata>	 thanks for using it!
[21:11:43] <ottomata>	 mforns:  just a thought: it might be nice to be able to use such a function to add isp data to other datasets too
[21:11:46] <ottomata>	 but for now this is fine
[21:12:02] <ottomata>	 i'm thining of cdan is' network error logging stuff
[21:12:07] <ottomata>	 but we can adapt this for that then later
[21:12:09] <mforns>	 ottomata: how would that be then?
[21:12:18] <ottomata>	 well, you have things like ip_src hardcoded in
[21:12:33] <ottomata>	 maybe it'd be possible to make the ip column configurable
[21:12:45] <mforns>	 ottomata: oh, you mean the transform function?
[21:12:50] <ottomata>	 yes
[21:12:52] <ottomata>	 hmm
[21:12:55] <mforns>	 ah I see
[21:13:01] <ottomata>	 would it be better to add this extra info as a single struct or map column
[21:13:04] <ottomata>	 rather than all top level?
[21:13:14] <ottomata>	 like we do for geocoded_data
[21:13:15] <ottomata>	 ?
[21:13:31] <mforns>	 don't know...
[21:14:08] <ottomata>	 i think it might, then we could more easily re-use it in other places, rather than adding a bunch of top level fields, right?
[21:14:09] <mforns>	 the fields are not really related to each other
[21:14:16] <ottomata>	 oh no?
[21:14:38] <ottomata>	 reading more...thought it was mostly ASN stuff
[21:15:04] <mforns>	 also, the transform function makes some assumptions about the format of the source dataset
[21:15:12] <ottomata>	 huh..
[21:15:12] <ottomata>	 yeah
[21:15:17] <mforns>	 it needs 7 fields
[21:15:17] <ottomata>	 hmmm
[21:15:19] <mforns>	 no 8
[21:15:25] <ottomata>	 yeahhh
[21:15:50] <ottomata>	 ok, i'm only going to add one naming nit then, this would be annohing to make generic
[21:15:53] <ottomata>	 i take it back
[21:16:12] <mforns>	 ip_src, ip_dst, comms, net_src, mask_src, net_dst, mask_dst, peer_ip_src
[21:17:50] <wikibugs>	 (03CR) 10Ottomata: [C: 03+1] "One minor naming nit, +1 LGTM" (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/634328 (https://phabricator.wikimedia.org/T254332) (owner: 10Mforns)
[21:18:19] <ottomata>	 OH YOU were asking me about the puppet patch
[21:18:25] <ottomata>	 sorry i dunno how i got to the reinfery patch
[21:18:47] <mforns>	 hehe, no it's good as well!
[21:18:53] <mforns>	 comment makes sense, will change
[21:19:05] <ottomata>	 ya that looks fine too
[21:19:10] <mforns>	 yea? :]
[21:19:12] <mforns>	 k
[21:19:22] <mforns>	 don't know how to test that, though
[21:19:55] <ottomata>	 running a PCC
[21:20:58] <ottomata>	 https://puppet-compiler.wmflabs.org/compiler1003/26219/
[21:21:06] <ottomata>	 https://puppet-compiler.wmflabs.org/compiler1003/26219/an-launcher1002.eqiad.wmnet/fulldiff.html
[21:21:19] <ottomata>	 hmm maybe we can rename that file mforns 
[21:21:26] <ottomata>	 network_infra_config not very apt, no?
[21:21:27] <ottomata>	 lets see
[21:22:00] <mforns>	 aha
[21:22:19] <ottomata>	 network_region_config
[21:22:20] <ottomata>	 ?
[21:22:21] <ottomata>	 maybe?
[21:22:48] <mforns>	 yes, fine by me! I just copied the current naming of the source data
[21:23:06] <mforns>	 but network_region_config sounds great
[21:25:16] <mforns>	 the output file looks good!
[21:39:08] <ottomata>	 oh hm
[21:39:47] <ottomata>	 hm
[21:40:07] <ottomata>	 i dunno, guess if that is what they call it too
[21:40:08] <ottomata>	 we should keep it
[21:40:14] <ottomata>	 mforns:  i'm fine either way then
[21:40:39] <mforns>	 whatever you prefer, I can change tomorrow!
[21:40:58] <ottomata>	 ok mforns  gonna sign off, ping me tomorrow and I can merge
[21:41:08] <mforns>	 ok! see yaa thanks!