[13:51:39] OTTOMATA~~ [13:52:01] mooorning! [13:52:14] mooooooooooorning [13:58:31] how was the wedding? [14:08:39] wedding was fun [14:09:37] hey milimetric! [14:09:41] hey guys [14:09:53] man I wish some of you were on linux [14:10:08] ottomata, why are there still warnings on locke for the banner impression filter? [14:10:13] this wireless driver is from hell, there are 20 bugs filed against it :) [14:10:23] haha welcome to linux :D [14:10:43] becuase the path is wrong! [14:10:46] i'm lookingat it right now [14:10:47] it's good to be back for the most part, but wireless has gotten much worse - stupid hw vendors [14:10:56] I am having it check the age of the C exec, not the log file [14:10:57] dohhh [14:11:08] k [14:19:06] hey milimetric: i think we actually do want to support external datasources, right now you have to upload data to our servers and that makes it much more complicated [14:19:27] oh lol, dsc was going to respond something much rougher [14:19:50] i agree with you but he seemed pretty convinced [14:19:54] really? [14:20:15] i thought he also wants to do that [14:20:24] k, i'll delete comment [14:20:41] tell him that he should merge my pull request :D [14:20:51] are you already admin on wikimedia/limn? [14:20:55] yep [14:20:57] k [14:21:02] i'll hash it out when he's online [14:23:02] ottomata, what else is on your plate? can you review my java patch for lucene? [14:27:14] yup, that, access for dan [14:28:10] hey dan, did you get an svn account yet? [14:28:15] from sumana? [14:28:24] no [14:28:24] that or gerrit? [14:28:27] nope [14:28:33] mk, what shoudl your unix username be? [14:28:43] dandreescu is fine [14:28:47] mk [14:29:03] ottomata, can you also do this username request: https://rt.wikimedia.org/Ticket/Display.html?id=3517 [14:29:22] i can do stat1, ja [14:29:46] cool [14:30:06] yeah, so mysql accounts for db42 are created by asher, right? [14:31:26] i thikn so, ja [14:32:08] what's maryana's last name? [14:32:15] oh pinchuk [14:32:15] ? [14:34:41] 1 sec [14:34:53] Maryana Pinchuk [14:35:42] ja danke [14:42:15] hey drdee, what do you think [14:42:17] would this check [14:42:21] » » if (filterResultLine(searchterm)) { [14:42:28] be better placed outside of the actual logResults function [14:42:30] at the call? [14:42:33] i think it is more elegant that way [14:42:39] good point [14:42:45] but if someone comes along later and calls logResult it wouldn't filter their data [14:42:49] as in [14:42:51] someone other than us [14:42:56] decides to use it somewhere else [14:43:22] (not very likely to happen) [14:43:24] aye [14:43:27] ok will leave that comment then [14:45:00] ok reviewd [14:50:18] thx otto [14:51:10] yuppers, it sill needs approved by ops [15:02:04] ottomata, done [15:02:35] ah, you probably shoudl have amended your previous commit [15:02:35] hmm [15:03:09] aaaargghhhhhh [15:03:27] hmmmmm argh indeed, i mean, its ok… kinda annoying [15:03:37] sorry [15:03:42] sok [15:03:53] amend should never have become part of the normal git workflow [15:04:00] that's what it never was designed for [15:06:10] ok reviewed that too [15:11:18] and sure enough now it won't push using amend [15:15:40] git commit -a --amend [15:15:41] ? [15:17:32] ty [15:17:41] that's for changing the commit message right ? [15:17:54] I use that when I made some mistake [15:17:58] yes [15:18:08] and we use it for our workflow with gerrit [15:18:09] yeah, and for actually changing the committed patchset in gerrit [15:18:22] before it gets committed to the real git repo [15:18:29] ottomata, meet stefan (average_drifter), stefan is working on wikistats [15:18:51] hi stefan! [15:18:57] he is based in Romania and will work with me and Erik Zachte [15:19:12] hello Andrew :) [15:19:28] buna seara Stefan :) [15:19:34] salut milimetric1 :) [15:19:43] wow that's nice, co-nationals :) [15:20:07] yep, but I'm in Philly [15:20:17] romania, seems to be very close to spanish and french [15:20:29] phonetically at least [15:20:31] and italian, it's all one language [15:31:19] drdee, I +1ed that [15:31:28] TY SIR [15:31:54] i'm reading through onboarding docs - do you guys have IRC cloaks? [15:32:01] yes [15:32:07] * drdee is googling [15:32:19] I don't [15:32:20] i'm not qualified, it says you need 250 edits on wikimedia projects [15:32:29] http://meta.wikimedia.org/wiki/IRC/Cloaks [15:32:36] not true, see link [15:33:05] you don't need a cloack but you can apply for one [16:00:45] milimetric, do you have access to asana? [16:03:41] i have an account on it, yes [16:03:41] but i don't think you added me to anything [16:08:51] yea they're retarded [16:08:55] oops [16:08:59] lol, nvm [16:10:03] <-- noob pidgin user [16:11:47] sent you an invite [16:11:54] saw [16:12:24] hammer? [16:12:59] brb getting coffee [16:13:06] k, what's hammer [16:13:25] grabbing lunch, brb too [16:19:49] brb, lunch [16:30:25] morning evan [16:30:31] i mean erosen [16:30:38] how was the wedding cake? [16:30:43] pretty good [16:30:49] NICE! [16:30:51] if only you were in sf [16:30:55] I brought in the top two layers [16:31:00] ohhhh.....darn [16:31:14] show it at standup! [16:31:21] now i am getting coffee for realz [16:31:34] http://www.flickr.com/photos/21137575@N05/sets/72157631545427560/with/7990438292/ [16:31:50] i was just organizing the flickr "set" [16:41:18] okay you are a baking king [16:41:22] that's crazy [16:43:55] wow, that is crazy [16:44:12] but it does look like that girl's doing all the work though. just sayin [16:44:55] :D [16:45:11] i was just the only one would cared about taking the pictures [16:45:15] but yeah, that is true [16:46:13] you should photoshop yourself in there [16:56:31] https://plus.google.com/hangouts/_/e63f312df95ca3425157004ec4ced10870ab89e8 [16:58:35] ottomata, another reason to visit toronto: http://www.nytimes.com/slideshow/2012/09/09/travel/20120909-SURFACING.html (my hood) [17:07:40] https://plus.google.com/hangouts/_/e63f312df95ca3425157004ec4ced10870ab89e8 [17:17:08] ottomata, not peter merged the lucene patch so fingers crossed :D [17:17:34] woo [17:17:40] brb [17:18:44] looking through git and asana issue lists [17:18:46] ha, well he also has to build the new .deb and deploy it [17:18:48] for limn [17:19:34] D3 support is a big change, I think it might make sense for me to work with dsc on that and then grab an issue [17:19:49] sounds good [17:19:55] but if anyone thinks there's something independent, i'll grab it [17:20:15] what do you mean by independent? [17:20:22] like outside limn? [17:20:38] outside the D3 upgrade [17:20:39] milimetric: packaging limn for production :D [17:21:02] that's painful but needs to happen sometime [17:21:45] ottomata, what is the IP6 cidr range that covers the entire IP6 address space? [17:21:57] what's left on that? It seemed from what David showed me that it was building a nice js file [17:22:08] one thought is to create the actual website which uses Limn as middleware, in to which we could incorporate things like csv uploading [17:22:39] milimetric: no, we need a deb package with all the dependencies, we can't run npm install on production servers [17:22:50] @drdee oh ok, cool [17:23:36] @erosen that goes in with supporting external urls for data [17:23:47] definitely [17:23:50] drdee, you think that should happen in limn right? [17:24:04] and i thought dschoon was more about separating it [17:24:22] like erosen's suggesting [17:25:14] drdee, is there an asana || github issue for deb package? [17:25:14] I would prefer it to be part of limn personally, but I could be convinced either way [17:25:48] yes, i think limn should support external URL's as datasource, not having a super strong opinion on where but i think in limn makes sense but dschoon should definitely weight on this as well [17:26:26] If it was part of Limn, we could make it as decoupled as necessary, we'll wait for dschoon's opinion [17:26:53] milimetric: yes limn deb task is in Servers & Configuration, just assigned it to you [17:28:31] drdee [17:28:33] re ipv6 [17:28:34] i think this [17:28:35] 0000::/0 [17:28:38] will cover all addies [17:28:46] ty ottomata [17:34:10] re: asana - do you use "Today" as in "I'll finish today" and "Upcoming" as in "I'm working on it but won't be done Today" and "Future" as in "I'll work on it later"? [17:34:50] back [17:35:02] milimetric: that's what the proxy package is [17:35:08] it's implemented, it just needs a UI [17:35:34] milimetric, feel free to use 'Today' etc, i don't think anybody is using it right now [17:35:39] i use it [17:35:48] :D [17:35:51] i do, sometimes. [17:35:53] and yeah, i use it like that [17:35:58] i have been all over the place recently though. [17:35:58] k [17:36:00] usually though [17:36:04] ignore what i said [17:36:04] today is I am currently workin gon it [17:36:05] so i haven't really been updating asana [17:36:08] and upcoming is I will start working on it soon [17:36:50] and Future is "snowball's chance"? :) [17:36:55] haha [17:37:06] yeah, haven't even thought about it [17:37:07] so yup [17:37:07] hah [17:37:43] actually it would be a good today for everybody to clean up asana tasks, close old ones and add new ones :D [17:38:17] reminder: github is now the primary point of work for limn [17:38:24] though... i guess you don't get reports then [17:44:04] ahhh drdee, you sure you wanted to abandon that? [17:44:11] your second patch depended on it [17:44:23] ottomata's too fast for me [17:44:37] what's asana? [17:44:43] asana.com [17:47:34] average_drifter: you have some clicktracking questions still? ask the wind, maybe someone has an answer [17:47:51] wow, crazy cake [17:48:03] who's wedding? [17:49:02] and who's the cook? [17:49:24] my cousin's [17:49:26] wedding [17:49:31] me and my gf, were the cooks [17:53:15] drdee, when you say "based on the sampled squid data" do you mean the reader should multiply by 1000 or you already did that? [17:53:35] i believe the reports of that data are all normalized [17:53:38] in which email thread [17:53:43] so if you got it from ezachte, it's normalized [17:53:46] the ip 6 you just sent [17:53:50] (as in limn for example) [17:53:54] that's already multiplied [17:53:56] maybe you should say "estimated based on sampled squid data" [17:54:11] drdee, did you confirm that that range does not include IPv4 addresses from the logs? [17:54:32] yes, by scanning at the output, i didn't see any ip4 address [18:00:13] erosen: can you mail me a slice? [18:00:17] ;-) [18:00:25] if only [18:00:36] i've got 140 servings woth [18:00:45] worth* [18:01:21] ok cool, drdee, that sounds like a good confirmation then [18:01:34] that is a huge increase in IPv6 requests since june though, no? [18:01:42] yeah it is shocking to be honest [18:01:57] maybe somehow it is wrong? maybe we weren't logging IPv6 requests before? [18:02:36] did timstarling push any ip6 related changes to udp2log since june 6th? [18:02:54] we certainly weren't logging IPv6 requests before nginx [18:03:12] * jeremyb only has half an idea what you're talking about [18:03:25] drdee: lol [18:03:30] yes, that is a reasonable patchset [18:04:00] ugh, /me stabs drdee!!! [18:04:23] ? [18:04:33] you already had reviews on the very same changes! why make a new gerrit change now? [18:04:39] i approve of not having social security numbers in search results. [18:04:53] ottomata is sometimes guilty of the same thing ;-( [18:05:14] jeremyb, because drdee accidentally abandoned a dependent change [18:05:18] is there a way to un-abandon? [18:05:20] i noticed [18:05:25] i'm testing right now [18:05:33] i didn't know it was dependent [18:05:49] 17 18:05:38 <+gerrit-wm> Change restored: Jeremyb; "this is a test" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/17041 [18:06:17] how'd you do that? [18:06:20] —amend? [18:06:28] to the abandoned commit? [18:06:36] no. just click the restore button and put in a comment [18:06:40] restore button!? [18:06:51] yes [18:07:22] where is that? [18:07:49] next to the other buttons? (e.g. next to where abandon would be if it were not yet abandoned) [18:07:54] * drdee facepalms [18:08:01] err, s/e\.g\./i.e./ [18:08:02] ah I guess I didn't abandon it so I don't see that [18:08:18] haha, thank you for escaping your periods for us in IRC [18:08:18] haha [18:08:34] sure! [18:08:52] umm, drdee, maybe the proper thing to do [18:08:53] if you can do it [18:09:00] is to un-abandon that first commit [18:09:03] and get notpeter to merge both [18:09:08] and then abandon your most recent one [18:09:13] and abandon the newest one. right [18:10:00] one reason i hate it when ya'll do that is sometimes I've gone out of my way to subscribe to one of the originals. so then I end up getting no mail on the new dupe [18:12:01] milimetric [18:12:06] you should have access to stat1 noe [18:12:07] now* [18:12:17] yes [18:12:23] i gotta change locations, want coffee and i'm about to change work gears [18:12:51] i mean, yes! [19:49:35] dschoon, you around? [19:49:42] yep [19:49:56] got disks all setup, looking into DSE configs [19:50:05] do we want to have some nodes for regular (real time) cassandra usage? [19:50:12] or should I just make them all for analytics (hadoop)? [19:50:25] this is just for now, btw, we can change this all later if we need to [20:06:56] dschoon? [20:07:04] sorry, missed your replies [20:07:06] reading [20:07:28] i believe the difference is that the "analytics" nodes run CFS [20:07:39] as don't do KV point queries [20:07:44] ottomata ^^ [20:08:38] right [20:08:45] so hm [20:08:46] so, for now, do we want any that aren't running CFS? [20:08:53] i guess make 1 of them not run cfs? [20:09:04] hmm [20:09:07] maybe 7/3 [20:09:08] ? [20:09:08] or [20:09:13] maybe just 10 CFS for now [20:09:19] and we can figure out if we want any non CFS later? [20:09:23] we still have the dells lying around [20:09:26] to play with too [20:15:29] dschoon? [20:15:38] hm. [20:15:46] the dells are going away soon tho [20:15:48] better not to rely. [20:15:57] i think 1-2 normal KV nodes is fine [20:17:17] we don't really have a use for them right now, do we? [20:17:25] that will just decrease the number of analytics nodes, ja? [20:20:37] btw, why don't we stick the the original plan and build a 5 node DES cluster and a 5 node CDH4 mini-cluster and experiment with both, and compare them in terms of performance, configurability, stability, and maintainability? [20:21:46] erosen: quick Skype call? [20:23:30] that was the original plan? [20:23:37] yes [20:23:47] for testing [20:23:50] i think the original plan was to do CDH4 on 10 nodes, perf test, then do DSE on 10 nodes, perf test [20:24:05] in my mind, it was always in parallel [20:32:57] i don't think that was really our plan. [20:33:04] i think you've said that many times, drdee [20:33:17] and ottomata and i have voted against [20:33:32] i agree with otto [20:33:49] that having stuff up in parallel doesn't help evaluate anything [20:33:52] but having a full cluster does [20:34:07] so it makes more sense even if we want to be cautious to do them one after another [20:34:24] but at this point, i agree with otto that our plan should be to start crunching numbers for reals. [20:34:42] basically to set up DSE, then point the udp2log stream at it [20:34:44] and see how things perform [20:37:00] so it'll be cool to have that data there and then test out hive & pig [20:37:11] that's the sort of perf test i think would be really interesting [20:52:43] brb food [21:23:52] back [21:25:09] dschoon: i don't agree that " having stuff up in parallel doesn't help evaluate anything", i am just worried that we will stick with whatever we install now [21:25:22] that's legit [21:25:27] but we're all smart chaps [21:25:35] but again i fully agree with getting real experience [21:25:41] so let's just move this ship [21:25:50] i think we are especially sensitive to some of the decisions around here that have been made out of ossification (gerrit) [21:26:14] umm, :D well it's a anti-pattern at WMF [21:26:31] but anyways, let's just be aware of it [21:28:02] we are of one mind. [22:11:15] drdee, i'll work on webstatscollector tomorrow if you like [22:11:26] I don't really know what Tilman is asking me to do though [22:11:33] yeah let's do that together! [22:15:21] dschoon, cassandra is running…although something is def not working [22:15:26] the nodes doesn't seem to be communicating properly? [22:15:27] not sure [22:15:30] i'm heading out soon [22:15:33] i'll look into it [22:15:39] i've seen that a billion times before [22:15:40] but [22:15:47] check to see which interface is default [22:15:48] before you go [22:15:52] oh [22:15:52] hm [22:16:00] for cassandra or in routes? [22:16:39] listen_address: 10.64.21.107 [22:16:40] ? [22:16:43] should be set to the IP of each node [22:16:44] by puppet [22:16:57] i mean in linux [22:17:07] in route tables? [22:17:17] i'm not ops, heh. i guess? [22:17:18] Destination Gateway Genmask Flags Metric Ref Use Iface [22:17:19] localnet * 255.255.255.0 U 0 0 0 eth0 [22:17:19] default vrrp-gw-1021.eq 0.0.0.0 UG 100 0 0 eth0 [22:17:36] i just recall that eth0 needs to be the IP we want them to chat over [22:17:43] yeah, that's true for all of the [22:17:44] though hopefully there's conf for that now [22:17:44] m [22:17:49] okay. [22:17:51] i'll look into it [22:17:54] and send an email [22:17:59] they each only have a singel IF [22:18:01] ok cool [22:18:06] so you know [22:22:01] latas alllll [22:22:02] ! [22:22:12] lates! [23:03:50] I've strategically shifted my timeschedule [23:03:55] alright, hi again everyone [23:04:05] jeremyb: yes you're right I have some curiosities about clicktracking [23:04:15] so I was able to set up a mediawiki on my vps [23:04:59] with clicktracking, wikieditor, userdailycontribs [23:05:41] Server-side Dependency: apparently the getUserEditCountSince() from userdailycontribs was used inside of clicktracking [23:05:52] average_drifter don't forget i am EST, not PST:) [23:06:37] Client-side actual usage: inside wikieditor extension (there are direct/indirect calls to $.trackAction calls) [23:07:05] and after all this has been set up, many console.log() statements throughout the code, reading the Talk page of ClickTracking extension, configuring my LocalSettings.php [23:07:09] I have a mediawiki with clicktracking [23:07:27] you could ask ori-l or spage (you can find both in #mediawiki) more about the clicktracking extension but wikistats is your thing, not the click tracking extension [23:07:45] drdee: alright [23:08:10] drdee: are they related ? [23:08:19] wikistats and clicktracking? [23:08:22] no, not all [23:08:33] i mean no, not at all [23:08:53] average_drifter: what are the questions re: clicktracking? [23:09:26] ori-l: well basically I was thinking, is there somewhere a TODO for them or a buglist ? [23:24:48] dschoon: probably a bit biased but could be useful: http://www.datastax.com/resources/whitepapers/hdfs-vs-cfs [23:34:43] cool, thanks [23:49:49] dschoon, DarTar, average_drifter: see you guys tomorrow! [23:49:59] c ya [23:49:59] lates [23:52:45] see you