[00:02:51] Ironholds: sure [00:03:19] TerrifiedPanda, the pageviews for mobile apps: you said it'd be trivial. Wanna put your money where your mouth is? ;) [00:03:27] able [00:03:31] that's not a trick question - I haven't found some mad gotcha or anything. [00:03:32] ah [00:03:38] I'm just busy trying to work out bloody UUIDs [00:03:47] all 36-character, except the one 32-character one. weird. [00:04:15] I could give it a shot, but I'm trying to reduce *any* dependency between me and the apps team [00:04:28] Oh, sure. It's just writing the cron task and the query [00:04:37] I'll not tell them you did it *patspats* [00:04:47] you write the query, I'll write the cronjob + puppetize it [00:05:09] and I'll do it in a way that makes adding future cron for hive queries simple [00:05:33] okie-dokes! [00:05:37] I'll email it to ya later? [00:05:55] sure [00:06:07] I mostly have it figured out how to write Hive queries from python [00:15:17] cool! [12:10:47] yo halfak :) [12:10:52] wanna chat here or continue via email to document? [12:11:00] Here is good. [12:11:04] Shouldn't you be sleeping? [12:11:14] That email came 5 hours ago! [12:11:16] I had a wonderful nightmare involving sleep paralysis, strangers and the outside. [12:11:34] https://en.wikipedia.org/wiki/Night_terror [12:11:40] oh, and lucid dreaming. So I was aware it was a dream but couldn't do anything, because sleep paralysis. Pretty sure I choked out a scream at one point though. [12:12:08] anyway. Kinda disrupted sleep ;p. [12:12:16] Okay: so, in order, re your questions... [12:13:37] the sample-gathering comes from grabbing in all the pageviews from Apps in [period] (I picked a day, because Hive partitions around days and so it was a nice testbed dataset. Came to about 5m rows) [12:14:14] Oh... day is not going to work. [12:14:19] whyso? [12:14:36] We need inter-time periods that can extend up to 1 year in order to see the common pattern. [12:14:41] the second bump is at 24 hours! [12:14:51] The third is at 6 months. [12:16:18] * Ironholds blinks [12:16:42] I don't get it. But I'm dumb ;p. [12:17:10] In order to see an inter-event time that is greater than 24 hours in length, we'll need events that span more than 24 hours. [12:17:15] absolutely [12:17:37] but I wouldn't expect to see a 'session' featuring pageviews 24 hours apart. [12:17:45] Agreed. [12:17:55] so...colour me confused. [12:18:04] (or, you know, off-white, which is sort of my default scheme) [12:18:53] I want to make this figure: https://meta.wikimedia.org/wiki/File:Inter-edit_time.png [12:19:05] OH. Gotcha. Yeah, totally. [12:19:34] that's the plan. The full run will consume 30 days of data (we don't have 6 months) - this was just a "can I write the code to be really efficient" test run. [12:19:48] (I got a 2om improvement out of using data.table for an operation instead of data.frame. Crazy.) [12:20:11] what's worrying me is that with the other datasets we saw that distribution centred around the hour, with the noted 430 second point [12:20:23] with this we just see a sort of...tremendous drop after about 10 seconds. [12:20:41] The distribution was never centered around an hour. [12:21:04] It is centered around 1-2 minutes. [12:21:25] gotcha. Sorry, misreading. [12:22:14] but yeah. 10 seconds with a tremendous drop after that feels...problematic. Which leads me to: re your second question about an example set of intertime periods, would you like a .RData of the uuid - timestamp combos instead? In case there's something tremendously wrong with my code. [12:22:27] I don't know why I'm asking exactly how you'd like to do me a solid, but ;p [12:22:50] What's the Var1 column? [12:23:12] seconds after the previous event [12:23:20] Ahh... bucketed on the second. [12:23:32] Yeah, that will make the lower end of a log distribution look funny. [12:23:56] RData of the inter-event times will rectify this. [12:24:38] gotcha [12:25:05] that is, the measurement in seconds, non-aggregated, or uuid/timestamp combos? [12:25:18] Sub-seconds would be great [12:26:41] we don't actually gather ms-level timings with the request logs, I don't think. [12:27:15] hokay [12:27:29] How are you subsampling within the day? [12:28:01] I'm not, actually: other than excluding people who only make 1 request, this is all apps requests in a 24-hour period. [12:28:17] * halfak observes the 89k events that happen within zero seconds of the last one. [12:28:35] yep. I'm imagiwait. [12:28:37] OK. That's just double-taps [12:28:43] yeah, probably. but..hrm [12:28:45] * Ironholds checks query again [12:29:04] okay, I should /probably exclude uncompleted requests/ [12:29:06] I am a dumbass. [12:30:12] will do that when my referer code completes and I have spare RAM on stat2, then regroup and see what happens. [12:30:24] Sorry for sucking in your cycles, dude! [12:32:40] Na. Fun data :) [12:33:37] halfak, if it helps, I just discovered a resource you're going to luuurve. [12:33:59] you remember your work around connection speeds as an influencer on user actions? [12:35:33] check yer email. [12:35:36] * Ironholds dances [12:35:43] I'm going to have so much fun looking at mobile behaviour :D [12:37:59] huh. That's interesting. [12:38:17] I note that 2g,3g and 4g are not listed as the speeds they report. [12:40:27] unfortunately not :(. But we can look at some fun stuff. For example, you know my circadian work and the hypothesis that the peaks are around transit to work, and lunch? [12:40:54] What happens if we traceback the mobile IPs and find that those periods also show a massive switch to cellular carriers for requests rather than various wifi connections? :D [12:41:13] * Ironholds schemes [12:41:22] Will it tell is the carriers? [12:41:25] *us [12:41:45] Good question; I don't know. But the existing setup already gives us ASN. [12:41:54] ASN? [12:41:54] I've been wondering for a while if we could get anything company or org-specific out of that. [12:42:13] https://en.wikipedia.org/wiki/Autonomous_System_%28Internet%29 [12:43:06] actually even if we don't have the specific carrier extractable from the ASN, if we combine ASN + netspeed type we can probably identify cell carries. But this is..me getting ahead of myself. Let's look at Netspeeds and see from there. [12:43:44] Have you been talking to ops people? [12:43:48] They'd be interested [12:43:54] And have some good questions. [12:44:09] I haven't, although I need to get back to Faidon about his IPv6 adoption rate request. [12:44:24] He's the dude to talk to about connectivity, right? [12:44:30] I think so. [12:44:49] We've had some good chats about measuring the /other side/ of our mission statement. [12:44:56] (consumption & availability) [12:45:51] * Ironholds nods [12:49:55] halfak: btw, I left some comments about making 'rule of 3' be rule of 4 in that google doc about the reorg [12:50:25] <3 [12:50:35] It's nice to have someone else say that. [12:51:08] I've been working on my arguments about why subject-matter-experts-in-the-functioning-of-wikis are consulted when changing the functioning of the wiki. [12:53:39] halfak, the sad thing is that you need arguments [12:53:48] "it would be nice if people who knew what the fuck they're doing were occasionally consulted" [12:54:07] I mean, we're still ignorant. Don't get me wrong. But.. [12:54:34] the wider movement, when it comes to predicting the future, consists largely of people with blindfolds on, in a dark room, trying to put a puzzle together. [12:54:56] * TerrifiedPanda pats Ironholds [12:54:56] Well... I wouldn't say we Know What We Are Doing(tm) so much as we know what's known and we have the tools to come to know the rest as necessary. [12:55:01] I'm also no longer terrified. [12:55:01] people like you and stuart and morten still have blindfolds but they're made of gauze. And in the land of the blind... [12:55:25] halfak: I made a follow up comment. please do expand on that [12:55:27] halfak, totally. Nobody knows everything. But yeah, you know some stuff and have the toolkit to get at a lot of the rest and the attitude to do that. [12:55:44] * YuviPanda thinks 'attitude' is perhaps the biggest oother problem [12:56:29] also looks like I might potentially move to another team before this all ends, since someone (Terry?) has put a 'consider services) under it [12:56:46] halfak: btw, my US Work visa was approved yesterday [12:56:52] +1 on attitude. If you knowledge as easily-attainable stuff that obfuscated and held hostage by the academics, it's going to be hard to work together. [12:57:00] Woo! Merca! [12:57:15] YuviPanda, where in the world are you right now? [12:57:36] * halfak can't type yet -- still working on coffee. [12:57:41] halfak: Glasgow, London tomorrow [12:58:11] halfak: indeed. but product loves research/analytics anyway, so I'm sure they'll represent those viewpoints there... [12:58:36] halfak, could not parse your second sentence [12:59:02] Trying again: If you see knowledge as easily-attainable stuff that is obfuscated and held hostage by the academics, it's going to be hard to work together. [12:59:07] ohh. gotcha. yep. [12:59:17] And, I would also put it the other way around too. [12:59:31] If you see knowledge as something that is either personally known by you, or false, same problem. [13:00:01] We should have an attitude of: if I don't know something I should try to find it out, and if I still can't work it out but trust the people who can, That's Okay. [13:03:38] I'll probably never grok Morten's algorithms on more than a conceptual level, but I know he knows what he's talking about, and that's enough. [13:03:52] Relying on sub-truths because their parent truth is a known is fine. [13:04:17] * Ironholds really wishes he'd studied more ontology as a student. It was fun. [13:04:32] Indeed. We rely on peer review and rebuttal to obtain scientific consensus. You can trust the consensus machine. [13:04:46] well, sometimes [13:04:53] Carsten Schaepers and his Paper of Doom (tm) [13:05:04] And by "trust", I mean "cautiously base new hypotheses on". [13:05:11] but the way peer review works is: I can access the tools and the datapoints and the background research to know that paper is terribly flawed. [13:05:15] and so I can correct for it. [13:05:26] and heavily judge the people who approved it >.> [13:05:33] Ironholds, peer review != scientific consensus [13:06:03] You don't really know if an idea really works out for years. [13:06:07] da. Peer review + rebuttal. I should work on that rebuttal. [13:06:16] +1 [13:06:17] :) [13:06:43] Scott keeps asking interesting questions! [13:06:52] As soon as I get that dataset for Brent out the door I'll have actual cycles back. [13:07:31] Ironholds is the prettiest girl at the dance. [13:07:46] must be a terrible dance :P [13:07:52] ...I'd instinctively respond negatively, because impostor syndrome, but it looks like I'm first author on a CHI note. [13:07:59] soo....okay, fine. You win this round. [13:08:06] They only want me for my datasets. You know what boys are like. [13:10:30] halfak, actually, do people read notes? [13:10:52] and, what's the normal result of...publishing something? By which I mean: am I going to get a ton of emails from interesting people, or no emails, or somewhere 'twixt the two? [13:10:55] because I would like fewer emails. [13:11:10] Ironholds, notes get read about the same as full papers. [13:11:28] Ironholds, somewhere inbetween depending on what you published. [13:11:30] read: "mostly by sad people such as Oliver" [13:11:33] gotcha [13:11:38] I get far more emails for media coverage than I do from publishing. [13:11:50] * YuviPanda gets no emails [13:11:50] ...I will resist the urge to think of my least favourite person at the foundation and stick their email address in the draft. [13:12:11] YuviPanda, https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service [13:12:22] Specifically https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service#Feature_extraction [13:12:53] I'm considering using a dependency injection-like strategy for feature extraction. [13:13:09] Might be over-engineered. [13:13:24] But it would allow many different models to share feature definitions. [13:14:05] halfak: so I read that and I'll be *super* happy to help. But I think I need to understand more about 'models' and the underlying work being done before I can think up something scalable [13:14:33] Ahh. That we can talk out pretty briefly. [13:15:04] model == magical black box that takes a long time to prime, but then can quickly make classifications. [13:15:18] So, we should expect the model building to be slow and happen offline. [13:15:26] But that use of the model can happen in real time. [13:15:28] aaaah [13:15:35] define 'long time' and also would it be updated? [13:16:02] Let's say 30 minutes to 5 hours to build a model. [13:16:12] We can rebuild it periodically as we get new training data. [13:18:11] halfak: ah, cool [13:18:35] halfak: and this model is usually just a custom datastructure stored on disk and then read into memory whenever it changes? [13:19:41] Yup [13:19:47] That's a good way to think about it. [13:21:10] For wikiclass, I built a simple datastructure that would store some metadata about the model and then pickle that onto the disk. [13:22:21] See ModelFile here: https://github.com/halfak/Wiki-Class/blob/master/wikiclass/models/model.py [13:22:27] halfak: hmm, right. how 'big' would the models be, on average? [13:22:30] gigs? [13:22:38] Megs. [13:22:40] https://github.com/halfak/Wiki-Class/blob/master/examples/train_model.py [13:22:43] aaaah, then no issues [13:22:51] Shows training a model and then storing it as a file. [13:24:02] Here's the enwiki assessment class model: https://github.com/halfak/Wiki-Class/blob/master/models/enwiki.rf_text.model [13:24:24] 18MB [13:26:47] cool [13:27:18] halfak: also, the stiki classifier is in Java. do you think we would want to port that, or have something in place that lets us run them in any language we want but provide a unified interfacae in the end? [13:27:42] I was going to ask you the same. [13:28:04] We could probably convince AGW to port his work to Python, but that's a lot of work. [13:28:17] indeed, and I'm ok with a heterogenous environment [13:28:43] However, with the whatever-language-you-want-just-have-a-common-interface strategy, this would allow us to work with Huggle's C++ too. [13:28:48] he can just add a layer on top that exposes a consistently defined set of network endpoints (HTTP probably, but we could look at 0mq as well?) and that should be enough. [13:29:04] actually forget 0mq, just http [13:29:13] My primary concern is that, if these things are black-boxy, then we'll be duplicating work. [13:29:30] Like gathering the text & metadata of a revision multiple times. [13:29:41] And generating the same feature multiple times. [13:30:00] indeed [13:30:12] that *will* happen if they're a heterogenous platform [13:30:13] *however* [13:30:33] we can fix that by having things like 'give me text + metadata for this rev on this wiki' themselves be available as services [13:30:39] that these black boxy things can then call [13:31:36] YuviPanda, indeed, but then we need to modify the black boxy things to call those endpoints. [13:31:39] At least we'll need to know what is going to be called. [13:32:01] indeed, and we need to modify them anyway to provide the http interface in a fashion we design [13:32:14] I'm kind of leaning towards building ourselves a solid classifier with AGW's help and then releasing ver. 1 with just that available. [13:32:25] that will indeed be simpler, yes [13:33:27] Feature extraction is relatively straightforward. Model building and testing is something I've got some experience with. [13:33:47] If we build a nice sandbox to build new features and classifiers, the model builders may just come to us. [13:34:11] if you think rebuilding the current classifier wouldn't be too much work, then yes, that is indeed the way to go [13:34:54] I'll make some simple examples first that we can play around with. [13:35:02] I'll also get AGW's take on the situation. [13:35:42] We should probably have a chat with Henrique too. He'll likely contribute some code. [13:36:12] yeah [13:37:20] * halfak wants 20% projects [13:39:39] halfak: I have soooo many... :) [13:39:55] Do you get to use 20% of your time on them? [13:40:15] For me, having the projects is not the problem. having the time for them is. [13:51:16] halfak: oh yeah, I do! [13:51:40] halfak: Tomasz always encourages that, and that's how I got to spend enough time in Ops over the last year [13:51:43] enough time to move there [13:51:49] * halfak is jealous [13:51:57] I get 120% time. [13:51:58] as long as you don't miss your sprint goals [13:52:03] but we didn't [13:52:34] halfak: looking at the other teams, I'm quite happy with how the mobile team was org'd [13:52:46] much lesser meetings, and the meetings also were smaller [13:55:24] Maybe less meetings is the ticket. [14:00:22] halfak: indeed. I calculated, I've a grand total of 8h of meeting a month [14:00:34] will reduce to 4 with the ops team move [14:00:41] I felt 8 itself to be a bit bigger than i'd have liked [14:02:04] I have about 8 hours of meetings per week. [14:02:20] wow [14:02:27] yeaaaah, that would have issues [14:02:58] halfak: there's also about 2h more of extra meetings I'm optional for and I really am optional so don't make it [14:05:37] I just finished up my 2 hours of 120% time and now I must prepare for the next meeting. [14:07:09] YuviPanda, what's a good term for "all models in python land" as opposed to "models can use whatever environment they want and provide a standard interface"? [14:08:03] models generated by python vs models generated by something else? [14:08:14] although I dunno what difference it makes how they are generated [14:09:48] Well, I'd like to propose two alternative design strategies. (1) model generation and use of the model will occur within the revision scoring service. [14:10:06] (2) model generation and use of the model will occur outside of the revision scoring service. [14:11:27] ah [14:11:28] I see [14:11:48] why would that matter? Model *using* is the part that needs to be homogenous [14:12:35] OK. that's fair. [14:12:58] I don't expect that C++'s model builder will output something that python's scikit-learn will be able to work with though./ [14:14:23] halfak: why not? [14:14:34] halfak: we *could* standardize on a format [14:14:37] JSON? [14:14:41] it's going to be read into memory anyway [14:14:46] Different model output formats. [14:14:54] can't we change that? [14:15:07] all datastructure serializations can be represented in JSON, no? [14:15:37] Uh. That sounds like a ton of work. Regardless the model is cheap. [14:15:46] If we can use it in python, we can generate it just fine in python. [14:15:51] then fine [14:16:22] The problem is (1) obtaining a training set and (2) getting the features extracted. [14:16:46] I suspect that we can borrow training sets. However, replicating feature extraction might be more difficult. [14:16:59] It seems that AGW has done a good job of documenting his feature set though. [14:17:07] hmm [14:17:13] what exactly is a 'feature' and a 'feature set'? [14:17:44] A feature is a single value that can be used when predicting a classification (e.g. damage vs. not-damage) [14:17:49] aaah [14:17:50] right [14:18:12] As you might imagine, a feature set is the whole collection of predictors. [14:20:35] right [14:20:42] so if they're trivial to port, we can port them [14:20:50] but we should standardize on a model format [14:20:50] I think that they will be. [14:21:02] Well, different models have different requirements. [14:21:07] one that's already well used in the 'industry' (if one such exists?) or something like JSON [14:21:10] halfak: examples? [14:21:12] Can we not just use whatever scikit-learn gives us? [14:21:39] we could, but then we've to write a parser for that format [14:21:44] and if it's a binary format that's going to be painful [14:21:51] Why would we need to parse it? [14:21:53] esp. when we've to do that for everything else [14:21:57] What's wrong with binary? [14:22:26] you've to write your own parser :P [14:22:29] and deal with things like: [14:22:30] 1. padding [14:22:32] 2. endianness [14:22:44] 3. padding is a bitch to deal with, not sure if I mentioned it before [14:22:50] Why would we need to parse? [14:22:51] 4. changing them is hard [14:22:56] hmm? [14:23:00] it's a model that's being used, no? [14:23:09] You don't usually change models. You build new ones to replace the old ones. [14:23:12] my idea of a model is a data structure that exists on disk and then is loaded into memory [14:23:17] It's unusually to check a model into version control. [14:23:19] and then used in some way [14:23:41] so you've to read them into some form of in-memory DS and then use it in some form [14:23:53] scikit-learn handles this for us. [14:24:13] handles what? loading into memory as well? [14:24:17] Sure. [14:24:19] what exactly is scikit-learn? [14:24:26] It's a package for python. [14:24:30] http://scikit-learn.org/stable/ [14:24:31] ooooh [14:24:32] damn [14:24:40] Contains lots of model builders. [14:24:56] I thought it had something to do with stiki :D [14:25:01] so yes, you are completely right [14:25:11] http://scikit-learn.org/stable/tutorial/machine_learning_map/ [14:25:13] ignore whatever I've said [14:25:16] yeah, I just saw that [14:25:17] yessir :) [14:25:47] So, i think we want a linear SVM, but I'd like to play around with Random Forrest too. [14:26:09] A Binary Tree might be nice too since it is readable. [14:26:20] But I doubt they'll perform as well as an SVM. [14:35:59] heh [14:36:12] if it's a python module, then go wild [14:36:45] :D [14:46:47] halfak: I'm reading the wiki page again [14:47:05] Please be bold :) [14:48:16] halfak: I shall. I also remember STiki asking users to rate them as well, and incorporating that into the model. That seems very powerful and something we should try to support rather than just depend on offline models [14:48:31] rely *only* on offline models, that is [14:48:46] Well, I think that stiki gathers data to generate new offline models that way. [14:48:56] So does ClueBot NG [14:49:33] one thing that is problematic with gathering data in that way is that it fails to be random. [14:49:42] And therefor can introduce biases in the model. [14:50:22] I'd much rather observe mistakes qualitatively and then try to develop features that will help the model builder build better models. [14:53:24] hmm, right [15:52:32] * Ironholds whistles [15:52:56] tnegrin, could you ask the sessions peeps to add ahangout? Otherwise no dan and no me :( [15:53:45] yeah -- will do [15:54:58] t [15:54:59] *ta [16:01:48] Ironholds: is this mobile app sessions? [17:11:26] https://en.wikipedia.org/wiki/Duck's_ass [18:11:47] tnegrin, you coming? [18:12:48] 2 mins - sorry [18:13:35] kk [20:50:18] hey halfak [20:50:29] Hey dude. [20:50:43] sorry about the last minute notice, any chance we could push our 1:1 to tomorrow? [20:51:23] That's fine by me. [20:51:37] Want to just find an open spot? [20:51:38] I had to postpone a lot of work due to stuff that happened this morning and I’d like to continue with ellery’s onboarding [20:51:48] Morning > afternoon [20:51:49] anything after Monthly Metrics [20:52:06] OK. Right after monthly metrics > later in the day [20:52:42] how about 3pm PT [20:53:20] Hmm.. I might have miscommunicated. Early in the day is easier. 3PM PDT is 5PM CDT. [20:53:55] I can do it if that's all you've got, but I was looking at my afternoon block there as a good opportunity to catch up on work. [20:54:11] got it, let me check [20:56:16] halfak the only other option is to do it over lunch PT (I can’t do earlier in the morning) [20:56:25] unless you and Ironholds can swap [22:08:30] I just ran into a single revision of Wikipedia that contains 32MB of data [22:09:35] I feel like there's some past vandal out there who thought that they might fill up our servers. Yet all they have succeeded in doing is trolling analysts. [22:09:52] Also, MongoDB should store whatever size JSON blob I want it to. [22:10:16] "We built this limitation in document size in order to encourage good behavior." [22:10:56] You don't know enough about what I'm doing to know what "good behavior" looks like. *grumble grumble* [22:12:04] http://docs.mongodb.org/manual/reference/limits/ [23:03:11] J-Mo, halfak: did you see stu’s reply re: derp [23:03:22] I did. Thoughts? [23:03:27] if you have bandwidth some quick clarification is in order [23:03:55] reading now [23:03:56] Tim acted in good faith and based on his last response I still believe that DERP wants to be an information broker [23:04:34] however, things were much messier at the time the press release was drafted and published [23:05:20] this is not to say that I think we should reconsider our decision any time soon, but I see Stu is worried about what signed up to [23:05:27] >what he signed up to [23:06:12] yeah, I don't think Stu needs to be worried. But perhaps it's okay to respond with a few of the pieces of the preliminary press release that gave us pause? [23:07:36] as I see it, the problem was that we saw some legal language that was concerning, but there wasn't adequate time to dig into what may (or may not) have been behind that language before the release went out. And being a big ORG with a legal dept and a lot of exposure, we thought it was safer to stay out for now. [23:08:34] of course, we still HAVEN'T been able to really post-mortem with Tim over what the language was supposed to signify, so we can't really offer any assurances to Stu, or make any stronger claims about what DERP is about. [23:09:16] sooo… not sure, DarTar, what we can actually say to Stu. [23:09:23] ^ sounds like we've got a volunteer [23:09:28] I think what you just said is solid. [23:09:47] * J-Mo goes to copy paste some stuff into the email body [23:09:50] halfak: agreed :) [23:10:40] thanks folks [23:11:12] J-Mo: I wouldn’t include references to the early drafts of the press release [23:11:21] as they never became public [23:11:32] ah, but that was the substance of our concern tho, yes? [23:11:46] the final press release didn't contain the language we objected to? [23:12:00] yes, but I don’t think it’s helpful to point at those drafts [23:12:01] (correct if I'm wrong, DarTar) [23:12:24] okay, so… do I refer to the press release at all? I'm not sure how to do this dance. [23:12:25] well, the final press release was amended after it was published [23:12:29] from what I recall [23:12:50] it’s been tweaked/edited after the launch [23:13:34] I don't think that anyone is questioning the sketchy language of the early press releases. [23:13:34] in other words, I don’t think it’s on us to prove that this language existed until the last minute [23:13:42] yeah [23:14:03] also, I didn’t mention other concerns that people like toby brought up, which are legitimate [23:14:10] so, I say something real vague like "in the days right before launch, we became aware of… FOO" [23:14:17] for example: some of these orgs share/sell user data to 3rd parties [23:14:33] nah, I'm not the one to write this, if we have to dance around the verbiage. Punting it to you, DarTar. [23:15:00] exactly, and we can’t answer that one without properly reviewing their respective privacy policies/terms of use [23:15:33] so J-Mo: if you’re going to send a note, plagiarize the reply that Tim sent [23:16:14] alright. Stu deserves a reply, so I'll try to put something together that doesn't complicate our legal situation or make it sound like we hate DERP [23:16:24] you’re the man [23:16:27] thank you [23:28:13] halfak: did you see lila’s note on editor retention? [23:29:23] I can’t provide a detailed answer now (I got signed up for an update on mobile uniques at tomorrow’s monthly metrics and I need to prep) [23:29:26] okay, DarTar, I said some stuff [23:30:16] DarTar: yes. StevenW CC'd me on his response. I think he did a good job describing the metric and what we know. [23:30:31] I talked to him about it a little beforehand. [23:30:49] I didn't see Lila's whole email though. [23:30:58] J-Mo: that’s perfect, thank you [23:31:14] all in a days DERP [23:31:59] halfak: he did, so there’s probably no need to follow up immediately, I’m forwarding you the original question [23:32:20] Thanks [23:32:53] it just strikes me that we have the sensitivity analysis up on that page, but not a few clean sample plots using the final parameter set [23:33:07] nbd, EEVS is meant to do that [23:33:50] Agreed. We should have that. [23:48:59] * halfak runs to dinner