[14:48:58] halfak, so we in search have a problem session reconstruction could be really good for [14:49:11] Oh? :) [14:49:35] namely: we want to work out whether a user got value, right? Use the example of the A-B-C-D chain of page clicks. [14:49:56] if we go for a simple "they had to click, our search isn't good enough" we ignore six degrees of wikipedia or value from the journey. [14:50:27] but how do we distinguish genuine browsing and journey-based value accrual from "they couldn't find it and moved on"? [14:50:48] one really dumb way of doing this quantitatively, but I can't think of a better way that's not computationally a Hard Problem: how much time did they spend on B and C? [14:51:36] I bet if we look at inter-time periods starting with search sessions we'll see some commonality and divide in time on page, and we can hypothesise, at least, that that's "immediately found the thing/couldn't find the thing and had to click through" versus "reading for the heck of it" [14:56:20] did that make any sense? [15:07:51] * halfak got distracted. Back to reading now [15:08:40] Ironholds, yes. [15:09:00] I think we need to understand reader's behavioral qualitatively before we can hope to understand it quantitatively though. [15:09:06] absolutely agreed [15:09:22] we need to work with design research on this or we're just..not going to understand what's going on [15:09:27] But I expect that we'll have qualitatively significant patterns emerge from the data. [15:09:31] and we need to do it in a way that isn't SF-centric [15:09:50] Ironholds, maybe. I'm not positive about that. [15:09:59] Some things aren't cultural/learned. [15:10:30] e.g. we might just have a bias in the relative rates of each reader behavior -- not totally different behaviors. [15:10:38] some things, yep, but I'd argue we're effectively looking at lived experiences here. Having a narrow frame on that can point us to a bad place, particularly when so much of our search infrastructure and its efficacy varies by language and culture [15:10:46] well, relative rates can be important [15:11:05] " so much of our search infrastructure and its efficacy varies by language and culture" citation? [15:11:29] So, as an example, Lucene's word splitter is the ICU piece, and in Chinese it splits by character [15:12:07] so instead of searching for "foo AND bar" when you type "foo bar" you actually get "f AND o AND o AND b AND..", as I understand it. Less extreme than that, because they are "multicharacter" characters, but still a very different search experience. [15:13:00] and that might just mean "our search sucks in multibyte character environments" or it might be "people have tried to compensate for this in a way we're not aware of in how they structure things" or... and we can't detect which unless we talk to people. [15:13:34] On relative rates: to use an example from previously, relative rates can be the difference between "most people are task-driven, let's deploy this person-agnostic recommender system" and "it's actually a much closer split between task-driven and learning-on-the-journey driven than we thought, let's not" [15:14:07] I think most of the key stories will still come out but how prominent they are, practically speaking, makes an impact on how we prioritise them. And if we prioritise wrong we could do some real damage. [15:14:25] because we don't have the resources to tackle all key stories, just the top N [15:52:59] Ironholds, I don't see a clear metric strategy here. On the contrary, I share your fear of just choosing one and optimizing it blindly. [15:53:40] When it comes to search, it seems like a better methodology would be to find out why it is that people seem to prefer google search over hours. [15:53:48] *better goal [15:54:02] halfak, or if they do, yeah. [15:54:02] A better methodology then would be interviews and grounded theory. [15:54:22] agreed! But I'd like those interviews to be pretty diverse in their subjects [15:54:49] Ironholds, first, interviews at all. [15:54:56] +1 [15:58:07] Does anyone know where the latest version of SuggestBot lives? [16:01:32] harej, Nettrom would. Where IS nettrom? [16:03:44] Isn't everyone in Oxford or something? [16:03:47] The alternative is that I spend the next week or two trying to manually re-implement it. [16:11:19] This is exactly what you want. You want harej to reimplement SuggestBot. [16:14:02] absolutely [16:15:29] harej, good luck with that [16:15:30] :P [16:15:54] There were some guys at the hackathon who were going to reimplement wikitrust [16:16:23] I showed them an API where they could get all of the info they need -- all they needed to do was build the UI. But it turns out there's already a UI. [16:16:33] And they partially reimplemented it! [16:16:41] harej, what do you need suggestbot for? [16:17:00] If you are trying to make within-WikiProject recommendations, then suggestbot already supports that. [16:17:29] Not when you try to invoke SuggestBot through a non-user page. I'm not even kidding. All I need is a one-line change. [16:34:56] So {{User:SuggestBot/get over here}} works on User:Harej/sandbox but not Wikipedia:WikiProject Women in Technology/Tasks