[00:17:07] does anyone want to talk me out of making 16,000 queries to grokse? :) [04:32:56] harej, yes, the data is terrible [15:04:24] Good morning, scientists. [15:14:55] morning guillom [15:15:11] hey leila [15:56:06] good morning Ironholds leila guillom et al. o/ [17:07:16] hey bearloga :) [17:20:56] yo yo [17:21:00] I am back from my holidays [18:02:54] hey, i'm looking for dev ops researcher checkpoint folks. not much to say this meeting so if we don't have it it would be ok. [18:41:06] hello J-Mo [18:41:19] soo, I have a message in fawiki that designresearch@... email doesn't work? [18:41:22] hi lzia. in a meeting. ping you when I'm done? [18:41:40] hello bearloga. Sorry I didn't respond earlier. I was in our QR. [18:43:59] J-Mo: for when you're back, is this the email: designresearch(at)wikimedia.org [19:12:56] hey lzia thanks for the ping. Could you send a test email to designresearch(at)wikimedia.org and I'll see if I receive it? [19:13:19] er, ^leila [19:18:04] A wild halfak appears. [19:18:11] o/ guillom [19:18:18] just got to DC and found some WiFi :) [19:18:25] Hey :) [19:21:04] welcome back halfak! [19:21:31] "back" :P [19:21:37] I was in MN for 7 hours last night [19:21:51] Enough time to sleep in my own bed and then go back to the airport [19:22:01] But I am back in the channel! [19:24:36] J-Mo: sent [19:24:47] hello halfak. [19:24:59] halfak: Grace has written notes that she will share with you and Dario. [19:25:12] o/ [19:25:12] The main comment about revscoring was: make sure product teams pick it up and use it. [19:25:13] thanks leila [19:25:17] notes on QR? [19:25:27] yes, halfak [19:25:41] Arg. I really want to push back on that. I can't be expected to do this kind of marketing work. [19:26:21] I told her that I don't know about the details of what Aaron and Dario have done on that front, so I'll transfer the message, and y'all should continue the discusison. [19:26:24] discussion* [19:26:32] I don't see why WMF product teams should take precedence over tool devs. Targeting tool devs was the original goal of the project. [19:26:46] I emphasized that part, halfak. [19:26:51] and the Huggle good news. [19:27:05] Thanks Leila :) [19:27:12] however, what I got from Lila's comment was that she wants the great result of this research to be user in product. [19:27:56] J-Mo: it seems it's working. I'll message the editor back. not sure what's going on. [19:28:52] yeah, me either Leila. checked my spam filter; no messages there. Thanks for following up! [19:29:03] no worries. thank you, J-Mo [19:29:06] leila, gonna have to chat about that with DarTar. [19:29:32] We're going to need to hire someone else to do the political work. [19:29:56] If Lila wants us to do that kind of political work. [19:30:52] The thing that gets me is that I have *no trouble* pitching this to tool devs. [19:31:03] Why would it be difficult to pitch to product teams? [19:31:25] Meh. Thanks for conveying. [19:31:34] * halfak develops his thoughts better. [19:32:14] leila, one more thing. I just wanted to make sure. Does Lila realize that there's already a very large userbase for revscoring? [19:32:22] That there are 10 products using it? [19:32:23] it's not difficult, halfak. I think she is saying that you should communicate it with product, so they know exactly what is out there for them, and how they can benefit from it. [19:33:24] Hmm... Maybe I should go on a product team campaign where I visit them specifically to talk about revscoring. [19:33:34] I don't call it political, halfak. It's finding more customers for a service you have developed. You may not want to do it as a researcher, but it's not very unexpected to receive that comment. Even in universities you get asked/ you receive such comments. [19:33:54] but anyhow, it's something for you to decide with her. Let me know if you want to chat about it, halfak. [19:34:22] halfak: I think many teams don't know what is revscoring. [19:34:23] Well in this case, she's not asking for impact. We've already got that. She's asking for impact through a particular route that isn't very tuned into the problems revscoring helps solve. [19:34:32] That, I would say, is entirely political. [19:34:51] fwiw, halfak I think your strategy of 'shopping around' rev scoring to particular product teams could work [19:34:54] leila, that's fair, but I did have a chat with most PMs about it at wikimania. [19:35:05] It's been presented about in metrics meetings. [19:35:07] so it's fair to say that it's important for them to know what it is exactly, halfak. if they know it and decide not to use it it's fine, but they have to somehow get what it does and how it can be useful for them. now, some of this is on us as researchers, some of it is on them as product owners. :-) [19:35:09] Research showcases. [19:35:52] you can tell her about the Wikimania chat, halfak. [19:36:13] I don't know what chat you had, and how much you transferred. it's good if she knows that they know what it is. [19:38:46] Makes sense. It seems that we need to have a conversation about what types of impact are legitimate and where the onus is on information flow from research to Product. [19:39:17] E.g. is showcase, metrics, etc. sufficient and reasonable or should PMs be courted and chased. [19:39:32] halfak: btw, I didn't get a sense that she is saying the impact is not enough. [19:39:49] halfak: she thought that this is awesome, and it can be used in more places, so you should advertise for it more. :d [19:40:17] I actually said the Huggle part as a bonus: something along the line of: we met our goal, but let me tell you where the true value comes from ... [19:40:18] :D [19:40:21] That's just it. I don't think that Lila understand that this *is* a product, it has users and it's being advertised like crazy :S [19:40:31] it's why I'm in DC right now! [19:40:44] Nice! [19:41:23] Until the WMF is ready to build a cluebot/huggle replacement, cluebot/huggle-like tools *are* the product teams I'm courting. [19:41:48] yeah, that totally can be. And I clearly said I'm not Dario or Aaron, and I don't know all the details of what they have done or what they are planning to do, I'll transfer the message and you all follow up. Take it as a positive comment, halfak. there was really nothing strange in it, and I think she gets it that you shouldn't become a PM. someone should take it from you and do nice things with it, as some people already have. [19:42:58] if you think product teams cannot use, communicate that with her, too, halfak. Wes mentioned something about his team wanting to use it. I'm sure you know about that. [19:44:18] (reading scrollback) leila, halfak: I think another part of Lila's comment was the ownership of the tool when it becomes a full-fledged product. As in, Research did an awesome job at creating this, but is Research expected to maintain it forever, or should that ownership be transferred to/shared with another group? [19:44:38] Yeah, I've been talking to Wes, but that seemed to fizzle. Either way, with the frwiki quality model, we're expanding in the direction Wes was interested in. [19:44:41] you are right, guillom. [19:45:21] guillom, good Q. That's something we should discuss internally I think. I think of ORES/revscoring as a research platform, so I think it is a product we should own. [19:45:28] But that we'll need engineering support to maintain. [19:46:05] leila, halfak: If I may abruptly insert myself into this conversation, it sounds to me like some pushback would be appropriate. [19:46:30] so halfak: what I said to that question, was that you are still adding the models for more languages, so I /think/ that part is owned by you, but she has to talk to you to figure out who owns the productized component. [19:46:37] leila, halfak: It's easy to throw around words like "Make people use it", but such words do have consequences, and I would consider asking Lila whether that's really what she wants relative to your other ongoing work. [19:46:40] Yes; my understanding of her comment was that there should be a plan in place so that all your great work isn't abandoned, and its maintenance doesn't prevent you from moving on to other research projects. [19:46:51] Deskana: were you in the meeting? [19:47:13] leila: No, so what I'm saying might be completely off-base. I just read the scrollback and filled in the blanks with my own experiences. [19:47:45] If I'm not being helpful because I've got no idea what I'm talking about, please do tell me. :-) [19:47:58] Not at all. Happy to have you commenting Deskana :D [19:48:05] And hanging out in -research :D [19:48:28] It's just I've had these kinds of experiences before, and I normally handle it by fully explaining the consequences and asking whether that's actually the tradeoff that is wanted. [19:48:51] Naturally, executives are not always aware of these details, and should not be because it's their job to focus on big picture. [19:49:07] Deskana: because that's not what Lila said really. She wants the work to get all the attention it deserves, if the work is designed only for a specific purpose, halfak can communicate this, but if halfak sees that the output of these models can be used for other purposes (like for example we're using one of them right now for a research we're doing), then the comment was that he should make sure he communicates this. Now, halfak says he ha [19:50:06] Deskana: please comment. My question was real, I wanted to see if your perspective is because you heard her comment in a different way or you have some other experience. :D [19:50:09] (Note to self: Murphy's law applied to IRC: Interesting discussions always happen when you're reading a paper and not looking at your screen.) [19:50:38] guillom: :D [19:50:48] But there's still that tradeoff. I could keep going with my campaign to advertise revscoring in whatever Wikimedia venue presents itself and developing the system to expand to more languages and better models -- or I could take a substantial cut out of that to hand-feed revscoring to PMs. [19:51:00] For what it's worth, we've incorporated revscoring-like things into our search brainstorming and mockups for long-term vision. [19:51:16] I highly doubt that the mockups are even close to what the system is currently capable of doing, which is fine because they're long term vision things. [19:51:18] halfak: my recommendatin: talk to her. We can spend hours speculating what she really meant, she can clearly tell you what she meant. :D [19:51:39] but I think I may have extrapolated well beyond the bounds of my error bars on this one. I should probably just go to Lila to have a chat about revscoring and raise the Q that Deskana suggests. [19:51:48] \o/ Deskana [19:51:59] halfak: My recommendation: Keep the interesting part of the work (the models?), get a vertical or someone to own the maintenance and the marketing. Problem solved! [19:52:00] yeah, halfak. do that. :-) [19:52:05] Happy to hear you guys have been planning [19:52:09] guillom: +! [19:52:10] :D [19:52:12] +1 [19:52:28] (I am not volunteering!) [19:52:58] We'd have to expand it somehow to do what we want. (where "we" is meant broadly) [19:53:15] Deskana, what sort of expansion do you have in mind? [19:53:34] halfak: Well, we were interested in putting it on search results as a kind of "article legitimacy" rating. [19:54:16] halfak: That seems like an interesting direction to me... but I think it's outside the current intended usage of the service. [19:55:06] Deskana, not totally. [19:55:19] Anyway, in terms of practicalities and the current conversation, +1 to the idea of having a conversation with Lila. :-) [19:55:27] It's more of a measure of completeness which is a good proxy for correctness. [19:55:44] But we can do some more to get a trustworthiness score. [19:55:50] halfak: Well, awesome! [19:56:05] Anyway, loop me in when you think it is apt. [19:56:29] While I push back on the idea of more advertising, I really want to talk to you about your use cases. [19:56:31] halfak: Will do. It's still early days so far, our first UX engineer only started a few days ago. [19:57:23] Gotcha. Don't wait too long though. A lot of research is opportunistic. the longer I have to think about something the more time I have to find an opportunity to do it! [19:58:39] This next meeting is going to be an awful lot of fun to do from a coffee shop. [19:58:44] /s [20:00:02] halfak: Then, we can totally have an early conversation soon to get the creative juices flowing. I just wanted to make it clear that realistically we're quite a few months out here. [20:00:25] Now, to my meeting! [20:00:27] Nice chatting. :-) [20:00:43] meetings [20:01:15] ! [20:01:19] wow so much backscroll [20:03:33] inorite‽ [20:04:38] lots of interesting conversation too [20:24:17] halfak: can you link it to me here too? [20:24:24] https://etherpad.wikimedia.org/p/open_infra_workshop [20:41:07] halfak: btw, I'll be out most of next week for the ops offsite [20:41:20] Gotcha. Thanks for the heads-up. [20:41:29] I don't have any big ORES plans coming up. [20:41:39] It's mostly going to be Wikilabels campaigns for a little while. [20:41:52] BTW, where should I stash backups of the Wikilabels DB. [20:41:59] ^? [20:42:23] I'm using my local server ATM. I assume there's a nice robust backup space in labs somewhere? [20:43:05] Oh! I suppose I might deploy the frwiki article quality model, but I'm pretty confident about deploys ATM. [20:43:26] halfak: hmm, so we put them in /data/project right now because labs has a shitty backups story [20:43:52] halfak: for the postgres db we had backups but that was broken when we quickly upgrades postgres at wikimania and haven't been fixed yet. [20:44:07] halfak: so... your private server seems ok atm :D but we should get that fixed at some point [20:44:08] I'm guessing I can't access /data/project from the wikilabels servers [20:44:16] halfak: yeah because we killed NFS [20:45:40] OK. Cool. I'll stick with this for now. [20:47:34] halfak: yeah +1 [20:47:43] halfak: there's also interesting backscroll in the channel I'll read [20:48:38] When you're done reading that, I'd like to discuss the lifecycle of a system like ORES and whether we can engineer ourselves away from regular maintenance or if we need to plan for some human infrastructure. [20:48:56] Might be a conversation for the services team too. [20:57:33] halfak: yeah, totally. ideally we'd not have to do any 'regular' maintenance at all [20:57:40] and just have alerts for exceptional situations that need monitoring [20:57:43] and alerting [21:00:20] So maybe long-term, we can "retire" services to "monitor" status rather than worrying about transferring ownership. [21:01:19] yuvipanda, ^ [21:02:04] halfak: yup [23:04:55] leila: hi [23:05:14] I'm in the article-rec meeting [23:05:16] cc yuvipanda [23:05:39] milimetric: oh shit yeah. [23:05:49] milimetric: I'm with leila let me go get into a room [23:06:03] k [23:11:17] milimetric: https://etherpad.wikimedia.org/p/wtf-is-production [23:11:20] leila: https://etherpad.wikimedia.org/p/wtf-is-production