[16:46:41] god evening, notconfusing [16:46:42] +o [16:47:02] Hey [16:54:34] o/ [16:59:07] we're setting up the virtual office for the upcoming Analytics+Research office hour which will start in a couple of minutes. [17:00:32] welcome everyone to office hours from Wikimedia Research & Analytics [17:02:35] We would like to learn about your research projects related to wikimedia; we hope to be able to provide support about questions and challenges you encounter regarding data, analysis, or other questions more generally. [17:02:41] No set agenda, simply drop your questions in the channel. we will try to address them individually as they come in. [17:04:37] * J-Mo waves [17:04:44] * leila waves back [17:06:46] * isaacj waves too! [17:07:22] The first person who asks a question will receive a Reearch coaster. :) [17:16:15] Hi. I have a question about the gender gap in participation, i.e. the percentage of editors who are female. I read a review article that said it is not possible to discern from the surveys whether the gender gap in participation has been increasing over time, decreasing, or staying the same, due to differences in survey methodology. Are there any [17:16:16] plans to set up surveys so that trends in the gender gap can be tracked longitudinally? [17:17:02] I can take a stab at this [17:17:54] no current plans (that I'm aware of). Although we have been doing some demographic surveys of readers recently, we tend to avoid gathering personal/demographic information when we can. [17:18:05] isaacj, what do you know about this? [17:19:20] in general: unless we see either a) substantial growth in or b) increased turnover in the active editor population (a lot of more established editors leaving and a higher rate of newcomers sticking around), we have no reason to believe that the gender distribution among editors will have changed, on a particular wiki [17:19:50] yeah, it's a good question -- we actually ran some editor surveys in August to look into the feasibility of this very question. we found that there are still a few technical details that would need to be worked out to have the right tool for setting up these surveys to run in a simple but non-invasive fashion: https://meta.wikimedia.org/wiki/Research:Surveys_on_the_gender_of_editors/Report [17:20:51] for context a blogpostfrom pointing out the issue of different methodologies in the surveys https://blog.wikimedia.org/2015/04/30/how-many-women-edit-wikipedia/ [17:21:06] Thanks. I think it's really important to be able to get longitudinal data on this issue. Otherwise how would we know that efforts to narrow the gender gap are having an effect? [17:23:23] Here's a question on editor retention: I'm curious about the differences between Template:Test1 and Template:Uw-vandalism1. The former, which is an older template, starts by thanking the vandal for experimenting with Wikipedia. Is there a particular reason that the newer template doesn't thank the user for experimenting? And in a related question, [17:23:23] has any formal research been done to compare how users respond to different wordings, e.g. to see which messages are associated with subsequent higher rates of unconstructive and constructive edits? [17:23:24] yeah, Clayoquot it's a really good point. One of the challenges too is simply one of statistics. many wikis have relatively few editors so it's hard to measure changes in the gender distribution and be certain that they are not random fluctuations. english wikipedia though has enough editors that longitudinal analyses should be possible, just difficult [17:24:25] clayoquot, yes, but it was a while ago. lemme dig up some links for you [17:25:01] here's one: https://www.aaai.org/ocs/index.php/ICWSM/ICWSM12/paper/viewFile/4657/4974 [17:25:46] I've also done some experiments with the wording of HostBot invites over the years. None of them have yielded substantially different results in the short term, but they tended to be small wording changes. [17:26:09] As to why some templates are nicer than others, that, no idea. Ask the editors who made them! [17:26:21] isaacj: Clayoquot 's question nudges me to pick up this conversation again. What you have in the recommendations in the meta page is a good place to start, especially when it comes to newcomers. That is, from the technical perspective and in some languages, a relatively easy step to start taking. [17:32:00] leila: yep, outside of the technical work that would need to go on for that, the big open question in my opinion is whether you focus on gender at account creation or some measure of retention. the latter is much harder from a design / privacy perspective but arguably more important than just who creates accounts [17:34:40] isaacj, that's an interesting question. Some huge percentage of people who create accounts never complete an edit, right? Do we know why people create accounts? One issue with focusing on account creation is that people could create accounts because they believe there is some reason to do so that has nothing to do with editing, e.g. they might [17:34:41] think it lets them read more stuff.. [17:35:49] regarding account creation, the Growth Team at WMF has done some really excellent work there in a few wikis. the overall page for that is here: https://www.mediawiki.org/wiki/Growth/Understanding_first_day [17:36:40] and here are some specific results from Czech and Korean Wikipedia: https://www.mediawiki.org/wiki/Growth/Analytics_updates/Welcome_survey_initial_report#Why_did_you_create_your_account_today? [17:36:58] Nettrom might be able to add more as he has led a lot of this work [17:40:16] @Isaacj, that research on registration is cool. Yeah, it looks like if you want to measure the gender gap in participation, you'd have to measure the people who at least attempt to edit. The population that creates accounts doesn't look like a very good proxy for the population that edits. [17:41:35] I understand the privacy challenges, totally! [17:42:22] Clayoquot: as you think about this topic, if you have ideas you want to share with us, please let us know. [17:43:09] Will do. Really appreciate this conversation and the office hours format. [17:43:31] Clayoquot: on whether measuring gender at registration is something we should aim, I'd argue that this is another important place where we should keep an eye on changes in demographics. For example, what we're seeing is that we have an imbalanced readership when it comes to gender. I would expect that to have impact on the demographics of account creation as well. [17:45:02] Clayoquot: check https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour/Demographics_and_Wikipedia_use_cases#Reader_Demographics for readership stats. [17:48:21] I should introduce myself: I'm Su-Laine Brodsky, and I'm in the early stages of writing a book about the English Wikipedia's processes and culture. Happy to send details by email to anyone who'd like to know more. [17:49:41] leila, the gender balance in readership is also interesting :) [17:50:03] good to meet you, Su-Laine [17:50:08] Clayoquot: that's really awesome. :) and nice to meet you here. Re your book details: If it needs to stay private, please send it to research-wmf@wikimedia.org (our internal, team-only mailing list). If it can be shared with others as well, wiki-research-l is the best place. [17:50:37] Folks: we're going to be here for 10 more min for the purposes of office hours. If you have questions, speak up now. :) [17:50:57] Really nice to meet you all. I'll email research-wmf. [17:52:00] sounds good. :) [17:52:11] nice to meet you as well -- thanks for the questions! [17:53:55] Has there been any research about how different genders respond to the experience of being reverted? E.g. are women or men more likely to quit editing if their edits are reverted? [17:54:20] Clayoquot: to your other question about how the text of the message can have impact on the users, the closest more recent work I can think about in the context of Wikimedia is https://arxiv.org/abs/1805.05345 . [17:56:55] J-Mo: have you run into any research re the above question? [17:58:56] If anyone finds something on how different genders respond to the experience of being reverted, please email me: sulaine@sulainebrodsky.com [17:58:59] I believe elizabeth whittaker's research that she presented in the showcase last July had some insights from interviews around the question of how to handle incivility: https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#July_2019 [17:59:04] Clayoquot: ^^ [17:59:18] Got it, thanks! [17:59:49] Ok, all! heading to my next meeting. Clayoquot: you will receive the coaster. ;) I'll reach out to you privately re that. And thanks for your questions and the conversation. [18:00:09] any last questions before we wrap up for today? [18:00:32] we are planning to have the next office hours in a month from now [18:01:47] thanks everyone for joining and for the questions. [18:44:35] my sincere apologies for missing office hours. I caught up on https://wm-bot.wmflabs.org/logs/%23wikimedia-research/20200122.txt and it looks like I wasn't needed. But very interesting that someone's writing a book on WP process and culture [18:45:03] I also have no idea why I wasn't joined here, but now I am and I'll stay that way [19:32:00] mgerlach: shall we grab lunch together? [19:32:27] milimetric: no worries. if it were urgent, we would find you somehow. :D [19:32:34] mgerlach: not at 11:30, of course. :D [19:32:50] leila: yes, sounds good. [19:33:09] mgerlach: I have a 13:00-13:15 PST meeting I need to take from the office. I can go before that or after it. [19:33:17] sounds good. you tell me when then (and no rush). [19:34:35] leila: after is good [19:34:49] oh, interesting, I also don't have the office hours on my calendar, adding now [19:34:49] mgerlach: ooki.