[00:13:43] leila, I'm not attending the coffe ... let me see what works for all of us... [01:04:50] dsaez: sounds good. [17:41:22] what time is next office hour? [18:29:21] RhinosF1: April 22nd at 1700 UTC per https://www.mediawiki.org/wiki/Wikimedia_Research/Office_hours [18:29:49] (excellent question!) [18:52:34] isaacj: wanted to make surr given the DST switch [18:53:34] Pinned my diary to UTC [18:54:47] :thumbs up: yeah that's been confusing for us too but we pin to UTC to try to simplify some of that [19:06:01] isaacj: o/ [19:06:34] isaacj: re accessibility for readership gaps, I have a few contacts I can connect you with. Do you want me to start with 3, 2, or 1? :) [19:09:42] leila: let's start with one given my bandwidth. we can reach out to the others then at a later date [19:09:58] isaacj: sounds good. [19:35:54] If a survey on the autism, adhd and editing was ran. How many editors would be acceptable to form a basic conclusion? What percentage do you think would reply? [19:58:43] there's some discussion and references on this talk page that might help guide that: https://meta.wikimedia.org/wiki/Research_talk:The_Construction_and_Application_of_Personality_Profile_Based_on_User_Behavior_in_Wikipedia#How_many_people_will_need_to_complete_your_survey? [20:02:05] isaacj: Based on that, we’d need to send out quite a lot. We might have the help that it’s a short questionaire that I’d look at. We might also have the benfit of asking more than just enwp [20:05:49] yeah, surveys are very tough unfortunately to balance getting good data with not spamming editors [20:07:32] Might be best to pick groups to outreach to [20:07:46] And then post on Village Pumps of the english wikis [20:24:27] well the issue is this -- if it's just a post in village pump, you're very very unlikely to receive a representative sample of editors. so i don't know if you'd be able to accurately answer the question "What percentage of editors identify as X". many researchers consider posting survey requests on a sample of users' talk pages thinking that they'll get a better sample, but that's something we obviously strongly caution against [20:24:27] because of how spammy it is. so i'd encourage doing any surveys via village pump or related outreach spaces, but i would consider adjusting the goals of the survey to be less about measuring what proportion of editors identify with a given group and more focused on their experiences editing given their identity [20:24:47] speaking personally, those surveys are usually much more interesting to respond to as well [20:26:59] I’d take a sample of editors and post on VPs [20:27:58] Or just do one [20:34:25] isaacj: I’m throwing out help with questioms and experiences of editors to the research list [20:38:11] leila, J-Mo: ^ [20:39:23] :thumbs up: as i said, i always strongly caution people from posting survey links on users' talk pages. some communities are going to be more or less open to this though. but yes, i'd ask the research list and also plan to post to a community's village pump beforehand to get a sense of their openness to the survey. this example isn't the same thing you're proposing, but obviously responses can be strong to put it nicely: [20:39:24] https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)/Archive_60#Research_project%3A_The_effects_of_specific_barnstars [20:41:22] +1 to focusing on understanding the experiences/motivations of editors who self-identify as being on the autism spectrum, rather than trying to get an overall count. For that, you could sample based on the userbox/category. [20:42:09] Yeah we could take a sample for the categories mentioned [20:42:27] J-Mo: how would we word them sort lf questions [20:43:39] I'm still uncomfortable with the idea of asking people who do NOT currently publicly identify as neurodivergent to disclose this information in any way that could be associated with their user account, IP address, etc. unless there is a very clearly articulated social benefit to knowing this information, AND the research study is subject to some kind of institutional review board. [20:44:49] RhinosF1 I've never deployed a psychometric survey before. Generally, the questions I ask in a survey follow directly from the overall goal of the research project--what do I want to know, and why [20:45:06] J-Mo: we’d probably want to destroy the username data if we took it in quickly [20:45:59] if the focus is on their experiences, and how those experiences might differ from someone without autism/aspergers, then it might be a matter of creating a survey that focuses on a) motivations for editing and b) typical editing activities. [20:46:47] the survey might also probe the social experience of being an editor; how do people participate in discussions, how do they represent themselves on their userpage, etc. [20:48:03] but this is a set of general themes I'm sketching out here; the first step is always articulating what you want to learn from the research. [20:48:54] Should we stick to questions like ‘How supported do you feel on Wikimedia Projects?’ , ‘How comfortable do you feel in discussions?’ [20:50:21] depends. what is the goal of the study? if it is to support this set of users better, then those kinds of questions may make sense. But I would generally make them more specific; "how" questions can be interpreted in different ways. [20:51:46] J-Mo: My aim is to understand their prevelance and what areas they prefer and how they feel in them areas [20:51:56] good start. why? [20:56:06] Partly because I fall into the (undiagnosed) non-neurotypical group myself, b) because I’m intrigued, c) I like finding out how wikis can better support their users. [21:01:12] I can certainly understand these motivations :) still uncomfortable with the idea of people providing sensitive health information to a third party without a clear personal and social benefit that outweighs the risks that participants incur by participating in the research (see the Belmont Report and especially the ethical concept of Beneficence https://en.wikipedia.org/wiki/Beneficence_(ethics) ) [21:09:24] I’d hope if the survey went out that we’d able to find a secure platform for data collection and that we’d be able to quickly destroy all data that links response to user. [21:30:07] RhinosF1: I'm trying to understand how best we can support you in Research (if our support is needed in any way). Can you keep us posted if you make the decision that you want to run such a survey? at the moment, my read of this discussion and what's happening in wiki-research-l is brainstorming. [21:31:02] leila: so far brainstorming and throwing ideas out to see what everyone thinks. Eventually, probably some help on design and outreach. [21:31:23] RhinosF1: as J-Mo says, we will most likely need to make sure Personally Identifiable Information is separated from the survey responses. If for some reason it can't be, then we will recommend a much tighter IRB like process. Let's talk specifics when you get to a point that you know you want to have this info. [21:32:18] RhinosF1: (and as J-Mo has said, we need to make sure we tighten the reasons for such a survey. this is something we can brainstorm with you and others about, especially in light of the interest in wiki-research-l thread. [21:32:18] leila: I'd like to avoid collecting it where possible [21:33:04] I think there's interest in it but a lot of reasons to be careful so I agree we should have a clear approach and data collection rules [21:35:33] RhinosF1: being careful is needed. for sure. [21:35:43] agreed. [21:36:20] I'm not sure, for example, why you may need to store the participants usernames at all. [21:37:17] The only valid reason I can see is to ensure same person doesn't submit more than once, but I'm leaning on not [21:39:52] right. you may need to tolerate some degree of error due to "vandalism" and stay on the safe side and not collect username at all. Especially if a large number of editors, organizers, admins, patrollers, etc. participate, and assuming that the majority will respond only truthfully (which is a safe assumption), you should be fine. [21:40:32] And btw, I assume a lot of time will be spent on finding the right questions. [21:40:49] anyhow, keep us in the loop if you decide that you want to proceed and there is some support for it. [21:43:30] I'll keep you in the loop, the main brainstorm now needs to be coming up with questions on how users feel when editing as well [21:43:34] isaacj: can the community insights survey be a good place for this type of question? [21:44:13] RhinosF1: what do you mean by feel? [21:48:36] leila: whether they're comfortable editing, how they're supported [21:49:27] RhinosF1: I see. I have a million more questions. ;) I'll stop here and we can dig deeper (maybe in a video call?) when you are ready to jump in this. [21:50:36] Throw anything you want at me, I'll try and answer them somewhere [21:52:30] ok. :) (and I just saw that you will join in the office hour. that'a great place to do sync discussion.) [21:53:59] leila: not sure. depends on the goal for the survey in a given year (e.g., whether it's focused on Medium Term Plan or something else). questions about an individual's identity are generally somewhat limited to basic demographics but they have asked about perceptions of the movement and inclusiveness before -- e.g., [21:53:59] https://meta.wikimedia.org/wiki/Community_Insights/2018_Report#What_is_the_current_health_of_Wikimedia_communities?_Related_to_community_health,_are_Wikimedia_communities_inclusive? [21:55:02] isaacj: makes sense. thanks. (I'd love piggybacking on the infrastructure and knowledge of that effort if it works in this case, because it is really about insights about the movement.) [21:55:58] I plan to be hear for the office hour [21:56:43] nice. [21:58:55] yeah i think the concern is that the existing survey is already quite long and so there's hesitation to add more (as that increases dropout and likely skews who does respond). but i haven't heard plans yet for the next survey