[17:09:02] miriam_: FYI https://phabricator.wikimedia.org/T155538 no action needed, unless you want to subscribe to it and/or follow up. [18:22:57] leila: just saw this [18:23:08] subscrbing it to it! [18:24:05] greetings! [18:25:38] ow hey mako1. [18:25:45] miriam_: sure. [18:26:34] hi mako1! [18:27:35] I will be the IRC host today, folks. Ping J-Mo with questions. We're going to be starting in a minute or two. [18:27:44] hola [18:27:51] hola groceryheist [18:30:39] alright, we're live! YouTube stream should be activating momentarily. [18:31:01] update the topic to the new youtube URL? [18:31:14] i was confused :) [18:31:56] what is the URL? [18:31:58] J-Mo: new URL is https://www.youtube.com/watch?v=Z1pa-pr6xis and is linked right from the wiki [18:32:25] Thanks :) [18:32:27] thanks mako. thought someone else had already done that. [18:32:52] halfak and or @here can someone else update the link? I never remember the series of commands :( [18:33:49] there is some background noise. Is this at the speaker end? :) [18:34:01] somewhere close to one of the mics something is happening. [18:34:13] DarTar: ^ [18:34:27] Got it J-Mo [18:34:31] I just muted Williams [18:34:32] should be good now [18:34:35] seems to have helped [18:34:49] no, it continues. It's on the speaker's end likely [18:35:11] let's wait a minute for the stream to catch up [18:35:11] ah, sorry about that [18:35:23] o/ [18:35:42] brendan_campbell: the background is quiet now. thanks. I'll report back if I hear it again. [18:35:55] thanks leila [18:48:12] question for Nicholas: how was the impact of Reddit/SO on new account creations calculated? (considering that these are not tied to a particular article linked from these sites) [18:49:18] got it HaeB [18:51:24] BTW, a review of the paper in the research newsletter is here: https://meta.wikimedia.org/wiki/Research:Newsletter/2018/February#Wikipedia's_one-way_relationships_with_Reddit_and_Stack_Overflow [18:53:57] J-Mo: comments for the speaker. I very much appreciate the effort to estimate the financial value of Wikipedia. This can help copyright discussions and negotiations Wikimedia Foundations or chapters take part in. I also want to highlight that this kind of citable information is key for some of our communities. I can speak to the Persian wikipedia community where I'm from. We do get the kind of "open/free license, so what?" ques [18:54:36] got it leila [18:59:27] no comment that needs to be relayed but this is a super cool study! [18:59:35] I agree ;) [19:00:29] rebooting my machine, Chrome frying [19:00:39] second presentation is starting now [19:00:53] WOO CDSC!! [19:01:04] thanks, J-Mo, for relaying. [19:01:44] <3 [19:01:46] RAD [19:01:47] funny aside: we are referred to as "the RAD authors" throughout the paper. [19:02:07] I <3 RAD. :D [19:02:07] aaron shaw gets full credit for discovering that acronym :) [19:02:12] not saying that calling WMF staff RAD guarantees you a Research Showcase slot, but it certainly doesn't hurt [19:02:20] I love that "RAD" stamp on the graph [19:02:23] hahaha [19:02:34] Ok, people! lets listen. ;) [19:03:26] "The RAD Theory" 😂 [19:07:14] exogenous factors influencing decline; maybe Wikipedia is a "special snowflake" [19:07:49] whether it not it's related to what happened in 2007, wikipedia is definitely a special snowflake [19:07:56] <3 [19:07:58] I’ve always wanted to see more work on exogenous factors and estimating their relative impact [19:08:10] tell Facebook to give us their historical data [19:08:20] * DarTar jumps on the phone [19:08:23] ty [19:09:06] aren’t historical DAU/MAU from FB public? Not that it would be the best data source [19:11:04] my favorite hypothesis is that exogenous factors didn’t really manifest themselves in hijacking attention but in shifting expectations about collaboration and online interaction and making the idea that “anyone can edit what you’re saying online” an alien concept to a larger and larger fraction of internet users [19:11:30] that's interesting. don't think I've heard that one before [19:12:50] BTW, here is the paper itself: https://teblunthuis.cc/outgoing/perm/rise_decline_replication_CHI_CR.pdf (in general, how about we try to include references in the announcements on https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase ?) [19:13:17] HaeB: yeah +1 [19:13:23] I usually link to it at the bottom of the abstract, when I'm the one updating the page. [19:13:29] thanks HaeB :) [19:15:45] J-Mo: Note that the survival rates in the RAD plot were for "good-faith and golden" newcomers. vandals were excluded and that could shift comparisons of survival rates. [19:15:58] Not a question, I guess [19:15:59] got it halfak [19:16:05] I don't understand that plot, std should decrease with time... I think. [19:16:13] halfak: aaah! [19:16:14] *vandals and goofballs were excluded [19:16:29] or I'm not understanding the concept of wiki age [19:16:33] dsaez, fewer wikis over time? [19:16:34] dsaez: it's not the SD, it's the mean in SD units [19:17:06] mako, makes sense [19:17:23] but halfak is right, the number of wikis at the end is smaller (which is why the error bars are bigger) [19:17:26] q: (for both Nate and halfak/J-Mo) didn’t the original RAD study have a number of language communities where the decline started around the same time (late 2006 / early 2007) despite the corresponding communities being in very different phases of their trajectory [19:17:49] DarTar, we only looked at enwiki [19:17:50] that wasn't convered in the paper [19:17:58] I thought that was refering to # revisions [19:18:11] weren’t there follow up analyses on this? [19:18:18] I've seen a few different plots of that, not sure how well it correlates with absolute age vs relative age. [19:18:50] the reason why I mention it is that – from what I remember – it’s been called out as one of the possible sources of evidence of exogenous factors [19:19:22] DarTar, I hear that and I think it misses an important source of endogenous factors. The wikis overlap and their norms/policies develop in parallel [19:19:30] E.g. Huggle was deployed to all wikis at the same time. [19:19:46] halfak: to dewiki too? [19:20:07] right, that’s a good point, but I haven’t seen a historical analysis of parallel development of norms, quality control tools, bots etc [19:20:18] *any historical analysis [19:20:19] good point. So when other language wikis adopt tools and policies from enwiki, they adopt them as mature/established entities [19:20:34] I'd love to do a comparative analysis of policy development across wikis [19:20:44] There's also CVN which is cross-wiki counter-vandalism [19:21:23] there's also a good paper about transfer of practices across wikiprojects that could be relevant to cross-language adoption of QC mechanisms [19:21:33] ....i'm wondering about the importance of huggle for dewiki, anecdotally the number of huggle edits there seemed very low (haven't checked the actual numbers though) [19:21:54] it's also the case that the original results were very similar between the GF edits and the non-GF edits in original RAD [19:21:59] sorry [19:22:16] the hand-coded GF edits, and the simple quantitative measure (i think non-reverted?) [19:22:24] not denying the existence of these cross-language tools, but I’d love to see more analysis to corroborate the results [19:22:25] i forget.. all edits? [19:23:09] J-Mo: I think that maybe the formalization of quality contol is a primary effect and the use of algorithmic support (and thus formalization of certain practices) exacerbate the issue. Do you think this could be the case? In other words, we'd expect Wikipedia to see declining retention (which we did) before the tools were involved, but far less -- and then a sudden shift when tools were involved (which we did). [19:23:22] another possible endogenous factor (which I don’t think anyone has looked at) might be related to multi-language editors, experiencing negative interactions on site A and thereby withdrawing from all languages [19:23:34] +1 to dartar, or to put it a bit differently: has anyone tried to replicate RAD's findings for Wikipedia other than english? [19:23:43] question for Nate: you mentioned Wikipedia has been in decline "ever since" 2007, but the RAD graph on that slide ended in 2012, more than half a decade ago - has the trend continued since then? [19:24:03] HaeB, we should do that once ORES gets some more traction ;) [19:24:21] HaeB: you should ask groceryheist! i think he may have at least started [19:24:40] ...or to wikidata? [19:24:45] got it HaeB [19:25:00] ...it would be nearing the inflection point now [19:25:37] I think Wikidata is just about to go quality crazy. I can feel it :| [19:26:57] mako: ah good, i wasn't sure if it would be a fair question based on the presentation ;) [19:27:39] question for Nate/groceryheist: have you (or anyone else you're aware o) looked at replicating RAD's findings for Wikipedia itself, in languages other than english? [19:27:41] HaeB: i know he at least started collecting the data. but i don't know if he ever ran the analysis [19:28:04] got it HaeB [19:28:20] * DarTar jumps on the queue [19:28:35] but I’ll ask first for qs from the room [19:29:01] got it DarTar [19:30:19] HaeB: to your question, I am not aware of any such attempts (and I get pings every time the paper is cited) [19:31:23] Boo dario. I disagree. I don't think we need to do that work. I'd be interested in seeing it, but I think it's safe to come to some tentative conclusions based on the robustness of this result. [19:32:05] wasn't there a great Pokemon character purge around 2007? [19:32:17] DarTar: i have some preliminary analysis on this :) [19:32:25] DarTar: i can share a presentation [19:32:51] the "gold rush" perspective DarTar mentioned is also known as/related to the "low-hanging fruit hypothesis" [19:32:59] eg. https://meta.wikimedia.org/wiki/Research:Newsletter/2012/August#%22Low-hanging_fruit_hypothesis%22_explains_Wikipedia's_slowed_growth? [19:33:26] Slowed growth != Sudden, abrupt shift from growth to decline :P [19:33:48] I think they were referring to growth in articles [19:33:51] thx mako [19:33:53] OH FAIR [19:33:56] Wait. [19:34:01] There is no slowed growth in articles. [19:34:08] DarTar: lets follow up on email [19:34:11] ah, then WTF [19:34:16] ....or https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2011-04-25/In_the_news ("on average, articles created early in Wikipedia’s life received many more hits than articles created more recently, suggesting that newer articles tend to be about low-interest topics. ") [19:34:48] Also: Lam, S. T. K., & Riedl, J. (2009, May). Is Wikipedia growing a longer tail?. In Proceedings of the ACM 2009 international conference on Supporting group work (pp. 105-114). ACM. [19:35:04] ^ which covered this 2 years earlier [19:36:33] tyranny of averages :) [19:36:59] HaeB of course that's self-reinforcing. Esp. as WIkipedia's audience was increasing and diversifying at a rapid clip during that period. Possible readers "learned" what kinds of things to look for on Wikipedia, and what to look for elsewhere, leading to a rich get richer phenomenon [19:37:50] i'm headed out! thanks everybody! [19:37:54] great job to both speakers! [19:38:02] thank you everyone [19:38:15] Good show, folks :) [19:38:22] i'll be around in this channel if anyone wants to talk more [19:38:28] o/ groceryheist [19:38:32] gotta do the honors with my guest here, thanks all! [19:38:36] Good to see you. I'm racing back to my CSCW submissions [19:38:55] halfak: i think that may be the same research by lam and riedl as quoted in the signpost article (from https://www.computer.org/portal/web/computingnow/0411/whatsnew/computer-r ) [19:39:05] Oh! must be :) [19:44:15] I didn't answer HaeB's question about non-english wikipedias [19:44:25] i actual did a version of this analysis that included non english wikipedias [19:44:51] but decided not to report it [19:45:06] my memory is that the results held up [19:45:06] groceryheist: i think we ran out of time, so no worries [19:45:13] any preliminary findings? [19:45:23] i see [19:46:03] i don't remember if there was a difference [19:46:25] between *-WP and wikia [19:46:37] i have a half hour meeting now, but i'll be around in the afternoon [19:48:54] groceryheist: also, regarding my earlier question, i was also asking because there had been a sense that the decline has been stopped or even slightly reversed in the last two years or so. just looked it up myself: https://stats.wikimedia.org/v2/#/en.wikipedia.org/contributing/editors (click "All") [19:50:09] i think there's also been a change in customer expectations over the past 15 years [19:50:16] customers here being customers of the internet [20:37:29] HareJ: Seems like the trend is indeed pretty flat since 2014 [20:37:53] hard to say if going up or if seasonal variation is getting smaller [20:39:13] I agree about 'customer expectations.' I think looking at multiple wikis that start at different points in time can help address that [20:39:37] but if the trend is the same through the entire period it would not [20:40:11] like if the change was just like something that happened in 2006 [20:40:40] looking at wikis created in 2008, we can say 'that doesn't explain it all' [20:41:03] but if it's a continuous change between 2002 and now, then we don't really know [21:58:29] 10Quarry: Add current git version somewhere in the interface - https://phabricator.wikimedia.org/T192506#4141022 (10Framawiki) [22:27:13] 10Quarry: Add current git version somewhere in the interface - https://phabricator.wikimedia.org/T192506#4141022 (10zhuyifei1999) Git version is complicated... The web interface (`quarry-main-01`) runs a copy of the code, and two more query runners (`quarry-runner-0[12]`) run two more copies of the code. The co...