[09:01:43] Quarry: Add a list/table of popular queries - https://phabricator.wikimedia.org/T71266#1444241 (Edgars2007) @He7d3r don't think this is a good idea to get top N queries by number of times they were executed. When one is testing the query, he probably executes it //to much//. But I support the idea itself to... [11:26:31] Quarry: Add a list/table of popular queries - https://phabricator.wikimedia.org/T71266#1444564 (He7d3r) ...maybe the most accessed (i.e. most page views)? [11:51:24] (PS12) Milimetric: Add global default report fields [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/217857 (https://phabricator.wikimedia.org/T74117) [12:28:35] Analytics-Tech-community-metrics: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1444622 (Aklapper) >>! In T56230#1443372, @Dzahn wrote: > How do you add channels to korma? Editing `/browser/data/json/irc-repos.json` in https://github.com/Bitergia/mediawiki-dashboard [14:22:46] (PS1) Mforns: Move pid and history files to project folder [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/224077 (https://phabricator.wikimedia.org/T103385) [14:29:47] (CR) Mforns: Move pid and history files to project folder (3 comments) [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/224077 (https://phabricator.wikimedia.org/T103385) (owner: Mforns) [14:58:37] hmnb. Where is ottomata? [14:58:46] (alternately can anyone else push new refinery snapshots to the cluster?) [15:24:58] Analytics, Analytics-Backlog, Performance-Team, Patch-For-Review: Collect HTTP statistics about load.php requests - https://phabricator.wikimedia.org/T104277#1445244 (Krinkle) > Change 224016 merged by @ori: > varnishrls: segment responses by cache-control max-age > > [[https://gerrit.wikimedia.org... [15:29:15] Ironholds: otto's on vacation [15:30:00] gotcha! [16:01:47] (PS13) Milimetric: Add global default report fields [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/217857 (https://phabricator.wikimedia.org/T74117) [16:13:05] Analytics-Backlog, Compact-Personal-Bar-(Beta): Delete all data from EventLogging:PersonalBar schema {tick} - https://phabricator.wikimedia.org/T105065#1445417 (ggellerman) [16:19:51] Analytics-Backlog, Deployment-Systems, Performance-Team, operations, Varnish: Verify traffic to static resources from past branches does indeed drain - https://phabricator.wikimedia.org/T102991#1445428 (Milimetric) Hey guys, so we are trying to get away from servicing ad-hoc data requests, but we... [16:22:48] Analytics-Backlog, Analytics-Dashiki: Improve the edit analysis dashboard {lion} - https://phabricator.wikimedia.org/T104261#1445440 (ggellerman) p:Triage>High [16:24:27] Analytics-Backlog, Analytics-EventLogging: Puppetize parallel eventlogging-processor {stag} - https://phabricator.wikimedia.org/T104228#1445448 (ggellerman) p:Normal>High [16:29:00] Analytics-Backlog, Analytics-Cluster: Epic: qchris transition - https://phabricator.wikimedia.org/T86135#1445457 (ggellerman) [16:29:01] Analytics-Backlog, Analytics-EventLogging: Investigate EventLogging Monitoring with Ops DBA {oryx} - https://phabricator.wikimedia.org/T86200#1445455 (ggellerman) Open>declined p:High>Normal [16:31:00] Analytics-Backlog: RUBICON - https://phabricator.wikimedia.org/T105515#1445474 (ggellerman) [16:39:15] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1445525 (ggellerman) [17:09:55] Analytics-Kanban: Bug: puppet not running on wikimetrics1 instance, Vital Signs stale {musk} [5 pts] - https://phabricator.wikimedia.org/T105047#1435170 (Milimetric) Fixed puppet by commenting out some lines in /etc/apt/sources.list that were causing puppet to error out. Verified that puppet runs are now pul... [18:59:27] o/ neilpquinn [18:59:40] I just finished my analysis of active editor counts and rates. [18:59:41] https://meta.wikimedia.org/wiki/Research_talk:Active_editor_spike_2015/Work_log/2015-07-09#Rates_of_activation [18:59:47] Nice, thanks! [19:00:40] It looks like there was a burst in registrations and a corresponding drop in new editor activation rates, but not in newcomer survival and old editor continued activation. I think that's what caused our temporary burst in actives. [19:01:15] So by temporary you mean it's dropped off again? [19:01:30] Hard to say, but it looks like it. [19:01:42] Well... I take that back. It seems like it might not have in big wikis [19:01:49] Top cell in this figure https://meta.wikimedia.org/wiki/File:Scaled_mae_counts.big_wikis.svg [19:02:24] It looks like July 2014 resulted in a 2*sd gain in registrations [19:02:56] There's two spikes -- one in Aug. 2014 (wikimania?) and one in April 2015. [19:03:03] I'm not sure what those could correspond to. [19:03:22] Actually, it could be commons Wiki Loves * [19:03:27] I'll split out commons. [19:03:32] Analytics-Cluster, Analytics-Kanban: Generate test data for Pageview API {slug} [5 pts] - https://phabricator.wikimedia.org/T101785#1446197 (mforns) **1) Page title hourly** The hive query looks good to me! I agree with you that K=100 seems too much in this case. I like the idea of having just daily gran... [19:03:55] Interesting. Looking at the mobile editors data, it looks like there's a substantial spike in mobile active editors between July 2014 and April 2015. [19:04:11] And persisting in between. [19:04:54] Do you know when we switched tablets to the mobile interface? [19:05:11] Not sure. I can look. [19:05:12] Deskana, ^ [19:06:08] By the by, when can we assume that the ServerSideAccountCreation eventlogging is capturing all creations? It looks like it got there in August 2013. [19:07:25] neilpquinn, good Q. I think that DarTar might know more about that. [19:07:31] We can run some quick analyses to check. [19:08:07] Let me run that query. [19:11:57] Okay, here's a simple count. [19:11:58] https://phabricator.wikimedia.org/P939 [19:12:37] Which shows that spike you're talking about. [19:13:03] Yup. I think this is mobile. [19:14:30] The totals for June aren't that high though. [19:14:47] You think what is mobile? [19:14:57] The spike? [19:15:13] I was going to say that, but now I'm second-guessing. [19:15:30] I need to plot these on top of eachother and do a seasonal decomposition. [19:16:03] Can you add your query to that phab? [19:17:07] Done. In case it wasn't clear, that was all events, not just mobile. [19:17:33] So I'm looking at just commons and that's where I see the spikes as most pronounced. [19:18:20] neilpquinn, make sure you filter by "event_isSelfMade" and "event_displayMobile" [19:21:15] What is "self made" about? If it's proxy creations, do we even have those on public wikis? [19:21:53] When you create an account on one wiki and browse to another, an account is created for you when you show up. [19:22:19] See https://meta.wikimedia.org/wiki/Research:Newly_registered_user [19:23:15] Woah! A lot of non-selfMade accounts happened in the months we're looking at through the mobile UI. [19:24:14] halfak: look at my updated paste. You can see mobile creations having a massive spike in the time we're talking about. [19:24:21] Woah indeed! [19:24:55] It looks like those are all autocreated. [19:25:21] See my last comment [19:25:24] neilpquinn, ^ [19:26:03] I didn't think the schema tracked autocreations, and that's what DarTar said too. [19:26:22] Oh, oh! self_made is about autocreations. [19:26:23] That's the whole point is event_isSelfMade. [19:26:46] Yeah. Nice to have a boolean field. Usually we need to deal with historical artifacts in the logging table ;) [19:28:52] neilpquinn, I think the answer to our question is going to be apparent when we find out what happened with this huge influx of autocreated accounts. [19:28:59] Could it be SUL? [19:30:20] That doesn't //sound// right. But I could certainly be wrong. [19:31:31] Here's my question about SelfMade: the schema page says "False if existing user created this account for someone else, true otherwise". [19:32:02] Also, I've had accounts autocreated for me since 2013, yet I don't show up in the table at all. [19:36:49] Is it sampled? [19:39:56] Hello! [19:39:58] I was pinged? [19:40:25] Yes, halfak and I were wondering when we switched tablets to the mobile interface. [19:43:13] We released the Android app, with anonymous editing, on 25th June 2014 [19:43:27] And the iOS app, also with anonymous editing, on the 31st July 2014 [19:43:41] Or when you said "mobile" did you mean "mobile web"? [19:44:47] Sorry, yes, I meant mobile web. [19:44:55] But those dates are helpful too. [19:46:25] neilpquinn, I believe that event_displayMobile works for apps too. [19:47:12] neilpquinn, I have to switch tasks, but I sent you a data file that has enough data to replicate my analysis (and you might be able to load it up in a spreadsheet) [19:47:19] Let me know if you have any questions. [19:47:27] Okay, thanks! [19:48:02] halfak: I did have the one above about SelfMade: the schema page says it's "False if existing user created this account for someone else, true otherwise". Also, I've had accounts autocreated for me since 2013, yet I don't show up in the table at all. [19:49:05] OK. Can you find it in the logging table of the apt. wiki? [19:49:31] I found it in the user table. [19:49:43] Should I check the logging table as well? [19:51:45] Yes. I want to know the log_action for the creation [19:53:39] Yep, it's in there as log_action = "autocreate". [19:58:14] halfak: I checked another autocreation (this one in Nov 2014). It's also in the log, but not in the EventLogging data. [19:58:31] I can give you a paste if it would help. [19:58:33] Can you give me a paste? [19:59:21] It would be nice to get the sequence of queries you ran to determine this. [19:59:52] halfak, neilpquinn: From what I can tell, the tablet redirect did take place at around the same time. [19:59:57] Don't quote me on that though... yet. [20:00:22] I have an email in my inbox titled "Re: PLEASE READ: Tablet Redirect, scheduled for Tuesday (6/17)" [20:01:24] halfak: on it. [20:01:30] Yeah, I'm 99% confident that that was the date it happened. [20:03:39] Deskana: Thanks! [20:03:49] Halfak: See https://phabricator.wikimedia.org/P941. [20:06:48] neilpquinn, indeed. I agree that the registration isn't there. [20:06:53] Not sure why. [20:07:02] Could you have been autocreated via the mobile app>? [20:07:28] Definitely not. I hadn't even installed the app at that point. [20:07:51] I tried another (eswikisource) and got the same result. And I definitely didn't go to the Spanish Wikisource on my phone. [20:07:59] neilpquinn, no idea why it wasn't caught then [20:09:45] halfak: How confident are you that the schema is set up to capture autocreations? [20:10:15] neilpquinn, I didn't program it, so as confident as I am in the docs meaning what they say. [20:10:25] Also, there's a huge amount of non-self-made accounts. [20:10:51] I know—and autocreations is the only thing I can think of that would account for that. [20:11:12] Given that you don't have any special knowledge about it, I'm happy to keep investigating and let you go on to other things. [20:15:48] Thanks neilpquinn [20:16:04] FWIW, I don't use that log in my analysis. [20:16:19] But I don't know a better way to break out mobile account creations :| [20:16:48] I'll keep crunching on it. Thanks for all the help! [20:23:52] milimetric, yt? [20:35:20] hey team, I'm going to prepare my baggage! see you tomorrow in mexico! :D [20:56:11] (CR) Milimetric: [C: -1] "I think there's a compatibility problem after the changed parameters with generate.py, take a look at the comment." (2 comments) [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/224077 (https://phabricator.wikimedia.org/T103385) (owner: Mforns) [20:58:35] I should get out of here and prepare for wikimania too. See you all tomorrow!! Can't wait :) [22:52:45] Deskana: Is it possible that when a user creates an account through the mobile apps, it gets event logged as a "non-self made" account creation? (i.e. the creating user is different from the created user) [22:53:13] neilpquinn: The app doesn't support you doing that. [22:53:32] neilpquinn: I'm impossible to get to the account creation page if you're logged in. [22:53:53] Analytics-Backlog, Analytics-EventLogging, MediaWiki-extensions-MultimediaViewer, Multimedia: Parse mediaviewer team's requirements for EventLogging {oryx} - https://phabricator.wikimedia.org/T90766#1446594 (ggellerman) [22:56:07] Deskana: Gotcha. Well, the account creation event logs have 2.7 million non-self creation events, 91% of which also have displayMobile = true. So I'm a bit mystified. [22:56:51] neilpquinn: I'll come over and take a look with you. [22:57:45] Analytics-Backlog, Analytics-Wikimetrics, Community-Wikimetrics, Easy, Need-volunteer: "Create Report" button does not appear when uploading a new cohort - https://phabricator.wikimedia.org/T95456#1446648 (ggellerman)