[04:28:51] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, LE-Sprint-84: language-reportcard.wmflabs.org shows wrong graphs - data swapped - https://phabricator.wikimedia.org/T92237#1107223 (Arrbee) [04:28:53] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, LE-Sprint-84: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1107226 (Arrbee) [04:28:57] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, LE-Sprint-84: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1107228 (santhosh) p:Triage>High [04:29:00] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, LE-Sprint-84: language-reportcard.wmflabs.org shows wrong graphs - data swapped - https://phabricator.wikimedia.org/T92237#1107230 (santhosh) p:Triage>High [05:33:44] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1107368 (Arrbee) [05:34:01] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: language-reportcard.wmflabs.org shows wrong graphs - data swapped - https://phabricator.wikimedia.org/T92237#1107369 (Arrbee) [09:26:58] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1108758 (Arrbee) a:KartikMistry [09:27:28] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: language-reportcard.wmflabs.org shows wrong graphs - data swapped - https://phabricator.wikimedia.org/T92237#1108760 (Arrbee) a:KartikMistry [12:28:31] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1109026 (Milimetric) When the ky and pa wikis were added, they were... [12:28:53] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1109027 (Milimetric) a:KartikMistry>Milimetric [12:37:44] (PS1) Milimetric: Fix select order [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/195885 (https://phabricator.wikimedia.org/T92236) [12:38:59] (CR) Milimetric: [C: 2] Fix select order [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/195885 (https://phabricator.wikimedia.org/T92236) (owner: Milimetric) [12:40:20] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 3 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1109093 (Milimetric) Open>Resolved Please take a look at the... [12:51:09] Analytics-Engineering, Analytics-Wikimetrics, Patch-For-Review: Story: WikimetricsUser reports pages edited by cohort - https://phabricator.wikimedia.org/T75072#1109135 (Milimetric) @Abit: "individual results" is what you think it is, but we also use that term to refer to the option of outputting "indi... [13:23:10] (CR) Joal: Adapt bin/refinery-drop-webrequest-partitions to work with refined table (3 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/195672 (https://phabricator.wikimedia.org/T89257) (owner: Ottomata) [13:37:45] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 3 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1109296 (KartikMistry) @Milimetric Thanks a lot! http://language-re... [13:38:11] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 3 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1109297 (KartikMistry) My bad. Sync is still needed. Sorry! [13:38:49] no prob kart_ [13:49:38] (PS1) Milimetric: Analyze page size impact on editing [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/195895 (https://phabricator.wikimedia.org/T89788) [13:50:11] milimetric: btw, my US visa is all done. I’ll book tickets tomorrow :) [13:50:16] and be in the US in 2-3 weeks [13:50:32] YuviPanda: that's awesome news [13:50:38] milimetric: :D yeah. [13:50:39] you took the race with Madhu seriously :) [13:50:41] milimetric: are you in NY? [13:50:43] milimetric: hehe :D [13:50:50] I'm in Philly [13:50:52] milimetric: we might actually sync up and take the same flight (or not). who knows. [13:50:58] ah, right [13:51:02] cool [13:51:08] are you both going to SF first? [13:51:12] I realized that the only thing I know about Philly is ‘cheese steaks' [13:51:21] I don’t even know what a cheese steak is. [13:51:22] :) that's pretty much all there is [13:51:36] my favorite cheesesteak place just got robbed at gunpoint last week [13:51:44] ow [13:51:51] robbed of cheesesteak or ..? [13:51:59] so basically cheesesteak is a beef and cheese hot sandwich with very high risk [13:52:04] :) [13:52:42] hahaha [13:52:43] :D [13:52:43] ok [13:54:52] if you're not going straight to SF, btw, you're always welcome to crash on the east coast. We've got representation in NYC and Philly [13:57:25] milimetric: I’ll be in NY for a couple weeks in May, before hackathon. [13:57:41] milimetric: my close friend is graduating out of NYUPoly, so I’ll go attend graduation. Hopefully will see a few people before that :) [13:58:04] milimetric: I don’t know if not going to SF straight is an option. I’m sure there’s paperwork to be done. [13:58:41] I might come out to SF if Madhu is starting out [13:58:49] but let me know when you're in NY as well [13:59:54] milimetric: ya, will do [14:00:00] milimetric: she hasn’t decided when she’s gonna move, I think [14:02:00] (CR) Ottomata: Adapt bin/refinery-drop-webrequest-partitions to work with refined table (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/195672 (https://phabricator.wikimedia.org/T89257) (owner: Ottomata) [14:18:03] Analytics-Kanban: Fix analysis that categorizes users (anonymous is bad) {lion} - https://phabricator.wikimedia.org/T92300#1109431 (ggellerman) a:Milimetric [14:19:21] Analytics-EventLogging, Analytics-Kanban, operations: Eventlogging JS client should warn users when serialized event is more than "N" chars long and not sent the event - https://phabricator.wikimedia.org/T91918#1109436 (ggellerman) a:Nuria [14:32:23] (PS1) Milimetric: Make adhoc slightly more generic [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/195900 [14:32:34] (CR) Milimetric: [C: 2 V: 2] Make adhoc slightly more generic [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/195900 (owner: Milimetric) [14:35:54] Analytics-Cluster, Analytics-Kanban: Read wiki dumps in Spark - https://phabricator.wikimedia.org/T92385#1109485 (JAllemandou) NEW a:JAllemandou [14:50:16] nuria: to be more homogeneous, should I move the functional code of UAParserUDF to the source/refinery-core module? [14:50:33] Didn't realize I wasn't awake :) [14:50:49] joal: I do not remember why ottomata want it on the other place ..... [14:51:19] joal: Sounds like yes, it shoudl be moved. I think since that was written we have a lot more structure [14:51:35] Yup [14:51:41] nuria: thx :) [14:51:58] ottomata: i guess you agree with nuria? --^ [14:53:07] the reason it didn't have a core module is because there was no functionality beyond what ua parser had [14:53:14] the UDF was just a thin wrapper for UA parser [14:53:21] so there was no reason to have a second layer wrapper in core [14:53:35] if we changed this, so that there is more custom logic in the way we use UA parser [14:53:50] then an interface in core would make sense [14:53:59] Right [14:54:06] get it [15:03:40] (CR) Ottomata: Adapt bin/refinery-drop-webrequest-partitions to work with refined table (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/195672 (https://phabricator.wikimedia.org/T89257) (owner: Ottomata) [15:04:51] (PS3) Ottomata: Adapt bin/refinery-drop-webrequest-partitions to work with refined table [analytics/refinery] - https://gerrit.wikimedia.org/r/195672 (https://phabricator.wikimedia.org/T89257) [15:04:53] woo internet is slow here [15:04:56] ther it goes [15:05:01] joal ^^ [15:05:11] ottomata: reading [15:09:44] (CR) Joal: Adapt bin/refinery-drop-webrequest-partitions to work with refined table (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/195672 (https://phabricator.wikimedia.org/T89257) (owner: Ottomata) [15:09:53] ottomata: one last thing :) [15:10:10] ottomata: Also, do you want me to test it on labs cluster? [15:10:42] naw, i've been testing it in my vagrant [15:10:56] it has a dry run mode that I will try in prod before puppetizing it [15:11:03] ok [15:11:21] What about the last comment, makes sense ? [15:12:04] oh, yeah, dict ther ebetter [15:12:06] on it... [15:12:52] ottomata: I suggest moving the 'NA to -' part of the code to core [15:13:27] Like that, it would also provide a base providing same data for non-hive systems [15:13:46] hm, yeah there is some custom logic here [15:13:52] ja, sounds good joal [15:13:56] yeah, the udf should be as thin as possible [15:14:35] almost not, only changing null to '-' [15:14:45] ok sounds good [15:17:48] joal: new patchset up, not sure where gerrit bot is... [15:17:52] https://gerrit.wikimedia.org/r/#/c/195672/ [15:20:48] Hey ottomata : you good to go :) [15:21:23] danke [15:23:58] joal: are you done backfilling mobile apps? [15:24:10] Yessir [15:24:25] ottomata: --^ [15:25:08] cool [15:43:48] !log deleting refined webrequest data older than 62 days for the first time. this will happen from here on out [15:45:25] Analytics-Cluster, Patch-For-Review: Improve hive partition deletion script to work with refined webrequest tables. - https://phabricator.wikimedia.org/T89257#1109658 (Ottomata) [15:47:51] Analytics-Cluster, Patch-For-Review: Improve hive partition deletion script to work with refined webrequest tables. - https://phabricator.wikimedia.org/T89257#1109664 (Ottomata) Open>Resolved Done! Running the drop script today on refined data for the first time: ``` 2015-03-11T15:44:36 INFO Dro... [16:01:28] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1109694 (dr0ptp4kt) [16:22:58] Analytics, MediaWiki-extensions-MultimediaViewer, Multimedia, Multimedia-Sprint-2015-03-11, Patch-For-Review: Set up varnish 204 beacon endpoint for virtual media views and use it in Media Viewer - https://phabricator.wikimedia.org/T89088#1109752 (Gilles) [16:23:52] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1109760 (dr0ptp4kt) There's an update. See https://lists.wikimedia.org/pipermail/analytics/2015-March/003583.h... [16:46:32] wikimedia/mediawiki-extensions-EventLogging#363 (wmf/1.20wmf21 - 3cee9e0 : Mukunda Modell): The build passed. [16:46:32] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/3cee9e06cace [16:46:32] Build details : http://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/53969557 [16:51:59] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1109863 (DarTar) @dr0ptp4kt, awesome, can you document this parameter scheme somewhere on mediawiki.org or wik... [17:01:29] ottomata: I wonder about my tests ... [17:01:43] (PS1) Joal: Move UAParser wrapper to refinery-core and update refinery-hive accordingly. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/195952 [17:19:41] Analytics, MediaWiki-extensions-MultimediaViewer, Multimedia, Blocked-on-Performance, and 2 others: Set up varnish 204 beacon endpoint for virtual media views and use it in Media Viewer - https://phabricator.wikimedia.org/T89088#1110021 (MarkTraceur) [18:04:42] lucnhy, back soon [18:23:01] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 3 others: Newly added language ky and pa doesn't reflect in Language Limn Graph - https://phabricator.wikimedia.org/T92236#1110347 (Nikerabbit) There was one additional issue of last three l... [18:34:34] wikimedia/mediawiki-extensions-EventLogging#364 (wmf/1.25wmf21 - 79b069f : Mukunda Modell): The build passed. [18:34:34] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/79b069fe2ec9 [18:34:34] Build details : http://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/53985661 [18:35:42] Analytics, MediaWiki-extensions-MultimediaViewer, Multimedia, Blocked-on-Performance, and 2 others: Set up varnish 204 beacon endpoint for virtual media views and use it in Media Viewer - https://phabricator.wikimedia.org/T89088#1027235 (Gilles) Now I just need to update the Media Viewer patch to p... [19:31:50] ottomata: yt ? [19:33:04] yup hey [19:33:09] batcave ? [19:44:46] (CR) Ottomata: Move UAParser wrapper to refinery-core and update refinery-hive accordingly. (1 comment) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/195952 (owner: Joal) [19:46:11] Bye guys [19:47:49] Analytics, Analytics-Kanban: Code changes in VCL to add a Last-Acess cookie to track "top domain" uniques - https://phabricator.wikimedia.org/T92435#1110744 (Nuria) NEW a:Nuria [20:04:19] Analytics, ContentTranslation-Analytics, Language-Engineering, MediaWiki-extensions-ContentTranslation, and 2 others: language-reportcard.wmflabs.org shows wrong graphs - data swapped - https://phabricator.wikimedia.org/T92237#1110805 (Nikerabbit) Open>Resolved This was fixed in T92236. [20:39:21] joal|night, when you wake up, you must know that your advice to try a trie was awesome! I am substantially more performant *and* I use far less memory! WOOO! [20:39:53] Also it was super freakin' easy :) [20:45:39] Analytics-Cluster, Analytics-Kanban: Mobile product managers should have reports on session-related metrics from the Wikipedia Apps - https://phabricator.wikimedia.org/T86535#1110939 (kevinator) p:Normal>High [20:45:48] halfak: a trie? [20:46:26] Yeah. It's a datastructure that builds a byte tree that is useful for doing fast lookups on tokens. [20:46:34] https://en.wikipedia.org/wiki/Trie [20:52:52] ottomata: trie tree they are called [20:53:28] hm, coool [20:55:20] Bah... It's breaks my position tracking to re-use tokens though. [20:55:23] * halfak grumbles [21:32:27] Analytics-Engineering, Analytics-Wikimetrics, Patch-For-Review: Story: WikimetricsUser reports pages edited by cohort - https://phabricator.wikimedia.org/T75072#1111153 (Abit) Thanks for the explanation, @Milimetric, it's super helpful. I think our program leaders and evaluators would find individual... [21:42:09] mforns: yt? [21:42:18] hi ottomata :] [21:42:23] https://gerrit.wikimedia.org/r/#/c/196073/ [21:42:30] this may help your varnishnsca woahs [21:42:32] woes* [21:42:47] no mor udp! [21:42:57] oh! [21:43:25] do you want me to review it? or just have a look? [21:43:38] well, at least, if we deploy this thing and run it, and set up a varnishkafka instance to log to eventlogging, we can maybe start moving away from varnishncsa for that [21:43:45] um, its ok, i'll get nuria or ori to review it [21:43:50] just wanted to know i was thikning about you [21:44:00] i think this is a first step to getting a more reliable backend system [21:44:02] for eventlogging [21:44:12] aha [21:44:15] this still relies on the 0mq pubsub system [21:44:19] so this should be very transparent [21:44:23] I see [21:44:27] just adds another way to get data into eventlogging [21:45:28] nice! [21:47:31] we'll see if anybody likes it :p [22:19:35] Bah. I have to take it back. I can't get the performance gains and still allow a token to be aware if it's position. [22:19:57] I tried the trie, but I'll need to do some more work before I can benefit from using it. [22:20:03] ottomata: of course everyone will like it [22:21:11] mforns: this is a way to get arround varnishncsa logging limits [22:23:19] (CR) Nuria: [C: 1] Move UAParser wrapper to refinery-core and update refinery-hive accordingly. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/195952 (owner: Joal) [22:29:26] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1111389 (Fjalapeno) I was going to pick this up but its not clear what the actual issue is from the ticket des... [23:14:27] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1111463 (dr0ptp4kt) I'll update the ticket description. [23:19:46] (PS11) Mforns: Reportupdater: Write/Update SQL reports into CSV files. [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/192319 (https://phabricator.wikimedia.org/T89251) [23:19:55] (CR) jenkins-bot: [V: -1] Reportupdater: Write/Update SQL reports into CSV files. [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/192319 (https://phabricator.wikimedia.org/T89251) (owner: Mforns) [23:22:20] (PS12) Mforns: Reportupdater: Write/Update SQL reports into CSV files. [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/192319 (https://phabricator.wikimedia.org/T89251) [23:22:26] (CR) jenkins-bot: [V: -1] Reportupdater: Write/Update SQL reports into CSV files. [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/192319 (https://phabricator.wikimedia.org/T89251) (owner: Mforns) [23:23:35] Analytics, Scrum-of-Scrums, Wikipedia-App-Android-App, Wikipedia-App-iOS-App, and 2 others: Avoid cache fragmenting URLs for Share a Fact shares - https://phabricator.wikimedia.org/T90606#1111532 (dr0ptp4kt)