[06:32:42] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663#c3 (nuria) We had alarms about traffic drop on Aug 16th 23:03 (packetloss), Aug 17th 00:11, Aug 17th 05:51, Aug 17th 07:51 (oxygen). Recovery was sent on Aug 17th 04:51 (packetloss) and Aug 17... [07:47:27] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663#c4 (nuria) Looked at the files for 17th and 18th again and the only event I can find with significant loss is printed below. 2014-08-17T06:24 3474 2014-08-17T06:25 3516 2014-08-17T06:26 1385 (... [07:49:45] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663 (nuria) a:nuria>christian [11:21:30] Analytics / Tech community metrics: Wrong data at "Update time for pending reviews waiting for reviewer in days" - https://bugzilla.wikimedia.org/68436#c9 (Alvaro) The problem with (In reply to Quim Gil from comment #4) > Ok, I'm sorry. I shouldn't send patches going beyond edits in text strings > with... [13:12:22] (CR) Milimetric: "A couple of general things:" (17 comments) [analytics/dashiki] - https://gerrit.wikimedia.org/r/154787 (owner: Nuria) [13:25:08] hi nuria - I'm around and looking at dashboard stuffs [13:25:22] we can talk anytime you like [13:26:04] specifically, these things are very cool: [13:26:06] https://github.com/SteveSanderson/knockout-triage/blob/master/src/js/lib/knockout-batch.js [13:26:22] (batches updates so when we're binding 1000 projects the UI doesn't freak out) [13:26:28] http://knockoutjs.com/documentation/textinput-binding.html [13:26:47] (handles text input better than the value binding, which typically has to be hacked to play nice) [14:01:44] (do we still have a logbot?) [14:02:04] !log changing group ownership and permissions on raw webrequest data in hdfs. You now must be in the analytics-privatedata-users group to access. [14:02:18] Ah it is 10! [14:22:33] No logbot for us. ... but having one would be great. [14:23:04] wm-bot2: can you please be a log bot too? [14:26:48] Yikes! analytics-bot is doing the logging. [14:26:53] https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:27:01] ottomata1: ^ [14:27:36] (But it seems it only has 4 entries in 2014 :-) ) [14:30:10] milimetric: did you try tried that dashboard sets up correctly (via bower json and npm install) [14:30:36] sorry milimetric|afk [14:31:29] oops [14:32:48] oh, it did work? [14:32:50] cool [14:32:56] (PS1) Yuvipanda: Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 [14:33:08] (CR) jenkins-bot: [V: -1] Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 (owner: Yuvipanda) [14:33:36] (PS2) Yuvipanda: Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 [14:33:45] nuria: it looks like it puts bower_components in the root instead of in /src [14:34:09] there are two sets of components in bower [14:35:01] one in tests one in src [14:35:36] yes, nuria I think you just need a .bowerrc in the root folder that tells it to put the packages in src/bower_components [14:35:39] amilimetric: Are both wrong ? [14:35:50] { [14:35:50] "directory": "src/bower_modules" [14:35:50] } [14:36:14] no, the test one seems fine [14:36:28] ok, that is something else to fix [14:36:50] yep, sorry I missed it on the CR [14:40:02] (PS3) Yuvipanda: Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 [14:46:09] (PS4) Yuvipanda: Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 [14:47:13] (CR) Yuvipanda: [C: 2] Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 (owner: Yuvipanda) [14:47:18] (Merged) jenkins-bot: Sanitize rows before inserting into sqlite [analytics/quarry/web] - https://gerrit.wikimedia.org/r/155263 (owner: Yuvipanda) [14:59:28] (CR) Milimetric: [C: -1] "Looks good. The factory+stub / mock thing is just a discussion I want to have. I'm leaning only slightly away from the factory. The res" (12 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/155003 (https://bugzilla.wikimedia.org/68507) (owner: QChris) [15:00:55] nuria: we have the predictable awesome support from SQLAlchemy when we want to change our mw connections to hit a single shard and issue "use blah;" statements: http://docs.sqlalchemy.org/en/rel_0_9/core/connections.html#sqlalchemy.engine.Engine.execution_options [15:01:19] nice! [15:01:32] sqlalchemy is really A_MA_ZING [15:01:59] (PS1) Gergő Tisza: Fix global optout counts [analytics/multimedia] - https://gerrit.wikimedia.org/r/155272 [15:08:22] milimetric, there is also jquery 2 which i did not know about: http://zurb.com/article/1293/why-we-dropped-zepto [15:09:20] benchmarks are ridiculous though [15:18:11] nuria: I'm all for jquery 2, zepto doesn't behave properly with Require [15:20:37] why not? [15:20:40] I had heard about it but wanted to try zepto. If you look carefully at dashiki, I think zepto's never being included. I request it in my module definition but require gets confused and $ just ends up being jquery [15:21:03] no, that is not because of require, it works fine [15:21:30] it is because you are loading jquery who defines window.$ = window.jQuery [15:21:31] if I do require(['zepto'], function(zepto){ // zepto does not work here [15:21:53] I thought that's what it was, but I was trying the above and always failing [15:21:57] it does [15:22:11] really, i have it all working with typeahead [15:22:23] very weird [15:22:26] zepto initializes itself excatly like jquery there is no difference [15:22:34] when it comes to initialization [15:22:56] you need this: [15:23:33] sorry, rather "something like this": shim: { [15:23:33] "typeahead": { deps: ["zeptoAdaptor"] }, [15:23:34] "zeptoAdaptor": { deps: ["zepto"] }, [15:23:34] } [15:23:54] if you look at scafolding for KO that is how it is done for jquery [15:24:04] and bootstrap [15:24:16] on require.config.js [15:24:37] nuria: forgetting about typeahead and other stuff [15:24:46] yes [15:24:51] I just tried the basic thing of requiring zepto and it comes into my function as undefined [15:25:15] because if you have jquery too they cannnot "share" "$" [15:25:20] this is my code: [15:25:20] define(['knockout', 'zepto', 'text!./wikimetrics-data-browser.html'], function(ko, zepto, templateMarkup) { [15:25:23] zepto [15:25:24] so I'm not using $ at all [15:25:37] zepto.get then throws an error [15:25:42] that is the problem [15:25:48] you need to use $ [15:26:10] the define [15:26:17] does not "set" the order [15:26:24] in taht case there [15:26:35] konckout could be loaded before zepto [15:26:38] well, the define should get me an object that fulfills "zepto" or else there's a problem with the module definition [15:27:23] sorry, I am not explaining myself clearly [15:27:41] substitute zepto by jquery in your define and try teh same thing, maybe taht is more clear [15:28:16] for a module w/o defines (zepto /jquery) [15:28:21] you need to use shims [15:28:26] taht is what tehy are for: http://requirejs.org/docs/api.html#config-shim [15:28:28] nuria: so that worked... [15:28:37] define(['jquery', 'knockout', 'text!./wikimetrics-data-browser.html'], function(zepto, ko, templateMarkup) { [15:28:53] but this fails: [15:28:53] define(['zepto', 'knockout', 'text!./wikimetrics-data-browser.html'], function(zepto, ko, templateMarkup) { [15:29:03] so look at your jquery module if it has "define" [15:29:07] will work w/o shime [15:29:13] if it doesn't [15:29:19] oh! [15:29:22] it likely will not [15:29:24] ok, yeah, so that's what I'm saying [15:29:29] zepto sux with require :) [15:29:44] my point wasn't that it's unusable [15:29:57] just that it wasn't a mature product that thought about super common things like require compatibility [15:30:34] I agree those performance tests the jquery 2 post links to is hilarious but I would bet that jquery is fast in general and more likely to get faster [15:31:13] mature and define are two differnt thins [15:32:08] not really - if a project lives in its own world where it pretends requirejs isn't popular, that's weird [15:32:13] and it does work fine with require.js, milimetric, iam using it right now. [15:32:38] with shims though, right? [15:33:12] take alook at your config: https://github.com/milimetric/dashiki/blob/master/src/app/require.config.js [15:33:20] jquery is on teh shim.. [15:33:53] well they're stupid too then :) [15:34:33] the version of jquery that you get from bower [15:34:35] wait no, that shim is for bootstrap - seems like jquery is defined properly no? [15:34:39] is not made for requirejs [15:34:43] k [15:34:46] it doesn't matter :) [15:34:49] top of file: [15:35:05] I'd rather stick with jquery 2, it seems like a better bet going forward [15:35:11] especially if we have to patch zepto [15:35:44] wait, so why do you need the .data patch though? [15:35:58] well, we have to see if it needs [patching then , so far i haven't patch it but it might need some things here and there [15:36:16] oh for typeahead [15:37:42] We will see, if it needs too much work , you are right, it will not be a best choice but has nothing to do with requirejs, if you remove jquery from your shim, you will get same behaviour you were seeing with zepto [15:37:58] sorry, *the best choice [15:45:25] (PS1) Ottomata: Add --location CLI opt to refinery-drop-webrequest-partitions [analytics/refinery] - https://gerrit.wikimedia.org/r/155281 [15:46:54] (PS2) Ottomata: Add --location CLI opt to refinery-drop-webrequest-partitions [analytics/refinery] - https://gerrit.wikimedia.org/r/155281 [15:54:26] Analytics / Quarry: Support queries containing non-latin strings - https://bugzilla.wikimedia.org/69786 (Yusuke Matsubara) NEW p:Unprio s:normal a:None When I use a non-latin string in a query, it apparently fails to match properly. An example of queries containing non-latin strings is: US... [16:47:59] ok, milimetric i found a pretty big thing that does work with zepto and typeahead so i will need to try jquery2 [16:48:15] but , same shims and all that does not change [17:07:23] any luck with jquery2 nuria? [17:07:37] ya , it works fine [17:08:16] as long as is loaded in the way we talked about before [17:08:33] I am going to install sublime to try it out [17:08:43] milimetric do want to use jshint? [17:09:22] nuria: no preference on js checker, whatever the cool kids are doing these days [17:10:20] does it have a gulp task? [17:17:25] milimetric, what did you wantd to do for spacing, so i have teh settings on sublime? [17:19:50] what do you prefer usually in JS? [17:19:54] I usually do 4 [17:20:24] nuria: ^ [17:20:36] sounds good [17:24:01] milimetric: and line width? [17:24:38] nuria: I usually don't set that for JS, it gets annoying [17:25:30] i saw a few indents on blank lines in your first patch, I think we should either have them all over or not at all [17:33:13] wamt to talk about components? I need to do still some cleanup of code but i can share screen [17:33:27] it's all stubs + typeahead [17:33:35] ^ milimetric [17:33:52] nuria: got scrum of scrums now, but i'll ping right after [17:35:12] ok [17:58:19] nuria: k, done [17:58:31] ok [17:58:42] i was still setting up sublime [17:58:52] do you have time now then? [17:59:16] yep [18:00:05] did you wanna talk in batcave or here? [18:00:17] batcave right? [18:00:34] I type wayyy to slowww [18:00:42] *too [18:00:44] see? [18:05:09] milimetric, batcave? [18:06:19] heh :) sorry, lots of people pinging me [18:06:19] brt [19:36:25] Analytics / General/Unknown: Thanks tool: How much does the confirmation button cost us? - https://bugzilla.wikimedia.org/69804 (WhatamIdoing) NEW p:Unprio s:normal a:None This is related to bug 61737. Can we find out how many thanks are not being completed on desktop? That is, the user... [19:36:35] Analytics / General/Unknown: Thanks tool: How much does the confirmation button cost us? - https://bugzilla.wikimedia.org/69804 (WhatamIdoing) [20:02:45] milimetric: I've been trying to set up a limn bar chart with multiple data sources but could not figure out how [20:02:48] https://gerrit.wikimedia.org/r/#/c/150750/3/graphs/uw_funnel.json [20:02:56] what would be the proper way to do that? [20:03:15] tgr: looking [20:05:36] tgr: I'll comment inline, ok? [20:05:44] thanks [20:08:21] (CR) Milimetric: Query UploadWizard funnel data (2 comments) [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/150750 (owner: Gergő Tisza) [20:08:35] tgr: ^ [20:08:45] http://mobile-reportcard.wmflabs.org/graphs/edits-monthly-active-2014 is a live example as linked there [20:09:00] http://mobile-reportcard.wmflabs.org/graphs/edits-monthly-active-2014.json?pretty shows you the details [20:09:59] thanks again [20:10:22] combining bar + ordinal-bar was just a weird workaround for two of the same type not working [20:47:37] Analytics / General/Unknown: geowiki data aggregation failed on 2014-08-19 - https://bugzilla.wikimedia.org/69812 (christian) NEW p:Unprio s:normal a:christian on 2014-08-19 a PoolWorker had MySQL connection [1] issues during the process_data.py job, and that took the whole job down. Since... [20:54:28] (PS1) Ottomata: Change mode of webrequest hourly imports when oozie add partition runs [analytics/refinery] - https://gerrit.wikimedia.org/r/155432 [21:11:01] (PS1) Ottomata: Set HDFS umask-mode=037 when Camus webrequest import runs [analytics/refinery] - https://gerrit.wikimedia.org/r/155439 [21:11:51] (Abandoned) Ottomata: Change mode of webrequest hourly imports when oozie add partition runs [analytics/refinery] - https://gerrit.wikimedia.org/r/155432 (owner: Ottomata) [21:14:30] (PS2) Milimetric: Move get_cohort_by_name() to CohortService [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154479 (owner: AndyRussG) [21:19:13] (PS2) Milimetric: Add validators module and use it in cohort_upload [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154482 (owner: AndyRussG) [21:20:20] (CR) Milimetric: [C: 2] Add validators module and use it in cohort_upload [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154482 (owner: AndyRussG) [21:23:32] AndyRussG: I submitted two of your changes but jenkins is acting up I think [21:23:34] or gerrit [21:23:49] Hey milimetric [21:23:51] I'm gonna have to go in a bit but I'll continue to review tomorrow [21:23:53] Yeah I saw, thanks! [21:24:08] Sure, feel free to ping any time if you have questions... :) [21:24:15] thanks, will do [21:30:07] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663#c5 (christian) Oxygen's alarms (see comment #3) around packet loss and udp2log from 2014-08-16 23:03:09 until 2014-08-17 07:51:08 were just artifacts of bug 69661. Loss in TSVs (from comment #... [21:32:02] (CR) Milimetric: [C: 2] Move get_cohort_by_name() to CohortService [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/154479 (owner: AndyRussG) [21:53:11] (PS19) Milimetric: Create a cohort from campaign participations [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/126927 (owner: Awight) [21:54:09] (CR) Milimetric: [C: -1] "I just did a basic rebase and made sure all the tests pass. Andy or Adam, mind going over that real quick and just +1 it if everything lo" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/126927 (owner: Awight) [21:54:21] AndyRussG: see my comment above ^ [21:54:56] it rebased more cleanly than I thought, but there might be still stylistic stuff that changed that I'll have to go through and fix [21:56:07] Analytics / General/Unknown: Packet loss alarm on oxygen on 2014-08-16 - https://bugzilla.wikimedia.org/69663 (christian) REOP>RESO/FIX [21:58:29] milimetric-afk: OK will do [22:44:51] Analytics / General/Unknown: geowiki data aggregation failed on 2014-08-19 - https://bugzilla.wikimedia.org/69812#c1 (christian) (In reply to christian from comment #0) > Since stat1003 had its distupgrade on the same day, maybe those two things > related? The stat1003 distupgrade looks unrelated. Th... [23:17:51] Analytics / EventLogging: Multiple user_ids per username in account creation events from ServerSideAccountCreation log - https://bugzilla.wikimedia.org/66101#c13 (Matthew Flaschen) (In reply to Aaron Halfaker from comment #12) > You probably shouldn't be able to register an account name if someone > al... [23:59:50] Analytics / EventLogging: Multiple user_ids per username in account creation events from ServerSideAccountCreation log - https://bugzilla.wikimedia.org/66101#c14 (Chris Steipp) (In reply to Matthew Flaschen from comment #13) > However, I think it's still better to not allow creation in this scenario....