[05:52:11] Analytics, MediaWiki-extensions-MultimediaViewer, Multimedia, Patch-For-Review: Investigate if pre-rendering images is having an impact on performance - https://phabricator.wikimedia.org/T76035#1613121 (Gilles) [06:45:12] Analytics-Dashiki, Analytics-Kanban, Browser-Support-Firefox: vital-signs doesn't display pageviews graph in Firefox 41, 42 {crow} [3 pts] - https://phabricator.wikimedia.org/T109693#1613180 (Nemo_bis) I still have the same issue. No chart works, other than LegacyPageviews. [08:55:30] Analytics: stats.grok.se doesn't offer mediawiki.org page view stats - https://phabricator.wikimedia.org/T111662#1613323 (Spage) [09:29:20] Analytics: stats.grok.se doesn't offer mediawiki.org page view stats - https://phabricator.wikimedia.org/T111662#1613372 (Schnark) Though not in the dropdown, the statistics actually do work: http://stats.grok.se/www.w/latest30/MediaWiki [10:52:23] Analytics-Cluster, Analytics-Kanban: Update UA parser for better spider traffic classification {hawk} [5 pts] - https://phabricator.wikimedia.org/T106134#1613510 (JAllemandou) a:JAllemandou [10:53:44] Analytics-Kanban: Create white list for pageview data {hawk} [8 pts] - https://phabricator.wikimedia.org/T110061#1613519 (JAllemandou) [10:54:01] Analytics-Kanban: Add a 'Guard' job for pageviews {hawk} [13 pts] - https://phabricator.wikimedia.org/T109739#1613520 (JAllemandou) [10:54:22] Analytics-Kanban: Provide the Wikimedia DE folks with Hive access/training {flea} [8 pts] - https://phabricator.wikimedia.org/T106042#1613522 (JAllemandou) [10:54:33] Analytics-Kanban: Change the agent_type UDF to have three possible outputs: spider, bot, user {hawk} [13 pts] - https://phabricator.wikimedia.org/T108598#1613523 (JAllemandou) [13:43:11] Analytics-Cluster, Analytics-Kanban: Update UA parser for better spider traffic classification {hawk} [8 pts] - https://phabricator.wikimedia.org/T106134#1613815 (JAllemandou) [13:46:15] Analytics-Cluster, Analytics-Kanban: Update UA parser for better spider traffic classification {hawk} [8 pts] - https://phabricator.wikimedia.org/T106134#1613823 (JAllemandou) Thanks @QChris I found ways to check differences between repos, and understood why we repackaged ua-parser. It seems now that ua-... [13:47:06] Analytics-Kanban: Change the agent_type UDF to have three possible outputs: spider, bot, user {hawk} [13 pts] - https://phabricator.wikimedia.org/T108598#1613830 (JAllemandou) a:JAllemandou [15:24:55] gudmoning [15:25:12] hulooo nuria [15:32:33] nuria: do we make a fake standup ? [15:32:37] :) [15:32:50] joal: if you are there sure [15:33:03] I am :)à [15:33:35] madhuvishy: Hello ! [15:34:03] If you are here, we are in standup [16:04:23] madhuvishy: not in standup anymore :) [16:15:09] joal: nuria morning :) [16:15:35] madhuvishy: holaaa [16:18:35] I'm a sleeper [16:22:53] madhuvishy: holaaa [16:23:04] hi nuria :) [16:23:24] joal: if you have time to deploy the restbase jobs ping me :) [16:38:31] Hey madhu [16:38:38] we can do that now if you want :) [16:38:44] madhuvishy: --^ [16:38:59] joal: cool, coming to batcave [16:45:20] (PS1) Joal: Update changelog.md before deployment. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/236565 [16:45:59] (CR) Madhuvishy: [C: 2 V: 2] Update changelog.md before deployment. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/236565 (owner: Joal) [16:51:31] (PS4) Madhuvishy: Add oozie job to schedule restbase metrics generation job [analytics/refinery] - https://gerrit.wikimedia.org/r/235519 (https://phabricator.wikimedia.org/T110691) [16:55:05] (CR) Joal: [C: 2 V: 2] "good! Merging :)" [analytics/refinery] - https://gerrit.wikimedia.org/r/235519 (https://phabricator.wikimedia.org/T110691) (owner: Madhuvishy) [17:08:29] (PS1) Joal: Bump core and job jar versions to 0.0.18 [analytics/refinery] - https://gerrit.wikimedia.org/r/236569 [17:09:22] (CR) Madhuvishy: [C: 2 V: 2] Bump core and job jar versions to 0.0.18 [analytics/refinery] - https://gerrit.wikimedia.org/r/236569 (owner: Joal) [17:39:14] !log deployed refinery-source and refinery [17:41:48] joal: question [17:41:51] sure [17:42:11] are we trying to make all changes to cluster code backwards compatible? [17:42:24] cause now i have to add 1 field to isPageviewUDF() [17:42:40] hm [17:42:41] joal: to make it backwards compatible i need to update the udf to be [17:42:46] That's a good question [17:42:46] a genericUDF [17:42:56] rather than the simpler one [17:43:13] otherwise when we deploy these chnages current oozie jobs will break [17:43:13] I can't recall having made a backward incompatible change [17:43:25] So .... [17:43:29] joal: cause this one shouldn't be [17:43:36] yup [17:43:45] joal: as an addition should not be non-backwards compatible [17:43:53] joal: it is because of the way udfs are coded [17:43:55] But changing from 'easy UDF' to 'Genric UDF' is not nice [17:44:03] joal: not because of the nature of the change [17:44:38] joal: not nice as in "too much work" ? [17:44:46] joal: or ...something else? [17:45:00] A good bunch of work yes [17:45:07] For bad reason in that case [17:46:20] joal: 'not enough of a good reason' you mean? [17:47:05] Nah, I mean, UDFs are really not nice to play with, so it's not cool to have to change from easy one to generic one just to prevent a breaking change in the API [17:47:08] That's all [17:47:15] But doable for sure [17:47:47] The clean way is to make the UDF a generic one and prevent breaking the API [17:48:02] But are we really that much after no breaking change ? [17:48:09] I can't tell for sure [17:48:39] I think most people use the field generated at refine time, not the udf [17:48:42] joal: "are we really that much ..." [17:48:57] joal: the counting jobs use the udf [17:49:23] joal: like "./webrequest/refine/refine_webrequest.hql" [17:49:32] nuria: That's the only one [17:49:42] others (in our stack) dopn't [17:50:07] But as you said, this change shouldn't be an API breaker [17:50:25] We can wait tomorrow for team opinion I guess [17:50:26] joal: right, it only is because of implementation [17:50:39] Or you can go for clean, which is always cleaner :D [17:51:15] nuria: implementation is important, you surelly know that better than I do :) [17:51:34] joal: i will do changes to "generify" udf and undo them if it doesn't pertain, but design wise that is the way to go I think. it is sad to have to stop the cluster rightttt? [17:52:02] nuria: we relaunch jobs regularly so it's no big deal [17:52:22] If a breaking change happen, that's really manageable in our side [17:52:41] the thing is about other people using our UDFs (if any) [17:53:18] madhuvishy: graphite updates, jobs are run fast, that looks awesome :) [17:54:22] nuria: I haven't yet bezen involved in discussions about API breaking changes in our team, so I might not be the one to go to for best advice ! [17:54:34] nuria: Let's ask ottomata and milimetric tomorrow :) [17:55:08] nuria: anything else ? If no, I stop my day :) [18:12:08] joal: sorry [18:12:40] joal: seeya tomorrow [18:15:30] np nuria :) [18:15:35] Tomorroooooow ! [18:21:55] good night :) [18:32:17] Analytics-Kanban, RESTBase, Patch-For-Review: Create a metric for overall RESTBase request rates from Varnish logs {hawk} [13 pts] - https://phabricator.wikimedia.org/T109547#1614458 (madhuvishy) This is done, and the job scheduled on production. Graphite will be updated for hourly numbers starting fro... [18:52:21] Analytics-Kanban, RESTBase, Patch-For-Review: Create a metric for overall RESTBase request rates from Varnish logs {hawk} [13 pts] - https://phabricator.wikimedia.org/T109547#1614525 (GWicke) @madhuvishy, thanks a lot! [21:48:24] Analytics, MobileFrontend: Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1614924 (JKatzWMF) @Jhernandez, I would check with @kevinator about who could help us with the purging. In the meantime, separating the search out from the rest of the clicks woul... [21:59:12] Analytics, MobileFrontend: Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1614941 (Jdlrobson) Given there is mobilewebsearch do we even need this? Can that be used to the same effect?