[01:07:19] (PS1) Madhuvishy: [WIP] Change config path in wsgi file [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/261604 [02:10:53] Analytics, MobileFrontend, Patch-For-Review, Reading-Web-Sprint-63-Ellip…, WMF-deploy-2016-01-12_(1.27.0-wmf.10): Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1909506 (dr0ptp4kt) A simple `count(*)` on events from November 2015 took 3.5 minute... [09:05:56] Analytics-Tech-community-metrics, DevRel-January-2016: gerrit_review_queue can have incorrect data about patchsets "waiting for review" - https://phabricator.wikimedia.org/T121495#1909721 (Aklapper) p:Low>Normal Currently the data isn't reliable to find older unreviewed changesets. Can this be inves... [12:25:21] Analytics, MobileFrontend, Patch-For-Review, Reading-Web-Sprint-63-Ellip…, WMF-deploy-2016-01-12_(1.27.0-wmf.10): Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1909801 (Tbayer) >>! In T108723#1909506, @dr0ptp4kt wrote: > A simple `count(*)` on... [15:09:55] Analytics-Tech-community-metrics, DevRel-January-2016: gerrit_review_queue can have incorrect data about patchsets "waiting for review" - https://phabricator.wikimedia.org/T121495#1909879 (Lcanasdiaz) a:Lcanasdiaz [15:10:10] Analytics-Tech-community-metrics, DevRel-January-2016: gerrit_review_queue can have incorrect data about patchsets "waiting for review" - https://phabricator.wikimedia.org/T121495#1880244 (Lcanasdiaz) Really strange, I'll have a look [15:14:08] Analytics-Kanban: Disable legacy tsv mobile, zero and 5xx-mobile jobs - https://phabricator.wikimedia.org/T122650#1909883 (Ottomata) NEW a:Ottomata [15:14:26] Analytics-Kanban: Disable legacy tsv mobile, zero and 5xx-mobile jobs - https://phabricator.wikimedia.org/T122650#1909892 (Ottomata) [15:16:30] Analytics-Kanban: Reorganize oozie jobs to not use mobile cache webrequest_source - https://phabricator.wikimedia.org/T122651#1909893 (Ottomata) NEW [15:17:07] Analytics-Kanban: Reorganize oozie jobs to not use mobile cache webrequest_source - https://phabricator.wikimedia.org/T122651#1909904 (Ottomata) [15:17:24] Analytics-Kanban: Reorganize oozie jobs to not use mobile cache webrequest_source - https://phabricator.wikimedia.org/T122651#1909893 (Ottomata) [15:18:03] (PS1) Ottomata: Removing webrequest legacy tsv mobile, 5xx-mobile and zero jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/261634 (https://phabricator.wikimedia.org/T122650) [15:19:20] (CR) Ottomata: [C: 2 V: 2] Removing webrequest legacy tsv mobile, 5xx-mobile and zero jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/261634 (https://phabricator.wikimedia.org/T122650) (owner: Ottomata) [15:23:14] !log killing oozie legacy_tsv job 0102159-150605005438095-oozie-oozi-B to restart it without mobile, 5xx-mobile and zero outputs [15:23:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [15:24:27] Analytics-Tech-community-metrics, DevRel-December-2015: Many profiles on profile.html do not display identity's name though data is available - https://phabricator.wikimedia.org/T117871#1909913 (Lcanasdiaz) Currently the people panel does display name. The issue here is that this name is not coherent due... [15:25:40] (PS1) Ottomata: Remove reference to coordinator_mobile_file in legacy tsv bundle.xml [analytics/refinery] - https://gerrit.wikimedia.org/r/261635 (https://phabricator.wikimedia.org/T122650) [15:25:55] (CR) Ottomata: [C: 2 V: 2] Remove reference to coordinator_mobile_file in legacy tsv bundle.xml [analytics/refinery] - https://gerrit.wikimedia.org/r/261635 (https://phabricator.wikimedia.org/T122650) (owner: Ottomata) [15:30:28] ottomata: holaaa [15:34:18] Analytics-Backlog, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API from the Event Bus perspective - https://phabricator.wikimedia.org/T112956#1909946 (Halfak) Chiming in here because this question came up in discussions regarding the #ORES API. It seems that we are discussing th... [15:35:51] hello! [15:42:25] Analytics-Kanban, Patch-For-Review: Disable legacy tsv mobile, zero and 5xx-mobile jobs - https://phabricator.wikimedia.org/T122650#1909961 (Ottomata) Open>Resolved New legacy_tsvs oozie bundle is 0182811-150922143436497-oozie-oozi-B [15:54:00] Analytics, MobileFrontend, Patch-For-Review, Reading-Web-Sprint-63-Ellip…, WMF-deploy-2016-01-12_(1.27.0-wmf.10): Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1909986 (Nuria) >@Nuria, is the size of the table too big for the old data such that... [16:00:04] o/ joal [16:00:05] Around? [16:00:16] ottomata: yt? I have a question [16:00:24] halfak: joal is on vacation until dev summit [16:03:59] yes! [16:04:00] i am [16:04:04] wassup nuria? [16:04:26] ottomata: on this query https://gerrit.wikimedia.org/r/#/c/216341/8/oozie/last_access_uniques/daily/last_access_uniques_daily.hql [16:04:38] that in order to insert data does a UNION at the end [16:04:48] ottomata: how are reruns handled? [16:05:21] ottomata: that is, reruns cannot override what's there .. correct? [16:06:05] hm? [16:07:10] hm like if the new data is alread in the archive table? [16:07:15] hm, i dunno [16:08:07] ottomata: seems that we would need to drop the table [16:08:31] ottomata: regenerate it with records that are not going to be re-run [16:08:44] ottomata: and re-run [16:08:53] ottomata: which ... ahem... seems hardly optimal [16:09:05] yeah seems so [16:09:16] hm yeah [16:09:29] i guess the table would have to be partitioned for overwrite to work properly? [16:09:35] so that a whole partition is overwritten [16:09:49] but, that defeats the purpose of the single file archive table [16:10:18] ottomata: this is the way mobile uniques are handled: https://github.com/wikimedia/analytics-refinery/blob/master/oozie/mobile_apps/uniques/daily/generate_uniques_daily.hql [16:10:27] ottomata: and i know joal has had to re-run those [16:10:55] ottomata: or .. wait.. are these any different? [16:10:56] same, right? [16:11:05] they look the same to be, but i'm not sure [16:11:36] ottomata: ya, they do. Ok, i will code the last access jobs in this style and ask joal about re-runs [16:12:47] nuria, thanks (re. joal vacation) [16:15:15] Analytics-Kanban: Gather phabricator metrics for quaterly review - https://phabricator.wikimedia.org/T122333#1910020 (Nuria) Related to https://phabricator.wikimedia.org/T122626 [17:24:13] nuria: another one for ya :) https://gerrit.wikimedia.org/r/#/c/261638/ [17:24:37] ottomata: looking [17:25:50] ottomata: i see, is the idea then than tokudb will come from puppet? [17:26:02] ottomata: the engine setting that is [17:26:08] sure, i guess, if we need to change it [17:26:13] i left tokudb as the default [17:26:17] but my test database didn't have tokudeb [17:26:20] and i didn't want to bother [17:26:23] so i wanted it to be configurable [17:33:26] ottomata: ah ok, so the driver for the change was testing [17:33:44] ottomata: it is more and more evident taht we have config mixed with code [17:33:59] ottomata: in EL all over the place [17:38:17] like those batch settings? aye [17:41:00] ottomata: ya, and config for db table [17:41:02] engine [17:41:13] ottomata: and other os settings [17:41:37] ottomata: but that is a minor point, just something to have in mind when designing a new service [17:42:51] nuria: the motivation isn't just testing, it'll be useful to insert meta style events in prod one day [17:43:10] e.g., if we wanted the new eventbus edit stream in the mysql eventloggign db too [19:25:55] nuria: spose you are cone, eh? [19:25:56] gone* [22:04:40] Hey folks. Can anyone tell me where I'd fine these files via stat1003? https://dumps.wikimedia.org/other/pagecounts-all-sites/2015/ [22:46:10] madhuvishy: turns out that country calculations can be done more efficiently.. i think doing them for daily is overkill but we can probably do monthly