[00:19:54] * average is lurking from Iasi in the north-east [05:29:48] erosen: ping. [07:55:54] Change abandoned: Rfaulk; "re-applying change with some corrections" [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62792 [07:56:21] Change abandoned: Rfaulk; "re-applying change with some corrections" [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62791 [08:48:29] New patchset: Rfaulk; "add. 'dist' aggregator for 'pages_created' metric." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/63110 [08:48:29] New patchset: Rfaulk; "add. Register 'dist' aggregator for 'pages_created' in API." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/63111 [08:50:30] New review: Rfaulk; "tested." [analytics/user-metrics] (master); V: 2 - https://gerrit.wikimedia.org/r/63111 [08:51:47] Change merged: Rfaulk; [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/63110 [13:11:47] morning everyone [13:39:31] Change merged: Milimetric; [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/63111 [14:05:46] mmmmmooooooooorrrrnnninnnnnggg [14:12:00] morning drdee [14:12:13] i'm at a coworking place [14:12:13] so it's kinda loud [14:12:23] and i forgot my power adapter! so i'm gonna try to mooch from someone [14:16:36] k [14:22:10] erosen [14:22:16] aroound? [14:22:45] we are running low on inodes on /home, [14:22:48] what is in /home/erosen/tmp/repos ? [14:22:55] and could you potentially delete that? [14:23:06] (or move) [14:34:57] hey [14:35:02] sorry, was showering [14:35:15] that is my cache of git repos representing articles [14:35:21] which i'm currently using [14:35:27] i can move it elsewhere [14:35:35] but it is on /a/ i thought [14:36:10] nvm, it's not [14:36:13] i'll move it /a/ now [14:36:34] ty, but how many repos do you have? [14:36:42] … 81k [14:36:52] 81k unique repos? [14:36:55] yeah [14:37:08] then moving to /a might create the same problem there [14:37:15] they each represent an article which I'm checkingin this complicated search procedure [14:37:19] true [14:37:25] can you reduce the number of repos? [14:37:33] i can easily cut it in half [14:37:37] if not more [14:42:32] that would be great! [14:44:09] milimetric: that's an ugly repo name ;( [14:44:21] indeed [14:44:38] but it's better! 'cause it's *2* [14:44:56] anyway, i'll just delete this when it's done [14:44:56] and merge into the normal one [14:44:59] rather overwrite [14:45:22] k [14:46:56] * erosen is deleting 70k repos... [14:49:44] drdee: are you sure it was inodes? [14:49:55] yes [14:50:04] Filesystem Inodes IUsed IFree IUse% Mounted on [14:50:13] /dev/mapper/stat1-root 915712 118754 796958 13% / [14:51:16] drdee: i'm just curious what tool you used [14:51:32] df -i [14:55:39] i guess it must be /dev/mapper/stat1-home [16:22:30] hey Snaps [16:22:36] how's it going? [17:20:50] hello [17:21:44] a few days ago I was discussing with average about this: https://github.com/steko/wikipedia-analytics-museums [17:22:11] but I couldn't find him anymore in this chatroom [17:22:40] he is on holidays [17:22:42] what's up? [17:25:29] milimietric; https://gerrit.wikimedia.org/r/63143 is waiting for ottomata to merge [17:28:36] drdee: thanks, I was just confused because he was very enthusiastic about my work and then he disappeared w/o notice. [17:30:52] rfaulckr, milimetric: here's a query that kills UMAPI (producing a 500) https://metrics.wikimedia.org/cohorts/test2/threshold?time_series&start=20111201&end=20120301&group=registration&aggregator=proportion&slice=24 [17:32:18] ciao steko [17:32:25] [Fri May 10 17:26:57 2013] [error] [client REMOVED Premature end of script headers: api.wsgi, referer: https://metrics.wikimedia.org/cohorts/test2/threshold?time_series&start=20111201&end=20120301&group=registration&aggregator=proportion&slice=24 [17:32:39] ciao DarTar [17:32:45] end of script headers...... [17:33:20] drdee: interesting, this also flushes the job queue [17:33:39] so possibly related to the "disappearing jobs" issue I was mentioning earlier [17:33:44] rfaulckr ^^ [17:34:24] a job is running now btw [17:35:26] job queue is empty [17:35:40] a job is running :) [17:36:03] yeah, just saying that it doesn't show up [17:36:49] steko: chi si occupa di opendata in Italia (a parte openpolis o OKFN-it)? [17:40:08] DarTar: dozzine di persone! c'è spaghetti open data, una comunità abbastanza attiva; GFOSS.it (geodati); ahref.eu ... solo per dirne alcuni! [17:40:39] ah non li conoscevo, mo' li guardo, grazie [17:41:45] DarTar: comunque guarda anche http://dati.gov.it/ [17:43:01] sì sì lo conoscevo, cercavo di trovare organizzazioni coinvolte in advocacy e hacking [17:43:55] e speravo qualcuno segnalasse al governo il decreto della casa bianca di ieri come un modello da seguire [17:47:58] wow, steko: vedo ora il post di m.vianello [17:56:17] drdee, rfaulckr: surprisingly ts requests break for that microscopic cohort but work like a charm for the massive "all" cohort, completed a time series in less than a minute [17:57:03] DarTar: è stato notato subito. dubito che questo governo possa o voglia fare qualcosa in merito agli open data [17:57:14] ^^DarTar, thanks. backlog of failed reqs would be helpful [17:57:29] steko: temo anch'io :( [17:57:54] rfaulckr: woah -> https://metrics.wikimedia.org/cohorts/all/threshold?aggregator=proportion&time_series=present&project=enwiki&start=20130501000000&end=20130509000000&slice=24&group=REGISTRATION [17:58:17] rather DarTar: grazie. arretrato di rich falliti sarebbe utile [17:58:26] rofl [18:00:43] rfaulckr: roundtrip translation to English: "DarTar, thanks, a backlog of bankrupt rich would be helpful" [18:00:52] :P DarTar was your "woah" one of despair or elation? [18:00:57] the latter [18:01:13] i love it [18:01:21] I'm going to produce a couple of things, stay tuned [18:01:25] some pretty sexy json [18:01:29] no need to wait for the client [18:01:30] cool [18:01:33] sexy data, rather [18:01:47] sexy data clothed in sexy json [18:02:37] ok watch out for pydata-style comments :p [18:02:48] * DarTar teasing [18:03:06] hah :) [18:13:56] DarTar, what are you running? [18:14:21] ah damn, another one failed [18:14:45] the load is over 100, and the all cores are maxed [18:14:53] interesting [18:15:39] that's one way of putting it :D [18:19:11] pm'ed you [18:22:05] drdee: so it's that a "don't do this again until further notice?" [18:22:44] well, i think this is inherent to the current architecture but it does worry me [18:22:51] particular for frank's demo [18:23:09] so we might want to put in hard limits on cohort size for the demo [18:23:11] yeah, we should find some way of "nice"-ing the threads [18:23:50] reducing the number of concurrent jobs is another option [18:24:00] I'm all for it (and different request limits on a per user/group basis) [18:24:26] this one completed like a charm in a couple of seconds https://metrics.wikimedia.org/cohorts/all/threshold?aggregator=proportion&time_series=present&project=enwiki&start=20130501000000&end=20130509000000&slice=24&group=REGISTRATION [18:24:37] (cached, you can look it up) [18:25:10] have all your jobs finished? [18:28:08] no one shows up any more, I lost a couple of them [18:28:25] see PM [18:29:09] quick fix (speaking of user roles), restrict access to the all keyword to admin-level users [18:29:26] none of the cohorts used by Frank will ever reach this size [18:29:40] we're talking of 4K users/day here [18:30:14] but this is one of the "product" use cases I wanted to talk about [18:33:17] damn, lost another one - drdee [18:55:30] drdee: when you get a chance can you share on the usermetrics list an excerpt from the log with failed jobs? [18:55:56] uhhmmmm not really [18:56:01] the logs are very long [18:56:19] don't know what to look for [18:56:26] grep it for the error above [18:56:43] every line contains 'error' [18:57:01] > [client REMOVED Premature end of script headers: api.wsgi, referer: [18:57:16] there should only be a bunch of those in the last hour [18:57:34] gotta run, bbl -> follow up by mail? [18:57:45] busy with other things, do you have access to the machine? [23:39:29] drdee?