[01:43:57] (CR) Yurik: [C: 2 V: 2] additional filtering for value formating [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153268 (owner: Yurik) [01:44:24] (PS1) Yurik: zero log (weblog) parsing, reorg [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153363 [01:44:35] (CR) Yurik: [C: 2 V: 2] zero log (weblog) parsing, reorg [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153363 (owner: Yurik) [01:54:03] (PS1) Yurik: unidecode workaround [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153364 [01:54:15] (CR) Yurik: [C: 2 V: 2] unidecode workaround [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153364 (owner: Yurik) [02:11:58] (PS1) Yurik: Handle duplicate URLs, api by demand only [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153367 [02:12:27] (CR) Yurik: [C: 2 V: 2] Handle duplicate URLs, api by demand only [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153367 (owner: Yurik) [02:49:55] (PS1) Yurik: error counting [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153371 [02:50:08] (CR) Yurik: [C: 2 V: 2] error counting [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153371 (owner: Yurik) [06:43:28] (PS1) Yurik: more stat parameters, errors, tsv results [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153382 [06:43:46] (CR) Yurik: [C: 2 V: 2] more stat parameters, errors, tsv results [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153382 (owner: Yurik) [07:33:55] Analytics / Wikimetrics: Backing up wikimetrics data fails if data is written while we back it up - https://bugzilla.wikimedia.org/68731 (christian) a:christian [08:52:33] (PS1) Yurik: split subdomain [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153386 [08:52:49] (CR) Yurik: [C: 2 V: 2] split subdomain [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153386 (owner: Yurik) [09:54:17] (CR) Nuria: ">As mentioned inline, chains fulfill the important requirement we have of >being able to configure the number of parallel connections we o" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/150475 (https://bugzilla.wikimedia.org/68840) (owner: Milimetric) [10:51:32] (PS3) Nuria: Removing usage of celery chains from report scheduling [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/150475 (https://bugzilla.wikimedia.org/68840) (owner: Milimetric) [10:57:28] (PS4) Nuria: Removing usage of celery chains from report scheduling [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/150475 (https://bugzilla.wikimedia.org/68840) (owner: Milimetric) [12:16:53] Analytics / General/Unknown: Package libcidr + libanon + libdclass for Ubuntu Trusty - https://bugzilla.wikimedia.org/68997#c4 (Antoine "hashar" Musso) This was detected when bootstrapping a Jenkins slave using Ubuntu Trusty. I have prevented the package from being installed on continuous integration... [12:56:23] qchris, do you have time for 1 question? [12:56:30] Sure. [12:57:25] where is the backup for wikimetrics logging failures? [12:57:56] Into my inbox. [12:58:07] I wanted to have monitoring implemented when we tasked backup. [12:58:12] It was pushed back. [12:58:27] So I set it up myself. [12:58:36] Feel free to take it :-) [12:58:59] I'd prefer to not have them in my Inbox. [13:00:04] better even, can we make monitoring part of the tasks of the next sprint? [13:00:28] Hey. I'd certainly not object :-) [13:00:37] how did you make the scripts report into your e-mail? cause they are root crontabs [13:00:59] Did you look at the crontab? [13:01:03] did you set up directly teh e-mail pipe for teh cron? [13:04:11] Analytics / Wikimetrics: Wikimetrics backup has no monitoring - https://bugzilla.wikimedia.org/69397 (nuria) NEW p:Unprio s:normal a:None Wikimetrics backup has no monitoring Monitoring ad minimum should be piping the cron output to the analytics-internal@ e-mail list. Or, if that is too s... [13:04:18] I cannot find a bug for the monitoring though, i just filed one. [13:04:45] I actually think that if we should do this this sprint [13:06:07] nuria: :-) Please let's not load more into this sprint. [13:06:36] The 55 points are way too much already. [13:09:27] ok, next sprint. When i saw the bug for 'backup not working'ni thought you guys did the monitoring when i was not here but after i though....wait a minute... [13:09:42] 'where is that monitoring going'? [13:09:57] can you tell me how are you piping it into your inbox? [13:10:14] Did you look at the crontab? [13:10:40] Analytics / Wikimetrics: Wikimetrics backup has no monitoring - https://bugzilla.wikimedia.org/69397 (nuria) p:Unprio>Highes [13:12:09] on prod? yes [13:12:41] but the file with crontab -l [13:13:05] The file that displays with crontab -l, that is, [13:13:29] Ok. Let me dpaste the relevant part of it then ... [13:14:01] http://dpaste.com/1JE57Q6 [13:14:18] nuria: That ^ is the relevant part of the "crontab -l" output [13:14:41] -> i am retarded [13:15:02] i was just looking at it [13:15:04] anyways [13:15:05] Hahahaha. No. Crontabs are just overloaded with comments. [13:15:20] It's too ease to not see the wood because of all the trees. [13:15:33] (Or however this proverb goes) [13:15:49] jaja. we probably need to create analytics-monitoring or something right? [13:16:09] Well ... that's something for future sprints. This one is way too busy for my taste. [13:16:46] ok, but on my mind w/o a backup we can rely on 100% we have nothing so i set priority of teh bug to highest [13:21:44] (CR) QChris: "Just nits." (3 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/152050 (owner: QChris) [13:38:47] milimetric, i am going to run "rolling active editors" on eswiki on dev to test my latest changes if that is Ok [13:45:42] (PS1) Yurik: Handle bad status code, proper zcmd parsing [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153402 [13:45:53] (CR) Yurik: [C: 2 V: 2] Handle bad status code, proper zcmd parsing [analytics/zero-sms] - https://gerrit.wikimedia.org/r/153402 (owner: Yurik) [13:54:05] nuria: totally fine, dev / staging / prod are really well separated now thanks to Christian [15:55:06] hello springle, are you arround? [16:08:23] Analytics / General/Unknown: zero.log contains duplicate host in logs - https://bugzilla.wikimedia.org/69371#c1 (nuria) Looking into these. Will look to see if there is anything common to all urls with this string 'orghttp'. We should probably entertain the possibility of thee being actual real requests. [16:10:37] (PS5) Nuria: Removing usage of celery chains from report scheduling [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/150475 (https://bugzilla.wikimedia.org/68840) (owner: Milimetric) [16:10:47] (CR) jenkins-bot: [V: -1] Removing usage of celery chains from report scheduling [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/150475 (https://bugzilla.wikimedia.org/68840) (owner: Milimetric) [16:18:59] (PS1) Milimetric: Add note about mobile unique visitors [analytics/reportcard/data] - https://gerrit.wikimedia.org/r/153414 [16:19:10] (CR) Milimetric: [C: 2 V: 2] Add note about mobile unique visitors [analytics/reportcard/data] - https://gerrit.wikimedia.org/r/153414 (owner: Milimetric) [17:09:38] Analytics / General/Unknown: zero.log contains duplicate host in logs - https://bugzilla.wikimedia.org/69371#c2 (nuria) This problem is also present on sampled logs, mobile and api since records are kept on stats1002. That means since about May 2013. I imagine there must have been another bug fixed on... [17:32:09] Analytics / General/Unknown: zero.log contains duplicate host in logs - https://bugzilla.wikimedia.org/69371#c3 (Kevin Leduc) If you compare the number of occurrences to the number of lines in the file, this seems to happen less than 0.025% of the time. This seems insignificant. Could this just be co... [17:57:24] Analytics / General/Unknown: zero.log contains duplicate host in logs - https://bugzilla.wikimedia.org/69371#c4 (Yuri Astrakhan) Sure, we could ignore it, but i am worried if this could point to some bigger issue. Parsing all zero.tsv* files i noticed a large number of other strange items - highly bro... [17:58:08] Analytics / General/Unknown: zero.log contains duplicate host in logs - https://bugzilla.wikimedia.org/69371#c5 (Yuri Astrakhan) Created attachment 16166 --> https://bugzilla.wikimedia.org/attachment.cgi?id=16166&action=edit Graph of the bad host counts per day as detected in zero logs