[07:13:27] (Abandoned) Hashar: Jenkins job validation (DO NOT SUBMIT) [analytics/kraken] - https://gerrit.wikimedia.org/r/114531 (owner: Hashar) [07:28:29] idenfity mwalker PZ6sraMLrW [07:28:46] ah damnit [13:39:28] http://www.reddit.com/r/wikipedia/comments/1yhwrl/wikimedia_has_proposed_an_amendment_to_the/ [15:18:34] qchris_meeting: https://github.com/wikimedia/operations-puppet/blob/production/templates/udp2log/filters.oxygen.erb#L10 [15:26:14] ottomata: Yes, sure. [15:26:31] ottomata: But that the zero= tag is only set on the mobile varnishes [15:26:42] So it is the same varnishes. [15:27:00] Hence zero \subseteq mobile-sampled-100 [15:27:10] (when ignoring logs from nginx) [15:58:31] task wall is now in shape [15:58:42] I decided to organize rows by person responsible [15:58:49] since we can have both defects and features on there [16:09:13] ottomata: about the kafkatee files ... You said that you have more data. Would it be possible to get a file spanning 2014-02-19T06:29:33 till 2014-02-20T06:29:36? [16:09:26] yes [16:09:39] Can I create that myself, or do you need to do that? [16:09:46] naw i need to get it for you [16:09:53] Ok. [16:09:56] its one big file on analytics1003 [16:10:00] don't think you have an account there [16:10:04] Then. I order 1 such file please :-) [16:10:09] haha, order up! [16:10:17] :-D [16:10:18] Thanks. [16:11:28] huhm, input kafka from topic foo partition 0-10 offset 2014-02-19T06:29:33 ... would be cool [16:11:32] (and possible!) [16:14:09] Snaps: Cool as in "qchris can do this by himself and need not distract otto mata"? [16:14:33] ;) [16:14:50] hahah [16:14:54] uhhh [16:15:17] I take that as a 'no' :-D [16:15:57] qchris, sorry,i could probably get this real fast but am trying to be fancy with sed [16:16:09] i can grab lines between two matches [16:16:23] but i want to get the last occurence of the end match :p [16:16:29] probably doesn't matter in this case [16:16:31] Hahaha. [16:16:33] but i'm trying for fun... [16:16:42] Sure. [16:19:04] oof, doesn't help that I have not rotated this file in a long time and it is 19G [16:19:30] Sed just zoooooooms through that :-) [16:20:48] But no hurry. The Wikipedia Zero scripts will run ~1 hour on the file anyways. [16:23:51] psshhh, i found the line number of the last match :/ [16:25:17] qchris: nah, the point was that kafka allows time-based consuming, so it would be technically cool to implement it for these rare qchris-occassions [16:25:33] kafka allows time-based consuming? [16:25:37] you'd have to know the offset, though, right? [16:26:03] qchris: stat1002:/a/otto/2014-02-19T062933-2014-02-20T062936.mobile-100.log.tsv [16:26:15] Snaps: Ok. I see. Thanks. [16:26:26] ottomata: Thanks. Copying it over. [16:26:59] ottomata: it allows you to query offsets using an absolute timestamp, which will return the offset of the message closest to that timestamp, then you start consuming form there. [16:27:13] https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI [16:28:38] that's awesome! [16:28:45] i did not know [16:28:50] ottomata: Should the last timestamp be 06:29:36 as the name suggests? [16:29:10] It is a bit afterwards ... is it safe to trim myself, or did something go wrong? [16:29:33] aaaafwooo [16:29:36] hm [16:29:44] it is def safe to trim >:( [16:29:50] Ok. Will do. [16:29:54] Thanks. [16:29:56] grrrrrrrr [16:30:00] stupid sed (user) [16:30:22] Damn you sed! [17:02:19] may I recommend s/sed/perl/ ? [17:04:02] ignore me.. I don't know the context :D [18:03:23] I want to bring the "If it didn't happen on the list, it did not happen" issue to the list. What value of "list" should I start with? [18:03:28] Is analytics@lists.wikimedia.org, analytics-internal@lists.wikimedia.org, [18:03:31] wikitech-l@lists.wikimedia.org, #wikimedia-analytics missing any obvious places [18:03:35] that we consider an appropriate medium? [18:04:28] average, csalvia, milimetric, nuria, ottomata: ^ [18:05:11] i vote for analytics@ [18:05:55] either analytics@ or analytics-internal@ count I think [18:05:58] You mean only a single list? [18:06:27] def not #wikimedia-analytics [18:06:36] Ok :-) [18:06:48] I would say analytics-internal@ [18:07:09] but agree with otto, not IRC [18:07:35] What about analytics@ and analytics-internal@ then we have a place for non-public stuff as well? [18:08:05] qchris, seems eminently sensible [18:08:15] +1 [18:08:20] as I might not be on irc every single minute (specially being on a different tz) [18:08:37] sounds good qchris [18:08:49] of course, we're making this decision on IRC. Oh the ironing! [18:08:53] Ok. Thanks. I'll take that then. [18:09:03] Ironholds: :-D [19:05:14] ottomata: you got time to help us troubleshoot ssh issues? [19:05:20] we're kinda blocked by it [19:05:24] and we're in the batcave [19:05:32] HELP! [19:06:33] sure! [19:07:03] 1 min [19:09:21] hey im' there [19:09:36] where'd yall go? [19:09:41] milimetric, csalvia ^ [19:09:50] :( [19:09:53] we got a meeting now [19:10:05] basically charles can't "git review" [19:10:06] to gerrit [19:10:19] because it's telling him permission denied, public key [19:10:28] ottomata ^ [19:12:35] but the diff of his public key and the one on wikitech is teh same [19:13:10] we sure that git review is using the correct key? [19:22:23] milimetric: did he ssh-add ~/.ssh/ ? [19:22:39] i believe so [19:22:43] both to average and ottomata [19:22:54] but we'll dig into that after this meeting [19:22:58] and yell for help again :) [19:25:08] k [20:36:34] (PS1) Csalvia: Test commit [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/114772 [20:36:47] (CR) jenkins-bot: [V: -1] Test commit [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/114772 (owner: Csalvia) [21:20:22] heyaaa somebody is committing! :) [21:25:27] :)