[07:40:56] (PS1) Adamw: WIP Create a cohort from campaign membership [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/126927 [07:41:03] (CR) jenkins-bot: [V: -1] WIP Create a cohort from campaign membership [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/126927 (owner: Adamw) [10:38:39] [travis-ci] master/c19d99b (#173 by Dan Andreescu): The build passed. http://travis-ci.org/wikimedia/limn/builds/23194706 [10:51:02] [travis-ci] develop/c19d99b (#174 by Dan Andreescu): The build passed. http://travis-ci.org/wikimedia/limn/builds/23195472 [12:52:19] milimetric: lmk if you're around :) [13:38:19] hey YuviPanda, what's up [13:41:45] gi11es: all limn instances seem to be affected by some weird network / labs instance problem [13:41:58] it seems that some of the files are chopped off (namely the limn.no-deps.min) [13:42:13] they look fine in source, but not when sent over the network [13:42:26] it's not due to what you deployed because an instance I haven't upgraded yet has the same problem [13:52:31] milimetric: the EL bug :) [13:55:07] YuviPanda: it's been prioritized as high and unless we have a million other "high" bugs, we'll start working on it today [13:55:30] but the person to bug is Kevin, as I do not make these decisions [13:55:42] kevinator is what he is known by in these parts [13:55:45] milimetric: sweet! [13:55:48] milimetric: ok. [13:56:10] but he be slumbering for he hails from the West [14:05:36] milimetric: :) I'll be off now and be back later, but it has repro steps in the bug (curl) [14:12:16] Yuvipanda [14:12:41] your event does not have a user agent as far as i can see [14:14:14] it comes from the apps right? [14:21:56] nuria: yeah. [14:22:14] so .. who is adding the capsule? [14:22:16] https://meta.wikimedia.org/wiki/Schema:EventCapsule [14:22:48] nuria: wait, isn't UA supposed to be added by the server? [14:23:19] nuria: that was my assumption, maybe that is wrong... [14:23:23] In js events (initiated by web-client) [14:23:30] it is added in vanadium [14:23:42] in server side events it is added via mediawikicode [14:23:54] but yours is neither [14:24:14] Makes sense? [14:24:29] nuria: right but this should follow the same pipeline js events follow, no? Since I'm hitting event.gif the same way JS events do, and so it should be added in vanadium too [14:25:28] nuria: IIRC JS EL also logs events by hitting the same URL [14:25:34] In teh case of UA if it i spresent in your request it will be grabbed from request headers [14:25:58] *if "user-agent" header is present [14:26:03] yes, U-A is present in request [14:26:13] when I grepped varnishcsaa logs I found the UA [14:26:18] but what about the rest of the stuff on the capsule [14:26:52] nuria: it had those too! let me copy paste a line from that [14:27:05] '?%7B%22schema%22%3A%22MobileWikiAppEdit%22%2C%22revision%22%3A8188113%2C%22event%22%3A%7B%22action%22%3A%22start%22%2C%22userName%22%3A%22yuvitest44%22%2C%22editSessionToken%22%3A%22544d99f3-4966-4304-90fa-3251560543ae%22%7D%7D cp1057.eqiad.wmnet 11519559 2014-04-16T20:44:49 208.80.154.76 curl/7.30.0' [14:27:20] nuria: so that's TS, IP, udp seq number, UA [14:27:27] nuria: so that's all taken care of by some EL process [14:27:40] (that's my IP and I'm ok with it being out in t he public) [14:28:02] it might be my dislexia but does your request have a wiki project? [14:28:10] like 'enwiki' [14:28:17] ah, hmm. no it does not. [14:28:23] I wonder if that is it [14:28:23] you are missing several [14:28:34] There is couple more missing i think [14:28:42] but you are right for UA [14:28:53] let me add wiki and test [14:28:54] if you hit end js endpoint it will be added [14:29:04] there is more than 1 field missing [14:29:33] wait ... what was the other one.... [14:29:33] what other field is missing? I'm looking at https://meta.wikimedia.org/wiki/Schema:EventCapsule [14:32:19] nuria: boom! it was just missing wiki! [14:32:21] nuria: works now [14:32:35] ok, i was looking looking but could not see other one missing [14:32:37] great [14:33:05] and UA is being added as teh one we agreed on from the app, right? [14:33:11] the "made up" one [14:33:11] nuria: yeah! [14:33:16] yup! [14:33:23] the Android change was merged yesterday [14:33:25] excellent, one less thing to worry about [14:33:34] :D [14:33:46] nuria: Tasking :-D [14:33:57] ah yes sorry ciao yuvipanda [14:34:06] nuria: ty! cia! [14:36:28] yuvipanda Would you close the bug? [14:36:55] nuria: already did! [14:36:59] :) [14:37:01] am off for food [14:37:02] brb [14:37:03] ty nuria! [15:27:08] qchris_meeting: ah meeting! I have another idea for deployment repo layout [15:47:45] milimetric: thanks for the heads up [16:31:11] ottomata: What's the new idea for the deployment repo layout? [16:31:46] what if: [16:31:58] we just had a deployment branch in kraken [16:32:02] that tracked master [16:32:11] or, no, doesn't track master [16:32:20] :-) [16:32:21] but, has an artifacts/ directory? [16:32:46] one repo, no submodules [16:32:58] And in that artifacts directory are the jars from the current commit? [16:33:28] yeah you'd have to commit to add the jars, but it would be acommit on the deployment branch [16:33:30] so yeah, i guess [16:33:34] code in master [16:33:36] when ready to deploy [16:33:41] checkout deploy [16:33:43] merge master [16:33:45] mvn package [16:33:51] git add jar files [16:33:55] git commit [16:33:58] ? [16:34:06] Does that mean having the huuuuge jars in plain git? [16:34:10] nono [16:34:15] git add == git fat add [16:34:15] Forcing gitfat? [16:34:17] with filter *(.jar [16:34:17] yea [16:34:22] :-( [16:34:23] that's in the .gitfat settings [16:34:24] or [16:34:27] artifacts/*.jar [16:34:34] the same way we have it in kraken/deploy now [16:34:52] I don't know. It sounds a bit twisted to me, to be honest. [16:35:05] Is this just to work around trebuchet issues, or [16:35:14] are you convinced it's the right approach? [16:36:10] no, its not to workaround trebuchet [16:36:13] i'm still going to fix that either way [16:36:17] Ok. [16:36:18] because that needs to work for other things too [16:36:30] but, its just annoying to have to do submodule commits if we don't have to [16:36:32] You'd be using the branches like two separate repositories. In the master branch, [16:36:40] you'd only be interested in the plain code. [16:36:45] And in the deployment branch, [16:36:56] you'd only be interested in the artifacts. [16:37:27] or tags even, you could use the deployment branch to make a tag [16:37:39] tag with git-fat jars [16:37:51] qchris, not just artifacts [16:37:57] artifacts + code that built artifacts [16:37:57] That would somewhat force us to add jars for every commit. wouldn't it? [16:38:15] hm, no? only when we want to make a deploy release? [16:38:28] or yeah, maybe that's annoying, dunno, tags or not [16:38:29] either way [16:38:30] same process [16:38:46] Let me think it through again. [16:38:54] the deploy branch commit would still tie the .jars together with the code that built them at a specific commit, right? [16:39:11] Only if you force to merge on release. [16:39:19] Which we should probably do. [16:39:22] right, it would be possible for someone to get around that [16:39:25] sure [16:39:31] but it is with the submodule thing too [16:39:36] Yes. [16:39:53] To me, branches would not [16:40:01] make the process simpler, but just [16:40:11] trade one sequence of commands to another. But in [16:40:26] addition to that, they would force two repos into one. [16:40:49] I think I like it better to have them separated and stick with submodules. [16:41:07] It makes analytics/kraken "pure code" and small. [16:41:09] I like that. [16:41:56] But meh. If you prefer branches. Let's do branches. [16:42:11] We can switch if either way once it becomes a problem. [16:42:19] ha, no i want your resistance! I am not totally sure [16:42:31] it just seems more inelegant with submodules [16:42:33] * qchris goes into resistance mode [16:42:37] NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO! [16:42:42] Only over my dead body! [16:42:48] haha [16:42:52] /srv/deployment/analytics/kraken/deploy/kraken/kraken-etl/camus [16:42:53] Really ... more inelegant? [16:42:58] that is the path to the camus script now [16:43:03] Ok. [16:43:24] wait ... "analytics/kraken/deploy/kraken" [16:43:28] yup [16:43:32] That's twice kraken. [16:43:39] the repo's name is [16:43:44] analytics/kraken/deploy [16:43:47] and it has a submodule [16:43:49] called kraken [16:44:13] Ok. [16:44:17] inelegant, see! [16:44:18] :p [16:44:23] So it takes the full project name. [16:44:24] one is the loneliest number that you'll ever do [16:44:27] i could make it clone at a different path [16:44:32] two can be as bad as one, it's the loneliest number since the number one [16:44:33] but then it wouldn't be clear [16:44:40] ha [16:44:48] i could make it clone at analytics/kraken [16:44:55] but that would match the existing repo name [16:45:00] and [16:45:06] analytics/kraken/deploy != analytics/kraken [16:45:30] right. [16:46:04] So branches could kill "deploy/kraken/" of the filename. [16:46:13] /srv/deployment/analytics/krakenkraken-etl/camus [16:46:20] That's still a crappy filename :-) [16:46:26] whoops ... /srv/deployment/analytics/kraken/kraken-etl/camus [16:47:18] yeah, it would really jsut remove deploy [16:47:23] i could probably get away with removing the analytics/ bit [16:47:32] that is fairly obvious, and I think I can do that in deployment.pp settings [16:48:06] Sooooo ... thinking about it again ... do humans have to use that path regularly, or is it mostly for machines anyways? [16:49:20] i use it while testing stuff [16:49:22] for sure [16:49:26] Hhmmm ... on the analytics machines we could even link it to a more convenient name. Because there we anyways know thaht we're dealing with analytics. [16:49:32] and sure [16:49:35] kraken-toolbelt [16:49:36] right? [16:49:39] :-) [16:49:41] yeah that's true [16:49:54] could make /srv/kraken symlink? [16:50:00] Yes. [16:50:12] to analytics/kraken/deploy/kraken? or to analytics/kraken/deploy? [16:50:20] /srv/kraken/kraken/kraken-etl [16:50:21] ung [16:50:47] maybe have a [16:51:04] Dunno. [16:51:09] /src/kraken-src [16:51:11] /srv/kraken-artifacts [16:51:18] To have both of them. [16:52:14] /srv/kraken/src -> analytics/kraken/deploy/kraken [16:52:14] /srv/kraken/artifacts -> analytics/kraken/deploy/artifacts [16:52:14] ? [16:52:24] might be nice actually [16:52:33] could even do /srv/kraken/bin eventually too, and add that to our paths [16:52:42] that way kraken-toolbelt stuff would just be available [16:52:50] Just braindumping here ... yes. [16:52:56] hmmmm [16:54:08] I think I like the idea of submodules, and having long descriptive paths, and symlinking them to nice names on analytics machines. [16:55:22] Oh ... next meeting in 5 and gotta get something to eat. [16:55:38] I'll be back after the sprint planning meeting. [16:55:44] ok [16:55:53] lunchtime for me soon anyway [16:55:54] thanks [17:00:02] gi11es: labs folks found a problem with the /var directory on the proxy and fixing it fixed our limn instances [18:30:18] !log switching erbium udp2log instance from consuming multicast relay to unicast direct from varnishes [19:31:07] hey Ironholds. sorry just saw your messages :( [19:31:14] Ironholds: I can't really read iOS code either... [19:31:19] Ironholds: but Adam should have it sorted [19:31:32] and as for the app, I am seeing ?%7B%22wiki%22%3A%22testwiki%22%2C%22schema%22%3A%22MobileWikiAppEdit%22%2C%22revision%22%3A8197809%2C%22event%22%3A%7B%22action%22%3A%22start%22%2C%22userName%22%3A%22yuvipanda%22%2C%22editSessionToken%22%3A%2266ca3699-0a4b-4ba8-a4c4-8a4a16eaa429%22%7D%7D cp1057.eqiad.wmnet 11617972 2014-04-17T19:30:42 208.80.154.9 WikipediaApp/2.0-alpha-2014-04-15 (Android 4.4.2; Phone) [19:31:41] in varnish, so I guess it's properly sorted out [19:31:46] qchris_away: ^ note, no v :) [20:24:23] zz_yuvipanda: \o/ no "v" :-) [20:24:36] ottomata: Back [20:24:50] Oh ... no ottomata around. [21:11:18] (PS1) Milimetric: Refactor metrics to take a project, not a session [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/127157 [21:15:02] (PS2) Milimetric: Refactor metrics to take a project, not a session [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/127157 [21:17:56] (PS1) Erik Zachte: new search function for (also extended) portal [analytics/wikistats] - https://gerrit.wikimedia.org/r/127159 [21:45:15] (PS1) Erik Zachte: monthly dump reports also in search results [analytics/wikistats] - https://gerrit.wikimedia.org/r/127164