[00:00:02] <James_F>	 It's background funnel instrumentation.
[00:00:17] <James_F>	 Later we may use it for deployments, but right now we just want to know where we are. :-)
[00:01:04] <James_F>	 halfak: https://meta.wikimedia.org/wiki/Schema:Edit
[00:02:15] <halfak>	 Why not simply store the editcount field and pick the cutoffs later?
[00:02:49] <James_F>	 We don't want to have to write insane queries for a relatively trivial piece of data.
[00:02:58] <James_F>	 If we really care, we can just join on userid to extract later.
[00:03:03] <James_F>	 This is just drive-by storage. :-)
[00:03:31] <nuria__>	 James_F: one more thing the schema does not need UA, it's passed in all schemas by default, you can remove it
[00:03:46] <halfak>	 Hmm... Not if you want the editcount at the time of the edit.
[00:04:04] <James_F>	 halfak: Fair, though it could be back-calculated.
[00:04:23] <James_F>	 nuria__: Only for things going through EventLogging rather than Wikimedia Events though?
[00:04:56] <nuria__>	 "Wikimedia events?" sorry i am not sure of what you mean (server side?)
[00:05:08] <nuria__>	 if so UA is capture just the same
[00:05:24] <James_F>	 nuria__: https://www.mediawiki.org/wiki/Extension:WikimediaEvents
[00:05:47] <James_F>	 Which is what VE is plugged into.
[00:06:29] <halfak>	 James_F: I agree that this will work fine.  I suspect that it will frustrate an analyst later though.
[00:06:41] <James_F>	 halfak: Ha. Fair concern.
[00:06:41] <nuria__>	 James_F: if it uses EL there is no way for it not to pass a UA
[00:07:12] <nuria__>	 James_F: anything on teh capsule is passes by default: https://meta.wikimedia.org/wiki/Schema:EventCapsule
[00:07:23] <nuria__>	 so there is no need to duplicate info
[00:07:27] <James_F>	 halfak: However, as the product person in charge, I'm going to let everyone blame me if it turns out to have been a mistake. :-)
[00:08:08] <nuria__>	 James_F: but from my conversation with Roan he was retrofiting VE to use EL directly as far as i understood
[00:11:07] <nuria__>	 James_F: you can of course verify that but I think that is the case
[00:11:41] <James_F>	 nuria__: Yeah, possibly.
[00:12:04] <James_F>	 nuria__: Really what I want is a sanitised {UA product, version} tuple I can query against.
[00:12:14] <James_F>	 nuria__: "WHERE browser=IE and version>10" or whatever.
[00:12:21] <halfak>	 James_F, fair enough.
[00:12:39] <James_F>	 Which this doesn't provide either, I'd note.
[00:13:13] <James_F>	 To do that we'd need a way of changing a UA string into a data structure pair or whatever on write.
[00:13:19] <James_F>	 Maybe I should file a bug. :-)
[00:13:37] <nuria__>	 James_F: you can do that with ua parser
[00:13:47] <nuria__>	 easily on the data you gather
[00:13:53] <James_F>	 nuria__: Can we get ua parser to run before the row hits the DB?
[00:14:07] <James_F>	 nuria__: After-the-fact processing sucks.
[00:14:53] <nuria__>	 James_F: we already have a backlog item for that
[00:15:21] <James_F>	 nuria__: Ah, cool.
[00:16:14] <nuria__>	 James_F: we are not getting to it this quarter though, this quarter you shall be able to do it in hadoop, the CR for that is pending
[00:16:26] <James_F>	 Oh well.
[00:16:30] <James_F>	 Anyway, must run.
[00:16:36] <James_F>	 Thanks for the help and tips.
[00:16:42] <James_F>	 halfak: Will sling you an e-mail.
[00:16:55] <halfak>	 Great.  Thanks!
[00:16:57] <nuria__>	 James_F: please update your schema to remove UA
[00:17:08] <nuria__>	 James_F: thank you
[00:17:28] <James_F>	 nuria__: Done.
[00:17:56] <nuria__>	 James_F: and anything else that might be in the capsule
[00:18:00] <nuria__>	 please
[09:37:12] <wikibugs>	 Analytics / Refinery: Raw webrequest bits partition for 2014-10-23T14/1H not marked successful - https://bugzilla.wikimedia.org/72473 (christian) NEW p:Unprio s:normal a:None The bits webrequest partition [1] for 2014-10-23T14/1H has not been marked successful.  What happened?   [1] _______...
[09:37:25] <wikibugs>	 Analytics / Refinery: Raw webrequest bits partition for 2014-10-23T14/1H not marked successful - https://bugzilla.wikimedia.org/72473#c1 (christian) NEW>RESO/FIX Commit 832ef481631f29216aee8f2f77f22727418409b4 got merged, which updated the varnishkafka configuration for the bits caches. This caus...
[09:37:26] <wikibugs>	 Analytics / Refinery: Raw webrequest partitions that were not marked successful due to configuration updates - https://bugzilla.wikimedia.org/72300 (christian)
[09:42:29] <qchris>	 !log Marked raw bits webrequest partition for 2014-10-23T14/1H ok (See {{bug|72473}})
[11:10:57] <grrrit-wm1>	 (CR) Yuvipanda: [C: 2 V: 2] Fix XSS vulnerability [analytics/quarry/web] - https://gerrit.wikimedia.org/r/168278 (owner: Yuvipanda)
[13:01:40] <grrrit-wm1>	 (PS14) Mforns: Add ability to global query a user's wikis [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 (owner: Terrrydactyl)
[14:01:45] <grrrit-wm1>	 (CR) Ottomata: [C: 1] Add UAParserUDF from kraken [analytics/refinery/source] - https://gerrit.wikimedia.org/r/166142 (owner: Ottomata)
[14:43:07] <milimetric>	 ottomata: I tried to puppetize my desktop and I gave up after a couple of days of pain
[14:43:44] <milimetric>	 I honestly would love to do a competition - me vs. puppet, 100 machines, replicate a configuration.
[14:43:49] <milimetric>	 I've got good money I'd win
[14:46:43] <ottomata>	 hah, you actually tried?
[14:46:55] <ottomata>	 ha
[14:47:24] <ottomata>	 i doubt it, not with an already set up puppetization i bet
[14:52:16] <milimetric>	 As long as I have a similar amount of time to work on my manual scripts as someone had to work on the puppetization, my claim stands
[14:52:34] <milimetric>	 ok, so - more relevant - ./setup.sh still hangs
[14:52:38] <milimetric>	 it's been going for 10 minutes
[14:53:21] <milimetric>	 vagrant 1.6.5
[14:53:28] <milimetric>	 virtualbox 1.4.3 18
[14:53:45] <milimetric>	 the ruby that comes with that vagrant is 2.0
[14:54:20] <mforns>	 milimetric, nuria__: when I modify puppet/modules/wikimetrics subproject, must I commit the submodule within puppet repo also?
[14:59:32] <milimetric>	 you don't have to submit that to gerrit until your main puppet change gets merged
[14:59:46] <milimetric>	 because the submodule pointer might change if you need to redo a patch
[15:00:03] <milimetric>	 mforns: ^
[15:00:47] <mforns>	 ok
[15:08:08] <milimetric>	 i'll be back after lunch - gotta run an errand
[15:20:06] <ottomata>	 halfak: hiya
[15:20:15] <halfak>	 o/ ottomata
[15:20:19] <halfak>	 (in meeting)
[15:20:21] <ottomata>	 have you looked at any of the existing java xmldump parsers out there?
[15:20:23] <ottomata>	 there are ton...
[15:23:23] <grrrit-wm1>	 (CR) Nuria: Add ability to global query a user's wikis (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 (owner: Terrrydactyl)
[16:03:07] <nuria__>	 ottomata: a puppet question ....
[16:03:14] <nuria__>	 if you may
[16:03:58] <ottomata>	 yes
[16:04:00] <ottomata>	 wasssuup
[16:04:45] <nuria__>	 ottomata: https://gerrit.wikimedia.org/r/#/c/167821/3/manifests/init.pp
[16:05:07] <nuria__>	 the "defaults" for db parameters are wrong (per our conversation this morning)
[16:05:10] <ottomata>	 aye, yeah, hard to choose a default there
[16:05:15] <ottomata>	 you could make them required if you like
[16:05:17] <nuria__>	 teh right ones are passed by teh role
[16:05:25] <ottomata>	 i.e. not specify defaults at all
[16:05:34] <nuria__>	 But, as far as we can see, if we remove the defaults it no work no more
[16:05:46] <ottomata>	 if you remove the defaults
[16:05:54] <ottomata>	 then puppet will error if you use the class without passing them manually
[16:06:09] <ottomata>	 but, afaik, these values are passed in manually in both vagrant and production roles
[16:06:10] <ottomata>	 right?
[16:06:23] <ottomata>	 (otherwise the mediawiki ones wouldn't work anyway)
[16:06:48] <nuria__>	 yes, but marcel just removed those from init.pp and got an error , lemme  try to repro
[16:06:56] <ottomata>	 did he add them in the vagrant roll?
[16:06:57] <ottomata>	 role*
[16:10:38] <nuria__>	 but the mediawiki ones are already on the vagrant role ottomata, right?
[16:11:32] <nuria__>	 ottomata: this is is already passed: " db_user_mediawiki     => $::mediawiki::db_user,"
[16:11:32] <ottomata>	 yes, afaik
[16:11:35] <ottomata>	 right
[16:11:46] <nuria__>	 but if I remove the (wrong) default
[16:11:47] <ottomata>	 so you should be able to safely remove defaults, and make those required parameters
[16:11:59] <ottomata>	 right now the default is being overridden by the role
[16:12:19] <nuria__>	 ottomata: ah .. how do you make them required?
[16:12:40] <ottomata>	 by removing the defaults
[16:12:56] <ottomata>	 $db_user_mediawiki,
[16:12:56] <ottomata>	 $db_name_mediawiki,
[16:12:56] <ottomata>	 ...
[16:13:00] <ottomata>	 instead of
[16:13:11] <ottomata>	 $db_user_mediawiki = 'default value',
[16:13:11] <ottomata>	 $db_name_mediawiki = 'default other value',
[16:13:11] <ottomata>	 ...
[16:15:52] <nuria__>	 ottomata: ah only naming th variables
[16:15:58] <nuria__>	 k
[16:17:00] <ottomata>	 yup
[16:20:24] <nuria__>	 ottomata: thank you, we will do just that otherwise it is confusing as it implies that defaults work when they do not
[16:20:42] <ottomata>	 sure
[16:20:46] <ottomata>	 makes sense to me! ;)
[16:20:48] <ottomata>	 :)
[17:34:48] <milimetric>	 kevinator: i just scheduled a meeting to deploy wikimetrics to production every Thursday
[17:35:06] <kevinator>	 cool!
[17:35:08] <milimetric>	 any other deployments will be to staging and won't impact production, unless you ask for an interim deployment
[17:35:18] <kevinator>	 ok
[17:35:27] <milimetric>	 i should've done this a long time ago, sorry :)  it was always implicit and bound to confuse everyone
[17:35:56] <kevinator>	 is the meeting shared? I don’t see it
[17:36:29] <kevinator>	 ie can you put it in the analytics calendar?
[17:37:16] <kevinator>	 ah! reload and it’s there!
[17:37:58] <kevinator>	 milimetric: thanks!
[18:10:05] <milimetric>	 my wifi card spontaneously decided to not work for 40 minutes :/
[18:20:14] <grrrit-wm1>	 (CR) Milimetric: "Looks like this change breaks 4 tests:" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/166157 (https://bugzilla.wikimedia.org/70714) (owner: Bmansurov)
[18:31:14] <mforns>	 nuria__: milimetric is reviewing the code also
[18:32:09] <nuria__>	 mforns: good, once ottomata reviews puppet changes all code should have been reviewed
[18:32:19] <mforns>	 ok
[18:33:57] <nuria__>	 ottomata: can we add json simple to archiva?
[18:35:37] <nuria__>	 ottomata: wait, maybe it's alredy there
[18:35:42] <nuria__>	 *already
[18:36:14] <ottomata>	 nuria__: i've got it in proxoy mode right now, because i'm working with it a lot
[18:36:21] <ottomata>	 if you try to install something it will likely just cache it
[18:36:47] <nuria__>	 ottomata: meaning i just need to edit the pom and we are good to go, right?
[18:37:21] <ottomata>	 yes
[18:37:28] <nuria__>	 ottomata: k
[18:37:32] <ottomata>	 for now, it won't always be like that, but it will likely work right now
[18:58:22] <kevinator>	 Hey, Ironholds
[18:58:45] <kevinator>	 I’d like to merge my user stories for page views into your document
[18:58:47] <kevinator>	 https://www.mediawiki.org/wiki/Analytics/Pageviews/Stories
[18:59:07] <kevinator>	 into https://meta.wikimedia.org/wiki/Research:Page_view
[18:59:25] <kevinator>	 That way the discussion on the talk pages can occur in one place
[19:00:00] <Ironholds>	 kevinator, so far, your definitions confuse me ;p
[19:00:18] <Ironholds>	 but sure, change mine to "Oliver's user stories", add yours with "Kevin's user stories"
[19:00:41] <kevinator>	 ok
[19:02:15] <Ironholds>	 and ping me when you have and I'll comment?
[19:02:38] <kevinator>	 sure… I was expexting to spend some time on that talk page today.
[19:09:29] <ottomata>	 nuria__: mforns, 1 more quick comment about defaults
[19:09:41] <ottomata>	 the defaults for name and host were fine i think, but if you really don't want them, that's fine too
[19:09:43] <nuria__>	 ottomata: yes.....
[19:09:44] <mforns>	 ok
[19:10:23] <nuria__>	 ottomata: I think it makes sense to pass "all together" as grants depend on user and host
[19:10:31] <ottomata>	 username and password, yes
[19:10:47] <ottomata>	 but db_name and db_host  could have sane defaults, i thikn...
[19:10:52] <ottomata>	 but, if you thikn not, then I don't mind
[19:10:52] <nuria__>	 ottomata: so there is no way for us to set a host default really
[19:11:00] <ottomata>	 in vagrant it will always be localhost
[19:11:02] <ottomata>	 that is sane
[19:11:20] <nuria__>	 ottomata: if it makes more sense we can add those back
[19:11:34] <nuria__>	 ottomata: your call (really not  a problem to add them)
[19:11:37] <ottomata>	 whatever you think is best there is fine, i'm not picky either way
[19:11:37] <mforns>	 aha
[19:11:37] <ottomata>	 haha
[19:11:38] <ottomata>	 my call!
[19:11:52] <ottomata>	 i would leave them, but i don't see a strong argument to do so
[19:11:55] <ottomata>	 so if you like them out that's fine
[19:12:06] <nuria__>	 ottomata: k then
[19:12:09] <mforns>	 I put them back
[19:12:24] <nuria__>	 ottomata: btw, archiva is working great
[19:12:28] <ottomata>	 awesome
[19:29:24] <grrrit-wm1>	 (CR) Milimetric: "I added some comments, I have to look closer at the way the form works now, I'll comment on that shortly." (5 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 (owner: Terrrydactyl)
[19:39:14] <ottomata>	 mforns: my mind is going bonkers with java stuff, and i worked on this late last night, so i'm going to peace out pretty soon
[19:39:25] <ottomata>	 Can I merge some stuff for ya real quick
[19:39:26] <ottomata>	 ?
[19:40:06] <grrrit-wm1>	 (CR) Milimetric: "ok, found the problem with the controller" (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858 (owner: Terrrydactyl)
[19:40:28] <mforns>	 ottomata: yep
[19:40:50] <mforns>	 just a sec, I'm pushing it now
[19:40:59] <mforns>	 I'm in the commit message
[19:45:08] <mforns>	 ottomata: I pushed it
[19:47:10] <ottomata>	 cool, the defaults shodul be documented too, like they were before
[19:47:14] <ottomata>	 if you are going to leave them in
[19:47:28] <ottomata>	 the doc still say s'. Just require.'
[19:47:32] <ottomata>	 mforns:
[19:47:55] <mforns>	 opa
[19:51:07] <mforns>	 ottomata: ok, it's there again
[19:53:54] <ottomata>	 ok, mforns do you know how to push the new submodule change to vagrant?
[19:54:14] <ottomata>	 that should go along as part of this commit https://gerrit.wikimedia.org/r/#/c/168600/
[19:54:21] <mforns>	 ok
[19:55:03] <mforns>	 should I just git-add the folder? and commit normally? and then push?
[19:56:25] <ottomata>	 ja
[19:56:26] <ottomata>	 thikn that should do it
[19:56:36] <mforns>	 ok
[19:59:12] <mforns>	 i pushed the submodule change also
[19:59:24] <mforns>	 ottomata: ^
[20:01:04] <ottomata>	 hmm
[20:01:09] <ottomata>	 mforns, not sure if that is the right sha
[20:01:28] <ottomata>	 patch set 8 was
[20:01:30] <ottomata>	 5f416bbce4188914f48a044a8df09f1c34b07183
[20:01:39] <ottomata>	 you set it at
[20:01:39] <ottomata>	 97cb1d45753b2c323976b757959d5a20cadd2823
[20:02:40] <ottomata>	 ah yeah
[20:02:42] <ottomata>	 that is patch set 7
[20:03:15] <mforns>	 my last commit at wikimetrics module is 97cb1d45753b2c323976b757959d5a20cadd2823
[20:03:15] <ottomata>	 cd puppet/modules/wikimetrics; git checkout master; git pull; cd ../../../; git add puppet/modules/wikimetrics
[20:03:22] <ottomata>	 ah
[20:03:23] <ottomata>	 it was rebased
[20:03:24] <ottomata>	 that's why
[20:03:29] <ottomata>	 before I merged i had to rebase it
[20:03:34] <mforns>	 ah!
[20:07:00] <mforns>	 ottomata: try now, I pushed the new link
[20:08:59] <ottomata>	 looks good
[20:09:20] <ottomata>	 merged.
[20:09:21] <ottomata>	 cool
[20:09:23] <ottomata>	 ok, i'm gonna run
[20:09:27] <ottomata>	 laaters
[20:09:48] <mforns>	 bye, thanks!!!
[20:21:09] <mforns>	 nuria__: I forgot how to link a changeset as dependenccy of another, can you refresh me, please?
[20:23:23] <grrrit-wm1>	 (PS2) Milimetric: [WIP] Add schema for edit fact table [analytics/data-warehouse] - https://gerrit.wikimedia.org/r/167839 (owner: QChris)
[20:23:46] <grrrit-wm1>	 (CR) Milimetric: [WIP] Add schema for edit fact table (2 comments) [analytics/data-warehouse] - https://gerrit.wikimedia.org/r/167839 (owner: QChris)
[20:24:39] <nuria__>	 mforns; for changes in different depots (our case) there is no way
[20:24:50] <milimetric>	 hey - I might be CRAZY but I love the gerrit "New Screen" setting.  Give it a shot
[20:25:02] <qchris>	 Nooooooooo! :-D
[20:25:07] <qchris>	 Hahahaha.
[20:25:07] <milimetric>	 nuria__: qchris has a way to do that
[20:25:08] <nuria__>	 mforns: unless you copy the gerrit url in the commit message, as to provide info
[20:25:29] <nuria__>	 ahhhhhh
[20:25:31] <mforns>	 ok
[20:25:50] <milimetric>	 qchris: I assume the Noooo was to the "new Screen"?
[20:25:56] <qchris>	 milimetric: Yes.
[20:26:00] <milimetric>	 why all the hate?  It makes a lot of things so much easier :)
[20:26:08] <nuria__>	 where is the new Screen?
[20:26:13] <qchris>	 I do not know how to link changes between arbitrary repositories.
[20:26:23] <milimetric>	 in that Preferences thing i was talking about yesterday
[20:26:27] <milimetric>	 (nuria__ ^)
[20:26:30] <nuria__>	 ahhh
[20:26:42] <qchris>	 The biggest issue with the new change screen is that the dependencies table is gone, and also the reviewer table.
[20:26:54] <milimetric>	 qchris / mforns / nuria__ : my apologies then, just pasting the URL to the repo is fine (I thought I got yelled at for doing that)
[20:27:13] <milimetric>	 qchris: but those are just tucked away, not gone
[20:27:34] <qchris>	 If you want to reference an unmerged change from another change's commit message, you can just use the Change-Id.
[20:27:34] <mforns>	 xD
[20:28:02] <qchris>	 If you want to reference a merged change from another change's commit message, use the commit's SHA-1
[20:28:14] <mforns>	 fine
[20:28:18] <mforns>	 thanks
[20:29:14] <qchris>	 milimetric: If you have changes with more reviewers, I miss the table a lot. The table is easier for me to parse.
[20:29:32] <qchris>	 But with the gerrit wmf runs ... it is a choice :-)
[20:29:41] <milimetric>	 oh, like more than 5-6, yeah, i can see that
[20:30:00] <milimetric>	 like, 50 people would look nasty in that single line there :)
[20:30:32] <qchris>	 But use it. The basic functionality is in the new change screen ... even for the version wmf runs.
[20:30:54] <milimetric>	 yeah, it's pretty cool, no scrolling on my tiny tiny screen
[20:31:03] <milimetric>	 12.5" 4evah! :P
[20:44:53] <grrrit-wm1>	 (PS4) Bmansurov: Add a timerange validator [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/166157 (https://bugzilla.wikimedia.org/70714)
[20:52:43] <Ironholds>	 kevinator, lemme know when it's in?
[20:52:53] <kevinator>	 It’s in
[20:53:12] <kevinator>	 guess you didn’t see my message earlier
[20:53:43] <kevinator>	 or maybe I never sent it
[20:54:03] <kevinator>	 d’oh, never sent it
[21:02:49] <grrrit-wm1>	 (CR) Milimetric: [C: 2] Add a timerange validator [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/166157 (https://bugzilla.wikimedia.org/70714) (owner: Bmansurov)
[21:02:56] <grrrit-wm1>	 (Merged) jenkins-bot: Add a timerange validator [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/166157 (https://bugzilla.wikimedia.org/70714) (owner: Bmansurov)
[21:04:23] <wikibugs>	 Analytics / Wikimetrics: time ranges should be validated to be start < end - https://bugzilla.wikimedia.org/70714#c3 (Dan Andreescu) PATC>RESO/FIX Thanks Baha, fix looks great, it will be deployed Thursday October 30.
[21:04:49] <milimetric>	 kevinator: fyi, Baha fixed the "start date has to be <= end date" bug, see above for details
[21:05:35] <kevinator>	 oooh cool!
[21:05:52] <kevinator>	 is Baha on this chanel?
[21:06:08] <bmansurov>	 hello kevinator, it's me ;)
[21:06:34] <kevinator>	 Hi Baha!  Thanks for helping out on wikimetrics
[21:06:55] <bmansurov>	 kevinator: np, I like wikimetrics
[21:07:39] <Ironholds>	 kevinator, I don't see it on the page
[21:07:46] <Ironholds>	 oh, wait
[21:07:47] <Ironholds>	 there we are
[21:08:03] <kevinator>	 https://meta.wikimedia.org/wiki/Research:Page_view#Kevin.27s_User_Stories
[21:09:56] <Ironholds>	 commentary left
[21:10:24] <Ironholds>	 would suggest sending an email internally to EZ and Christian too.
[21:10:38] * qchris reads backscroll
[21:10:50] <qchris>	 Oh the user stories are in?
[21:11:52] <qchris>	 Mhmm. That'll have until next week...
[21:12:01] <qchris>	 s/have/have to wait/
[21:14:23] <grrrit-wm1>	 (PS9) Nuria: Add UAParserUDF from kraken [analytics/refinery/source] - https://gerrit.wikimedia.org/r/166142 (owner: Ottomata)
[21:16:38] <mforns>	 have a nice weekend folks
[21:16:51] <mforns>	 :]
[21:17:17] <nuria__>	 mforns: ciao
[23:24:27] <grrrit-wm1>	 (PS10) Nuria: Add UAParserUDF from kraken [analytics/refinery/source] - https://gerrit.wikimedia.org/r/166142 (owner: Ottomata)