[11:57:32] nuria: hello [11:57:35] hola...kind of late eh? [11:57:39] I was about to send you another CR as I realized [11:57:46] i did not send you last batch of changes but teh prior one. Sorry [11:57:53] will do now [11:57:57] it's ok, gerrit is horrible [11:58:04] open a JS console on a wiki page and type: $.client.profile() [11:58:23] any wikimedia page? [11:58:29] i wasn't aware of it until a day or two ago, it might tilt the scale in favor of logging the UA in JS as opposed to getting it from the header [11:58:40] any article, yeah [11:59:45] i rather do it from the header so i know it works 100% same no matter teh browser [11:59:58] yeah, probably true [12:00:10] if we used it we'd also have to have an equivalent implementation on the server anyway for server-side events [12:00:18] so it's not really saving us any work [12:01:19] right [12:01:29] doing it server side is 100% consistent [12:01:53] even if people start using opera mini all arround you know [12:02:35] or amazon silk in compress mode, many of mobile browsers will have js issues [12:03:32] yep [12:03:35] i would not update the schema id in python yet, i think [12:08:01] i would simply add "event.pop('userAgent', None)" to eventlogging-processor [12:08:01] right before the call to validate(event) [12:08:24] (the second param to dict.pop makes it not fail if the key is not present) [12:09:12] this lets you deploy the PHP change and ensure the field is present in the raw stream [12:09:20] amusingly, i don't think you need to do anything to protect against the client-side change [12:09:20] if you look at parser.py, it uses re.match to parse each log line [12:09:43] re.match succeeds if the pattern matches the start of the string; it doesn't care about any garbage afterwards [12:11:55] but since it is not required it should validate regardless right? (i just added unit tests to test that on php side, was going to add them to python side too) [12:12:16] >>> re.match('a b c', 'a b c 1 2 3') is not None [12:12:16] True [12:12:39] it will validate, sure, but what happens at the database level? [12:12:45] we create columns for optional fields [12:12:49] ah ok [12:13:20] ok, will remove the new schema from python class [12:14:30] will send you a cr with tests (and no spaces) [12:14:46] re.match makes it handy [12:14:46] yeah, it's arguably a bug, i should have constructed the regex to end with a '$' probably [12:14:46] to match end-of-line [12:14:46] but it's a very convenient bug [12:15:07] i think you understood my point about the database, but just to make it explicit: [12:15:28] each incoming event specifies its schema revision, so there's no need for schema migrations with normal event schema changes [12:15:40] right [12:16:02] but the capsule schema is declared "for all time", so to speak -- eventlogging doesn't have any notion of older events having been declared while another capsle schema was in effect [12:16:03] but the capsule has no automatic [12:16:09] update [12:16:28] right [12:16:28] not very elegant, but c'est la vie [12:17:03] is no issue really, i will ask dan to help me add the column to the db, it will be good to schedule an outage right? [12:17:28] yeah definitely [12:17:32] how do we do that? [12:18:09] the outage I mean [12:18:12] man is late we can talk tomorrow [12:18:29] (for you), late today (for me) [12:18:49] yeah, i'm going to log off in a sec [12:18:53] you can e-mail eventlogging-alerts@lists.wikimedia.org and analytics@lists.wikimedia.org [12:19:10] you'll need to subscribe to the former [12:19:20] which you can do here: https://lists.wikimedia.org/mailman/listinfo/eventlogging-alerts [12:19:31] is there a setting that turns off the loging somewhere [12:20:10] the logging of what? [12:20:17] the event logging client side or server side [12:20:30] everything? you can run 'eventloggingctl stop', but the varnishes will continue running varnishncsa [12:20:30] like a "global disable config" [12:20:46] and the mediawiki hosts will continue sending UDP events too [12:20:46] so when we say "outage" is that things keep logging they just do not go to db [12:21:11] we will just turn off the consumers, ok [12:21:26] we might be able to do that, but i think we should give ourselves a break and just say data for this period will be incomplete [12:21:51] if we commit to having everything turned off during the entire duration of the update, we can't bring thigns back up to test until we're 100% sure they're working [12:22:06] if we commit to having some things still logging, it requires that we be delicate and cautious [12:22:51] outages are pretty rare, so i really think it's okay to just tell people to mark some day on their calendar as "crappy eventlogging data day" [12:23:07] ok, we will talk later. [12:23:07] these are not transactions, they're analytic data, so there's no reason to make it a military operation [12:23:12] kk, see ya. [12:23:33] ciao [13:05:00] morning analytics :) [16:18:56] oook nuria [16:19:16] where is your new key? The link from the hangout isn't working for me [16:31:38] hola [16:42:46] [travis-ci] develop/bcc44f1 (#148 by nuria): The build has errored. http://travis-ci.org/wikimedia/limn/builds/16147713 [16:51:58] nuria: try stat1 or vanadium with your new key now [16:52:11] ok, going [16:52:41] milimetric: try stat1 [16:54:31] k [16:54:46] i have to change my ssh config hang on [16:56:39] awesome, thanks it works ottomata [16:56:55] cool [16:58:24] ok works [16:58:24] i did it this way [16:58:24] ssh -i ~/.ssh/wikimedia-production nuria@stat1.wikimedia.org [16:58:24] to make sure it was using the actual key, excellent [16:58:34] yup, cool [17:01:26] thanks much, shoudl i try vanadium too? [17:01:27] BTW, I have updated this doc so it is clear you need two different keys: https://www.mediawiki.org/wiki/Analytics/Onboarding [17:06:59] cool thanks [17:06:59] vanadium should be good [17:06:59] go ahead and try [17:06:59] oh but bastion [17:06:59] it might take a bit for puppet to run on bastion [17:07:00] which bastion do you use? [17:07:00] bast1001? [17:07:08] running puppet there now [17:09:53] i think so, let me confirm [17:10:26] Yes, that's it [17:10:46] # use bast1001 as main bastion for both pmtpa and eqiad [17:10:46] # (fenari can be used if needed) [17:10:46] Host *.pmtpa.* *.eqiad.* [17:16:08] ok yeah, you can try vanadium now [17:16:11] bastion should be good too [17:16:31] ok, trying [17:18:55] vanadium works too THANKS! [17:20:33] great