[03:23:26] New patchset: Diederik; "Fixtures and scripts to load MW database with a small set of test data for UMAPI." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [08:27:08] New patchset: Rfaulk; "add. 'dist' aggregator for 'pages_created' metric." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62791 [08:27:08] New patchset: Rfaulk; "add. Register 'dist' aggregator for 'pages_created' in API." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62792 [09:18:47] If I get a proportion=3.35.. in GstMessageQOS from ffdec_h264, does that mean the source element should be feeding data to the decoder at 3 times the current speed? [09:25:43] whoops, wrong channel! [12:44:45] mooooooorning [13:02:28] hey morning average [13:02:44] thanks for creating the debian package@ [13:05:48] morning drdee [13:05:48] np [13:32:42] morning fellas [13:34:11] morning milimetric [13:34:22] hi stefan [13:34:26] where you going on vacation? [13:35:29] milimetric: at EOD [13:36:02] no not when, but where [13:37:26] milimetric: I'll be in a Iasi. A friend offered to teach me some C++, so I'm going to take the occasion to learn more. [13:37:56] oh cool, nice city [13:38:21] so for your vacation you're gonna code? Make sure you get some rest and relaxation too :) [13:38:28] milimetric: check https://gerrit.wikimedia.org/r/#/c/62450/ [13:38:57] oh cool, ok, i'll test [13:39:15] can we quickly squash those upload bugs? [13:40:54] yeah, the two files? [13:41:11] yup [13:41:19] i had them working before, not sure what changed, i'll get to them right after i review the seed.sql [13:41:27] (provisioning that right now) [13:47:39] drdee: ok if i make it delete everything from page, revision, and user first? [13:50:54] maybe a simple bash script that asks to do it instead of doing it automatically [13:50:55] ? [13:51:11] are you using ajax for the cohort upload functionality? [13:52:03] only a small bit of ajax [13:52:39] hm, i don't know - the seed should only run if the database is to be seeded. And in that case it should start it from scratch [13:54:23] maybe a big fat warning saying it will wipe the tables; continue yes/no? [13:55:32] hello! [13:55:59] well no - this is run by puppet, so how can you choose to not continue? [13:57:41] heh, one of the pages you're importing has "u so stupid protect yr fucking page" in it [14:01:37] hello steko [14:09:15] milimetric, okay go for it [14:09:32] ok found the problem - everything was wrapped in `` [14:09:37] so the script failed [14:10:07] i replaced everything manually and couldn't find a reason stuff was wrapped - usually if something has a space or isn't safe or something, just wrap it in [] [14:10:42] hm, there's something else too [14:11:59] what's the error? [14:14:23] Table storage engine for 'revision' doesn't have this option [14:14:24] ERROR 1064 (42000) at line 3 [14:14:43] (line 3 because i commented everything out to only run the insert into revision) [14:14:52] if you run the script as a whole it just says there are errors [14:15:43] that's odd; the script was generated using mysqldump [14:19:02] yeah, so that's just a note apparently, but the real error is: ERROR 1136 (21S01) at line 96: Column count doesn't match value count at row 1 [14:19:14] and i added 4 lines for deleting first, so lemme see... [14:19:25] it's the insert into page [14:51:52] New patchset: Diederik; "Fixtures and scripts to load MW database with a small set of test data for UMAPI." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [15:21:38] New patchset: Diederik; "Fixtures and scripts to load MW database with a small set of test data for UMAPI." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [15:24:12] New patchset: Diederik; "Fixtures and scripts to load MW database with a small set of test data for UMAPI." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [15:48:57] New patchset: Diederik; "Fixtures and scripts to load MW database with a small set of test data for UMAPI." [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [16:55:15] drdee: amit wants to try out the maxmind ISP database as a replacement for manually curated cidr ranges [16:55:18] thoughts? [16:55:29] let's try it! [16:55:42] cool [16:55:50] what is the best way to buy something like this? [16:55:54] it costs $25 [16:59:50] no scrum today right ? [17:00:20] xyzram: nope :) [17:00:32] wednesdays are no-scrum days [17:05:00] xyzram; ping me if you need me [17:05:18] erosen: we are an existing customer of maxmind [17:05:27] yeah [17:05:36] do they have a card on file? [17:05:41] yes [17:06:01] is it a WMF card? [17:07:50] email robla and rachel and describe the use case and ask for permission, then we can buy a one off db for $25 [17:07:54] please cc me [17:11:43] k [18:03:54] xyzram: sprint planning [18:04:03] average: sprint planning [18:04:10] milimetric: sprint planning [18:04:20] https://plus.google.com/hangouts/_/5b70172d0f7418695ff6d98f3cb53bbb7097e020 [18:04:54] oh, ok, that wasn't on my calendar for some reason ... anyway, I'll be there. [18:19:04] sorry, late [18:30:07] the door to my appartment was unlocked and a dog entered my house [18:30:11] brb, closing door [18:34:50] average: like a random dog from the streets? [18:37:58] erosen: yes [18:48:47] far out [18:52:45] So testing my changes to UserAgent.java -- I'm assuming PageviewTest.java is the right place to add unit tests. Is that right ? [19:06:48] hey xyzram, were you able to push your new code? [19:11:15] guys, can you access gmail? [19:13:27] New patchset: Milimetric; "adding test files and fixing bugs: encoding and exception handling" [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62858 [19:18:19] Change merged: Erosen; [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62858 [19:26:36] New review: Milimetric; "awesome!" [analytics/user-metrics] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/62450 [19:26:37] Change merged: Milimetric; [analytics/user-metrics] (master) - https://gerrit.wikimedia.org/r/62450 [19:27:16] drdee: ^^ yay! great job [19:27:27] so when's the next meeting, i'm now confused [19:27:29] 3 min? [19:27:46] dunno :) [19:37:20] rounce123: how does one attach a file to mingle? [19:38:20] at the bottom of the card in edit mode there is an attach option. [19:38:43] you have to be editing the card to see it and you can upload almost any format to my understanding [19:54:54] gotcha [20:20:23] importing the kraken project in eclipse is giving me errors in kraken-etl about the clojure plugin; anyone know how to fix that ? [20:21:04] kraken-etl is not needed for now; so i would say ignore that if possible [20:22:05] drdee: I was looking for unit tests for the existing WMF_APP_PATTERNS and don't find any [20:22:20] milimetric: is that correct ^^ [20:22:21] at least in PageviewTest.java [20:22:36] DarTar: the two 500 errors you were getting should now be fixed [20:22:47] no drdee / xyzram, there are tests [20:22:48] sweet [20:23:01] they're in UserAgentFilterTest something, i don't have it up [20:23:22] can't test right now but I'll give it a try later this afternoon, was that an encoding problem? [20:23:56] it was, yes [20:24:10] I don't find any files with find . -name 'UserAgentFilterTest*' [20:24:13] the current mysql connection assumes that latin-1 can handle everything [20:24:17] one sec ram [20:25:28] https://github.com/wikimedia/kraken/blob/master/kraken-pig/src/test/java/org/wikimedia/analytics/kraken/pig/UserAgentClassifierTest.java [20:26:38] I see that file, thanks. [20:26:49] Does "maven test" run that test ? [20:27:02] yes [20:27:07] maven test runs everything though [20:27:13] i'd recommend trying out intellij IDEA [20:27:19] there's a fully open source version [20:27:30] Is there a command line way of running just that test ? [20:28:58] mvn test (i belief) [20:29:06] mvn -Dtest=UserAgentClassifierTest test [20:29:14] yup,that's it [20:29:16] I tried that and got errors. [20:29:31] [INFO] dClass JNI Wrapper ................................ FAILURE [0.712s] [20:29:40] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project kraken-dclass: No tests were executed! (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1] [20:29:55] did you install the dclass-debian package? [20:30:02] average …. around? [20:30:04] xyzram: are you on a different machine than the one you installed the libdclass-dev deb ? [20:30:08] drdee: I am here [20:30:12] cool [20:30:19] can you help xyzram with dclass [20:30:19] ? [20:30:23] drdee: yes [20:30:40] xyzram: can you please make notes of all the steps involved to setup a dev env? [20:30:41] average: hang on one second [20:30:42] xyzram: did you copy the GeoIPCity.dat I sent you yesterday ? [20:30:48] xyzram: ok, waiting [20:31:23] yes GeoIPCity.dat is present in /usr/share/GeoIP [20:31:48] drdee: how can mvn be run with verbose cranked up to maximum ? [20:31:56] -vv ? -vvv ? [20:32:02] --verbose ? [20:32:05] probably different [20:32:09] google :) [20:32:31] xyzram: mvn -x please [20:32:36] to enable full debug output [20:32:49] average: I'm on the same machine but different repo (I forked the github repo) [20:33:02] xyzram: shouldn't be a problem [20:33:09] mramanath@xyz mediawiki: dpkg -l | grep libdclass [20:33:09] ii libdclass-dev 2.0.12 am [20:33:16] So that is installed. [20:33:36] xyzram: do you have the appropriate Nexus settings in ~/.m2 ? [20:33:39] But the above mvn command to run just one test is failing. [20:33:47] What are those ? [20:33:56] xyzram: can you run mvn -x and please put the output on http://gist.github.com so we can look at it ? [20:34:14] I have the settings.xml file copied from examples [20:34:48] xyzram: ok, please run mvn with the -x switch [20:35:04] mvn -x just prints a usage message and says: Unable to parse command line options: Unrecognized option: -x [20:36:04] xyzram: capital X [20:36:08] xyzram: mvn -X [20:36:51] Ok, that gets an exception ... I'll put it on gist ... [20:37:02] thank you [20:38:26] https://gist.github.com/xyzram/724aac04cfe83e9731ed [20:38:58] xyzram: thanks [20:39:36] xyzram: what command have you run [20:39:38] ? [20:39:50] drdee: can you confirm that the command Ram run is the right way to run Maven please ? [20:39:55] I remember I had the same problem [20:40:30] For the unit test you mean ? [20:40:37] xyzram: yes [20:41:05] As given earlier: mvn -Dtest=UserAgentClassifierTest test [20:41:17] drdee: ^^ is this the right way to test ? [20:41:32] in meeting [20:42:00] xyzram: what happens if you ommit the -D argument ? [20:42:08] xyzram: with the param and value of course [20:42:11] xyzram: so just mvn test [20:42:28] xyzram: I remember stuff was passing yesterday. Have you made substantial changes since yesterday ? [20:42:58] xyzram: ok I identified one problem [20:43:03] xyzram: you are using jdk 1.7 [20:43:26] xyzram: Diederik , Andrew and David , we agreed at a point that jdk 1.6 will be used for this [20:43:30] java -version [20:43:31] java version "1.7.0_17" [20:43:33] xyzram: please switch your jdk to 1.6 [20:44:20] drdee: please consider confirming this after the meeting ( that jdk 1.6 is needed for Kraken development ) [20:44:26] Just "mvn test" works. [20:44:45] yes you need jdk 6 [20:44:54] xyzram: so all tests pass, but if you try to run just one, it doesn't pass. Is this correcT ? [20:45:22] correct [20:45:34] odd [20:45:38] xyzram: alright, then there must be a problem with the parameters you're using for maven [20:45:41] I'm now installing jvm 1.6 [20:46:11] xyzram: thank you [20:46:28] xyzram: also consider using update-alternative to switch the JVM on your system to 1.6 [20:46:32] xyzram: I remember Diederik mentioning that IDEA is much more easy to use to run the tests for Kraken [20:46:39] xyzram: IDEA is the IDE from IntelliJ [20:47:02] xyzram: I remember running individual tests with IDEA and it was much easier as opposed to convincing Maven to run them on console [20:47:07] does it come with its own JVM or use the installed one ? [20:47:35] xyzram: IDEA allows you to choose which jvm to use [20:47:37] Does IDEA not use maven ? [20:47:44] yes it uses maven [20:48:12] how do I use the alternatives mechanism ? [20:48:19] xyzram: moment [20:48:33] xyzram: you're using Ubuntu, is this correct ? [20:48:39] right [20:48:56] 12.10 [20:49:35] perfect [20:49:36] sudo update-alternatives --config java [20:50:03] There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java [20:50:03] Nothing to configure. [20:50:18] xyzram: did you install jdk 1.6 ? [20:51:12] I was going to but normally I do it from the Oracle download file; I guess I'll need to do it from the Ubuntu package manager for this to work. [20:51:32] Looking to see how to do that ... [20:51:37] xyzram: I'll show you how [20:51:40] there is a ppa for it [20:52:00] xyzram: you need oracle jdk 6; not openjdk 6 [20:52:12] deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main [20:52:13] deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main [20:52:27] xyzram: add these two to your /etc/apt/sources.list [20:52:54] xyzram: after that sudo aptitude update [20:53:02] after that sudo aptitude search oracle-java6-installer [20:53:08] sudo aptitude install oracle-java6-installer [20:53:53] xyzram: after that look on this article to see the few steps that remain http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html [20:54:44] (remember to replace 7 with 6 in that article to get jdk 6 ) [20:54:53] after this you will have jdk 6 [20:55:04] after that install IDEA, choose jdk 6 and try to run that test with IDEA [20:58:41] W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY C2518248EEA14886 [20:59:09] not a problem [20:59:15] ignore that [20:59:26] ok [21:01:04] ok the install of jdk 6 is happening ... [21:01:31] * drdee wonders whether we are making notes.... [21:01:39] drdee: we have IRC logs [21:01:42] yes indeed. [21:01:58] let's clean that up and make it a readable read me :) [21:02:15] ok [21:02:54] (so include geoip database, dclass-lib, java etc etc ) [21:07:13] Installing that automatically switched the default: /usr/bin/java -version [21:07:20] java version "1.6.0_45" [21:08:47] xyzram: perfect [21:08:51] xyzram: have you installed IDEA ? [21:09:23] working on that now [21:19:10] milimetric: has this been fixed: https://mingle.corp.wikimedia.org/projects/analytics/cards/657 [21:22:05] average: what is the status of https://mingle.corp.wikimedia.org/projects/analytics/cards/557 ? [21:23:59] drdee: the status on the mingle card said Dropped [21:24:02] drdee: but it's done [21:24:10] drdee: so I marked it "Done" a second ago [21:24:22] ok [21:24:24] ty [21:26:33] yes drdee: the only link works [21:26:38] it's marked as done in mingle [21:26:45] arrghh:) [21:29:57] average: I've got IDEA running; how do I run a selected test ? [21:30:53] xyzram: you open the test [21:31:33] xyzram: and you click the green button [21:32:41] drdee: ping; meeting [21:34:16] xyzram: Alt+Shift+F10 or Shift+F10 [21:34:24] xyzram: or in the menu Run > Run [21:34:27] average: I chose run from the "Run" dropdown and got a small window to edit config [21:34:49] after that I don't see any results. [21:35:34] The run button (triangle) is actually greyed out [21:35:49] xyzram: is it ? did you select your jdk at the beginning ? [21:36:11] xyzram: you should have a default jdk selected [21:36:11] Yes. Also, lots of red boxes in the right margin. [21:36:30] xyzram: screenshot please, upload on http://imgur.com [21:36:58] coming ... [21:37:08] drdee , xyzram install documented so far here ==> https://gist.github.com/wsdookadr/42f9a305682091a0d53b [21:37:40] awesome! [21:38:47] average: can you push that read me to github.com/wikimedia/kraken ? [21:38:56] drdee: yes [21:42:53] http://imgur.com/jsPfCzw [21:43:16] average: that's the screenshot [21:43:28] looking [21:43:41] drdee: please look also at the screenshot. I am less experienced with the IDEA IDE [21:44:32] drdee: information for setting up added to README.md in kraken repo https://github.com/wikimedia/kraken/blob/master/README.md [22:12:06] uhm [22:13:28] [INFO] Reactor Summary: [22:13:28] [INFO] [22:13:28] [INFO] Kraken ............................................ SUCCESS [0.296s] [22:13:30] [INFO] dClass JNI Wrapper ................................ SUCCESS [1.844s] [22:13:34] [INFO] Kraken Base Library ............................... SUCCESS [19.742s] [22:13:37] [INFO] Kraken EventLogging ............................... SUCCESS [1.075s] [22:13:40] [INFO] Kraken Pig Library ................................ FAILURE [3:41.196s] [22:13:43] [INFO] Kraken Funnel ..................................... SKIPPED [22:13:45] [INFO] Kraken ETL ........................................ SKIPPED [22:13:52] this is what I get.. [22:14:08] with what command ? [22:15:43] mvn package [22:20:54] That was giving me success yesterday. [22:22:49] OK, now I've added some unit tests to UserAgentClassifierTest and I'm getting "BUILD SUCCESS" [22:23:22] So I'm going to push this change to my fork on github. [22:24:04] great [22:26:22] nice! [22:28:34] drdee: Just pushed my change to https://github.com/xyzram/kraken.git [22:28:40] awesome! [22:30:45] xyzram: you can make a pull request on github now for kraken :) [22:31:07] but maybe you have other things to add before you do [22:31:09] not sure [22:31:54] Let drdee decide [22:32:28] xyzram: i assume it passes the unit-tests? [22:32:51] As noted above: BUILD SUCCESS [22:32:52] if that's the case then copy the kraken-pig-.0.0.2 jar to analytics1010 [22:33:15] To which path ? [22:35:06] drdee: what's the full domain ? analytics1010.wikimedia.org doesn't seem to work [22:35:31] you have to connect through a bastion host like bast1001.wikimedia.org [22:35:42] ok [22:36:33] ram@bastion1:~$ ssh analytics1010 [22:36:33] ssh: Could not resolve hostname analytics1010: Name or service not known [22:37:35] ram@bast1001:~$ ssh analytics1010 [22:37:35] Permission denied (publickey). [22:38:19] Ok, finally I'm able to get to analytics1010. [22:38:30] Needed the -A on ssh [22:41:33] drdee: analytics1010:/tmp/kraken-pig-0.0.2-SNAPSHOT.jar [22:41:33] now exists. [22:43:15] ok, I'll have to get a train in 3h, I'm out [22:43:26] good night [22:43:55] will be missing in action for 2 sprints [22:43:59] bbl [22:44:07] ok, thanks for all your help. [22:44:10] np