[12:55:09] morning milimetric [12:55:30] morning average_1rifter [12:55:33] morning drdee [12:55:43] or i should say good afternoon [12:55:52] yes, it's 16:00 here [12:55:58] ready to submit some code :) [12:56:02] 9AM [12:56:03] here [12:56:29] drdee: I want to put it on the build1, I've used iconv for conversion to UTF-8 [12:56:42] great! [12:56:44] drdee: have you used iconv ? I got into trouble [12:56:58] user@garage:~/wikistats/webstatscollector$ file dumps/pagecounts-20120925-124700 [12:57:01] dumps/pagecounts-20120925-124700: Non-ISO extended-ASCII English text, with LF, NEL line terminators [12:57:09] drdee: although I've used iconv [12:57:17] scp-ing code to build1 [12:57:26] ok [12:57:48] you can read https://labsconsole.wikimedia.org/wiki/Help:Access#Using_ProxyCommand_ssh_option [12:58:07] to login directly to a labs instance without connecting first to bastion [12:58:47] yeah, I was thinking of scp local => bastion1 ; scp bastion1 => build1 [12:58:52] but now I can use that [12:58:59] don't do that :) [12:59:06] yes, I'll use the proxy [12:59:10] cool [13:11:42] drdee: morning [13:11:57] morning milimetric [13:14:46] drdee: I'm in a screen on build1.pmtpa.wmflabs [13:15:00] ok [13:15:30] ok [13:15:32] i am in [13:15:40] wow, dschoon started a whole policy investigation on my account (my user page got auto-deleted) https://www.mediawiki.org/wiki/User_talk:Jasper_Deng#How_is_having_a_user_page_grounds_for_deletion.3F_19172 [13:16:04] pretty cool, people have my back :) [13:16:36] drdee: screen -x please [13:18:35] no screen to bet attached [13:18:48] oh I was writing and [13:19:09] drdee: now ? [13:19:16] nope [13:19:28] drdee: can you make one and I'll connect ? [13:19:41] done [13:19:56] drdee: you're connected as your user [13:20:02] drdee: and the screens are per user ? not sure [13:20:10] yep [13:20:18] drdee: you can su - spetrea if you want [13:20:23] drdee: then we'll be able to have a shared one [13:20:48] ok [13:21:19] mmmm [13:21:23] doesn't work [13:21:29] let me first finish with erik zachte [13:21:32] drdee: ok [13:39:28] morniiiinnnnggggg [13:47:23] mooorning ottomata [13:53:08] yo ottomata, could you do two things? give stefan access to stat1 (see asana) and create a 1:1 banner impressions filter on oxygen (see also asana)? [13:54:23] ottomata: I'm here if you need any info from me, I'd be glad to provide it [13:54:49] a lll info is in the RT ticket [13:54:57] oh ok [13:55:08] yup can do [13:56:43] thx [14:04:48] any of you guys putty experts? [14:08:17] milimetric: ^^ [14:08:33] average_drifter: screen share [14:08:35] ? [14:08:37] i've used putty... [14:08:49] what you tryin to do [14:08:51] i am trying to get erikz running [14:08:58] get very weird error [14:09:10] oh, so SSH key hell through putty? [14:09:22] No supported authentication methods available [14:09:24] yes [14:09:32] drdee: yes [14:09:34] where are you authenticating to? [14:09:40] gerrit [14:09:49] just upgraded putty [14:09:54] so that only accepts SSH keys, right? [14:09:58] drdee: do you have mikogo ? [14:10:13] else i will have him regenerate his keys [14:10:23] mikogo? [14:10:34] drdee: http://www.mikogo.com/download/windows-download/ [14:10:41] drdee: http://www.mikogo.com/download/linux-download/ [14:10:42] i am on osx [14:10:48] why not screen [14:10:49] ? [14:10:52] drdee: oh, ok http://www.mikogo.com/download/mac-download/ [14:11:07] drdee: well ok, screen then, let's try [14:11:15] ok [14:11:35] drdee: can you login su - spetrea please ? [14:12:01] done [14:12:11] screen -x [14:12:24] Cannot open your terminal '/dev/pts/2' - please check. [14:12:59] drdee, i had that problem too! [14:13:02] when doing a screen with you [14:13:06] on labs [14:13:09] right? [14:13:14] weird...... [14:13:20] i think if you chmod 777 /dev/pts/2 [14:13:22] it'll just work [14:13:29] or if stefan does his [14:13:36] drdee: on build1 [14:13:38] whoever started the screen [14:13:40] ottomata: I'll do it now [14:14:01] drdee: try now please [14:14:22] nope [14:14:30] hm [14:14:37] sorry drdee, I was never able to get putty to authenticate me with SSH keys to git, I just read through it again and it seems straightforward enough so I'm not sure where it fails. [14:15:10] :( [14:15:26] man i already spent so much time getting this to work [14:15:34] this is insane [14:15:38] drdee: ok, let's just try mikogo ? [14:15:42] drdee: that one just works [14:15:53] ok [14:17:08] installing [14:18:10] ok [14:18:13] ready [14:18:21] please check pm [14:18:25] for the session number [14:18:25] ok [14:18:54] ok [14:21:05] is it fluent ? [14:21:37] drdee: is the image fluent ? or is it cuasing problems ? [14:22:08] it is fine [14:22:13] alright [14:23:59] very weird [14:29:02] drdee: I will close the session now, and look on the "unable to flush" error [14:29:05] ok [14:38:48] drdee, you want me to run to run the 1:1 banner impressions on locke? [14:39:00] i guess oxygen is safer [14:39:17] hmm, ok [14:39:23] can you gimme some background? [14:39:26] what's up? [14:41:00] 1 se [14:41:02] 1 se [14:41:05] 1 sec [14:46:14] peter wants to run the two banner impressions filters in parallel at 1:1 and 1:100 sampling so he can compare the results and see how much discrepancy there is between the two [14:46:30] 1:1 is obviously more intense [14:48:27] so oxygen is more safe [14:48:33] ottomata : ^^ [14:48:51] ok cool [14:49:02] how long does he want to run it? [14:49:17] if just for a few hours today or something, I can just turn it on without committing to puppet for a little bit [14:51:16] i think at least 24 hours [14:56:42] i am okay with not puppetizing it [14:57:27] how about ~11 hours? [14:57:29] heheh [14:57:43] there is a cron job that restarts puppet if it is off every night at 2am UTC [14:58:12] i guess 24 hours is most useful [15:09:28] brb lunch [15:09:34] I mean ETA 1h [15:11:08] ok [15:23:34] drdee, tis running [15:23:35] on oxygen [15:24:10] ty [15:57:48] ssh + putty + windows != fun [16:01:35] mocking up a time series in d3 took longer than I thought: http://jsfiddle.net/milimetric/g2qSV/1/ [16:07:26] if anyone's interested, you get what you expect from d3's time scale only if you specify a min and max date for the domain. I was passing the array of all the dates and confused myself for a while. [16:07:40] * milimetric lunch [16:09:23] * average_drifter away [16:11:36] average_drifter: around? [16:28:46] drdee: here [16:29:01] ok, can you push your wikistats changes? [16:29:23] drdee: yes [16:29:33] ok [16:39:29] drdee: can I add the sample log too ? [16:39:42] ? [16:39:51] drdee: sampled-1000.log-20120701.txt [16:39:52] which sample log and for what project? [16:39:55] no [16:39:58] oh ok [16:40:51] user@garage:~/wikistats/wikistats/squids$ git review [16:40:51] No '.gitreview' file found in this repository. [16:41:01] drdee: shall I copy it from webstatscollector? [16:41:42] copy and modify :) [16:42:48] ok [16:49:43] drdee: I'm missing a Change-Id in the commit message [16:49:48] drdee: is that generated by something ? [16:50:00] run git commit -a --amend again [16:50:32] drdee: https://gerrit.wikimedia.org/r/25036 [16:51:15] okay, next time more and smalller commits :) [16:51:28] this is massive :\ [16:53:29] can you collect all the todo's from the source code and email them to me (add filename / file number for each todo as well) [16:54:41] drdee: so the current tasks I'm working on wikistats that haven't been finished ? [16:54:51] drdee: the code shows just one TODO [16:54:52] EzLib.pm [16:54:52] 493: exit 0; # just for debugging(TODO:clear this exit) [16:55:03] oh okay, i thought there were more [16:55:07] never mind in that case [16:55:11] ok [17:00:43] oh so rooorrrreeeyyyyy [17:00:44] in the thangout [17:00:47] https://plus.google.com/hangouts/_/2e8127ccf7baae1df74153f25553c443bd351e90 [17:00:47] https://plus.google.com/hangouts/_/2e8127ccf7baae1df74153f25553c443bd351e90 [17:07:14] man. why is 10a so early in the morning?? [17:19:21] average_drifter: finished review for https://gerrit.wikimedia.org/r/#/c/25036 [17:30:32] average_drifter: can you also push your changes for webstatscollector, i wanna take a look at the utf8 stuff [17:32:43] OOH WOOOOW WOW WOW WOW http://ieeelog.com/ [17:33:32] "the ieee.org account username and plaintext password of around 100,000 IEEE members were publicly available on the IEEE FTP " [17:35:04] dschoon, ottomata, milimetric: ^^ [17:35:12] lol [17:39:09] average_drifter: also, do not include the browsecap.ini file in git [17:39:23] "several undoubtedly grave mistakes" :) [17:42:27] such as, "uploading passwords to an FTP site" [17:45:06] ha, they cited us [17:49:37] this person's unbridled desire to use periods as commas drives me nuts. damn europeans. [17:51:45] drdee: ok, about browsercap, should I write something to wget it ? [17:52:15] average_drifter, something like that but you don't need to worry about it right now [17:52:34] please commit webstatscollector first ;) (i know it doesn't work but that's okay0 [17:53:43] ok [18:00:38] hey stefan, you should be able to log into stat1.wikimedia.org now [18:00:41] lemme know if you have troubles [18:00:44] or need help [18:00:53] average_drifter^ [18:01:59] average_drifter: if it works then ask me for a tour :D [18:02:17] thanks ottomata! [18:51:46] anyone know of a good machine-readable list of all of the different wikipedias? [18:55:55] drdee: http://docs.oracle.com/cd/E17076_02/html/gsg/C/BerkeleyDB-Core-C-GSG.pdf [18:56:03] drdee: is this the documentation for Berkeley DB ? [18:56:07] drdee: I was reading from it [18:56:13] drdee: what docs did you use ? [18:56:18] the same [18:57:04] altough i prefer http://docs.oracle.com/cd/E17076_02/html/gsg/C/index.html [19:00:18] hey ottomata, are you experimenting with pig right now? [19:01:04] hive [19:01:07] or, at least [19:01:12] reading about it [19:01:12] 'coz if you are, maybe you can try to get piggybank to work [19:01:15] https://cwiki.apache.org/confluence/display/PIG/PiggyBank [19:01:26] which contains user defined functions for pig [19:01:41] like geocoding of ip addresses and other frequent use-cases [19:03:43] brb, lunch [19:08:50] cool [19:18:06] brb lunch [19:23:46] drdee: got a question [19:23:58] drdee: http://docs.oracle.com/cd/E17076_02/html/articles/inmemory/C/InMemoryDBApplication.pdf [19:24:18] drdee: says there on page 5 that filename should be ommited in db->open if the DB is going to be in-memory [19:24:24] drdee: do we want it in-memory or on the disk ? [19:32:26] average_drifter: on disk [19:34:52] To ssh://spetrea@gerrit.wikimedia.org:29418/analytics/webstatscollector ! [remote rejected] HEAD -> refs/publish/master/fixing_page_flush_and_utf8 (change 24379 closed) [19:34:58] ! [remote rejected] HEAD -> refs/publish/master/fixing_page_flush_and_utf8 (change 24379 closed) [19:35:11] drdee: I want to git review [19:35:15] drdee: but I get this erro [19:35:31] grumble [19:35:35] let me have a look [19:35:40] drdee: ok [19:36:12] yes that change has already been merged [19:36:28] if you want to start a new change set then don't use --amend [19:36:36] use --amend only for existing changesets [19:36:52] alright [19:36:55] uh ok [19:43:31] drdee: problem is my changes are on a branch [19:43:43] drdee: and now I tried to merge --squash them into a new branch for review [19:43:49] and now I get rejected [19:44:08] or there's another way around where I can send 2 commits per review [19:44:14] but I don't think that's acceptable [19:44:18] I mean allowed [19:44:32] try it [19:44:56] rejected again [19:44:56] wow [19:45:10] change 24379 closed [19:45:15] can't you create a new local branch [19:45:21] and make a new commit [19:45:23] ? [19:45:31] I can branch out from master [19:45:37] but my changes are all on a branch [19:45:51] I can copy them manually, or just git merge --squash them from the branch I've worked on [19:45:51] branch out of master [19:46:03] merge changes from old branch to new branch [19:46:06] create new patchset [19:48:15] hey guys, [19:48:23] quick idea: how about introducing demo friday? [19:48:37] where every friday afternoon (or maybe biweekly) [19:48:46] we demo some of the stuff that we worked on [19:48:55] limn, kraken, adhoc stuff etc. [19:49:41] +1 [19:49:50] (to demo friday) [19:50:23] I wish I could demo stuff but considering I'm like many many miles away, not sure how streaming would work here [19:50:33] uh, apart from that, what if a feature is not done by friday ? [19:51:36] but I'm attracted to the idea of demo :) [19:51:43] you buy beer for the team :D [19:52:02] hehe :) [19:52:15] yeah I could do that :) [19:52:33] 'fraid of not getting you guys drunk :) [19:52:59] from that much beer [20:02:59] dschoon: i suggested to do an internal team biweekly demo friday, like it? [20:03:13] totally! [20:03:20] ...what are we demoing? [20:03:30] limn, kraken, whatever we can demo [20:03:37] cool [20:03:54] i'm sure milimetric will be excited to show off his new stuff ;) [20:03:56] NO PRESSURE [20:04:04] :D [20:04:48] * milimetric doesn't feel pressure [20:07:46] drdee: https://gerrit.wikimedia.org/r/25169 [20:09:23] thx [20:09:52] does this fix the flush problem? [20:11:13] drdee: not yet [20:11:19] k [20:17:18] ohhhh average_drifter [20:17:23] about title length [20:17:38] titles in wordpress blogs can be for sure longer than 255 characters [20:17:52] and also make sure that collector and filter have the same expected title length [20:18:00] i think there is currently a mismatch [20:44:28] average_drifter: about the flush problem, you can also look at an older version of the code, around 2010 to see how it then was [21:14:16] hey guys, do we put anything up here: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/September [21:14:17] ? [21:14:27] yes we shoul [21:14:28] d [22:59:03] wow you wikimedians are good salesmen [23:15:59] whatup? [23:20:45] ? [23:24:33] milimetric: we are? [23:24:39] because i'd think we'd make more money if so [23:24:59] well, I think if money is your concern I can definitely help you out there. [23:25:13] but your sales acumen seems powered by your love of freedom :) [23:25:31] I'm referring to Asher's counter pitch [23:25:38] (response to otto) [23:36:10] off to dinner, may be done for the night [23:36:26] later everyone