[08:16:12] hi [13:32:03] * drdee stumbles into the IRC room, knocks over some chairs and tables and hits his knee, bad start of the day :( [13:32:21] ha [13:32:24] go home, Dr. Dree, you're drunk! [13:35:56] of sleep deprivation [13:37:26] * YuviPanda has been up for 26 hours now [13:37:31] more like 27 already [13:43:20] Hi drdee. [13:43:27] hey [13:43:35] * qchris casts knee-healing spell upon drdee [13:43:53] But I am out of mana to cure sleep-deprivation as well. [13:44:04] thanks! [14:40:58] ok dokey, moved to Windows. GASP, I know, I know [14:41:06] but I was about to throw my Linux comp out the window [14:41:09] uh oh, socat process on gadolinium is dropping packets [14:41:17] :( [14:42:07] yeah :( [14:42:17] yesterday as well [14:44:43] yeah saw that for about an hour [14:45:48] drdee, I think I can calm this by disabling filters on gadolinium [14:45:56] which are less important here? [14:45:56] https://github.com/wikimedia/operations-puppet/blob/production/templates/udp2log/filters.gadolinium.erb [14:48:04] maybe you can move line 51 and line 57 to oxygen? [14:48:08] the rest is really important [14:48:13] yeah, ergh [14:48:18] or maybe we need a dedicated socat relay host [14:48:36] yes that's also an option [14:48:44] is emery still up? [14:49:09] yes it is [14:49:18] maybe use that as socat relay host? [14:49:26] yes, but it is being used too [14:50:15] i thought we had moved everything from there to oxygen and gadolinium [14:50:15] the stuff that is running is from ram IIRC [14:50:55] hm? no [14:50:59] you are right [14:51:04] https://github.com/wikimedia/operations-puppet/blob/production/templates/udp2log/filters.emery.erb [14:51:08] you are probably thikning of locke [14:51:18] how about locke [14:51:22] that's also still up [14:51:26] and nothing is rnning [14:51:51] it can use a precise upgrade :) [14:52:56] ja maybe? whats strange is this is a different behavior than we're used to seeing [14:53:10] usually we see a udp2log process drop packets due to load or blocking or something [14:53:23] this is the socat dropping packets while udp2log is running on the same host [14:53:26] if I stop udp2log, it work sok [15:03:56] yarghghh [15:06:59] yeah not good [15:07:11] even if udp2log is running on gadolinium with 0 filters enabled [15:07:17] socat drops packets [15:07:23] it only is stable if I stop udp2log altogether [15:07:43] and, drdee, I can't just move socat to another host, that requires a config change to all frontends :/ [15:07:50] i could move the gadolinium udp2log stuff to other hosts [15:08:12] at least the webstatscollector stuff should be fully puppetized now…except apergos' dumps rsync cron [15:09:31] yargh, and jeff's FR stuff is on gadolinium [15:09:33] that's hard for me to move [15:12:59] hello ppl [15:13:21] is there a demo for user metrics ? [15:15:42] yes tomorrow [15:16:06] ottomata, how about moving line 51 and line 57 to locke? [15:16:36] doesn't matter righ tnow [15:16:45] i temporarily commented out all filters [15:16:48] on gadolinium [15:16:50] and started up udp2log [15:16:52] if udp2log is running [15:16:55] the socat drops packets [15:16:58] that's where i'm at righ tnow [15:17:15] i'm moving as many as I can to oxygen right now, I think oxygen can deal with some more filters now that we only have one for x-analytics [15:17:28] then we'll have to see what we can do about webstats collector and FR filters [15:17:35] k [15:57:46] ok, drdee, i've got all filters except for FR moved over to oxygen [15:57:56] niiiiiiiice [15:58:00] webstatscollector 'filter' process moved to oxygen [15:58:07] collector can keep running on gadolinium [15:58:09] so that's good [15:58:16] don't have to mess with dumps rsync jobs [15:58:17] but [15:58:17] are you sure? [15:58:19] FR stuff is tricky [15:58:21] it seems so [15:58:26] if I stop udp2log [15:58:27] socat is fine [15:58:33] collector can keep running [15:58:58] and i can see traffic being beamed to the collector from oxygen to gadolinium [15:59:01] so that looks fine [15:59:12] but FR stuff, i dunno [15:59:14] its really custom [15:59:23] NFS mounts, custom cron job to rotate stuff really fast [15:59:42] i think they should have their own dedicated box [15:59:57] yeah hm [16:00:03] we can give them locke :p ? [16:00:27] :D [16:08:58] hi [16:09:12] trying to upload a new gerrit patchset for kraken today earlier [16:09:17] uhm getting this scp error dunno why [16:09:26] scp error? [16:09:44] yeah this one https://gist.github.com/wsdookadr/36d20a0972c0fedd3088 [16:10:10] what command are you running? [16:10:14] git review -r origin [16:10:25] try [16:10:30] "scp -P 29418 gerrit.wikimedia.org:hooks/commit-msg .git/hooks/commit-msg" [16:10:31] its probably trying to scp the hook [16:10:36] yes it is [16:10:42] but the port is 'None' [16:10:54] if you refresh I also put my .gitreview file [16:12:16] maybe my remote's wrong [16:13:00] (PS1) Stefan.petrea: Updating Kraken to cope with libdclass 2.2.2 [analytics/kraken] - https://gerrit.wikimedia.org/r/75349 [16:13:04] ok, worked [16:13:10] it's still in progress [16:13:14] have to add more things [16:13:45] modified pom.xml to include /usr/share/libdclass/dclass.jar [16:14:22] I noticed that dclass.jar's contents get bundled inside ./kraken-generic/target/kraken-generic-0.0.2-SNAPSHOT-jar-with-dependencies.jar [16:19:56] mmmm probably should be excluded [16:20:42] drdee: what should be excluded ? [16:21:32] what exactly gets bundled inside kraken-generic-0.0.2-SNAPSHOT-jar-with-dependencies.jar ? [16:23:50] 0 Sat Jul 20 08:28:56 CEST 2013 dclass/ [16:23:50] 4767 Sat Jul 20 08:28:56 CEST 2013 dclass/dClassMain.class [16:23:50] 2763 Sat Jul 20 08:28:56 CEST 2013 dclass/dClassLoader.class [16:23:50] 2937 Sat Jul 20 08:28:56 CEST 2013 dclass/dClass.class [16:24:23] drdee: I added a dependency in kraken-dclass/pom.xml to the /usr/share/libdclass/dclass.jar [16:24:47] okay that looksgood [16:24:55] please loop qchris in as well [16:25:04] and make sure he understands what 's going on [16:25:08] ok [17:00:29] qchris ? [17:01:09] erosen: scrum [17:01:14] qchris: scrum [18:41:24] (PS1) Milimetric: added about page [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75430 [18:41:34] (CR) Milimetric: [C: 2 V: 2] added about page [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75430 (owner: Milimetric) [18:41:35] (Merged) Milimetric: added about page [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75430 (owner: Milimetric) [20:00:08] hey erosen, I was interviewing [20:00:15] done now [20:00:30] i'm gonna code for a while, but keep me updated on any news [20:07:22] milimetric: hey [20:07:43] milimetric: I'll be in hangout if you want to chT [20:07:49] cool [20:07:56] brt, eating crazy late lunch [20:08:03] woah [20:08:10] maybe I'll sneak out and grab lunch now then too [20:20:25] erosen: has this been done? https://mingle.corp.wikimedia.org/projects/analytics/cards/780 [20:44:01] drdee: yes [21:25:34] ty erosen [21:25:39] np [21:29:35] milimetric: did you implement this by any chance: https://mingle.corp.wikimedia.org/projects/analytics/cards/664 ? [21:29:59] or erosen :) [21:30:08] hehe [21:30:15] i saw you were looking ;) [21:30:25] hehe [21:30:28] on mingle [21:30:40] and the answer is .... [21:32:12] erosen: are we 'Enforce cohort description as part of the upload process " ? [21:32:12] https://mingle.corp.wikimedia.org/projects/analytics/cards/711 [21:35:26] not sure, dan would know [21:35:40] milimetric: ^^ [21:37:20] sorry drdee, was just kung-fuing wikimetrics in the hangout [21:37:29] you can always stop in there to yell at me if i'm being unresponsive :) [21:37:48] no worries [21:37:51] just doing some cleanup [21:37:54] 664 is sorta handled by the autocomplete [21:38:00] mark it done? [21:38:04] we're not quite using the language selector but I'm not at all sure how that would work [21:38:12] the language selector doesn't have support for project selection really [21:38:29] i think autocomplete is good enough [21:38:33] let's close it [21:38:36] what about 711? [21:39:32] milimetric: ^^ [21:39:42] and how about 429? [21:39:44] k on 664 [21:39:46] https://mingle.corp.wikimedia.org/projects/analytics/cards/429 [21:39:47] looking at those two [21:42:02] k [21:43:45] done 711 [21:44:07] k [21:44:10] will be done with 429 by tomorrow [21:44:37] the stack exception for the job is not necessary anymore i hope [21:44:41] but i can expose that if needed [21:44:55] k [21:48:26] final q: milimetric: is this still an issue: https://mingle.corp.wikimedia.org/projects/analytics/cards/663 [21:48:58] drdee: i don't think so, but millimetric should have the final word [21:49:02] he is out for a bit though [21:49:39] k [22:09:56] (PS1) Milimetric: job management working and fit to demo [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75516 [22:10:02] (CR) Milimetric: [C: 2 V: 2] job management working and fit to demo [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75516 (owner: Milimetric) [22:10:03] (Merged) Milimetric: job management working and fit to demo [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/75516 (owner: Milimetric) [22:14:16] drdee: wikimetrics should be beta quality now [22:14:20] feel free to beat the crap out of it [22:15:14] re: 663, we're getting the same list of projects so no, it's still an issue [22:15:29] but I don't think it's an actual issue since nobody can see anyone's cohorts. [22:15:48] so we're not increasing the visibility of anything beyond what's available today. [22:16:09] I'm going to spend the next week fixing any bugs and writing any tests I can think of, but for now, feature freeze :) [22:16:14] later guys [22:16:20] oh! [22:16:59] special thanks to Evan for being an awesome teammate on this project. We're gonna miss you! [22:17:00] :D [22:17:02] laters [22:17:14] thanks milimetric :) [22:17:30] you were a pretty awesome teammate yourself [22:33:51] ;-)