[00:03:29] brb meeting [00:35:52] http://imgs.xkcd.com/comics/star_trek_into_darkness.png [00:36:08] I was just informed of this xkcd [00:36:34] hehe, nice [00:37:11] I'm definitely more the sort of person who has a favorite edit war than one who actually edits (and wars) [00:40:07] I was involved in one and promptly decided that I should calm down and chill out [00:40:17] long time ago [14:26:13] mooorniiinnngg! [14:34:26] hey ottomata [14:34:41] heya [14:35:46] so by the looks of things I'll be getting to puppetizing lab instances tomorrow or early next week [14:35:56] do you have any spare cycles to help with that? [14:38:33] cool! [14:38:35] sure! [14:38:37] especially next week [14:38:39] but yeah probably [14:38:54] i'm on RT duty next week, which basically i'm in administrivia and helper dude mode [14:39:00] basically means* [14:55:35] average_drifter, you around? [14:55:56] i am around [14:57:34] are you average_driftaaah!? [14:57:39] but! maybe you know [14:57:48] .debs for udp-filter? are they cooking? [15:00:52] just pushed a change to filter adding wikidata support and tab support [15:00:59] so that can be compiled [15:01:04] i am pinging him [15:01:49] cool [15:02:29] i mean, since what I did is in a branch, it won't affect the debianize work that is done ahead of it [15:02:31] i could probably just build them [15:06:38] let average_drifter do it [15:08:52] hokay [15:23:38] morning milimetric [15:23:45] good morning! [15:35:29] erosen: fixed labels to read from datasource labels if they're not specified: http://gp-dev.wmflabs.org/graphs/active_editors_by_region [15:36:30] yay [15:49:03] milimetric, i turned kraken into a well organized multi module project [15:49:14] woo! [15:49:37] i'll pull [15:52:00] havent' pushed [16:42:47] milimetric: pushed [17:36:58] ottomata1: solved the distro name for the package (I mean lucid-wikimedia or precise-wikimedia ) [17:37:01] we're getting there [17:38:09] nice! [17:45:06] !log Released version 0.6.0 of Limn [17:45:08] Logged the message, Master [18:00:19] woo! standup! woo! [18:00:20] https://plus.google.com/hangouts/_/2da993a9acec7936399e9d78d13bf7ec0c0afdbc [18:00:39] drdee, ottomata1, erosen ^^ [18:00:50] ohp standup! [18:10:42] ottomata1: please link me up with the wikitech regulations to package names etc [18:10:55] ottomata1: I need that so I can refer to it in the docs of debianize [18:11:38] i think wikitech is closed, we'd have to get you an account :/ [18:11:45] ottomata1: this ? http://wikitech.wikimedia.7val.com/view/APT_repository ? [18:12:01] ottomata1: wikitech is close d? [18:12:04] uhm [18:12:35] ottomata1: I just need a link to the documentation which states what you wrote in the github issue [18:12:42] Don't need access to wikitech itself [18:13:06] ottomata1: btw, I'm closing your issue now as it's solved :) [18:13:09] right, the documentation is in wikitech [18:13:36] ok [18:13:54] oh I take it back [18:13:57] i thought it wasn't readable [18:13:58] here you go [18:14:03] http://wikitech.wikimedia.org/view/Reprepro#Importing_packages [18:59:29] problems encountered: build1 is quite slow(dunno why). every time I build, I need to transfer files build1 => locally , locally => stat1://home/spetrea/www/releases/ with scp in two steps. [18:59:53] i think there are labs issuing with the filesystem [19:00:06] a build which takes 10s locally , takes 8m on build1 [19:00:15] or more [19:00:25] i think the fs is doing some self-healing tuff [19:00:29] that sucks [19:00:34] yes, it does [19:14:12] ha, i have to do that too, labs and prod systems are not connected [19:14:21] 8m? [19:14:29] i built udp2log there pretty quickly [19:14:30] hm [19:14:49] http://www.jasondavies.com/maps/transition/ [19:14:54] mercator is SO TRIPPY [19:15:14] dschoon: the projection ? [19:15:20] ottomata: yes, 8m+ [19:15:22] the animation [19:16:22] [ALERT] udp-filters 0.3.21 is ready here http://stat1.wikimedia.org/spetrea/releases/udp-filters/ [19:16:36] yeehaw [19:16:54] data cowboy [19:17:47] is this really 0.3.21? [19:17:52] i thought it was 0.26 still [19:18:00] damn [19:18:18] there is one problem [19:18:30] the precise deb is empty, have to go and find out why [19:18:57] is the lucid package ready? [19:19:02] ottomata: yes it's 0.3.21 , the git history decides our version [19:19:06] drdee: the lucid one is ready yes [19:19:12] but this is a backport release [19:19:20] please give it to ottomata [19:19:20] i mean, i guess it doesn't mattter [19:19:23] i have it [19:19:24] drdee [19:19:26] ok [19:19:35] do you need a precise version as well? [19:19:44] ottomata: ^^ [19:19:48] and if so for which machines? [19:19:53] not for this deploy, but we will for analytics [19:19:55] ok [19:20:08] i think all udp2log machines are lucid [19:20:19] think so too [19:20:29] let's chat on the etherpad [19:20:37] average_drifter feel free to join [19:24:39] how do I hcange my etherpad name ? [19:24:42] *change [19:25:11] ottomata: stat1001 got magically fixed by itself again (unless you sprinkled some magic powder this morning) [19:25:30] the metrics api, that is [19:25:46] i think it was magic powder [19:25:56] Hmmmm, drdee [19:26:05] isn't udp-fitler package name supposed to be 'udp-filter' not 'udp-filters'? [19:26:31] it is udp-filter in puppet [19:26:37] average_drifter :? [19:26:38] :/ [19:27:15] ottomata: erm, I'll try to fix it, moment [19:27:25] ottomata: ETA 30m [19:27:29] i thought that was fixed long time ago [19:28:06] ok, I'll try to fix it, it will be another 30m ... [20:18:34] how goes it average_drifter? [20:19:33] ottomata: got the problem solved, double checking, will push to build1, then I have to build it there, and then transfer it to stat1 [20:19:43] ETA ~10m [20:20:08] ok [20:20:08] danke [20:30:32] average_drifter : how much more time? [20:31:19] I'm waiting on the filesystem now.. [20:31:20] ~5m [20:31:29] the filesystem on build1 [20:39:17] 0.3.22 ready for lucid [20:41:01] yeehaw [20:41:33] thanks average_drifter! [20:42:11] np [20:43:08] ottomata, average_drifter, let's get back to the ether pad chat [20:43:22] you putting up on stat1? [20:43:37] oh i see it [20:43:38] sorry [20:43:41] was looking at the bottom fo the list [20:43:46] but now udp-filter sorts before -s [20:43:47] coool [20:43:48] k checking [20:53:09] can't get the tests to pass [20:53:21] I did cat example.log | perl -ne 's/\ /\t/g; print' > example.log [20:53:30] then I added -F '\t' to all the tests [20:53:36] all fail except 2 [20:54:17] oh tests [20:54:17] yeah [20:54:19] they don't pass [20:54:26] someone changed the number of lines in example.log [20:55:14] ottomata: revert back example.log to a very long time ago perhaps ? [20:55:19] maybe that would solve it [20:55:28] maybe :) [20:55:34] but i manually tested a bunch of the stuff [20:55:40] it works pretty good, the tests are just poopy [20:55:46] I fixed some of them [20:55:51] mainly the IP filtering tests [20:55:59] but anyway, its looking good [20:56:04] average_drifter I think you can cherry pick some of the later patch sets and back port them [20:56:12] I will even restart udp2log and make it use the same non tabbed udp-filter but with the new version [20:56:15] and watch it for a min [20:56:16] before we deploy [20:56:19] doing that now [20:57:25] drdee: yes, but not right away. today we can have them, but if you need to deploy let's deploy and then I'll figure the cherry-picking [20:57:37] ok [20:57:51] deploy first [21:01:51] i would just fix the tests [21:01:53] will be way easier [21:02:02] just fix the number of lines that each test actually expects [21:02:03] to se [21:02:04] e [21:02:41] !log deployed new version (0.3.22) of udp-filter on emery, locke and oxygen. This version accepts —field-delimiter flag. [21:03:30] ok let's plan, so we fixed the urgent problem, we deployed [21:03:36] for the next release we'll have tests fixed [21:03:45] and we'll cherry pick which features ? [21:04:27] drdee: ^^ [21:04:57] no just fix the tests, don't worry about it for now [21:05:56] ok [21:08:55] ottomata, i think the command is !log-bot [21:09:03] at least in this channel [21:09:10] naw, always has been log here and ops [21:09:56] 19:45 < milimetric> !log Released version 0.6.0 of Limn [21:09:56] 19:45 < analytics-logbot> Logged the message, Master [21:10:17] analytics-logbot: are you reading this ? [21:10:18] I am a logbot running on bots-analytics. [21:10:18] Messages are logged to www.mediawiki.org/wiki/Analytics/Server_Admin_Log. [21:10:18] To log a message, type !log . [21:10:21] hey average_drifter [21:10:25] hi milimetric :) [21:10:42] why you re-pasting my messages? [21:10:57] :) [21:11:00] milimetric: ottomata just wrote a !log command and I was expecting the analytics bot to pick it up [21:11:26] !log deployed new version (0.3.22) of udp-filter on emery, locke and oxygen. This version accepts —field-delimiter flag. [21:11:26] !log deployed new version (0.3.22) of udp-filter on emery, locke and oxygen. This version accepts —field-delimiter flag. [21:11:29] haha [21:11:30] LOGBOT [21:11:33] HELLOOOOO [21:11:40] he is dead [21:11:45] Don't make me go talk to cleverbot [21:11:49] analytics-logbot: are you ok ? [21:11:49] I am a logbot running on bots-analytics. [21:11:49] Messages are logged to www.mediawiki.org/wiki/Analytics/Server_Admin_Log. [21:11:49] To log a message, type !log . [21:11:57] !log hiii [21:12:29] Logged the message, Master [21:12:36] :) [21:12:37] !!! [21:14:36] !log deployed version 0.3.22 of udp-filter on emery, locke and oxygen. This version accepts —field-delimiter flag. [21:14:57] hey everybody, I just got arbitrary visualization working for world country datasources (the first one in that list is an example): http://dev-reportcard.wmflabs.org/datasources [21:15:01] !log deployed version 0.3.22 of udp-filter on emery, locke and oxygen. This version accept field-delimiter flag. [21:15:03] Logged the message, cap'n [21:15:20] yep, that — was the problem [21:15:24] yep [21:15:24] so apparently the logbot chokes on — [21:15:26] good find [21:15:32] what is that? two -- [21:15:34] no [21:25:14] does it work with quotes? [21:26:05] milimetric: heh, "See as Table" doesn't work out too well for json [21:26:25] oh no where? [21:27:04] oh geojson! [21:27:07] lol, yeah, oops [21:27:34] it should be a GraphNode instead of a piece of GraphView [21:28:24] but yeah, it doesn't work for anything but timeseries right now [21:33:35] just an fyi :) [21:41:22] ottomata, dschoon, drdee, looks like ops is having problems with our tab separator thing [21:41:36] why? [21:41:39] ottomata is on top of it in #ops, i believe [21:41:43] ok [21:41:48] need any help? [21:41:59] they're in a hangout [21:51:12] we are all good, things are recovering [21:51:26] site didn't go down, pfffeeeww [21:53:22] yay [21:54:36] hey dschoon - get that limn converter pig script working? [21:54:51] i'm working on mobile's varnish header [21:55:11] ok, let me know when you work on it [21:55:22] i could use a break from the horrible task of building browserify automatically [21:55:30] totally. [21:55:55] are you really doing that now? [21:56:02] god, that was a nightmare when i tried myself [21:56:18] because the browserify executable gives different results than the middleware does for reasons i cannot fathom [21:57:20] yeah, I have to do it - it's a bottleneck for debianizing/puppetizing limn [21:57:25] *nod* [21:57:53] it might be wise to upgrade browserify first and see if it breaks everything [21:58:04] it probably will. it has in the past. [21:58:10] maybe it'll help to talk things out [21:58:34] so there's our sources - easy, coco compiles them [21:58:40] there's vendor sources - easy, we bundle them [21:58:44] yep. [21:58:56] and then there's browserify which is a bundle of node_module stuff right? [21:59:10] I was thinking just dynamically figure out what that "stuff" is and bundle that [21:59:19] sort of like findBrowserifySources or something [21:59:26] er [21:59:30] it's recursive. [21:59:32] but i'm lost there [21:59:34] ew [21:59:37] EW [21:59:39] besides. [21:59:46] that's *exactly* what browserify does. [21:59:50] right ok [21:59:56] so the executable should do its job [22:00:00] it walks the AST and builds the `require` graph [22:00:01] yes. [22:00:02] when you do what? require ... [22:00:06] you go: [22:00:24] so [22:00:28] line 75 [22:00:30] in the cokefile [22:00:36] it's out of date [22:00:41] but that (maybe) worked before. [22:01:16] console.log ' Bundling Browserify bundle' [22:01:16] bundle = browserify exports:<[ require process ]> [22:01:16] .require <[ seq d3 events ]> [22:01:16] .bundle() [22:01:16] write 'static/vendor/browserify.js', bundle [22:01:16] @ok() [22:01:19] i just pushed the updated list [22:01:24] but what's the magic list [22:01:25] oh! [22:01:27] :) where'd you get it? [22:01:41] line 197 in middleware.co [22:02:01] middleware [22:02:02] there's a todo to pull out all the constants into a config file or something [22:02:03] ok, i thought so [22:02:13] i could use a break from varnish docs [22:02:24] i'll do that right quick :) [22:02:29] couldn't view_helpers return the constants in a function? [22:02:44] or can middleware not require that? [22:03:04] sure. but i want them all in one place so we don't go hunting around [22:03:07] i was thinking in the root, as constants.co [22:03:12] Y/n? [22:03:24] i think they'd make sense either in the modules.yaml [22:03:30] or view_helpers [22:03:39] because that's where other "sources" type things are [22:03:48] yea, but it's a lot of things [22:03:54] in fact [22:04:00] we could eliminate modules.yaml! [22:04:06] and replace it with a value in constants! [22:04:15] ooo. i've been wanting to kill that directory for a while [22:04:20] :) [22:04:23] ok, cool [22:04:25] we could move the css to static [22:04:27] that sounds efficient [22:04:31] and rename static to public, or something [22:04:34] then www could go away [22:04:36] i'll do that part [22:04:40] aiight [22:04:46] well [22:04:49] you moving modules to constants.co then? [22:04:52] i have to move modules.yaml in first, yes? [22:05:23] uh, ok, I'll make css work out of static [22:05:30] and then you can delete www once you're done with modules [22:06:05] it can be broken inbetween [22:06:12] just move it and i'll not touch www [22:06:18] you can delete, i have a local copy [22:15:23] ottomata: hey [22:15:33] ottomata: how do I test sqstat.pl ? [22:16:14] ok I'll just fiddle with localhost [22:16:49] heya, uhh, dunno, never used it before :p [22:17:05] take a log sample file [22:17:09] and pipe it into sqstat [22:18:20] I'm gonna do just that :) [22:28:28] I would like to adopt sqstat.pl [22:28:41] is it available for adoption ? [22:28:58] nawwwwwww its asher's, i wouldn't modify it much [22:29:01] just change the tabs! [22:29:04] hehe [22:29:09] it'll probably be deprecated one day [22:29:18] you can ask him i guess [22:29:27] where's asher ? [22:29:29] but it soudns like it is important for monitoring big production stuff [22:29:34] binasher on IRC [22:29:41] you can reply to that email I just sent too [22:29:48] Asher is en-route to Fosdem. [22:29:53] but for now, I just need to use tabs [22:29:54] So he's over the Atlantic right now, iirc. [22:30:27] average_drifter [22:30:31] can't we just change these lines [22:30:31] if ($line =~ /^\S+ \S+ \S+ \S+ \S+ \S+ \S+ \S+ http:\/\/upload.wik/) { [22:30:31] $d{'upload_requests'} += $mult; [22:30:32] } elsif ($line =~ /^\S+ \S+ \S+ (\S+) \S+ \S+ \S+ (GET|POST) http:\/\/([\w.]+org)\/(wiki|w)\/(\S+) /) { [22:30:37] and put \t where ther are spaces? [22:30:38] yes, we can [22:30:40] that should be it, right? [22:30:43] let me test it [22:30:45] ok [22:30:48] so I can make sure it works [22:30:51] should it be \t or [ \t] ? [22:31:01] \t [22:31:08] we're trying to avoid the spaces in the content type [22:33:41] yeah. [22:40:14] ok I was able to test it as it is right now [22:40:16] now replacing tabs [22:41:48] ottomata: got a question ! [22:42:01] ottomata: so lines that come to sqstat.pl are now tab-separated right ? [22:42:10] ottomata: and for this to happen you needed to patch squid configurations ? [22:42:23] ottomata: I just want to check so I understand what's going on [22:43:29] ottomata: ^^ [22:45:17] yes [22:45:18] yes [22:45:24] we had to patch squid, varnish and nginx configs [22:45:32] everything is done except for the sqstat bit [22:46:45] ok [22:47:04] !log deployed tab as separator to all frontend cache servers, modified udp2log server filters to DEAL WITH IT! [22:47:06] Logged the message, Master [22:47:36] soooo uhhh, can I change sqstat.pl to \t ? :) [22:47:45] I'm changing it :) [22:47:50] please be patient [22:47:50] you can commit to puppet? [22:47:51] haha [22:47:53] :) [22:47:54] sorryyYYEEE [22:47:57] i'm getting ready to head out for the day [22:48:02] and want to do it before I go [22:48:17] 2m ! [22:48:30] ok ok ok ok [22:56:43] I tested it [22:57:25] ok! [22:57:37] can you commit to puppet gerrit via git-review? [22:58:49] yes, moment [23:11:05] ottomata: https://gerrit.wikimedia.org/r/47036 [23:11:10] yeehaw [23:11:25] sorry, had git review problems [23:11:31] would've done it much faster otherwise [23:11:37] also had to test it [23:11:39] great [23:11:40] naw its cool [23:11:54] merging deploying [23:17:56] git review still doesn't work on my laptop, although I put the ssh key in it [23:17:59] amazing [23:18:01] will have to solve it [23:20:53] thanks average_drifter! deployed and running. [23:23:23] you're welcome [23:23:30] ottomata: how do we find out if it's working properly ? [23:23:38] how or when [23:24:01] how and/or when [23:29:37] uhh no idea [23:29:41] i don't know what it does [23:29:46]