[08:26:27] !logs [08:26:28] http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs/ [11:11:34] When signing up for an account on wikitech.wikimedia.org, which LDAP fields get set to sign-up form's "Username", and which to "Instance shell account name"? [11:11:45] It seems it's "Username" -> cn, sn. [11:11:56] "Instance shell account name" -> uid [11:12:37] Is this somewhat correct? Or do we have documentation where I could read up on that? [11:48:15] Coren|Away: what is firefox doing on exec nodes? o.o [11:48:38] !log tools someone installed firefox on exec nodes, should investigate / remove [11:48:41] Logged the message, Master [11:49:02] :0 [11:49:43] petan: what do you recommend as a decent free win c# compiler? [11:49:45] !log tools petrb: syncing packages of -login with exec nodes [11:49:46] Logged the message, Master [11:50:05] T13|needsCoffee mcs [11:50:32] or csc [11:51:08] http://www.microsoft.com/en-us/download/details.aspx?id=8279 [11:52:50] Meh... looks like mcs because your link there is for win7 [11:54:46] which OS you need it for [11:57:17] Vista... [11:57:25] Don't judge... :p [11:58:51] I'll read the page for and dl mono Monday. Got classwork to finish today. [12:04:41] Technical_13, how free must it be? Free-as-in-beer gets you the fairly decent VisualStudio Express with the mainline compiler. Otherwise go msc from Mono [12:06:30] VisualStudio "Express"? What are the limitations of it not being full version? Nag lines (I could live with) or technical restrictions like only compile programs up to 64Kb? [12:07:04] in the compiler none [12:07:07] in the IDE many [12:09:34] no nag lines. technical restrictions in using may different features, profiling, shim/stub/mock framework, static analysis, automated UI tests, built in continuous integration are all missing [12:09:44] but, like I said, the compiler is the same [12:14:46] !todo is https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Todo [12:14:46] Key was added [12:14:54] @notify Coren|Away [12:14:54] This user is now online in #wikimedia-labs. I'll let you know when they show some activity (talk, etc.) [12:15:45] !log tools petrb: making pv from /dev/vdb on new nodes [12:15:47] Logged the message, Master [12:17:43] !log tools petrb: test [12:17:45] Logged the message, Master [13:03:16] petan: Whazza? [13:04:29] Coren: I wanted to ask you something but I forgot :/ [13:04:33] why is firefox on exec nodes? [13:04:39] it makes little sense to me [13:04:46] also this: http://tools.wmflabs.org/admin/packages.html [13:04:48] and [13:04:49] !todo [13:04:49] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Todo [13:04:50] :P [13:05:07] we should list all packages that are missing in puppet [13:05:14] and insert them [13:33:24] !todo | scfc_de [13:33:24] scfc_de: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Documentation/Todo [13:33:32] scfc_de: why there is firefox on exec nodes http://tools.wmflabs.org/admin/packages.html [13:39:55] petan: Probably because it was an automatic suggestion through apt. [13:43:05] Isn't jobutils on exec nodes?! I've installed Emacs sometimes, but that doesn't need to be puppetized. [13:48:21] scfc_de fix it then it's a wiki [13:49:54] petan: What packages are missing on exec nodes at the moment? (A patch for python-oursql is pending.) [13:50:06] http://tools.wmflabs.org/admin/packages.html [13:50:35] "'missing' not found" [13:50:38] scfc_de that TODO was made of packages listed on that html page (some of them are already gone as I install them by hand) [13:50:59] that means, if they are not found, puppet never installed them, which means they are missing in puppet [13:51:19] or puppet is broken? who knows... :o [13:53:30] petan: No, I mean someone complained recently that python-oursql was missing on exec-node-something, I installed it and submitted a Puppet patch. Have there been other complaints? [13:53:44] no idea [13:53:47] I wasn't here [13:54:21] Then there were none, and work's done. [14:01:22] scfc_de: there is still that todo list I guess [14:01:49] exec nodes should be identical in all ways or we can get in unexpected troubles [14:02:04] at least when it comes to packages and their versions [14:06:26] petan: Eh, you wrote that todo list. I have no idea why for example ed is installed, but I don't think we should puppetize that or other dependencies unless a user explicitly asks for it and we want to commit to it. [14:07:25] scfc_de: that list was made from the generated report I linked you to. It is a list of packages that are somewhere but are not somewhere else. That is a huge problem. I don't insist on having these packages, but if we don't want them, they shouldn't be anywhere, exec nodes must be 100% same [14:07:52] ed is standard UNIX component, it should be on every unix system [14:08:00] just as sed or grep [14:09:48] petan: And I disagree with that this was "a huge problem" based on the experience with exec-node-07 and exec-node-08 which were set up perfectly (except for python-oursql). [14:10:30] it is not a huge problem for you, but for target users it is. Some of their jobs may need these packages, and these jobs will work randomly and randomly crash because environments will differ [14:11:01] Coren: do you think it makes sense to have exec nodes all same? I think it does... [14:11:35] otherwise we produce very unreliable and unpredictable exec environment [14:12:25] btw that script that check consistency of exec nodes was made by anomie not me... so at least there is 1 more person who has similar opinion on this [14:12:29] petan: It's not as important as you think; you only want to puppetize explicit user requests. Anyone who relies on a package that is only indirectly installed (by default, say, or because it's a recommended or dependency) is asking for trouble anyways since we don't wipe and reinstall everytime we change something. [14:12:56] I.e.: the nodes will diverge except where explicitly specified. [14:13:46] ok but what if some user doesn't know they need some package, they test run their job, it will get submitted to exec-03 which has the package they need. Then they decide everything is ok, they cron in and stop watching it. The job will randomly crash. That's not a good system. [14:14:41] we should make sure that all these 8 OS are identical in every possible way if you really want to distribute jobs randomly and not stick them on same server all time [14:15:01] basically jobs should see any difference in these environments... [14:15:05] * shouldn't [14:15:18] petan: That's not how it works in practice, short of wiping and reinstalling at interval. [14:15:51] It's possible iff all the nodes were installed simultaneously, from the same image, and are always updated exactly at the same time. [14:16:13] If a tool has a dependency, then that dependency needs to be made explicit and not rely on happenstance. [14:16:31] if you really want to keep doing this you will have more confused users with randomly crashing jobs than just that guy who couldn't run their job on some nodes (few days ago) [14:16:51] but I am not talking about automatic updates, nor the default packages etc [14:16:58] I am talking about packages that were installed in past by hand [14:17:06] petan: There are none. [14:17:11] did one of you even had a look on that huge list? [14:17:13] @notify valhallasw [14:17:13] I'll let you know when I see valhallasw around here [14:17:21] http://tools.wmflabs.org/admin/packages.html [14:17:27] csh [14:17:40] petan: Yes. What you are seeing is the effect of normal drift, packages that were installed to test things but were not made part of the setup and so on. [14:17:41] this was requested many times, I even think I have had insert that to gerrit [14:17:49] it probably got lost somehow [14:18:04] these are even packages that people requested [14:18:06] petan: Changes don't "get lost somehow" [14:18:13] petan: Are they in puppet? [14:18:19] cython - I remember someone wanted me to install this in past [14:18:25] I don't know [14:18:41] I think I submitted them but got never merged for whatever reason [14:18:46] petan: Well, if someone asked you to install it then you should have put it in puppet. And that shouldn't be on exec nodes anyways. [14:19:22] these exec nodes don't even have mysql client [14:19:24] petan: Where is the bz asking for cython on exec nodes? [14:19:34] petan: Because they have mariadb-client. [14:19:40] I am almost sure if you add 7 and 8 to grid now lot of jobs will start crashing [14:19:53] Coren: people asked on irc, not in bugzilla [14:20:15] anyway, I see no mariadb client on 07 [14:20:29] * Coren is still setting those up. [14:21:09] php5-cgi is missing on 07 [14:21:17] this is surely mandatory component for many users [14:21:36] tcl is missing on both [14:21:56] there is batton of things missing, why don't we just make all nodes same? it's pretty simple task to do [14:23:54] Actually, php-cli missing is interesting. It /is/ in puppet yet has not been applied. [14:23:58] well, I don't really care, my jobs don't need python libraries nor php, tcl, csh or whatever, but if people start coming here and complaining about tools project being unstable I will point them to you :P [14:24:15] petan: BECAUSE IT IS IMPOSSIBLE TO DO WITHOUT WIPING AND REINSTALLING THE NODES EVERY TIME YOU ADD ONE. [14:25:12] why not... you can always install ed, csh and all missing packages? no need to wipe or reinstall anything [14:25:23] I am not talking about system packages [14:25:29] some of them will always differ that makes sense [14:25:31] The versions, dependencies, and recommends vary over time. [14:25:44] yes I know, but I am talking about important ones, like interpretors [14:25:45] calm down, Coren. The trick is to not take petan that seriously :) [14:25:46] If you need something on all nodes, then it must be in puppet. Period. [14:26:01] hmm [14:26:24] !whyismypackagegone:'( is If you need something on all nodes, then it must be in puppet. Period. [14:26:25] Key was added [14:26:37] !why [14:26:37] If you need something on all nodes, then it must be in puppet. Period. [14:26:42] here we go [14:26:56] universal answer to people who will come soon :P [14:26:57] * Coren nows goes to figure out why puppet seems to forget php-cli and tcl for some reason. [14:27:45] Hmmm? php5-cli is installed? [14:28:07] That's only a list of discrepancies. [14:28:23] YuviPanda: Hi! How's the painkiller working? [14:28:39] scfc_de: nothing today! I can move my head around without it messing up my ears [14:28:41] so yay! :) [14:30:54] YuviPanda: Perfect :-). I spent a huge amount of time last night to use the Ubuntu nginx packages, then found out that they don't allow coroutines (as you said), then looked at the Raring version, but its Perl dependency is missed by three patchlevels IIRC. So after spending all this time to avoid packaging, today I'll do exactly what you asked for: Package nginx + lua module in a recent version. [14:31:12] scfc_de: woohoo! :) [14:31:28] scfc_de: yes, the ubuntu packages seem a bit... large? nginx-extras has a lot of stuff I don't need [14:32:32] YuviPanda: My first try will be the shipped Ubuntu source package and upgrade the Lua module there. [14:33:39] YuviPanda: BTW, noticed https://gerrit.wikimedia.org/r/#/c/78002/ of course only *after* I copied the files manually from project-proxy ... proxy-project? You know what I mean :-). [14:33:49] * Coren frowns at puppet. [14:34:26] scfc_de: hehe, I keep them up to date (except for changes in file paths) [14:42:51] Aha. Ze fault, she is Ubuntu [14:43:38] 'tcl' isn't a real package, it's a dependency that doesn't actually provide an update-alternative install. [14:44:15] YuviPanda: BTW, so that my time wasn't wasted :-), the way to force Puppet to pull a package from a particular repo is create a File Puppet resource in /etc/apt/preferences.d/ that pins the package from "Pin: release a=precise" with "Pin-Priority: 2001", then in the package requirement for nginx-extras "require" the File resource. That means the *.pref is created before the package is installed. [14:44:52] scfc_de: hmm, so we make a deb and put it in puppet? [14:45:18] petan: And no, php5-cgi isn't supposed to be a dependency for anything. [14:46:18] YuviPanda: That'll be the question after I packaged it. I'm not sure what's needed to push something into the apt.wikimedia.org repo. But first things first. [14:46:28] heh, true [14:56:45] Coren: If you want to resolve those metapackages, libxslt-dev and libpng3-dev occur in every puppet run. [16:13:43] !ping [16:13:43] !pong [16:13:47] okay, i'm connected. [16:14:05] hi YuviPanda and welcome to being connected :) [16:14:20] sumanah: heh, *all* the channels I'm in were just eerily silent for about...2 minutes [16:14:37] I should start being on some nonwiki ones... [16:15:12] I like #geekfeminism myself [16:15:47] sumanah: I am just on #guardianproject and #nixal. [16:56:14] [bz] (8RESOLVED - created by: 2Tim Landscheidt, priority: 4Unprioritized - 6normal) [Bug 52258] Obsolete scripts in /usr/local/bin need to be "managed" - https://bugzilla.wikimedia.org/show_bug.cgi?id=52258 [16:57:12] Coren: so tools-proxy should replace tools-webproxy, and should send to 'one of the apache boxes' when it doesn't find a mapping for current URL it is being hit with [16:57:15] so for users, nothing will change [16:57:44] That sounds about right. [16:58:29] alright [16:58:51] Coren: so it already doesn't pass off IP addresses, should it strip anything else? [16:59:18] PandaInABox: No, the rest is done at the apache level. [16:59:25] oh! [16:59:30] so that's fine, then [16:59:40] Coren: well, this will also proxy things to non-apache things :) [16:59:41] so [16:59:48] this should also strip whatever it is that we need stripping [17:00:11] * Coren ponders. [17:00:42] Coren: so we should move it out of apaches into the proxy, I'd think :) [17:00:45] for consistency [17:00:46] IP is really the only critical one that I filtered at the proxy level. [17:00:54] The rest is cosmetic. [17:00:59] well, okay then! [17:01:02] it's good enough :) [17:03:08] Coren: I'll setup a role once scfc_de is done packaging :) he's close, I hear [17:06:37] (03CR) 10coren: [C: 032] "Yeay proper packaging!" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71112 (owner: 10Tim Landscheidt) [17:11:18] (03CR) 10coren: [V: 032] "Yeay proper packaging!" [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71112 (owner: 10Tim Landscheidt) [17:11:52] (03PS2) 10coren: Move test suite from debian/rules to autotools [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71114 (owner: 10Tim Landscheidt) [17:15:19] (03CR) 10coren: [C: 032 V: 032] "Works fine." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71114 (owner: 10Tim Landscheidt) [17:15:42] (03PS4) 10coren: Add tests for jsub [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71115 (owner: 10Tim Landscheidt) [17:18:54] (03CR) 10coren: [C: 032 V: 032] "changelog will need to be updated before we can deploy, though." [labs/toollabs] - 10https://gerrit.wikimedia.org/r/71115 (owner: 10Tim Landscheidt) [17:25:58] [bz] (8RESOLVED - created by: 2Steinsplitter, priority: 4Normal - 6normal) [Bug 52502] deleting of service group - https://bugzilla.wikimedia.org/show_bug.cgi?id=52502 [17:43:43] [bz] (8ASSIGNED - created by: 2Krinkle, priority: 4Normal - 6enhancement) [Bug 49350] Tool Labs: Change logo - https://bugzilla.wikimedia.org/show_bug.cgi?id=49350 [17:45:25] [bz] (8ASSIGNED - created by: 2Carl Fürstenberg, priority: 4Normal - 6normal) [Bug 49366] MariaDB lacks help - https://bugzilla.wikimedia.org/show_bug.cgi?id=49366 [17:47:34] [bz] (8ASSIGNED - created by: 2Daniel Schwen, priority: 4Unprioritized - 6blocker) [Bug 52944] Please enable mod_fastcgi support - https://bugzilla.wikimedia.org/show_bug.cgi?id=52944 [17:55:53] scfc_de: What is https://gerrit.wikimedia.org/r/#/c/77144/ intended for? Exec hosts already do HBA and your patch seems to turn /Kerberos/ on. :-) [17:57:06] Coren: *Outgoing* HBA. Otherwise, from a fresh bastion you can't access other hosts. [18:10:47] scfc_de: any luck packaging? :D [18:15:53] heh. yeah. why is GSSAPI enabled? :) [18:16:09] GSSAPI? [18:16:20] Coren, scfc_de: ^^ [18:16:28] YuviPanda: in the ssh config [18:16:43] also, won't that ssh config clash with the system one? [18:16:51] or do we not manage that in the cluster? [18:21:44] Ryan_Lane: No, /etc/ssh/ssh_config isn't managed by Puppet. (Ha, a typo in the commit message!) [18:21:52] heh [18:23:45] Re GSSAPI, that's standard Ubuntu. I only changed the two options mentioned in the commit message (HostbasedAuthentication and EnableSSHKeysign). [18:24:00] Ryan_Lane: do you know if I can find someone who will volunteer to write the wikitech patches for the proxy? :) [18:24:09] YuviPanda: Haven't fully returned yet :-). [18:24:48] scfc_de: I was trying to get him to volunteer himself :P [18:25:35] YuviPanda: wikitech patches = UI? [18:25:41] scfc_de: yes!@ [18:25:45] UI, in PHP [18:34:42] YuviPanda: I think andrewbogott_afk will be willing to do so [18:34:48] woo, super. [18:34:52] but he's not around [18:34:56] he may still be on vacation [18:35:07] Ryan_Lane: yeah, i think he is. we spent time together after everyone left. [18:35:28] Ryan_Lane: but yeah, I hopefully can get everything else in order before he comes back. Just the package now. [18:38:52] cool :) [19:28:04] hey scfc_de [19:28:14] Hey YuviPanda! :-) [19:28:19] hey Ryan_Lane - scfc_de has finished packaging it, now 1. where do we put it? 2. naming? [19:28:30] this is a backported version? [19:29:04] i think so [19:29:15] although we'd probably want to make more changes to it later [19:29:17] I want to see what's been changed. it's a possibility we'll want to use this in production [19:29:20] (specifically replace liblua with luajit) [19:29:29] Ryan_Lane: No, to the future (nginx_1.2.6). [19:29:40] oh. this is an unreleased version? [19:29:52] I'd say use a local repo for now [19:29:52] scfc_de: wait, what? 1.2.6? that doesnt' support websockets! [19:30:01] I think. let me confirm [19:30:10] Ryan_Lane: No. The source package from Raring recompiled on Precise. [19:30:14] ah ok [19:30:26] scfc_de: yeah, need at least 1.3 [19:30:44] eh? why wouldn't it support websockets? [19:30:53] Ryan_Lane: websocket support wad added in nginx 1.3 [19:30:54] it supports HTTP 1.1 to the backend [19:31:02] not websocket proxying. [19:31:08] that should be all that's necessary for websockets, right? [19:31:22] Ryan_Lane: nope. It needs to handle the 'Upgrade' header properly, and then switch *out* of HTTP [19:31:29] into websockets, which are persistant long-lived connections [19:31:42] Ryan_Lane: http://nginx.com/news/nginx-websockets/ is their 'press release' announcing how 1.3 supports websockets. [19:31:55] Okay, then I'll have to forthport. [19:32:23] scfc_de: is wikistream-tools running your package? [19:32:26] or the one I hand-compiled? [19:32:58] I only tested with "nc -C localhost http" on proxy-pure-2. [19:33:04] ah, okay. [19:33:19] Is that accessible via instance-proxy or does that kill the stream? [19:33:27] scfc_de: should kill the stream, I'd think. [19:33:44] YuviPanda: At the instance-proxy point? [19:33:46] scfc_de: of course, we can proxy from our current proxy to the new proxy so we can proxy from there to the instances... :P [19:33:58] scfc_de: yes. instance-proxy is nginx but isn't setup for websocket proxying, IIRC [19:34:17] YuviPanda: It's just for testing whether the packaged version supports websockets or not (as far as I understand the problem). [19:34:30] scfc_de: yeah, but we know for a fact that it isn't supported until 1.3 [19:34:36] http://nginx.com/news/nginx-websockets/ [19:36:01] Ryan_Lane: Other question re naming: Having Ubuntu's nginx, WMF's nginx and nginx-1.4 all be named "nginx" is very confusing. To avoid conflict, I'm thinking about naming it "wmflabs-labsproxy-nginx{,-common,-extras,...}" with debian/controls "Conflict: nginx". WDYT? [19:36:14] why? [19:36:20] just stick it into a local repo [19:36:52] if by 'local repo' you mean a repo on the machine, wouldn't that be lost when we kill and re-create it? [19:37:00] or... am I missing something? [19:37:09] https://wikitech.wikimedia.org/wiki/Help:Using_debs_in_labs [19:37:21] ah, of course. documentation! [19:37:23] there's no reason to name it something else [19:37:31] our repo is pinned [19:37:42] and this repo would be pinned higher than ours [19:37:56] so, when you go to install nginx, if you use a local repo it'll install your version [19:38:00] if not, it'll install ours [19:38:09] that's the reason we use the same name [19:38:24] we may want to use this same version in production [19:38:36] if so, we'll import the package and you can remove the local repo [19:40:16] Ryan_Lane: Okay. Coren: There's a standard for local repos! :-) We should convert /data/project/.system/deb/ some time. [20:24:48] scfc_de: Except that I don't want non-tools directories in /data/project, hence .system [20:25:10] scfc_de: how's the forwardporting(?!) going? [20:36:05] Coren: +1; I'll look into whether a parameter would be useful for labsdebrepo. [20:37:11] YuviPanda: Picked misc::package-builder while I was at it, and that took *long*. Now getting to know the lay of this land. [20:39:42] scfc_de: :D [20:44:25] valhallasw: Hi! Is the Gerrit Reviewer Bot down? [20:49:02] scfc_de: let me check. [20:49:32] scfc_de: I had plans to move it to labs becaus the process seems to die at twn.net [20:50:04] scfc_de: ok, running again. [20:50:35] valhallasw: Will it process the changes "missed"? [20:51:38] scfc_de: yes, except for merged/abandoned changes [20:53:07] valhallasw: Perfect. Thanks! [20:57:32] When signing up for an account on wikitech ... in which LDAP fields do the form's �Username� and �Instance shell account name� get stored in? [21:01:46] Is that �Username� -> sn, cn. �Instance shell account name� -> uid? [21:07:38] Ryan_Lane: ^ [21:12:06] qchris: That's basically it. [21:12:29] Coren: Awesome. Thanks. [21:30:27] qchris: wiki username = cn [21:30:33] shell account name = uid [21:30:55] all those question marks make your questions basically unreadable :D [21:31:04] Ryan_Lane: Cool. Thanks. [21:31:18] Ryan_Lane: Which question marks are you talking about [21:31:36] Is that ?Username? -> sn, cn. ?Instance shell account name? -> uid?" [21:31:48] s/^/"/ [21:31:49] Oh ... those are quotes .. [21:31:56] At least on my screen. [21:32:01] heh. not on mine :D [21:32:09] Sorry :-/ [21:32:29] FreeNode mangles my utf-8 quotes? [21:32:38] Harr :-( [21:33:01] :D [22:14:42] scfc_de: any luck? :) [23:02:49] * YuviPanda pokes scfc_de