[02:49:56] 3Wikimedia Labs / 3tools: "webservice stop" doesn't stop php-cgi processes - 10https://bugzilla.wikimedia.org/63878#c2 (10Tim Landscheidt) Idea: Replace "qdel -j $job" with "ssh $WEBGRIDHOST 'kill -TERM $(cat /var/run/lighttpd/wikilint.pid)'". This will make lighttpd shut down in an orderly fashion taking t... [05:59:26] !channels [05:59:26] | #wikimedia-labs-nagios #wikimedia-labs-offtopic #wikimedia-labs-requests [08:40:09] Hoi, the webservice was stopped again, had stopped again.. Is there a way to automatically restart it ? [08:59:11] 3Wikimedia Labs / 3tools: Add support for Java Servlets on Tool Labs - 10https://bugzilla.wikimedia.org/54845#c7 (10Silke Meyer (WMDE)) Hi Marc-André, any news on this? Robin can't migrate his MMP without this feature. [09:01:26] GerardM-: the web service for what project? [09:01:45] and it should automatically restart, actually. [09:02:00] valhallasw the webservice for WDQ [09:08:03] GerardM-: https://wdq.wmflabs.org/wdq/ works for me? [09:13:24] valhallasw it is because Magnus started it by hand [09:13:52] what I am looking for is that it will start automatically [09:15:55] that's something the WDQ admins will have to set up [09:16:28] i.e. Magnus [11:23:59] steinsplitter@tools-login:~/bot$ wget http://tools.wmflabs.org/pywikibot/core.tar.gz [11:23:59] --2014-04-24 11:21:45-- http://tools.wmflabs.org/pywikibot/core.tar.gz [11:23:59] Resolving tools.wmflabs.org (tools.wmflabs.org)... 208.80.155.131 [11:23:59] Connecting to tools.wmflabs.org (tools.wmflabs.org)|208.80.155.131|:80... failed: Connection timed out. [11:23:59] Retrying. [11:24:01] hm :/ [11:25:06] *using cp now* [12:26:39] Coren: it appears pathinfo is no longer supported? [12:28:16] Coren: nope it's something else [12:48:26] 3Wikimedia Labs / 3tools: Add support for Java Servlets on Tool Labs - 10https://bugzilla.wikimedia.org/54845#c8 (10Marc A. Pelletier) It's working, but currently requires a lot of manual work to get it running. I'm hoping I can have a script that makes it similar to the lighttpd setup. [12:59:01] RPiotrowski: From inside Labs, you need to connect to tools-webproxy. [12:59:23] ah, k. thx [13:05:35] Hm.. I just got a "Failed to create instance" error [13:06:03] and again [13:06:07] https://wikitech.wikimedia.org/w/index.php?title=Special:NovaInstance&action=create&project=cvn®ion=eqiad [13:06:15] cvn-dev, m1.large, [x] default [13:07:25] Krinkle: Have you reached your quota? I think it can be displayed on the Manage Projects page. [13:07:45] !log tools tools-mail: rm -f /var/log/exim4/paniclog (relay_domains bug) [13:07:48] Logged the message, Master [13:20:21] scfc_de: I have quota 3/10 [13:25:07] Ah, it's the RAM [13:25:27] and CPU [13:25:31] Interesting, that's done separatly [13:25:41] Krinkle: You can ask andrewbogott or Coren to up the quota or delete an instance that you don't longer use. [13:27:02] Yeah, it doesn't have to be large actually. This is just a dev instance where I'll keep my screen and do compilation of stuff (instead of doing that on app servers directly) [13:27:27] will make writing documentation easier (ssh into cvn-dev instead of cvn-app{4,5}) [13:29:31] Coren: Can I have 2 extra CPU quota on project cvn? [13:29:49] Krinkle: How many you got now? [13:29:52] 20 [13:31:55] Bumped. [13:32:23] ty [13:47:41] 3Wikimedia Labs / 3tools: Soften qdel behaviour from KILL - 10https://bugzilla.wikimedia.org/61102#c1 (10Tim Landscheidt) *** Bug 63878 has been marked as a duplicate of this bug. *** [13:47:42] 3Wikimedia Labs / 3tools: "webservice stop" doesn't stop php-cgi processes - 10https://bugzilla.wikimedia.org/63878#c3 (10Tim Landscheidt) 5NEW>3RES/DUP The problem with this approach would be the dependence on "webservice stop" being the only way to kill a job. If for example the grid would transfer th... [13:48:41] 3Wikimedia Labs / 3tools: Soften qdel behaviour from KILL - 10https://bugzilla.wikimedia.org/61102#c2 (10Tim Landscheidt) We need to use "qsub -notify" in webservice as well. [14:07:22] scfc_de: 61102 should be a dependency, not a duplicate for 63878, right? [14:15:58] valhallasw: In principle you're correct; I had thought about that before changing, but thought it easier to handle it all in one bug. [14:44:08] Hm.. what is the recommended way to deal with permissions errors in labs? [14:44:21] I'm looking for something similar to what we have in production on tin.eqiad [14:44:35] where I can act as myself, but thigns aren't unreadable by other people int he same group [14:44:45] * ^d had same problem yesterday [14:44:58] I'm thinking either a sticky flag on the root of the /data or something bash that does umask. [14:45:01] Or maybe both is needed? [14:45:05] I'm not sure. [14:45:29] Coren: andrewbogott: [14:45:36] ^d: Did you find a work around? [14:45:53] <^d> Briefly chowned the file to myself. [14:46:00] <^d> Which is not a solution. [14:46:04] ha [14:46:44] Krinkle: I'm confused by "aren't unreadable" [14:46:50] Nikerabbit: what is twn channel? [14:46:51] writable rather [14:46:59] ah, ok. [14:47:13] andrewbogott: e.g. there is a git checkout somewhere in /data on cvn-* instances. However none of the other people can do git pull [14:47:26] because it's mostly 755 / 644 krinkle:project-cvn [14:48:06] I don't want to do -R 775/644 on everything either. I think there's a few files (including those managed by git) that intentonally have different permissions [14:48:19] but I can fix that up once if needed [14:49:05] Krinkle: You want SGID on the directory. [14:49:45] Is that what we use for /a/common in production? [14:56:36] Krinkle: I think you need to set umask 002 instead of the default 022 [14:56:54] Do we have a global bashrc shared between a project's instances? [14:57:15] if not I can symlink it to /data I supose [14:57:22] from /etc/ [14:57:30] Krinkle: Coren's SGID solution doesn't require this. [14:57:56] I've found in practice though that SGID didn't work for me. But I'm willing to give it a try [14:58:11] There's always be files messed up when you least expect it [14:58:34] SGID on directory has the net effect that any file created under that directory take the directory's group; and directories get SGID. As long as you don't break this by hand, it'll always work. [14:58:43] Err. SGID sets *which* group will be set for files, not what the permissions for that group will be. [14:59:00] Well yeah, you still want your umask to allow group write. :-) [14:59:51] So that's just chmod g+s /data/project ? And then a one-time pass to fix the existing files I suppose [15:00:05] And it is maintained recursively? [15:00:37] Coren: Hm.. so it's not just safer with both umask and sgid, but both is required for the desired effect? [15:00:41] That's OK, just checking. [15:01:13] Right, any new directories will get the group and SGID to keep this recursively correct. [15:03:25] so chmod g+s on all directories in /data/project, umask 002 from bashrc. [15:03:34] And then a chmod as well, or is the above two things enough? [15:04:29] Did something change with the ApiLogin on beta? It seems that after the two-step login process the session is not remembered anymore (worked till April 17th, started failing from April 18th on). The login is successful but for the following API call I'm logged out again.. [15:04:54] I did not change anything in my code. [15:27:14] 3Wikimedia Labs / 3tools: user_password_expires column is missing - 10https://bugzilla.wikimedia.org/64369 (10Liangent) 3NEW p:3Unprio s:3normal a:3Marc A. Pelletier Other private columns in user table are still kept and just nullified. [19:14:12] 3Wikimedia Labs / 3tools: user_password_expires column is missing - 10https://bugzilla.wikimedia.org/64369 (10Marc A. Pelletier) 5NEW>3ASS [20:49:29] 3Wikimedia Labs / 3tools: Let 404 responses pass through the proxy if they contain contents - 10https://bugzilla.wikimedia.org/64393 (10Liangent) 3NEW p:3Unprio s:3normal a:3Marc A. Pelletier MediaWiki generates useful 404 pages which get caught by the proxy. [23:17:25] (03PS1) 10GeorgeBarnick: Adding configuration for #brickimedia [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 [23:25:40] (03CR) 10Legoktm: [C: 04-1] "Sure. You just need to also add the repo to #wikimedia-dev so it'll show up there as well." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 (owner: 10GeorgeBarnick) [23:26:36] (03CR) 10GeorgeBarnick: "Sure, I'll fix that." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 (owner: 10GeorgeBarnick) [23:30:09] (03PS2) 10GeorgeBarnick: Adding configuration for #brickimedia [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 [23:31:21] (03CR) 10Legoktm: [C: 032] Adding configuration for #brickimedia [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 (owner: 10GeorgeBarnick) [23:31:24] (03Merged) 10jenkins-bot: Adding configuration for #brickimedia [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 (owner: 10GeorgeBarnick) [23:31:27] (03CR) 10Yuvipanda: "Thanks for the patch!" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/129610 (owner: 10GeorgeBarnick) [23:33:47] !log tools restarting grrrit-wm https://gerrit.wikimedia.org/r/129610 [23:33:50] Logged the message, Master [23:38:52] !log tools restarted greg-g after cherry-picking aec09a6f669bc1806557576212aa218bfa520c35 for auth of IRC bot [23:38:54] Logged the message, Master [23:38:59] err [23:39:12] ??? [23:39:19] !log tools restarted grrrit-wm, not greg-g. greg-g does not survive restarts and hence care must be taken to make sure he is not. [23:39:20] * greg-g reboots [23:39:20] Logged the message, Master [23:40:32] greg-g: :) [23:40:44] greg-g: sorry about the accidental ping, and then the three further pings. [23:42:21] YuviPanda: s'ok, it's nice to remind people to take care when restarting me [23:42:58] greg-g: :D [23:43:11] legoktm: cherry-pick for auth is in SAL now [23:43:39] legoktm: actually I created a branch called 'auth'. So you can now just always cherry-pick auth and it should just work [23:43:53] oh perfect [23:44:12] YuviPanda: want to update the wikitech docs now? [23:45:36] legoktm: can you? it's 5:15 AM and I'm in a code haze that if I snap out of I'll fall asleep [23:45:44] ok :P [23:48:55] YuviPanda: https://wikitech.wikimedia.org/w/index.php?title=Grrrit-wm&diff=110735&oldid=110222 [23:50:16] legoktm: cool! :)