[00:21:02] Negative24: yeah [00:21:17] Negative24: still reading docs though [00:21:34] einyx: Good. What happened to be the problem? [00:22:39] Negative24: not sure, I think someone mentioned that my home directory was missing, then after removing/readding my key it worked [00:23:15] Ah, yes. I saw that your home dir wasn't created but you were in /etc/passwd [00:23:25] didn't think much of it though [00:23:48] Well let me know if you need anything [00:24:51] cheers, I'll do [00:44:57] 6Labs: Validate puppet variables of Labs instances - https://phabricator.wikimedia.org/T93460#1137613 (10Tgr) 3NEW [02:48:23] twentyafterfour: you around? [02:55:08] 6Labs: Validate puppet variables of Labs instances - https://phabricator.wikimedia.org/T93460#1137745 (10scfc) The user gets feedback, just not immediate :-). There are two issues: # Most configuration values somehow get pasted in a configuration file. It is virtually impossible to detect all invalid values.... [03:08:12] Negative24: I'm here [03:09:09] twentyafterfour: could I be added to the phabricator labs project? [03:09:22] ok [03:09:36] wiki username? [03:09:42] Negative24 [03:09:46] nice :) [03:09:57] yeah I try to be simplistic :) [03:12:29] could you also build a m1.small instance named phab-02 [03:14:36] Negative24: so can I help you with testing? I read the buffer on wikimedia-devtools and I see you were wanting to test the security extension. I wrote that one so I can probably be of help [03:15:23] err no not -devtools, wait which chat was that :) [03:15:31] ops [03:16:13] I'm actually testing the infamous herald and security extension bug. I'm more looking at the security model with checking the SAL [03:17:43] I just need a phab installation with the extension running [03:18:56] the set security to none? [03:19:26] No the leak private info one. Although that one is quite annoying [03:23:59] So somehow I can't figure out how to add an instance on phabricator project [03:24:16] Negative24: I added you to the project, trying to figure out why I can't find the add instance link anywhere [03:24:33] Sidebar "Manage Instances" [03:24:41] do a ctrl-F [03:28:13] yeah the link is not there [03:29:30] I don't quite understand why, since I'm in the admin group for that project [03:29:41] Yeah you are sysadmin [03:30:01] Try skipping the link by going to https://wikitech.wikimedia.org/wiki/Special:NovaInstance [03:31:29] twentyafterfour, try logging out and back in? [03:32:27] Krenair: will do, though if that is frequently required then something needs fixing [03:32:45] Indeed. [03:32:55] and that worked [03:33:07] :) [03:33:12] Nova is funny at time [03:33:15] *times [03:36:56] twentyafterfour, yeah, unbreak now - https://phabricator.wikimedia.org/T89808 [03:37:06] unfortunately, it's in Wikimedia-Labs-wikitech-interface [03:38:08] 6Labs, 10Wikimedia-Labs-wikitech-interface: Wikitech registration requires labs shell access - https://phabricator.wikimedia.org/T88092#1137755 (10Krenair) >>! In T88092#1003889, @Tgr wrote: > Also the registration form is enclosed in ``/`` - don't know if that's intentional or a bug, but... [03:38:47] 6Labs, 10Wikimedia-Labs-wikitech-interface: Wikitech registration requires labs shell access - https://phabricator.wikimedia.org/T88092#1137757 (10Krenair) Is it not possible to do this check with an LDAP query? [03:42:19] 10Wikimedia-Labs-wikitech-interface: Include role::analytics::hadoop roles in default list of labs puppet groups - https://phabricator.wikimedia.org/T70391#1137762 (10Krenair) [03:45:19] 10Wikimedia-Labs-wikitech-interface, 10Wikimedia-Site-requests: Add "Nova Resource" to default search namespaces on wikitech.wikimedia.org - https://phabricator.wikimedia.org/T67132#1137764 (10Krenair) [03:47:37] 10MediaWiki-extensions-OpenStackManager: "Failed to create service group" error when creating service groups but group appears in list on Special:NovaProject - https://phabricator.wikimedia.org/T49336#1137771 (10Krenair) [03:49:53] 6Labs, 10Wikimedia-Labs-wikitech-interface, 5Patch-For-Review: Keystone tokens truncated when wikitech stores them - https://phabricator.wikimedia.org/T92014#1137775 (10Krenair) Did this work? [03:50:31] 6Labs, 10Wikimedia-Labs-wikitech-interface: Keystone tokens truncated when wikitech stores them - https://phabricator.wikimedia.org/T92014#1137776 (10Krenair) [03:53:59] twentyafterfour: hows it going? [03:58:57] Negative24: phab-02 is up and running [03:59:11] phab-02.eqiad.wmflabs [03:59:13] twentyafterfour: Thanks [03:59:29] twentyafterfour: I see you had a bit of trouble :) https://gerrit.wikimedia.org/r/#/c/198461/ [03:59:35] it's already puppeted and the phabricator schema applied [03:59:43] Great! [03:59:53] yeah not a big problem but I wanted to fix it in puppet while I'm thinking of it [04:02:04] 10Wikimedia-Labs-wikitech-interface, 6operations: wikitech instances list is blank - https://phabricator.wikimedia.org/T89808#1137778 (10mmodell) a:5mmodell>3None [04:02:41] 10Wikimedia-Labs-wikitech-interface, 6operations: wikitech instances list is blank - https://phabricator.wikimedia.org/T89808#1045882 (10mmodell) p:5Unbreak!>3Triage Just encountered this bug again myself. [04:05:45] twentyafterfour: Thanks for your help. I'll probably get to it tomorrow [04:06:16] I have to sleep sometime. What timezones are you guys on? [04:10:15] Negative24: I'm in UTC-5 (us central) [04:11:31] Ah well its only 8 there. I'm on MDT [04:12:40] 23:12 here actually [04:13:34] Negative24: I should be around part of the day tomorrow [04:14:08] twentyafterfour: Yeah my bad [04:14:19] twentyafterfour: I will see you tomorrow then [04:16:57] 6Labs, 10Wikimedia-Labs-wikitech-interface: Wikitech registration requires labs shell access - https://phabricator.wikimedia.org/T88092#1137791 (10scfc) @Krenair: It is, cf. http://git.wikimedia.org/blob/mediawiki%2Fextensions%2FOpenStackManager.git/master/nova%2FOpenStackNovaUser.php#L508. At the moment, the... [06:04:49] 6Labs: Validate puppet variables of Labs instances - https://phabricator.wikimedia.org/T93460#1137852 (10yuvipanda) 5Open>3declined a:3yuvipanda I think my preferred solution is for Wikitech / OpenStackManager to die :D And puppet code should validate its inputs as well - setting a puppet variable wrong sh... [06:05:56] 10Wikimedia-Labs-wikitech-interface: Include role::analytics::hadoop roles in default list of labs puppet groups - https://phabricator.wikimedia.org/T70391#1137856 (10yuvipanda) @ottomata Is this still something you think we should do? :) [06:07:54] 10Wikimedia-Labs-wikitech-interface: Include role::analytics::hadoop roles in default list of labs puppet groups - https://phabricator.wikimedia.org/T70391#1137858 (10yuvipanda) a:3Ottomata Ah, so I've made Otto cloudadmin, and he can add the groups himself :D BWAHAHA :D [06:08:09] 10Gerrit-Patch-Uploader, 7Easy: Serve static resources from //tools-static.wmflabs.org or /static/ project - https://phabricator.wikimedia.org/T86354#1137861 (10Krinkle) Afaik that never worked. Such directory didn't exist. Where is that url being used? The proper url to that resource is (and afaik always ha... [06:09:10] 10Gerrit-Patch-Uploader, 7Easy: Serve static resources from //tools-static.wmflabs.org or /static/ project - https://phabricator.wikimedia.org/T86354#1137862 (10Krinkle) [06:14:25] 10Wikimedia-Labs-wikitech-interface, 6operations: wikitech instances list is blank - https://phabricator.wikimedia.org/T89808#1137864 (10yuvipanda) p:5Triage>3Normal [06:15:10] 10Tool-Labs: Enable "Access-Control-Allow-Origin: *" header on tools-static.wmflabs.org - https://phabricator.wikimedia.org/T93466#1137866 (10Krinkle) 3NEW [06:21:49] 10Tool-Labs: Enable "Access-Control-Allow-Origin: *" header on tools-static.wmflabs.org - https://phabricator.wikimedia.org/T93466#1137875 (10yuvipanda) Should be trivial, but remember to not restart any of the nginx instances before T93212 is done :) [06:34:10] 6Labs, 10Tool-Labs: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1137879 (10yuvipanda) Done on tools-webproxy-01, -02 and -static. Need to do dynamicproxy-gateway. [06:35:57] PROBLEM - Puppet failure on tools-webgrid-tomcat is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:36:36] [13flask-mwoauth] 15philiptzou opened pull request #7: Changed hard code `name` to param with default val (06master...06changed-hard-code-name) 02http://git.io/hPYo [06:47:46] PROBLEM - Puppet failure on tools-exec-09 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [07:00:56] RECOVERY - Puppet failure on tools-webgrid-tomcat is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:53] RECOVERY - Puppet failure on tools-exec-09 is OK: OK: Less than 1.00% above the threshold [0.0] [15:11:06] 6Labs: milimetric and halfak would like postgresql database access - https://phabricator.wikimedia.org/T91267#1138126 (10Halfak) @yuvipanda, did this happen yet? I'm hoping to get started with postgres soon. [15:52:17] twentyafterfour: have you seen this phd daemon start problem before? [15:55:55] Negative24: what start problem? [15:56:26] twentyafterfour: phd isn't started as the user phd and can't because of labs sudo policies [16:00:32] Negative24: no, I'm not sure what to do about that. Is it causing problems? [16:01:05] twentyafterfour: it isn't that big of a deal in labs but in production i'm sure it would be a bit of a problem [16:01:07] I don't see any way to grant sudo to phd because it only lets me add ldap users that are in the project [16:01:24] Negative24: in production I'm pretty sure that problem doesn't occur [16:01:33] phd doesn't need sudo. ldap users need to be able to sudo to phd [16:02:08] I'll research a bit further and tell you what I find [16:02:48] I was just seeing if you've had this issue before [16:09:37] 10Tool-Labs: Unattended upgrades are failing from time to time - https://phabricator.wikimedia.org/T92491#1138167 (10scfc) Not stricly unattended upgrade, but maybe related: ``` From: root@tools.wmflabs.org (Cron Daemon) Subject: Cron test -x /usr/sbin/anacron || ( cd / && run-parts --report... [16:47:13] twentyafterfour: Do we still need the dummy redirector on fab-01? https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/manifests/role/phabricator.pp;0ae5511d1f22957892d10c0d2f6aec99a84a54e0$205 [16:47:35] ln 205 [16:52:56] [13flask-mwoauth] 15valhallasw pushed 3 new commits to 06master: 02http://git.io/h1QG [16:52:56] 13flask-mwoauth/06master 140951771 15Philip Tzou: Changed hard coded `name` to parameter... [16:52:56] 13flask-mwoauth/06master 1442b2177 15Merlijn van Deen: Merge branch 'philiptzou-changed-hard-code-name' [16:52:57] 13flask-mwoauth/06master 14194a3fb 15Merlijn van Deen: Bump version to 0.1.37 [16:53:45] [13flask-mwoauth] 15valhallasw closed pull request #7: Changed hard code `name` to param with default val (06master...06changed-hard-code-name) 02http://git.io/hPYo [19:05:03] Negative24: nope [20:18:03] 10Tool-Labs: Add script_path to meta_p.wiki database - https://phabricator.wikimedia.org/T93483#1138365 (10Krinkle) 3NEW [20:29:58] twentyafterfour: sorry I was gone. Should I remove it? [20:56:08] Negative24: I don't think it's hurting anything but it's not really helping anything either ... it's a judgement call, I don't have much opinion ;) [20:56:31] removing for the sake of less clutter isn't a bad idea I guess [20:56:49] twentyafterfour: ok. I just looking for dependencies of the class now [20:56:55] *I'm [20:57:18] and getting sidetracked :)