[07:41:35] akosiaris jynus still ok for m1 failover in 20 minutes? [07:44:23] one second I check bacula status [07:44:44] I got interrupted on preparations due to clinic duty (surprise) [07:45:13] marostegui: +1 [07:47:09] jynus: We can delay it if you need more time to stop bacula and all that jazz [07:47:23] it is ok, only offsite jobs are running [07:47:37] those are not as important to cancel [07:48:22] do you have the ticket handy? [07:48:34] yep [07:48:39] https://phabricator.wikimedia.org/T256717 [07:48:43] thank you! [07:53:18] so that should be it [07:53:27] are you going to restart etherpad quickly this time? [08:20:00] akosiaris: so what is the aproximate schedule for work on otrs? [08:20:48] as in, are you going to work on that right away? [08:21:52] jynus: I am mostly exploring what needs to be done these days. Chances are I 'll be ready sometime next week [08:22:14] ok, that actually fits better with blockers on our end [08:22:22] \o/ [08:22:40] with our current db automation, copying db stuff around should not be a huge issue [08:23:17] although let me sync with manuel about idp befor commiting to something concrete [08:24:53] ^sounds a reasonable deadline, marostegui, me taking over db1077 next week? [08:25:31] I won't definitely be able to help with idp or otrs this week [08:25:44] jynus: Sure, you can take db1077 next week [08:25:59] jynus: I will move idp database from db1077 to misc this week [08:26:07] not pressuring you, ok? [08:26:14] just asking [08:26:20] 0:-) [08:26:37] :) [08:26:38] I won't be able to do nothing this week [08:27:32] akosiaris: so after I get the hardware, no blocker on me just setting up a snapshot, right? [08:27:39] btw, I can do it as well. my mysql skills are pretty rusty, but I 'll probably be able to do a mysqldump and feed it to another host [08:27:51] well, I do not doubt that [08:27:58] but we have not more modern tools [08:28:10] if you want to learn those, I would be happy to show you too [08:28:23] s/not/now/ [08:28:38] I keep making that type arg [08:28:52] sure, why not? Maybe I 'll get something out of this OTRS pain [08:29:02] got to look at the bright side. /me learning! [08:29:10] so we use now mydumper instead of mysqldump [08:29:31] but "snapshotting" with xtrabackup will be just easier and faster [08:31:07] we also have a more streamlined (although net yet finished) provisioning service [08:31:34] will ping you when db is ready [09:07:27] FYI, I'll switch graphite.w.o to CAS shortly [09:18:25] !log remove eqord-eqiad tunnel - T254877 [09:18:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [09:24:04] !log renumber eqord NTT link - T254877 [09:24:07] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [09:31:17] hi, for the puppet catalog compiler we can add a Hosts: field in the commit message. There are a way to select a profile or role or even regex, but I could not find a doc about all those parameters. Does anyone know about it? ( jbond42 maybe? ;) ) [09:31:33] hashar: yes [09:32:02] i see an example of `Hosts: P:puppetmaster::frontend` used by jbond [09:32:57] yeah then I have seen usage of Hosts: O:idp and I am not sure what 'O' stands for [09:33:07] hashar: i expect it's related to the cumin query language [09:33:12] O in that case would stant for rOle [09:33:16] maybe that is for the roles, which is what I am looking for ;] [09:33:22] OH [09:33:23] https://wikitech.wikimedia.org/wiki/Cumin#PuppetDB_host_selection [09:33:50] R was already owned by Resource :-) [09:34:32] I didn't know that the puppet compiler used cumin underneath [09:34:33] yes O is for roles, PCC supports the following https://wikitech.wikimedia.org/wiki/Help:Puppet-compiler#Host_variable_override [09:34:38] I don't even want to know about how the compiler manages to understand cumin selectors hehe [09:35:12] perfect excellent, thank you for the documentation links! [09:35:42] for the record it dosn't use cumin underneath although i plan to add it (and i can here vol.ans reminding me i should have done it to begin with :).) it currently just supports a simplified syntax [09:36:57] yeah no complains :] profile/role/hostname regex are already a very very nice improvement [09:47:26] Hosts: O:deployment_server works like a charm [10:46:11] <_joe_> jynus: it does not [12:01:24] !log renumber eqiad NTT link - T254877 [12:01:27] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [12:08:13] godog: xionox-tmp: [12:17:37] !log rollout less frequent option-refresh-rate - T240658 [12:17:41] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [12:17:42] T240658: fastnetmon spamming /var/log on netflow hosts leading to disk saturation - https://phabricator.wikimedia.org/T240658 [12:42:45] that's one supremly useless error, pyprospector: vulture: failure / Tool vulture failed to run (exception was raised) [12:54:39] TIL: openssl s_client -starttls mysql [12:55:13] jbond42: huh! [12:56:03] neat [12:59:34] fyi cdanis i updated the known_hosts CR creating a seperate file for each algo. i think they can all go in one file however sshd(8) has the following "It is permissible (but not recommended) to have several lines or different host keys for the same names." [12:59:55] yeah I haven't tried that myself, not sure how it picks amongst them if there are multiple [13:00:41] no me neither i figure have all, update the known_host_update script to pick the strongest but then the others are there if people want them [13:01:34] moritzm ^^: may know [13:08:59] mmmh, no idea about it's matching strategy there [14:15:41] fyi about to switch icinga authentication to CAS SSO [14:16:50] ldap-icinga.wikimedia.org is avalible using the old method just in case [14:18:03] but now restricted to cn=ops compared to the old setup [14:19:34] ack, thanks for the heads up [14:23:56] updated let me know if you see any issues [14:24:34] works for me [14:25:40] hmmm [14:25:46] I think I have a problem with icinga and the CAS... [14:26:02] * vgutierrez trying to confirm [14:26:03] if you're having a redirect loop vgutierrez you can clear cookies for icinga.wm.o and idp.wm.o [14:26:26] it is working for me though [14:27:19] hmmm nope, that's not my problem [14:27:44] so for the CAS I'm "vgutierrez" but I need to be "Vgutierrez" or I get read-only access to icinga :( [14:28:08] icinga has a separate list of usernames to give read/write access to, right? [14:28:09] vgutierrez: yes thats an issue with icinga/ldap [14:28:13] maybe that just needs to be updated? [14:29:24] yes you could fix it by adding both to the cgi.conf file. however my understanding is that issue already exists with the basic auth ldap? [14:30:06] that said i do have this ticket https://phabricator.wikimedia.org/T256656 [14:34:36] yeah, the same issue also affected the LDAP auth before, I reproed it with my user last week [14:34:57] the permissions in the cgi config are case-sensitive and the LDAP auth is case-insensitive [14:40:43] yup