[00:08:56] RECOVERY - Check unit status of drop_event on an-launcher1002 is OK: OK: Status of the systemd unit drop_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [06:56:13] !log Rerun cassandra-daily-wf-local_group_default_T_pageviews_per_article_flat-2021-5-29 [06:56:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:47:11] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Request to delete test_gsc_* datasets from Druid (& Superset/Turnilo) - https://phabricator.wikimedia.org/T283536 (10JAllemandou) Done [12:28:56] morning joal :) [12:29:10] i think it’s just the two of us today :) [12:29:11] Hi fdans :) [12:30:08] fdans: That's right [12:30:16] all good for you fdans ? [12:30:47] joal: yep! just back from rowing, great start of the day [12:30:52] \o/ [13:37:59] fdans: I'll be at the vaccination center at standup time - we can talk later or earlier :) [14:42:30] Hi there, [14:42:31] I'm a formal collaborator from EPFL, have had Wikimedia cluster access for several years, but have gotten rusty after not having accessed the cluster for several years. It seems some things have changed since I last used it, so I was hoping someone could help me get re-onboarded... I'm dumping my questions here as a list, but feel free to pick only [14:42:31] those to which you know the answer. Context: I can already access the cluster via stat1006.eqiad.wmnet. [14:42:32] (1) While I can access stat1006, I cannot access stat1004 and stat1008. Is this a bug or a feature? [14:42:32] (2) It seems that, in order to access HDFS, I need a Kerberos ticket, but when I do `kinit`, I get this: "kinit: Client 'west1@WIKIMEDIA' not found in Kerberos database while getting initial credentials". How do I need to proceed to get the necessary credentials? [14:42:33] (3) I'm trying to use Jupyter Hub via an SSH tunnel (`ssh -N stat1006.eqiad.wmnet -L 8000:127.0.0.1:8880`). The tunnel works, but I'm not sure which credentials to use in order to log on. Should I already have them, or do I need to create them somewhere? [14:42:33] (4) I would like to run R on the cluster. Is this possible via Jupyter Hub? My student Tiziano Piccardi told me that there used to be an R kernel, but that he now only sees a Python kernel. [14:42:34] Sorry for the many questions -- any help is welcome! [14:42:34] Bob West [15:11:18] joal: so excited that you're getting vaccinated, you want to talk around your 7pm? [15:17:19] cervisiarius: most of the team is out because of Memorial Day, but I can answer some of your questions: [15:17:19] 1) both stats 1004 and 1008 are up, maybe try removing them from your ssh/known_hosts list? [15:17:19] 2) You need to submit a phab task like this: https://phabricator.wikimedia.org/T283710 [15:17:19] 3) Have you tried your wikitech credentials? AFAIK the LDAP login matches those but I might be wrong [15:17:31] no idea about R sorry [15:31:46] Thanks so much for the quick response, fdans! [15:33:20] (1) I think it's something else, since I didn't even have an .ssh folder in my (currently blank) home dir. I can ping stat1004, but when I try to ssh, nothing happens [15:33:32] (2) Thanks, I just created the ticket! [15:34:59] (3) The Wikitech credentials don't seem to work here. I can log into wikitech.wikimedia.org, but not into Jupyter Hub. From this it seems that I might need Kerberos access before being able to get into Jupyter Hub? [15:35:19] (4) Ok, so I'll wait to see if someone else has any info on this [15:35:49] cervisiarius: for 1, where is your ssh config file located if you do not have a .ssh directory? and what's the exact error message? [15:36:41] majavah: oh, I do have the .ssh dir on my local machine, but was trying to ssh into stat1004 from stat1006. [15:36:51] there's no error message; it just stalls [15:37:50] I just checked, and I can actually ssh into stat1004 directly from my local machine, so I guess I get it now [15:38:41] I guess I need to always go via bastion, and that's not set up on stat1006 [15:44:59] firewalls only allows ssh traffic via the bastions, yes, and there is no host-based ssh authentication setup on production [15:46:05] ok, thanks, makes sense [17:31:36] 10Analytics, 10Event-Platform, 10MediaWiki-Blocks, 10Platform Team Workboards (MW Expedition): extensions / EventBus: replace uses of AbstractBlock ::getTargetAndType() and ::getTarget() - https://phabricator.wikimedia.org/T284027 (10Vlad.shapik) [17:32:58] 10Analytics, 10Event-Platform, 10MediaWiki-Blocks, 10Platform Team Workboards (MW Expedition): extensions / EventBus: replace uses of AbstractBlock ::getTargetAndType() and ::getTarget() - https://phabricator.wikimedia.org/T284027 (10Vlad.shapik) p:05Triage→03Medium [18:18:23] Hi fdans :) [18:18:32] I'm later than I would have liked