[00:48:32] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Dzahn)
[00:53:21] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Dzahn) I see on https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hue " If you already have cluster access, but can't log into Hue, it...
[01:34:36] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10SNowick_WMF) Thanks, yes it is a manual sync process:  The ticket attached to this one says "Currently, Hue users are manually synced from...
[01:39:19] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Dzahn) a:03elukey
[02:11:40] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Xqt)
[02:22:05] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Xqt) An all usage statistic for 2019 is given here:  https://www.jetbrains.com/lp/devecosystem-2019/python/ where one of 10 users still takes...
[03:49:38] <wikibugs>	 (03PS2) 10Milimetric: Encode to pagecounts-ez format with a UDF [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/562597 (https://phabricator.wikimedia.org/T192474)
[03:52:15] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Encode to pagecounts-ez format with a UDF [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/562597 (https://phabricator.wikimedia.org/T192474) (owner: 10Milimetric)
[04:03:19] <wikibugs>	 (03PS3) 10Milimetric: Encode to pagecounts-ez format with a UDF [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/562597 (https://phabricator.wikimedia.org/T192474)
[05:59:43] <fdans>	 good morning team
[07:49:33] <elukey>	 good morning fdans 
[07:50:02] <fdans>	 elukey: helloooo whenever you want we can do the wikistats move
[07:50:57] <elukey>	 fdans: sure, gimme ~30 mins to bootstrap ok? 
[07:51:05] <fdans>	 sounds good!
[07:51:07] <elukey>	 and then I'll be ready :)
[07:51:35] <elukey>	 fdans: yesterday I was thinking to make a list of stat v1 urls to check after the move, to see if they are broken or not
[07:51:49] <elukey>	 sort of unit testing :D
[07:52:03] <elukey>	 can you come up with a good list or urls?
[07:52:05] <fdans>	 yeah that's a good idea
[08:05:03] <wikibugs>	 (03PS3) 10Fdans: Add vue-i18n integration, English strings [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/558702 (https://phabricator.wikimedia.org/T240617)
[08:05:24] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add vue-i18n integration, English strings [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/558702 (https://phabricator.wikimedia.org/T240617) (owner: 10Fdans)
[08:06:19] <wikibugs>	 (03PS4) 10Fdans: Add vue-i18n integration, English strings [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/558702 (https://phabricator.wikimedia.org/T240617)
[08:07:29] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add vue-i18n integration, English strings [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/558702 (https://phabricator.wikimedia.org/T240617) (owner: 10Fdans)
[08:12:50] <fdans>	 elukey: I have a bunch of urls!
[08:15:46] <elukey>	 \o/
[08:15:56] <elukey>	 I am sending emails to people using too much space on notebooks again :(
[08:35:47] <fdans>	 elukey: ban everybody
[08:40:38] <elukey>	 I am wondering if the new 'stat1008' should be 'notebook1005', with a GPU and a lot of disk space like a stat node
[08:40:51] <elukey>	 so people can work on it while we experiment with newpyter
[08:43:24] <elukey>	 anyway, stats.w.o
[08:44:36] <elukey>	 ah snap we haven't cleaned up the old stuff yet
[08:44:40] <elukey>	 just remembered
[08:46:13] <elukey>	 I am re-reviewing the task, I still have some doubts
[08:57:51] <fdans>	 elukey: cleaned up the old stuff?
[08:58:20] <elukey>	 fdans: yes the old geowiki things etc..
[08:58:26] <elukey>	 from the vhost config 
[09:01:42] <elukey>	 fdans: time to bc for a sec?
[09:01:43] <wikibugs>	 10Analytics, 10Analytics-Kanban: Make stats.wikimedia.org point to wikistats2 by default - https://phabricator.wikimedia.org/T237752 (10elukey) >>! In T237752#5667573, @elukey wrote: > The other idea that me and Fran had about moving on with the stats.w.o -> v2 transition is the following: >  > 1) Add somethin...
[09:01:49] <elukey>	 see --^
[09:03:09] <fdans>	 elukey: omw
[09:16:03] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Urbanecm) a:03Urbanecm Maybe I'm doing something wrong, but I logged to [Turnilo](https://turnilo.wikimedia.org) and sub...
[10:01:11] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Dvorapa) This seems similar to core/compat times. One of the reasons is that help pages and docs suggest `python pwb.py`,...
[12:13:21] <neilpquinn>	 ottomata: re https://wikitech.wikimedia.org/w/index.php?title=Analytics/Systems/Superset&diff=next&oldid=1837211, does this mean we can remove the part about accounts not being automatically created from the access section?
[12:15:30] <elukey>	 neilpquinn: the account is created when you first log in by superset now (do you remember the horrible http 500 used to be returned for first time users?)
[12:16:12] <elukey>	 so it is sufficient to be in wmf/nda to access superset to have your account created
[12:16:57] <elukey>	 fdans: I came up with https://phabricator.wikimedia.org/T237752#5785128
[12:17:23] <elukey>	 that is a little bit more verbose in httpd's settings, but probably easier to follow/maintain
[12:19:59] <fdans>	 elukey:  it does seem more clear on its intent
[12:20:04] <fdans>	 fine with me!
[12:21:46] <wikibugs>	 10Analytics: Kerbereros Identity for Gabriel Birke - https://phabricator.wikimedia.org/T242215 (10gabriel-wmde)
[12:27:36] <elukey>	 fdans: ack!
[12:29:34] <wikibugs>	 10Analytics: Kerbereros Identity for Gabriel Birke - https://phabricator.wikimedia.org/T242215 (10elukey) 05Open→03Resolved a:03elukey ` elukey@krb1001:~$ sudo manage_principals.py create gbirke --email_address=gabriel.birke@wikimedia.de Principal successfully created. Make sure to update data.yaml in Pupp...
[12:38:48] <elukey>	 fdans: ah no wait, there is one thing still missing, namely index.html
[12:39:00] <elukey>	 in theory it should be looked into v2 first
[12:39:06] <elukey>	 but I'd need to check
[12:39:08] <elukey>	 forgot about it
[12:43:18] <elukey>	 probably not, uff
[12:43:46] <elukey>	 back to square one
[12:58:17] <elukey>	 fdans: sorry for the hassle, updated the task with a new version
[12:58:34] <elukey>	 that involves the creation of /v1 but not /v2, that is moved to the main root dir
[12:58:35] <joal>	 Hi team - siesta time :)
[12:59:32] <fdans>	 nono elukey thank you for doing this
[13:05:55] <wikibugs>	 10Analytics: Kerbereros Identity for Corinna Hillebrand - https://phabricator.wikimedia.org/T242217 (10CorinnaHillebrand_WMDE)
[13:05:59] <elukey>	 with this version we shouldn't need any symlink etc..
[13:06:38] <fdans>	 elukey: sounds good to me!
[13:06:57] <wikibugs>	 (03PS5) 10Fdans: Add vue-i18n integration, English strings [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/558702 (https://phabricator.wikimedia.org/T240617)
[13:12:07] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy later today" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562325 (https://phabricator.wikimedia.org/T242015) (owner: 10Joal)
[13:12:42] <elukey>	 fdans: I'd need your consult if you have time in bc
[13:14:33] <fdans>	 elukey: omw
[13:15:48] <wikibugs>	 (03PS2) 10Joal: Reduce mediawiki-history oozie accepted SLA delay [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562500 (https://phabricator.wikimedia.org/T242015)
[13:17:53] <wikibugs_>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy later today" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562500 (https://phabricator.wikimedia.org/T242015) (owner: 10Joal)
[13:29:31] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Xqt) The result is surprising me a bit:  - good news: Python 3.4 isn't used and we can drop support for that release soon....
[13:35:38] <wikibugs>	 10Analytics: Kerbereros Identity for Corinna Hillebrand - https://phabricator.wikimedia.org/T242217 (10elukey) ` elukey@krb1001:~$ sudo manage_principals.py create cohi --email_address=corinna.hillebrand@wikimedia.de Principal successfully created. Make sure to update data.yaml in Puppet. Successfully sent email...
[14:10:45] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Investigate sporadic failures in oozie hive actions due to Kerberos auth - https://phabricator.wikimedia.org/T241650 (10elukey)
[14:11:41] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10User-Elukey: Investigate Hue alarms - https://phabricator.wikimedia.org/T241649 (10elukey)
[14:12:40] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10User-Elukey: Investigate Hue alarms - https://phabricator.wikimedia.org/T241649 (10elukey) I am inclined to mark this as done and re-open if the issue re-occurs (wasn't able to repro it).
[14:22:52] <elukey>	 ottomata: o/
[14:22:55] <ottomata>	 ehllooooo
[14:23:10] <elukey>	 do you have a min for a chat about an idea? (on irc I mean)
[14:23:21] <ottomata>	 sure!
[14:23:28] <elukey>	 thanks :)
[14:23:51] <elukey>	 today I checked the work to add user quotas to notebooks, and since there's nothing in puppet (afaics) it is a bit of a hassle
[14:24:06] <elukey>	 so I was wondering if we could do something like the following
[14:24:20] <elukey>	 1) order stat1008 with the GPU, and move stat1004's homes to it
[14:24:49] <elukey>	 2) reimage stat1004 as notebook1005, and allow users to use notebooks with more space until newpyter is ready
[14:25:32] <wikibugs>	 10Analytics: Kerberos password for user mepps - https://phabricator.wikimedia.org/T242222 (10mepps)
[14:25:37] <elukey>	 just as interim solution to facilitate everybody's job
[14:26:27] <ottomata>	 sounds fine to me elukey!
[14:27:00] <elukey>	 ack then, I'll propose it to the team later on then :)
[14:27:21] <elukey>	 also, how do you prefer to proceed with newpyter?
[14:28:23] <elukey>	 possibly start from https://phabricator.wikimedia.org/T188275 ?
[14:29:56] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Services (watching): Switch all eventgate clients to use new TLS port - https://phabricator.wikimedia.org/T242224 (10Ottomata)
[14:29:57] <elukey>	 ah I found https://phabricator.wikimedia.org/T224658
[14:30:04] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Services (watching): Switch all eventgate clients to use new TLS port - https://phabricator.wikimedia.org/T242224 (10Ottomata)
[14:30:30] <ottomata>	 yeah!  elukey  i'll have headspace hopefully by next week to start thinking about that stuff!
[14:30:39] <ottomata>	 excited to work on that with ya!
[14:30:55] <wikibugs>	 10Analytics: Newpyter - First Class Jupyter Notebook system - https://phabricator.wikimedia.org/T224658 (10Ottomata)
[14:36:24] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Ottomata) Manual syncing is still needed for Hue (users are in MySQL, not SQLite, syncing is still needed).
[14:36:45] <elukey>	 ottomata: I'll start reading docs then, super ignorant in the subject, let me know if I can help on this
[14:37:21] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Ottomata) Done.  Use your shell username and ldap password to login.
[14:37:30] <ottomata>	 ok great
[14:37:41] <ottomata>	 elukey:  i'm ignorant too, we will learn together!
[14:40:34] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Ottomata) Also hi and welcome! :D
[14:41:36] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Urbanecm) >>! In T242157#5785695, @Xqt wrote: > The result is surprising me a bit: >  - good news: Python 3.4 isn't used a...
[14:43:04] <ottomata>	 oh elukey i have more tls analytics vlan changes!
[14:43:17] <ottomata>	 we are doing a migration, so this should be adding a rule
[14:43:19] <ottomata>	 not just changing
[14:43:19] <ottomata>	 for now
[14:44:18] <ottomata>	 https://phabricator.wikimedia.org/T242224#5785876
[14:44:46] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Services (watching): Switch all eventgate clients to use new TLS port - https://phabricator.wikimedia.org/T242224 (10Ottomata) Ah @elukey, we need to allow port 4192 for eventgate-analytics from analytics VLAN.
[14:47:46] <elukey>	 ottomata: is it needed now or can it be done around standup? 
[14:48:07] <ottomata>	 can wait elukey  ty
[14:48:25] <elukey>	 super, because I'll ask Arzhel to use https://wikitech.wikimedia.org/wiki/Homer !
[14:48:36] <ottomata>	 oh mny
[14:48:40] <ottomata>	 cooool
[14:50:24] <elukey>	 ottomata: the port needs to be added to the range or swapped with the current one?
[14:50:53] <ottomata>	 we need to change configs for clients to use it too
[14:51:03] <ottomata>	 so wee need to add it for now
[14:51:06] <elukey>	 ack
[14:51:09] <ottomata>	 we can remove the old one after
[14:54:02] <elukey>	 running an errand for ~1h, ttl!
[14:56:16] <dsaez>	 hi elukey
[14:56:52] <dsaez>	 qq: are the notebook machines working? I can open the notebook, but can't run any command there
[15:04:57] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Switch all eventgate clients to use new TLS port - https://phabricator.wikimedia.org/T242224 (10Ottomata)
[15:06:20] <ottomata>	 hm, dsaez  what do you mean?
[15:18:00] <milimetric>	 joal / mforns: I'm meeting with PA after standup to talk about canonical data / refinery/static_data, let me know if you're interested
[16:12:05] <dsaez>	 ottomata, I can open the notebooks, but can't excute commands there
[16:12:33] <dsaez>	 ottomata, this for pyspark notebook, python is working ok.
[16:29:17] <elukey>	 dsaez: o/ - what notebook host?
[16:31:35] <elukey>	 dsaez: did you kinit? I see errors on 1003 for Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt
[16:53:25] <sbassett>	 Hey analytics - do individual users need their own kerb tokens to use beeline now?  https://phabricator.wikimedia.org/T242236
[16:55:21] <wikibugs>	 10Analytics: Problem connecting to database from stat1007.eqiad.wmnet - https://phabricator.wikimedia.org/T242236 (10sbassett)
[16:59:52] <mforns>	 sorry team 2fa
[17:01:50] <nuria>	 ping fdans staduppp
[17:09:34] <elukey>	 sbassett: yep, we enabled kerberos in December :)
[17:13:44] <wikibugs>	 10Analytics: Problem connecting to database from stat1007.eqiad.wmnet - https://phabricator.wikimedia.org/T242236 (10elukey) We enabled kerberos authentication in December, more info about how to get new credentials in https://wikitech.wikimedia.org/wiki/Analytics/Systems/Kerberos/UserGuide#Get_a_password_for_Ke...
[17:14:41] <elukey>	 sbassett: I added some info to the task, please ping me if you need clarification or have doubts etc..
[17:15:10] <sbassett>	 elukey: I'm creating a kerb creds request task rn.  I imagine dsharpe will be doing the same soon :)
[17:15:57] <elukey>	 sbassett: yes sorry I just realized that it is completely separate, I thought you were working on the same problem :)
[17:17:47] <elukey>	 hello dsharpe :)
[17:17:57] <dsharpe>	 Hello there!
[17:18:19] <wikibugs>	 10Analytics: Kerberos credentials for sbassett - https://phabricator.wikimedia.org/T242242 (10sbassett)
[17:18:21] <wikibugs>	 10Analytics: Kerberos credentials for sbassett - https://phabricator.wikimedia.org/T242242 (10sbassett)
[17:21:16] <elukey>	 dsharpe: do you need a kerb principal?
[17:21:49] <elukey>	 (I am doing one know this is why I am asking)
[17:23:46] <dsharpe>	 Yes please.  I just submittted https://phabricator.wikimedia.org/T242244.  I don't know how to do check that "Your shell username is in analytics-privatedata-users." from https://wikitech.wikimedia.org/wiki/Analytics/Systems/Kerberos/UserGuide#Authenticate_via_Kerberos
[17:24:29] <elukey>	 dsharpe: basically if you are in the analytics-privatedata-user posix group (you are), so I'll add it
[17:27:52] <wikibugs>	 10Analytics: Kerberos credentials for dsharpe - https://phabricator.wikimedia.org/T242244 (10elukey) ` elukey@krb1001:~$ sudo manage_principals.py create dsharpe --email_address=dsharpe@wikimedia.org Principal successfully created. Make sure to update data.yaml in Puppet. Successfully sent email to dsharpe@wikim...
[17:32:05] <wikibugs>	 10Analytics: Kerberos credentials for jfishback - https://phabricator.wikimedia.org/T242245 (10JFishback_WMF)
[17:36:25] <wikibugs>	 10Analytics, 10Analytics-Cluster, 10Product-Analytics: Improve Hue user management - https://phabricator.wikimedia.org/T127850 (10Dzahn)
[17:36:28] <wikibugs>	 10Analytics, 10Operations, 10Product-Analytics, 10SRE-Access-Requests: Access to analytics infrastructure for SNowick_WMF - https://phabricator.wikimedia.org/T242026 (10Dzahn) 05Open→03Resolved Cool, thanks, Ottomata. Closing ticket.
[17:36:28] <dsharpe>	 elukey: Thanks!
[17:40:48] <wikibugs>	 10Analytics, 10Android-app-Bugs, 10Wikipedia-Android-App-Backlog: EventLogging sees MobileWikiAppFindInPage parsing errors - https://phabricator.wikimedia.org/T147196 (10Dbrant)
[17:41:52] <elukey>	 dsharpe: yw!
[17:43:39] <wikibugs>	 (03PS6) 10Mforns: [WIP] Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[17:47:14] <wikibugs>	 10Analytics: Problem connecting to database from stat1007.eqiad.wmnet - https://phabricator.wikimedia.org/T242236 (10Dsharpe) 05Open→03Resolved a:03Dsharpe I am all set now.  Thanks!
[17:51:39] <wikibugs>	 10Analytics, 10Patch-For-Review: Kerberos credentials for sbassett - https://phabricator.wikimedia.org/T242242 (10elukey) >>! In T242242#5786883, @sbassett wrote: > @elukey - thanks.  Can I create a keytab for myself on `stat1007` or does #analytics need to do that for me?  Wasn't quite certain of the process...
[17:52:31] <joal>	 !log Rerun webrequest-load-wf-text-2020-1-8-15 with updated thresholds after frontend issue
[17:52:33] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[17:52:38] <wikibugs>	 (03PS1) 10Elukey: Add the 'response_size' dimension back to webrequest_sampled_128 druid indexation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562888 (https://phabricator.wikimedia.org/T240681)
[17:53:32] <elukey>	 joal: not sure if correct or not --^
[17:53:47] <elukey>	 I can quickly test it with my test dataset
[17:53:50] <elukey>	 just to be sure
[17:54:05] <joal>	 elukey: I feel conformata
[17:54:06] <elukey>	 (I mean launching a coord for 1 hour and see in turnilo)
[17:54:18] <joal>	 confortable - but if you can test easily that's even better :)
[17:54:40] <joal>	 elukey: you tell me :)
[17:54:54] <elukey>	 joal: if you think the change is ok please go ahead :)
[17:55:05] <elukey>	 I just merged the two puppet changes
[17:55:12] <elukey>	 let me know when you want to test aqs
[17:55:47] <joal>	 elukey: now!
[17:55:48] <joal>	 :)
[17:55:53] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy later today" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562888 (https://phabricator.wikimedia.org/T240681) (owner: 10Elukey)
[17:57:39] <wikibugs>	 (03Merged) 10jenkins-bot: Correct MW XML dumps converter parameter parsing [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/560833 (owner: 10Joal)
[17:58:06] <milimetric>	 mforns: any idea why reportupdater would get a Permission denied error when trying to access wmf_raw data?  Are files owned as analytics:hdfs not accessible to the RU user on stat1007?
[17:58:33] <mforns>	 milimetric, I think ru runs with analytics user no?
[17:59:09] <mforns>	 is analytics user able to access those files?
[17:59:22] <mforns>	 oh! of course
[17:59:32] <mforns>	 I though you were saying hdfs:hdfs
[17:59:58] <mforns>	 milimetric, the permission denied error might be when trying to access a pid file though
[18:00:03] <elukey>	 joal: aqs1004 depooled and ready
[18:00:06] <mforns>	 or the history file
[18:00:14] <joal>	 elukey: gone for a test :)
[18:00:34] <mforns>	 milimetric, or the rerun directory if any
[18:00:52] <mforns>	 milimetric, in any case those temp files can be deleted and RU restarted
[18:01:11] <joal>	 elukey: good for me :)
[18:01:19] <milimetric>	 hm...
[18:01:27] <joal>	 elukey: do you do it, or do you prefer me to do it ? (using scap)
[18:01:54] <milimetric>	 mforns: no pid file in this case, it seems like it can't access the data somehow
[18:02:25] <joal>	 milimetric: do you want me to wait for https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/555578/ ?
[18:03:09] <elukey>	 joal: I can do it via cookbook
[18:03:19] <joal>	 elukey: works for me :) As ou prefer
[18:03:37] * joal feels a new keYboard will be needed soon
[18:06:38] <milimetric>	 joal: oh shoot, sorry, I can do it in a few minutes is that ok?
[18:06:52] <joal>	 milimetric: it is :)
[18:07:32] <elukey>	 joal: done!
[18:08:07] <joal>	 also milimetric, I guess we move to using canonical table being sure we'll use that in the future (getting info from your meeting ;)
[18:08:11] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Keep canonical_data.wikis updated - https://phabricator.wikimedia.org/T241741 (10Milimetric) A productive meeting with Connie and Neil resulted in the following draft proposal:  * create a separate repository and iteratively migrate the contents of refine...
[18:08:22] <joal>	 awesome - Thanks elukey - Testing  UI
[18:09:12] <wikibugs>	 10Analytics, 10Patch-For-Review: Kerberos credentials for sbassett - https://phabricator.wikimedia.org/T242242 (10sbassett) >>! In T242242#5786945, @elukey wrote: > You can use the one that all the members of `analytics-privatedata-users` can access, sudoing as the `analytics-privatedata` users. We don't creat...
[18:10:50] <joal>	 all good for me elukey - Thanks a lot :)
[18:11:03] <joal>	 !log AQS deployed with new druid datasource (2019-12)
[18:11:04] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[18:13:37] <wikibugs>	 10Analytics: Kerberos credentials for sbassett - https://phabricator.wikimedia.org/T242242 (10elukey) 05Open→03Resolved
[18:13:39] <wikibugs>	 (03PS5) 10Milimetric: Add and update columns on edit_hourly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:13:55] <milimetric>	 ok joal ^
[18:14:30] <milimetric>	 yes, we'll be using canonical_data, but it'll be updated regularly, so best of both worlds
[18:14:30] <wikibugs>	 (03PS1) 10Joal: Bump changelog.md to 0.0.110 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/562893
[18:15:37] <joal>	 milimetric: two things - Do we want the is_redirect field in druid?
[18:16:07] <joal>	 milimetric: And, maybe can we name it is_redirect_currently ? in order to have is_redirect abailable later for the real one?
[18:16:24] <milimetric>	 ooops
[18:17:02] <milimetric>	 joal: will add to druid template, but I think we can replace it in place when we have a historified one
[18:17:05] <milimetric>	 or use _historical
[18:19:56] <joal>	 milimetric: it feels kinda weird, since all other fields use historical values when possible - Since it;s the only one not being so, I'd rather have it explicit - Also - This is my last push, if you feel strongly it should be is_redirect, let's keep it :)
[18:20:44] <milimetric>	 joal: I kind of support your idea but _currently is confusing because it's a relative term so it could easily mean currently as in relative to that event
[18:20:53] <milimetric>	 that's why we went with _latest and that confused everyone
[18:20:54] <milimetric>	 I donno
[18:21:03] <joal>	 right milimetric :( hm
[18:21:39] <joal>	 I don't know either :(
[18:22:35] <joal>	 milimetric: given it'll be the only 'currently' one, maybe it's explicit enough?
[18:22:43] <milimetric>	 I was just thinking that
[18:22:44] <milimetric>	 sold!
[18:22:48] <joal>	 \o/
[18:25:08] <wikibugs>	 (03PS6) 10Milimetric: Add and update columns on edit_hourly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:25:08] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/562893 (owner: 10Joal)
[18:25:44] <wikibugs>	 (03PS7) 10Milimetric: Add and update columns on edit_hourly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:26:07] <joal>	 milimetric: while we are it: add the is_redirect to ocmmit-message please?
[18:26:16] <milimetric>	 np
[18:27:13] <joal>	 Ah! found an error - sending a comment on CR
[18:27:19] <joal>	 (comment only)
[18:29:04] <joal>	 milimetric: just sent a comment about a comment
[18:29:14] <joal>	 (on patch 7)
[18:29:17] <wikibugs>	 (03CR) 10Joal: "1 comment + add is_redirect to commit message and we're good :)" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:30:49] <joal>	 !log Releasing refinery-0.0.110 to archiva using Jenkins
[18:30:51] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[18:33:58] <wikibugs>	 (03PS1) 10Joal: Update oozie wikidata article placeholder jar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562905
[18:34:55] <joal>	 milimetric: have you seen my last comment?
[18:35:08] <joal>	 also, if you don't mind looking at the above milimetric --^
[18:35:19] <joal>	 no rush, jenkins is releasing ;)
[18:35:51] <milimetric>	 yep, removing the .org and already updated the commit msg, will look at the placeholder thing in a sec
[18:36:47] <wikibugs>	 (03PS9) 10Milimetric: Add and update columns on edit_hourly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:37:15] <wikibugs>	 (03CR) 10Milimetric: [V: 03+2 C: 03+2] Update oozie wikidata article placeholder jar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562905 (owner: 10Joal)
[18:38:19] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Urbanecm) Do you think we should inform the user about it?
[18:48:00] <elukey>	 milimetric: if you have some time we can discuss wikistats, otherwise tomorrow :)
[18:48:29] <milimetric>	 elukey: yeah, sorry I should monitor this deploy and get some lunch
[18:48:48] <milimetric>	 elukey: can we chat on IRC?
[18:48:53] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/555578 (https://phabricator.wikimedia.org/T232659) (owner: 10Conniecc1)
[18:50:02] <elukey>	 milimetric: sure, but we can do tomorrow without rushing
[18:50:04] <elukey>	 no problem
[18:50:26] <milimetric>	 either way, I'm happy to chat on IRC while stuffing my face :)
[18:50:36] <milimetric>	 just doing it over video seems mean :)
[18:53:27] <elukey>	 ahahah
[18:53:40] <elukey>	 ok so very briefly, I tried the solution that we discussed
[18:53:54] <elukey>	 more info in https://phabricator.wikimedia.org/T237752#5785128
[18:54:09] <elukey>	 but it doesn't work for the /dist dir issue that I mentioned during standup
[18:54:22] <elukey>	 so I thought another solution
[18:56:07] <elukey>	 also outlined in the task
[18:56:20] <elukey>	 buuut no idea if it diverge too much from what you want t odo
[18:56:37] <elukey>	 so just wanted to re-sync and see if we can find a better solution or similar
[18:56:53] <elukey>	 I don't have any strong opinion, just trying to make all the use cases to work
[18:58:37] <milimetric>	 elukey: that seems ideal, no?
[18:59:01] <milimetric>	 this way we can easily migrate the root anytime (via changing the symlink) and all the old links work
[18:59:52] <milimetric>	 elukey: so if I get it right, the "fallback look into v1" will mean that even if a link is absolute as in /some-link-to-old-wikistats.html Apache will look for it in /v2/some-link-to-old-wikistats.html and then /v1/some-link-to-old-wikistats.html
[19:00:37] <milimetric>	 it kind of doesn't scale if we add a v3 and v4 but hopefully we really don't do that :)
[19:00:49] <milimetric>	 so I say +1 if my understanding above is correct
[19:00:52] <milimetric>	 nice idea
[19:03:41] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Xqt) >>! In T242157#5787298, @Urbanecm wrote: > Do you think we should inform the user about it?   Sure.
[19:04:11] <joal>	 !log Deploy refinery using scap
[19:04:12] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:04:26] <wikibugs>	 (03CR) 10CDanis: "Oh cool, so now it's a metric *and* a dimension? That's great!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562888 (https://phabricator.wikimedia.org/T240681) (owner: 10Elukey)
[19:04:52] <elukey>	 milimetric: the caveat is that we'll need to add a symlink called 'index.html' -> /v2/index.html in the root, IIRC you didn't like the idea
[19:05:08] <elukey>	 but I might not remember correctly, if so I'll proceed with that :)
[19:05:16] <elukey>	 in this way we can keep the current v2 deployment
[19:05:29] <milimetric>	 elukey: no, that idea's fine, I didn't love the complicated apache stuff
[19:05:30] <wikibugs>	 (03PS7) 10Mforns: [WIP] Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[19:05:38] <elukey>	 milimetric: ack then, thanks!
[19:05:42] <milimetric>	 :) thank you
[19:10:12] <wikibugs>	 10Analytics, 10Event-Platform, 10MediaWiki-Maintenance-scripts, 10WMF-JobQueue, 10Wikimedia-General-or-Unknown: showJobs.php maintenance script useless and misleading in production - https://phabricator.wikimedia.org/T221224 (10Pchelolo) Perhaps we could make it log a warning and error-out if JobQueueEve...
[19:12:41] * elukey off!
[19:12:48] <joal>	 bye elukey 
[19:12:51] <elukey>	 o/
[19:14:22] <wikibugs>	 (03PS8) 10Mforns: [WIP] Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[19:16:53] <joal>	 !log Deploy refinery on HDFS
[19:16:55] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:29:04] <joal>	 !log Kill-restart webrequest-druid-daily-coord and  webrequest-druid-hourly-coord after deploy
[19:29:06] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:42:47] <joal>	 !log Kill-restart mediawiki-history-load-coord,
[19:42:49] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:44:34] <joal>	 !log Kill-restart mediawiki-history-load-coord, mediawiki-history-denormalize-coord,  mediawiki-history-check_denormalize-coord, mediawiki-history-metrics-coord, mediawiki-history-reduced-coord, mediawiki-history-dumps-coord
[19:44:35] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:48:22] <joal>	 !log Kill-restart wikidata-articleplaceholder_metrics-coord
[19:48:24] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:50:12] <wikibugs>	 (03PS9) 10Mforns: [WIP] Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[19:51:31] <wikibugs>	 (03PS10) 10Mforns: Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[19:56:39] <joal>	 !log Kill-restart edit-hourly-coord and edit-hourly-druid-coord
[19:56:41] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:56:52] <joal>	 !log kill wikidata-articleplaceholder_metrics-coord as it is buggy
[19:56:54] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:59:05] <wikibugs>	 (03CR) 10Joal: "This patch doesn't work in prod - Let's have another patch for next week deploy." (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/556988 (https://phabricator.wikimedia.org/T236895) (owner: 10Ladsgroup)
[20:00:14] <wikibugs>	 10Analytics, 10ArticlePlaceholder, 10Wikidata, 10wikidata-tech-focus, and 3 others: ArticlePlaceholder dashboard stopped tracking page views - https://phabricator.wikimedia.org/T236895 (10JAllemandou) The patch merged by @Nuria had a bug. I commented on the already merged patch on a solution. For the momen...
[20:00:35] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10ArticlePlaceholder, 10Wikidata, and 4 others: ArticlePlaceholder dashboard stopped tracking page views - https://phabricator.wikimedia.org/T236895 (10JAllemandou)
[20:06:00] <joal>	 !log Prepare and start learning-features-actor-hourly-coord
[20:06:01] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[20:17:53] <joal>	 !log rerun  edit-hourly-wf-2019-12 after having updated the underlying table <facepalm />
[20:17:54] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[20:21:36] <joal>	 git st
[20:21:38] <joal>	 oops
[20:32:16] <mforns>	 joal, is there an example of oozie spark action where we collect the job's output?
[20:32:20] <mforns>	 in refinery?
[20:32:37] <joal>	 I don't think so mforns 
[20:32:39] <mforns>	 I mean, oozie collects it an uses it in a decision
[20:32:48] <mforns>	 I see, you think that is possible?
[20:33:05] <joal>	 possible - we do it in hive, whu mnot in spark
[20:33:14] <joal>	 pff - tired - fatty fingers
[20:33:24] <mforns>	 hehe
[20:33:35] <mforns>	 ok, will look for the hive example
[20:33:49] <joal>	 mforns: IIRC webrequest is one of them
[20:33:54] <mforns>	 ok, cool :]
[20:34:11] <joal>	 mforns: oh and actually mediawiki-history-check is another, from spark
[20:34:26] <joal>	 I didn't think about it
[20:34:48] <mforns>	 oh! great
[20:44:17] <joal>	 !log Restart webrequest-load-bundle to update queue to production
[20:44:18] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[20:47:47] <wikibugs>	 (03PS1) 10Joal: Fix oozie learning/features/actor/hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562957
[20:48:28] <wikibugs>	 (03CR) 10Joal: "Fixed in prod by overwriting CLI parameter." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562957 (owner: 10Joal)
[20:51:48] <wikibugs>	 (03PS2) 10Joal: Fix oozie learning/features/actor/hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/562957 (https://phabricator.wikimedia.org/T238360)
[20:52:55] <joal>	 ok team, deploy is done - One failure (wikidata article-placeholder see T236895), and 2 jobs still running (https://hue.wikimedia.org/oozie/list_oozie_coordinator/0028067-191216160148723-oozie-oozi-C/ and https://hue.wikimedia.org/oozie/list_oozie_coordinator/0028069-191216160148723-oozie-oozi-C/)
[20:52:56] <stashbot>	 T236895: ArticlePlaceholder dashboard stopped tracking page views - https://phabricator.wikimedia.org/T236895
[20:53:03] <joal>	 I'm gonna get diner
[20:53:06] <joal>	 See y'all tomorrow
[20:58:32] <mforns>	 byeeeee
[21:21:24] <wikibugs>	 (03PS11) 10Mforns: Add Spark/Scala module for anomaly detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/561674 (https://phabricator.wikimedia.org/T235486)
[21:49:11] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, and 2 others: Create and use new schema repositories - https://phabricator.wikimedia.org/T240985 (10Ottomata) Ok, new images and configs deployed to all staging instances.  I tested POSTing an example of each event type that each...
[22:38:47] <wikibugs>	 10Analytics, 10Pywikibot, 10Wikimedia-Site-requests, 10User-Urbanecm: Provide some Pywikibot usage statistics for Python2.7 and Python3.x - https://phabricator.wikimedia.org/T242157 (10Multichill) @Urbanecm thanks for looking into this. Appreciated. Not sure if bot usernames are a privacy issue, I think no...
[23:03:44] <wikibugs>	 10Analytics, 10Datasets-Archiving, 10Research-Backlog: Make HTML dumps available - https://phabricator.wikimedia.org/T182351 (10leila) @ArielGlenn thanks for engaging on this ticket further. From the current conversation, it doesn't look to me as we can have a solution for having HTML dumps on a regular basi...