[00:25:04] <wikibugs>	 (03CR) 10Joal: "Tested on cluster" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584602 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal)
[00:25:26] <wikibugs>	 (03CR) 10Joal: "tested on cluster" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/578373 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal)
[05:51:12] <wikibugs>	 10Quarry, 10DBA, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Marostegui) labsdb1011 has been running a big alter table in the last few hours, so that's why it has been lagging s4 a bit - s4 is commons but it of course affects the whole...
[06:05:46] <wikibugs>	 10Analytics, 10Patch-For-Review: Analytics Kerberos Welcome Email contains hostname typo - https://phabricator.wikimedia.org/T249103 (10elukey) 05Open→03Resolved a:03elukey Fixed, sorry for the trouble. Really appreciated the feedback :)
[08:23:30] <elukey>	 !log kill/restart netflow realtime druid indexation with a new dimension (peer_ip_src) - T246186
[08:23:32] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[08:23:32] <stashbot>	 T246186: can aggregated netflow data include the router it was sampled from? - https://phabricator.wikimedia.org/T246186
[08:29:38] <wikibugs>	 (03PS1) 10Elukey: Update Netflow's realtime indexation druid spec with a new dimension [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585432 (https://phabricator.wikimedia.org/T246186)
[08:55:33] * elukey interview, afk
[10:28:08] * elukey lunch!
[10:45:20] <addshore>	 o/ Would it be possible to get those 2 accounts access to Hue too? :D (from yesterday), ((would ping elu_key but dont want to disturb lunch)) :D
[11:13:48] <elukey>	 addshore: added tarrow, itamar asked yesterday in a task. Any more? :)
[11:13:59] <addshore>	 thats it, thanks!
[11:14:02] <elukey>	 super
[11:28:50] <elukey>	 since it seems quite I am going to extend my lunch and go buy some groceries, will take (hopefully) ~1h
[11:28:57] <joal>	 later elukey :)
[12:40:45] <awight>	 nuria: lexnasser: Hello, nice to meet you and please feel free to ping me any time :-)
[12:45:30] <joal>	 hashar: hello :) I'm 1 day late, but would you feel like deploying refinery today?
[12:45:37] <wikibugs>	 10Analytics, 10Commons, 10Epic: Provide download statistics of files on Wikimedia Commons - https://phabricator.wikimedia.org/T218076 (10Aklapper) p:05Triage→03Lowest Adding #Analytics per last comment
[12:45:59] <hashar>	 joal: sure :)
[12:46:02] <joal>	 \o/
[12:46:30] <joal>	 hashar: I'm preping and then ping you (5/10 minutes?) - or later?
[12:46:43] <hashar>	 5-10 minutes sound perfect
[12:46:50] <hashar>	 that gives me time to finish my coffee 
[12:46:51] <hashar>	 ;]
[12:46:52] <joal>	 :)
[12:46:58] <joal>	 ack hashar :)
[12:48:51] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10nshahquinn-wmf) 05Open→03Resolved >>! In T244547#6001049, @nshahquinn-wmf wrote: >>>! In T244547#6000890, @Nuria wrote...
[12:56:57] <wikibugs>	 (03PS1) 10Joal: Update changelog for v0.0.120 and v0.0.121 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/585488
[12:57:29] <joal>	 hashar, elukey (if you're here) --^
[13:06:21] <hashar>	 o/
[13:06:34] <hashar>	 joal: sorry I had kids screaming downstair
[13:10:14] <hashar>	 joal: https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/585488/1/changelog.md has plenty of trailing white spaces :D
[13:17:14] <hashar>	 joal: seems good now :]
[13:19:24] <joal>	 np hashar - I posted a CR for our changelog (for triple checking)
[13:19:25] <joal>	 wow indeed! will fix that hashar :)
[13:19:25] <wikibugs>	 (03PS2) 10Joal: Update changelog for v0.0.120 and v0.0.121 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/585488
[13:19:26] <joal>	 Should be better
[13:19:27] <wikibugs>	 (03CR) 10Hashar: [C: 03+1] Update changelog for v0.0.120 and v0.0.121 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/585488 (owner: 10Joal)
[13:19:27] <joal>	 Here we go then :)
[13:19:28] <wikibugs>	 (03CR) 10Joal: [C: 03+2] "Merging for deploy (using jenkins-docker !!! :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/585488 (owner: 10Joal)
[13:19:28] <joal>	 hashar: once the above is merged, ready we are :)
[13:19:28] <joal>	 hashar: how do you want to proceed? You deploy as a test and then we merge your patches? or the opposite?
[13:20:05] <hashar>	 hmm
[13:20:46] <hashar>	 the couple changes I have proposed for analytics/refinery are for the jar updater
[13:20:54] <hashar>	 and I have crafted a docker based job for that one yet
[13:21:02] <hashar>	 the maven release job should work though
[13:21:29] <hashar>	 so I guess we can now run https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release-docker/
[13:22:16] <wikibugs>	 (03Merged) 10jenkins-bot: Update changelog for v0.0.120 and v0.0.121 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/585488 (owner: 10Joal)
[13:22:58] <hashar>	 joal: wanna do it with meet? ;]
[13:23:23] <hashar>	 joal: https://meet.google.com/mud-gnpx-rhe if you feel so
[13:25:27] <joal>	 hashar: now is the time my wife uses most the bandwisth thtrough meetings - will do dound only :)
[13:25:37] <hashar>	 AHAH
[13:25:43] <hashar>	 I know that effect
[13:30:18] <joal>	 !log Releasing refinery-source v0.0.121 using new jenkins-docker :)
[13:30:20] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[13:33:04] <hashar>	 \o/
[13:43:59] <elukey>	 back! sorry but it took 2h to get groceries
[13:44:25] <hashar>	 joal: succes :)
[13:44:43] <hashar>	 and the job should have send an email to analytics-alerts
[13:44:57] <hashar>	 we might make it to send an irc notification as well 
[13:45:17] <elukey>	 confirm the email to alerts
[13:45:28] <elukey>	 hashar: the irc notification would be awesome
[13:45:42] <hashar>	 will add it
[13:45:45] <elukey>	 <3
[13:45:58] <hashar>	 then I think Joal poked you for some good old shell script review
[13:46:15] <hashar>	 I need some teak to the refinery script bin/update-refinery-source-jars which hardcode a gerrit url
[13:46:33] <hashar>	 so there is a first pass to double quote a bunch of variables to please shellcheck https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/584949/
[13:46:53] <joal>	 checking the deploy for correctness, but this looks awesome :)
[13:47:15] <hashar>	 and a second one which installs the Gerrit commit-msg only when it is not there and over https instead of ssh https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/584984/
[13:47:50] <hashar>	 !log test 1 2 3
[13:47:51] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[13:48:01] <joal>	 huhu :)
[13:48:29] <hashar>	 ah and it is properly relayed to https://tools.wmflabs.org/sal/analytics 
[13:48:35] <joal>	 it is indeed :)
[13:48:36] <cdanis>	 hello analytics friends, can I ask how long after the top of the hour that webrequest_sampled_128 data is usually available for the prior hour?
[13:48:54] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10akosiaris) >>! In T210271#6020856, @hashar wrote: >>>! In T210271#6017936, @...
[13:49:24] <joal>	 cdanis: hello - There usually is ~2h delay (a bit less) if you consider the beginning of the hour you're interested in
[13:50:05] <joal>	 archiva deploy is a success, and way easier then it was - Much love sent to ou by the team hashar :)
[13:50:20] <cdanis>	 ah I thought it was usually more like 45 minutes -- so that data for 12:00-13:00 UTC would be available about now :)
[13:50:31] <joal>	 elukey: would you mind having a look at the patches hashar mentions? you're far better in bash than I am :(
[13:51:11] <hashar>	 bash is like java
[13:51:18] <hashar>	 without all the boilerplat of factories and utility classes
[13:51:24] <cdanis>	 ah there it goes
[13:51:24] <hashar>	 just plain raw commands!
[13:53:20] <elukey>	 joal: sure
[13:53:38] <joal>	 hehe hashar :)
[13:54:00] <wikibugs>	 (03CR) 10Elukey: [C: 03+1] Double quote variables in update-refinery-source-jars [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584949 (owner: 10Hashar)
[13:55:29] <wikibugs>	 (03CR) 10Elukey: [C: 03+1] Fetch commit hook over https and skip if already present [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584984 (https://phabricator.wikimedia.org/T210271) (owner: 10Hashar)
[13:55:34] <joal>	 \o/
[13:55:37] <elukey>	 seems good!
[13:55:59] <joal>	 hashar: ready when you want to test or merge-run the second job :)
[14:00:29] <hashar>	 and irc notifications are added by https://gerrit.wikimedia.org/r/#/c/integration/config/+/585497
[14:00:46] <hashar>	 joal: just run the existing update jar job https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/
[14:00:56] <hashar>	 I haven't prepared the docker based version yet
[14:01:03] <hashar>	 I might have a draft locally though
[14:01:38] <joal>	 as you wish hashar - running the old one and let you prepare for the next one is fine by me :)
[14:01:48] <hashar>	 great :]
[14:01:59] <hashar>	 I am not too worried about the second one. It is straightforward
[14:02:09] <hashar>	 I would just need my couple changes to analytics/refinery to land
[14:02:13] <hashar>	 which we might want to do now
[14:02:24] <hashar>	 before triggering the old https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/
[14:02:25] <elukey>	 joal: if you are ok I'd like to test G1 for hive
[14:02:34] <hashar>	 at lest that will confirm the shell changes I have made work properly
[14:02:55] <joal>	 ack elukey - works for me
[14:03:11] <elukey>	 https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/585498/
[14:03:21] <elukey>	 I also reviewed some heap settings
[14:03:30] <elukey>	 the major consumer of memory on the node is the mysql instance
[14:03:50] <joal>	 yes
[14:04:36] <elukey>	 but we can't do much about it
[14:04:50] <joal>	 hashar: I didn't understand how you wish us to proceeed :) Shall I wait for the changes you want?
[14:04:51] <elukey>	 I would start removing all the recurrent jobs and move them to an-launcher
[14:05:00] <joal>	 makes sense elukey 
[14:05:45] <elukey>	 !log temporary stop timers on an-coord1001 to facilitate hive daemons restarts
[14:05:46] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[14:11:03] <hashar>	 joal: we should get the shell script changes merged first, then run the legacy  update-jars job. This way we will now for sure that my changes work properly
[14:17:57] <joal>	 works for me hashar 
[14:18:34] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584949 (owner: 10Hashar)
[14:18:59] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584984 (https://phabricator.wikimedia.org/T210271) (owner: 10Hashar)
[14:19:11] <joal>	 Commits merged
[14:20:21] <hashar>	 great
[14:20:32] <joal>	 hashar: launching the job now
[14:20:33] <hashar>	 so https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/  should still work after that hopefully
[14:20:45] <joal>	 https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/131/
[14:20:59] <joal>	 \o/
[14:21:03] <hashar>	 it works
[14:21:04] <hashar>	 still
[14:21:08] <hashar>	 I am disappointed :]
[14:21:18] <joal>	 :)
[14:22:06] <joal>	 hashar: So only the other job to finish dockerization, right?
[14:22:13] <hashar>	 the hook did not work
[14:22:20] <hashar>	 Installing Gerrit commit-msg hook ...
[14:22:20] <hashar>	 .git/hooks/: Is a directory
[14:22:23] <hashar>	 that is a silent error
[14:22:31] <hashar>	 since the script hmm.. does not run with set -e
[14:22:31] <joal>	 hah
[14:22:52] <joal>	 hashar: has the script done what was expected nontheless?
[14:23:01] <hashar>	 yeah the commit got pushed
[14:23:05] <joal>	 it has yes
[14:23:11] <hashar>	 but it lacks a change-id :]
[14:23:16] <joal>	 Ah meh :(
[14:23:19] <hashar>	 it is not a big deal since it is directly pushed to gerrit
[14:23:30] <hashar>	 we can actually change that behavior
[14:23:40] <hashar>	 and first push for review to have Gerrit to create a change
[14:23:43] <hashar>	 then push the commit
[14:24:00] <hashar>	 (which Gerrit will notice and cause the change to be automatically closed)
[14:25:13] <wikibugs>	 (03PS1) 10Hashar: Fix wget/chmod to use commit-msg, not the directory [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585503
[14:26:42] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Move netflow to TLS encryption/authentication via librdkafka - https://phabricator.wikimedia.org/T248980 (10elukey) For the moment I am happy with TLS encryption only, since we'll probably move to kerberos authentication soon and it doesn't make much...
[14:26:52] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Move netflow to TLS encryption/authentication via librdkafka - https://phabricator.wikimedia.org/T248980 (10elukey)
[14:28:07] <wikibugs>	 (03PS1) 10Hashar: Make update-refinery-source-jars more strict [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585505
[14:29:47] <wikibugs>	 (03CR) 10Hashar: "https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/585505/ makes the script stricter (set -e) which would have caused a failure when " (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585503 (owner: 10Hashar)
[14:30:02] <wikibugs>	 (03CR) 10Hashar: "That would have caught the issue fixed by parent change https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/585503/" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585505 (owner: 10Hashar)
[14:30:18] <hashar>	 I have proposed some amendments
[14:32:30] <joal>	 ack hashar - Will review and ask elukey to triple check
[14:33:34] <wikibugs>	 (03CR) 10Joal: [C: 03+1] "LGTM :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585503 (owner: 10Hashar)
[14:34:41] <wikibugs>	 (03CR) 10Joal: [C: 03+1] "LGTM !" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585505 (owner: 10Hashar)
[14:36:05] <joal>	 elukey: look at that deployment process steamline: https://wikitech.wikimedia.org/w/index.php?title=Analytics%2FSystems%2FCluster%2FDeploy%2FRefinery-source&type=revision&diff=1861979&oldid=1853936
[14:36:39] <elukey>	 wow it looks awesome
[14:36:41] <elukey>	 one nit: wehther
[14:37:05] <joal>	 elukey: nothing to check (as long as the master branch is on ok state) - click build, done :)
[14:37:13] <joal>	 Ha!
[14:37:44] <joal>	 Symlinks to be changed next time :)
[14:38:56] <joal>	 nuria: if you're up and have some time, can you do a final check of https://gerrit.wikimedia.org/r/#/c/584602/ for me to deploy please?
[14:42:41] <hashar>	 AHAH
[14:43:15] <nuria>	 joal: on meeting, we can do it after stand up
[14:43:19] <joal>	 sure nuria 
[14:43:20] <hashar>	 OKR: [X]  further automate refinery release
[14:43:28] <joal>	 \o/ :)
[14:43:44] <hashar>	 I am still assuming lot of the extra steps were due to the Jenkins maven release plugin
[14:43:59] <hashar>	 what would be very cool
[14:44:02] <joal>	 hashar: I really the team owe you one for that help - streamlining the process will make our life a lot easier :)
[14:44:08] <hashar>	 is to reuse the same workflow for all other maven based projects
[14:44:18] <joal>	 indeed hashar :)
[14:45:03] <hashar>	 I have to migrate wikidata/query/rdf to the same model
[14:45:25] <hashar>	 and I guess I will ping some folks from the maven cabal (eg gehel ) 
[14:45:27] <joal>	 ! log Kill restart wikidata-json_entity-weekly-coord  and wikidata-item_page_link-weekly-coord after wikidata dumps got fixed (jobs waiting for data that will never happen
[14:46:01] <joal>	 Sounds great hashar - Please keep us in the loop if new stuff happens is the real of CIJVM
[14:46:04] <joal>	 :)
[14:46:55] <gehel>	 hashar: maryum and zpapierski both have had a look into our release process lately...
[14:47:18] <gehel>	 lot of people interested! (and interesting)
[14:50:24] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Vertical: Virtualpageview datastream on MEP - https://phabricator.wikimedia.org/T238138 (10Ottomata) Oof.  I think VirtualPageView is a bad choice for our first port. https://phabricator.wikimedia.org/T187...
[14:51:19] <hashar>	 moara new names
[14:51:40] <gehel>	 hashar: things have continued to move while you were not looking :)
[14:52:25] <cdanis>	 is it a known issue with turnilo that selecting "Show Druid query" gives a "General error / Unexpected error occurred" message?
[14:53:22] <joal>	 I have never used that feature cdanis - I can't say
[14:56:25] <elukey>	 cdanis: it is an easter egg for your user! :D
[14:56:57] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Vertical: Virtualpageview datastream on MEP - https://phabricator.wikimedia.org/T238138 (10Nuria) @Ottomata  ae you thinking next quarter we will do teh instraumentation of session length but also an addit...
[14:57:27] <elukey>	 cdanis: lemme check the logs on an-tool1007, might be a turnilo bug
[14:57:35] <elukey>	 do you need to query druid directly?
[14:57:48] <ottomata>	 nuria:  no but i want to work on the EL backwards compat
[14:57:57] <ottomata>	 not that we would actually switch it
[14:58:03] <ottomata>	 but i want to use something as an example
[14:58:07] <ottomata>	 i was going with VirtualPageView
[14:58:13] <ottomata>	 but that one doesn't even use EventLogging!!!!
[14:58:33] <ottomata>	 actually nuria  we had a KR to do VirtaulPageView this quarter
[14:58:38] <ottomata>	 not going to make that for sure
[14:58:54] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10JAllemandou) `refinery-source-release` job has been successful today. Our de...
[14:59:00] <ottomata>	 nuria:  do you have another suggestion for an EL schema to start with?
[15:01:44] <cdanis>	 elukey: I'd like to try! not something I've done before
[15:04:38] <elukey>	 cdanis: it is not that great, there is some doc in wikitech.. but if I may I'd divert your efforts to Presto :)
[15:04:47] <cdanis>	 ok!
[15:05:07] <elukey>	 https://wikitech.wikimedia.org/wiki/Analytics/Systems/Presto
[15:05:27] <elukey>	 because if you like it, then you'll be able to use it in superset to make cool dashboards
[15:05:33] <joal>	 cdanis: https://superset.wikimedia.org/superset/sqllab
[15:05:39] <elukey>	 and this --^
[15:05:59] <joal>	 cdanis: pick database `presto_analytics_hive` and you're good
[15:06:15] <joal>	 cdanis: only limit is, not for webrequest please (or 1h no more)
[15:06:28] <cdanis>	 1h window would be fine for my purposes right now
[15:06:34] <joal>	 you can try
[15:06:50] <elukey>	 cdanis: we might even do more but step by step since it is a lot of data :)
[15:07:03] <elukey>	 super happy to help if you need anything
[15:15:45] <ottomata>	 heya ebernhardson!  what produces the SearchSatisfication eventlogging events
[15:15:45] <ottomata>	 ?
[15:15:53] <ottomata>	 and, can I do it in vagrant easily?
[15:16:03] <elukey>	 he may be on holidays IIRC
[15:16:06] <ottomata>	 oh
[15:16:10] <wikibugs>	 (03CR) 10Joal: Add dimensions to druid's pageview_hourly (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns)
[15:16:11] <ottomata>	 dcausse: do you know? ^^
[15:16:58] <dcausse>	 ottomata: it's WikimediaEvents
[15:17:06] <ottomata>	 oh great
[15:17:16] <ottomata>	 perfect
[15:17:17] <ottomata>	 thank you
[15:17:20] <dcausse>	 yw!
[15:19:49] <ottomata>	 dcausse:  the event gets emmiitted when, when  a user clicks on a search result?
[15:20:45] <dcausse>	 ottomata: search click, search click and when the clicked page is visited
[15:21:15] <dcausse>	 err, when searching, when clicking a result and when opening the clickec result
[15:21:22] * dcausse can't type :/
[15:22:32] <dcausse>	 any problems with these events?
[15:23:15] <ottomata>	 dcausse:  we are looking for an event to port to eventgate using eventlogging extension for backwards compat
[15:23:25] <ottomata>	 we were going to do virtualpageview, but it does custom code to bypass DNT
[15:23:26] <ottomata>	 ...
[15:23:43] <ottomata>	 and i am seeing that searchSatisfaction.js also does custom code to bypass EL?
[15:23:51] <ottomata>	 			 * This is a direct copy of mw.eventLog.logEvent. It is necessary
[15:23:51] <ottomata>	 			 * to override the call to sendBeacon.
[15:24:18] <dcausse>	 DNT is do not track me?
[15:24:20] <ottomata>	 yeah
[15:24:25] <dcausse>	 I think we respect that
[15:24:29] <ottomata>	 yes, right
[15:24:49] <ottomata>	 i just mean, i'm trying to find a candidate to work with to make EL ext backwards  compatible
[15:24:53] <ottomata>	 so instrumentation code works as is witih EL client
[15:24:58] <ottomata>	 but with eventgate
[15:25:04] <ottomata>	 instead of GET requeust to varnish
[15:25:21] <ottomata>	 and i think maybe also SearchSatisfactiion is a bad choice, because you guys also don't use eventlogging
[15:25:53] <ottomata>	 do you know what this is about?
[15:25:54] <ottomata>	 https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikimediaEvents/+/master/modules/ext.wikimediaEvents/searchSatisfaction.js#346
[15:26:16] <ottomata>	 milimetric:  is this relevant to the queue and page unload problems that someone reported?
[15:27:08] <dcausse>	 ottomata: I think struggled to get coherent events and had to add few hacks
[15:27:17] <dcausse>	 Erik struggled
[15:27:18] <ottomata>	 OH, it looks like in most places you return mw.eventLog
[15:27:18] <ottomata>	 ?
[15:27:47] <ottomata>	 do you think we can just put your fixes into EventLogging somehow?
[15:28:37] <dcausse>	 ottomata: this is too specific for me, I'd have to ask Erik
[15:28:54] <ottomata>	 OHHHH
[15:28:58] <ottomata>	 you use the img hack
[15:29:05] <ottomata>	 we can't support that anymore
[15:29:08] <ottomata>	 events have to go via POST
[15:29:47] <dcausse>	 then we'll probably have to change this
[15:30:07] <ottomata>	 ya it won't work without sendBeacon i think
[15:30:42] <wikibugs>	 10Analytics, 10Operations, 10ops-eqiad: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10Cmjohnson) These are failing during install.  @elukey can you verify the raid configuration please   Failed to partition the selected disk             │...
[15:32:57] <wikibugs>	 (03PS9) 10Joal: Add dimensions to druid's pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns)
[15:34:01] <wikibugs>	 (03CR) 10Joal: Add dimensions to druid's pageview_hourly (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns)
[15:34:16] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns)
[15:35:00] <dcausse>	 ottomata: fyi Erik is on vacation for two weeks
[15:47:38] <ottomata>	 hm k
[15:47:49] <ottomata>	 dcausse:  ok can you help me get an event emitted in mw vagrant?
[15:47:59] <ottomata>	 i've got cirrussearch and wikimediaevents roles enabled
[15:48:11] <ottomata>	 i just did a search and i have a results link like
[15:48:12] <ottomata>	 http://dev.wiki.local.wmftest.net:8080/wiki/Test?wprov=srpw1_0
[15:48:26] <dcausse>	 sure
[15:48:45] <ottomata>	 OH i think maybe that is doing it, I see event.gif? in network
[15:48:47] <ottomata>	 consoe
[15:48:55] <ottomata>	 hmmm where does that go....
[15:49:14] <dcausse>	 I think there's a eventlogging.log log file somewhere
[15:49:47] <elukey>	 joal: ok to restart hive?
[15:49:48] <ottomata>	 ya hm
[15:49:55] <elukey>	 there are some jobs running including mediawiki_history-2020-03
[15:49:59] <joal>	 looking at jobs running elukey 
[15:50:38] <joal>	 good to go elukey - only some backfilling jobs left - will restart them :)
[15:51:14] <dcausse>	 ottomata: should see events in that file, if not it means you're not selected
[15:51:38] <dcausse>	 but iirc we no longer do sampling client side
[15:51:59] <elukey>	 !log restart hive server2/metastore with G1 settings
[15:52:01] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[15:52:09] <elukey>	 joal: done!
[15:52:19] <joal>	 Will check for jobs elukey - Thanks
[15:52:26] <ottomata>	 i don't see them but am trying to figure out why, i see the browser emitting events...will figure it out, I am already farther than I thought I'd be dcausse  thank you!
[15:52:49] <dcausse>	 yw :)
[16:01:47] <nuria>	 hip
[16:02:37] <joal>	 elukey, ottomata, dcausse - might of interest for you folks: https://www.ververica.com/blog/how-openssl-in-ververica-platform-improves-your-flink-job-performance
[16:03:20] <dcausse>	 joal: thanks, looking!
[16:12:13] <elukey>	 !log re-enable timers on an-coord1001 after maintenance
[16:12:14] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[16:23:03] <wikibugs>	 (03CR) 10Nuria: [C: 03+1] "Looks good, virtual +2 as we need to deploy the UDF before this code is usable" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584602 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal)
[16:24:17] <nuria>	 cdanis: did you found what you were looking for?
[16:26:13] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "UDF deployed earlier today, merging for deploy :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/584602 (https://phabricator.wikimedia.org/T238363) (owner: 10Joal)
[16:27:35] <elukey>	 nuria: are we doing grooming?
[16:27:44] <nuria>	 elukey: not today
[16:27:54] <elukey>	 ack thanks, I have it in my cal
[16:28:04] <joal>	 ok - everything that needed to be merged has been, starting refinery deploy
[16:28:04] <elukey>	 wanted to triple check
[16:28:17] <nuria>	 elukey:  ya, did not cancel it but will move it up now, to be after standup
[16:30:50] <joal>	 !log Deploy refinery using scap
[16:30:51] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[16:32:57] <fdans>	 nuria: elukey didn't we dump groskin on thursdays for the time being?
[16:41:59] <nuria>	 fdans: yes,   we did. 
[16:55:55] <joal>	 !log Deploy refinery onto HDFS
[16:55:56] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[17:07:30] <cdanis>	 nuria: got sidetracked by a meeting and some other things, but yes for now :) I'll ask if I get stuck
[17:07:34] <cdanis>	 thanks!
[17:09:00] <nuria>	 cdanis: k
[17:12:47] <wikibugs>	 10Analytics, 10Developer-Advocacy, 10MediaWiki-API, 10Product-Infrastructure-Team-Backlog, and 3 others: Metrics about the use of the Wikimedia web APIs - https://phabricator.wikimedia.org/T102079 (10bd808) 05Open→03Resolved a:03bd808 >>! In T114017#6023382, @bd808 wrote: > Closing as "resolved" rath...
[17:43:02] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Import regularly via sqoop mediawiki_imagelinks table - https://phabricator.wikimedia.org/T249113 (10JAllemandou) I merged the patch without realizing that monthly job has not been updated (hourly and daily, but not monthly). Moving...
[17:48:29] <joal>	 !log Kill/restart virtualpageview-hourly-coord after deploy
[17:48:30] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[18:01:41] * elukey off!
[18:42:39] <wikibugs>	 (03PS1) 10Joal: Fix virtualpageview_hourly bug [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585574
[18:42:45] <joal>	 nuria: --^
[18:42:53] <joal>	 If ok I'll merge that and redeploy
[19:04:21] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10Krinkle) >>! In T226986#5992337, @Ottomata wrote: > Ah we haven't yet resolved the x-request-id stuff. >...
[19:09:51] <wikibugs>	 (03CR) 10Nuria: [C: 03+2] Fix virtualpageview_hourly bug [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585574 (owner: 10Joal)
[19:15:11] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform: Vertical: Migrate SearchSatisfaction EventLogging event stream to Event Platform - https://phabricator.wikimedia.org/T249261 (10Ottomata)
[19:15:54] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10Ottomata)
[19:16:56] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Vertical: Virtualpageview datastream on MEP - https://phabricator.wikimedia.org/T238138 (10Ottomata) Not to necessarily port another event, but at least prepare to do so.  If things turn out to be easy, th...
[19:18:18] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform: Vertical: Migrate SearchSatisfaction EventLogging event stream to Event Platform - https://phabricator.wikimedia.org/T249261 (10Ottomata)
[19:27:30] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for quick patch" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585574 (owner: 10Joal)
[19:29:03] <joal>	 !log Manually patching last deploy to fic virtualpageview job - code merged
[19:29:04] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:31:10] <joal>	 !log restart paegviewhourly job after manual patch
[19:31:11] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[19:36:52] <wikibugs>	 (03PS1) 10Joal: Fix VirtualPageview hourly (2nd) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585582
[19:39:38] <joal>	 another one nuria  :( sorry --^
[19:42:36] <wikibugs>	 (03CR) 10Joal: [V: 03+2 C: 03+2] "Tested on cluster - merging for prod patch" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/585582 (owner: 10Joal)
[21:02:38] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Event-Platform, 10Patch-For-Review: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10Ottomata) Ok, in https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/585581 I went with namespacing e.g.:    /l...
[21:13:14] <wikibugs>	 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Produce an instrumentation event stream using new EPC and EventGate from client side browsers - https://phabricator.wikimedia.org/T241241 (10Ottomata) I'm working on {T249261} and I think that figuring out how we wa...