[01:06:45] (03PS1) 10Nuria: Updating mvn surefire plugin [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/618419 [01:07:47] (03CR) 10jerkins-bot: [V: 04-1] Updating mvn surefire plugin [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/618419 (owner: 10Nuria) [03:42:31] 10Analytics: Check home/HDFS leftovers of nathante - https://phabricator.wikimedia.org/T256356 (10leila) @Groceryheist I checked with Legal and your MOU is active with us until the end of September and that timeline covers your needs. Here is my ask to you: please work with the data that you need, and update the... [03:42:47] 10Analytics: Check home/HDFS leftovers of nathante - https://phabricator.wikimedia.org/T256356 (10leila) a:05leila→03MoritzMuehlenhoff [07:56:27] RECOVERY - Hue Kerberos keytab renewer on an-tool1009 is OK: PROCS OK: 1 process with command name python2.7, args /usr/lib/hue/build/env/bin/hue kt_renewer https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hue/Administration [07:57:36] this is me testing --^ [08:24:01] RECOVERY - Hue CherryPy python server on an-tool1009 is OK: PROCS OK: 1 process with command name python2.7, args /usr/lib/hue/build/env/bin/hue runcherrypyserver https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hue/Administration [08:38:57] 10Analytics: Check home/HDFS leftovers of nathante - https://phabricator.wikimedia.org/T256356 (10MoritzMuehlenhoff) a:05MoritzMuehlenhoff→03akosiaris >>! In T256356#6361901, @leila wrote: > @MoritzMuehlenhoff could you please re-grant the access to the user until the end of September? Reassigning to @akosi... [08:51:30] 10Analytics-Radar, 10Datasets-General-or-Unknown, 10Product-Analytics, 10Structured-Data-Backlog: Set up generation of JSON dumps for Wikimedia Commons - https://phabricator.wikimedia.org/T259067 (10ArielGlenn) This would only partly be me; once the code is available in the extension to produce output of t... [09:32:42] !log set ticket max renewable lifetime to 7d on all kerberos clients (was zero, the default) [09:32:44] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:53:33] 10Analytics-Clusters, 10Patch-For-Review, 10User-Elukey: Upgrade Druid to its latest upstream version (currently 0.19.0) - https://phabricator.wikimedia.org/T244482 (10elukey) All done! [09:54:08] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade Druid to its latest upstream version (currently 0.19.0) - https://phabricator.wikimedia.org/T244482 (10elukey) a:03elukey [09:56:22] 10Analytics-Kanban: Neflow data pipeline - https://phabricator.wikimedia.org/T257554 (10elukey) [10:09:10] 10Analytics-Clusters, 10Discovery, 10Discovery-Search, 10Patch-For-Review: Move mjolnir kafka daemon from ES to search-loader VMs - https://phabricator.wikimedia.org/T258245 (10elukey) [10:36:56] * elukey lunch! [11:34:29] ottomata: fwiw I’m down to add enabled: false as a universally recognized stream config setting. I’ll make a phab task but we should see how things are working in practice and whether that first approach turns out to be too confusing or messy as folks start actually using this stuff. [11:35:27] Up there with debug mode as topics to revisit once we’ve gotten enough anecdotal data for identifying pain points and roll [11:35:37] And areas for improvements [12:20:18] (03PS10) 10Mforns: Configure Oozie job for loading geoeditors data into Cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [12:22:24] (03CR) 10Mforns: [C: 04-2] "Still testing this, please don't merge." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [12:35:37] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) [12:40:04] 10Analytics, 10Better Use Of Data, 10Event-Platform: Produce an instrumentation event stream using new EPC and EventGate from client side browsers - https://phabricator.wikimedia.org/T241241 (10jlinehan) [12:48:25] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics, and 2 others: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10jlinehan) [12:48:31] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: EventLogging MEP Upgrade - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:48:57] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) [12:50:26] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: EventLogging MEP Upgrade - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:53:16] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 2 others: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10jlinehan) [12:54:00] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:55:35] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Product-Infrastructure-Team-Backlog (Kanban): MEP Client MediaWiki (PHP) - https://phabricator.wikimedia.org/T253121 (10jlinehan) [12:55:55] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:56:09] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Product-Infrastructure-Team-Backlog (Kanban): MEP Client MediaWiki PHP - https://phabricator.wikimedia.org/T253121 (10jlinehan) [12:56:32] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:58:54] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) [12:59:09] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS - https://phabricator.wikimedia.org/T238544 (10jlinehan) [12:59:11] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Event-Platform, and 3 others: Set up an instance of EventStreams in beta that will allow for consuming any stream - https://phabricator.wikimedia.org/T253069 (10jlinehan) [12:59:40] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Product-Infrastructure-Team-Backlog (Kanban): MEP Client MediaWiki PHP - https://phabricator.wikimedia.org/T253121 (10jlinehan) [12:59:45] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS - https://phabricator.wikimedia.org/T238544 (10jlinehan) [13:03:34] !log set yarn_scheduler_minimum_allocation_mb = 1 (was zero) to Hadoop to workaround a Flink 1.1 issue (namely it doesn't work if the value is <= 0) [13:03:36] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:04:14] !log restart yarn resource managers on an-master100[12] to pick up new Yarn settings - https://gerrit.wikimedia.org/r/c/operations/puppet/+/618529 [13:04:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:10:41] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS - https://phabricator.wikimedia.org/T238544 (10jlinehan) 05Open→03Resolved [13:11:18] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS (MVP) - https://phabricator.wikimedia.org/T238544 (10jlinehan) [13:12:28] ottomata: when you have moment, can you check if I forgot anything in https://gerrit.wikimedia.org/r/c/operations/puppet/+/611168/6/hieradata/role/common/kafka/jumbo/broker.yaml ? [13:12:39] (even in two hours, not urgent) [13:14:30] elukey: if i'm on kafka-jumbo1001 will I be able to talk to kafka-jumbo1002? [13:14:56] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics, and 2 others: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10jlinehan) [13:15:26] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics, and 2 others: EventLogging MEP Upgrade Phase 3 (Stream cc-ing) - https://phabricator.wikimedia.org/T256165 (10jlinehan) [13:15:56] ottomata: this is a very good point [13:16:08] :) [13:16:17] I forgot the most simple thing, sigh [13:16:19] thanks! [13:16:39] i only barely thought of it! [13:22:06] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS (MVP) - https://phabricator.wikimedia.org/T238544 (10jlinehan) [13:22:58] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, and 4 others: MEP Client MediaWiki JS (MVP) - https://phabricator.wikimedia.org/T238544 (10jlinehan) [13:23:34] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Infrastructure-Team-Backlog, and 2 others: EventLogging MEP Upgrade Phase 1 (Submit) - https://phabricator.wikimedia.org/T244521 (10jlinehan) [13:24:40] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) a:03jlinehan [13:34:30] 10Analytics-Radar, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10jlinehan) [13:34:53] 10Analytics-Radar, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10jlinehan) [13:35:30] 10Analytics-Radar, 10Better Use Of Data, 10Product-Infrastructure-Data, 10Wikimedia-Logstash, and 4 others: Documentation of client side error logging capabilities on mediawiki - https://phabricator.wikimedia.org/T248884 (10jlinehan) [13:35:50] 10Analytics-Radar, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10jlinehan) [13:36:32] 10Analytics-Radar, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10jlinehan) [13:36:47] ottomata: yep same thought, in theory mirror maker should consume from main right? [13:37:00] yes, and eventually also from logging too [13:37:08] 10Analytics-Radar, 10Better Use Of Data, 10Desktop Improvements, 10Product-Infrastructure-Team-Backlog, and 7 others: Client side error logging production launch - https://phabricator.wikimedia.org/T226986 (10jlinehan) [13:37:21] but we colocate the mirror maker processes witih the dest cluster [13:37:24] so they should just be pulling [13:37:44] and pushing to local, in theory ok for ferm [13:37:56] but worth to double check when deploying the ferm rules [13:37:59] just to be sure [13:38:00] well, pushing to all kafka jumbo brokers [13:38:04] but you just added that rule :) [13:38:16] right right [13:38:43] all brokers, I was assuming the produce locally in the wrong way [13:38:59] okok should be good [13:39:22] ya safe to assume that anything that talks to one kafka broker talks to them all [13:41:15] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10mpopov) [13:42:30] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10mpopov) p:05Triage→03Low This is low priority until we observe h... [13:47:07] ottomata: I looked all over https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingOnBetaCluster and mediawiki's EL docs, couldn't find that role you were talking about, if you happen to find it let me know [13:47:19] (the role I need to create a Schema: page on beta) [13:47:49] milimetric: i think it is autoconfirmed [13:47:50] whatever that is [13:47:50] https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/EventLogging/+/master/extension.json#27 [13:48:02] i had to comment that out locally to be able to edit in mw-vagrant too [13:48:10] oh! ok [13:48:42] I don't think it's needed in vagrant if you login with the admin account [13:49:13] autoconfirm I think is 10 edits but I did that... maybe there's some job that's supposed to run and add me the role and it only runs every 24 hours or something [13:54:56] ah I have to wait $wgAutoConfirmAge, whatever that is [13:58:59] what is admin account? [13:59:01] i just know 'vagrant' [14:00:27] ottomata: from mw-vagrant's doc "You can login with user Admin and password vagrant." [14:00:46] well well well [14:00:48] :) [14:01:07] wow milimetric this revert logic somehow detects manual reverts? [14:01:08] hopefully that's still the case! [14:01:15] i thought it would have just been via some UI use? [14:01:54] ottomata: I think that new PHP class they built to handle reverts knows more about how the revert was triggered [14:02:06] but yeah, that's why I'm excited about it, it's more information than we currently have [14:02:18] it's actually too good! I can't do the incremental stuff with it :) [14:02:25] https://phabricator.wikimedia.org/T216297#6362837 [14:02:55] oh ottomata no that's easy, that's just looking at the sha1 [14:03:03] if you have the exact same sha1, it's a revert anyway [14:03:12] yeah but...it goes and re-looks up the previous sha1s and compares during the php request? [14:03:19] what if there are a billion revisions? [14:03:24] oh yeah, I don't know... [14:03:34] that'd be interesting to find out [14:03:52] I think it must have a history of the page loaded somewhere, because it also sends you a list of revisions reverted, right? [14:04:00] yeah! [14:04:03] oh let me try several [14:04:33] yup [14:04:36] "rev_reverted_revs": [ [14:04:36] 9251, [14:04:36] 9252, [14:04:36] 9253 [14:04:36] ], [14:04:53] ah but i guess it only finds the most recent rev with the same sha [14:05:29] if there is an early rev with the same sha it doesn't include it as the original rev id [14:08:35] (03PS11) 10Mforns: Configure Oozie job for loading geoeditors data into Cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [14:12:34] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10Ottomata) This would also be useful for eventgate and event ingestio... [14:12:39] (03PS12) 10Mforns: Configure Oozie job for loading geoeditors data into Cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [14:13:46] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10mpopov) [14:13:58] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10jlinehan) [14:14:01] 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Goal, and 3 others: Modern Event Platform (TEC2) - https://phabricator.wikimedia.org/T185233 (10jlinehan) [14:21:11] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10mpopov) > This would also be useful for eventgate and event ingestio... [14:22:14] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10Ottomata) Very low, right now it would be equivalent to just removin... [14:47:02] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Set up environment for Product Analytics system user - https://phabricator.wikimedia.org/T258970 (10mpopov) [14:49:40] 10Analytics-Radar, 10Product-Analytics, 10Release-Engineering-Team, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10mpopov) @nshahquinn-wmf: That's an excellent point! I updated the task description with the `analytics/wmf-pr... [14:51:18] 10Analytics-Radar, 10Technical-blog-posts: Story idea for Blog: The Best Dataset on Wikimedia Content and Contributors - https://phabricator.wikimedia.org/T259559 (10Milimetric) [15:01:55] 10Analytics-Radar, 10Product-Analytics, 10Release-Engineering-Team, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10nshahquinn-wmf) >>! In T230743#6362955, @mpopov wrote: > @nshahquinn-wmf: That's an excellent point! I update... [15:02:47] ping milimetric [15:20:34] 10Analytics-Radar, 10Product-Analytics, 10Release-Engineering-Team, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10mpopov) [15:26:11] 10Analytics-Radar, 10Product-Analytics, 10Release-Engineering-Team, 10Repository-Admins: Create a repository and user for Product Analytics Oozie jobs - https://phabricator.wikimedia.org/T230743 (10mpopov) Requested `analytics/wmf-product/jobs` Gerrit repo [15:33:15] 10Analytics, 10Analytics-EventLogging, 10Better Use Of Data, 10MediaWiki-extensions-WikimediaEvents, and 2 others: Limit the number of mw-client-errors from a single client (suggested: maximum 5 errors should be recorded for a single IP on a single page) - https://phabricator.wikimedia.org/T259371 (10LGoto... [15:58:02] 10Analytics-Radar, 10Better Use Of Data, 10MediaWiki-extensions-WikimediaEvents, 10Product-Infrastructure-Data, 10Product-Infrastructure-Team-Backlog (Kanban): Limit the number of mw-client-errors from a single client (suggested: maximum 5 errors should be record... - https://phabricator.wikimedia.org/T259371 [16:00:10] nuria: this is the task for Kafka firewall - https://phabricator.wikimedia.org/T204957 (forgot about it yesterday) [16:00:35] elukey: got it [16:46:20] ottomata: ok if I attempt to firewall jumbo? [16:49:42] nevermind, seems that the search team needs more time to verify [16:59:26] ok [17:20:10] I added https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/AMD_GPU#Reset_the_GPU_state [17:20:37] Miriam found the two gpus not able to run anything with TF, and it was due to (I suspect) an unclean shutdown of previous jobs [17:20:59] upstream suggests to reset the gpu when this happens, but I wasn't able to recover a good state, so I had to reboot [17:21:19] we are also missing some metrics, I opened https://github.com/RadeonOpenCompute/ROC-smi/issues/90 [17:21:33] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10jlinehan) [17:23:24] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) p:05Triage→03Low [17:23:38] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data: Obtain evidence-based guidance on capacity for event streams - https://phabricator.wikimedia.org/T259155 (10jlinehan) p:05Triage→03Low [17:33:22] (03PS13) 10Mforns: Configure Oozie job for loading geoeditors data into Cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [17:33:42] * elukey afk! [17:35:07] (03CR) 10Mforns: [V: 03+2] "OK, I have tested this against Cassandra, and seems to work!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [17:36:32] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Goal: BUOD: Require that all new schema/instruments are created with the MEP system - https://phabricator.wikimedia.org/T259157 (10jlinehan) [17:37:10] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Data, 10Product-Infrastructure-Team-Backlog: Develop test environment solution for MEP analytics events - https://phabricator.wikimedia.org/T238837 (10jlinehan) [17:37:12] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10jlinehan) [17:54:30] 10Analytics, 10MediaWiki-REST-API, 10Platform Team Sprints Board (Sprint 0), 10Platform Team Workboards (Green), 10Story: System administrator reviews API usage by client - https://phabricator.wikimedia.org/T251812 (10Pchelolo) **Where do we start from** API Gateway envoy instance is able to log the acc... [17:58:44] 10Analytics, 10MediaWiki-REST-API, 10Platform Team Workboards (Green): Unify access log schema for Action API and API Gateway/REST API - https://phabricator.wikimedia.org/T259736 (10Pchelolo) [18:05:34] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10Nuria) Just saw this ticket on passing, maybe we want to rework the timeline/text of ticket ? (as deliverables were set for la... [18:06:14] 10Analytics, 10MediaWiki-REST-API, 10Platform Team Sprints Board (Sprint 0), 10Platform Team Workboards (Green), 10Story: System administrator reviews API usage by client - https://phabricator.wikimedia.org/T251812 (10Ottomata) Alternatively, we could create a sidecar that reads from a file/stdin and POS... [18:09:40] 10Analytics, 10MediaWiki-REST-API, 10Platform Team Sprints Board (Sprint 0), 10Platform Team Workboards (Green), 10Story: System administrator reviews API usage by client - https://phabricator.wikimedia.org/T251812 (10Pchelolo) > You mean kafkacat? :p :) why not? [18:14:45] nuria: what next with event-utilities? [18:15:05] shall i just get all the CI set up and figure out how to release it? [18:21:29] elukey: FYI refinery-source built with regular hadoop dependencies [18:22:24] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10jlinehan) [18:22:26] 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Goal, and 3 others: Modern Event Platform (TEC2) - https://phabricator.wikimedia.org/T185233 (10jlinehan) [18:22:35] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175 (10jlinehan) >>! In T228175#6363685, @Nuria wrote: > Just saw this ticket on passing, maybe we want to rework the timeline/text o... [20:19:58] (03CR) 10Nuria: [C: 03+2] Configure Oozie job for loading geoeditors data into Cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/582638 (https://phabricator.wikimedia.org/T248289) (owner: 10Lex Nasser) [20:25:44] ottomata: i think releasing event utilities seems the way to go yes [20:25:57] ottomata: we would it on gerrit ya? [20:26:10] https://gerrit.wikimedia.org/r/c/wikimedia-event-utilities/+/618378 [20:26:22] nuria: ^ [20:27:29] ottomata: i do not think i will be able to review this with the detail that I would like [20:29:47] aye, joseph reviewed most of it [20:30:06] that was here: https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/603582 [20:30:11] (going to abandon that one) [20:30:24] before he left he told me i could fix his last comments and merge [20:31:06] ottomata: I would like to look at it though [20:32:40] ottomata: did dan or marcel CR-ed? [20:34:17] marcel did [20:37:37] ottomata: ok, if mforns Cr-ed and he can give one last pass we can merge [20:37:58] ok [20:38:41] (03Abandoned) 10Ottomata: Add classes to use EventStreamConfig with EventSchemaLoader to aide in event ingestion tasks [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/603582 (https://phabricator.wikimedia.org/T251609) (owner: 10Ottomata) [20:40:12] ottomata: i just have too many comments and i feel like I should contribute to the work if we change this that much [20:40:25] :) [20:58:26] mforns: yt? [20:58:36] mforns: we can talk about session length schema if you want [22:32:23] (03PS1) 10Nuria: Requests with app user agents should not be evaluated as app pageviews [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/618635 (https://phabricator.wikimedia.org/T257860) [22:34:25] (03CR) 10Nuria: "Take a look at the fix for the long standing bug that some app requests were evaluated as pageviews using the desktop criteria. Need to te" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/618635 (https://phabricator.wikimedia.org/T257860) (owner: 10Nuria) [22:59:55] 10Analytics: Add authentication and encryption to Druid Analytics clients - https://phabricator.wikimedia.org/T250484 (10Nuria) [22:59:57] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade Druid to its latest upstream version (currently 0.19.0) - https://phabricator.wikimedia.org/T244482 (10Nuria) 05Open→03Resolved [23:00:15] 10Analytics, 10Analytics-Kanban: RU reportupdater-ee-beta-features keeps logging a lot of daily errors to its logs - https://phabricator.wikimedia.org/T256195 (10Nuria) 05Open→03Resolved [23:00:53] 10Analytics: Investigate why netflow hive_to_druid job is so slow - https://phabricator.wikimedia.org/T254383 (10Nuria) [23:00:55] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Alarm when druid indexation fails - https://phabricator.wikimedia.org/T254493 (10Nuria) 05Open→03Resolved [23:01:06] 10Analytics, 10Analytics-Kanban: Fix geoeditors job heisenbug - https://phabricator.wikimedia.org/T257397 (10Nuria) 05Open→03Resolved [23:02:24] 10Analytics-Clusters, 10Analytics-Kanban, 10Cassandra, 10User-Elukey: Cassandra3 migration plan proposal - https://phabricator.wikimedia.org/T249756 (10Nuria) 05Open→03Resolved [23:02:27] 10Analytics, 10Cassandra: Cassandra3 migration for Analytics AQS - https://phabricator.wikimedia.org/T249755 (10Nuria) [23:02:52] 10Analytics-EventLogging, 10Analytics-Kanban, 10Analytics-Radar, 10Event-Platform, and 5 others: Refactor EventBus mediawiki configuration - https://phabricator.wikimedia.org/T229863 (10Nuria) 05Open→03Resolved [23:02:57] 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10MW-1.34-notes (1.34.0-wmf.15; 2019-07-23), and 4 others: Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main - https://phabricator.wikimedia.org/T211248 (10Nuria)