[00:00:38] (03CR) 10Razzi: "Here's my attempt at adding pylibmc to the superset virtualenv." [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [00:02:10] 10Analytics, 10Product-Analytics, 10Epic: Readership Retention: New vs. Returning Unique devices - https://phabricator.wikimedia.org/T269815 (10kzimmerman) [00:16:39] 10Analytics-Radar, 10WMDE-Analytics-Engineering, 10Patch-For-Review, 10User-GoranSMilovanovic: Downscale Wikidata-analysis pyspark scripts to analytics limits - https://phabricator.wikimedia.org/T268684 (10GoranSMilovanovic) 05Open→03Resolved Ok. Everything we do in Pyspark is now put under the scaling... [00:55:35] PROBLEM - Check the last execution of monitor_refine_eventlogging_legacy_failure_flags on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_eventlogging_legacy_failure_flags https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [07:01:24] isaacj: yep it was me, glad that you liked it, next is auto-renew but it will take a bit :) [07:06:34] (03CR) 10Elukey: [C: 03+1] "> Patch Set 1:" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [07:18:37] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-41] - https://phabricator.wikimedia.org/T260445 (10elukey) Thanks a lot for all the work! To recap: +2 servers in A2 +2 servers in A4 +2 servers in B2 +2 servers in B4 +1 servers in B7 +2 servers i... [07:22:49] so we have some space (12 slots) for the first new workers [07:23:07] finding the other 12 will be probably really difficult.. [07:23:08] sigh [07:23:10] good morning [07:29:58] Good morning elukey [07:30:56] I saw the ticket about racking yesterday - Not enough space in 10g racks :( [07:33:55] yes it is a big problem [07:35:40] joal: adding the second metastore! :) [07:35:44] \o/ [07:38:40] also joal I think that we should be checking http://dharmeshkakadia.github.io/presto-event-listener/ as soon as possible [07:39:03] yessir - I backloged and was going to investigate [07:39:09] <3 [07:39:24] it is annoying that something like that it is not already available [07:39:31] for sure ( [07:39:42] in theory we could create a jar and then package it in the .deb [07:40:17] About yesterday problem, I'd really like to understand if it is one big query or plenty smaller - I actually assume it is the second cause I have witnessed presto doing a good at mitigating problems with big queries [07:40:37] Instrumentation needed! [07:40:43] the main issue is that on the logs we don't have any trace :( [07:41:08] * joal think of hacking XKCD 'Citation needed' with 'Instrumentation needed' for nerds [07:49:46] Morning, I'm following up on some messages from last night about the WikibasePingback EventLogging schema. I've made the required schema changes and I was wondering whether there was anything additional to be to ensure the events' refinement? [07:50:16] **to be done [07:51:42] Hello itamarWMDE - Schema got blacklisted from our refinement process yesterday to stop sending error emails - We'll need to remove it from the blacklist and see if refinerment goes fine (we don't have test refine environement) [07:53:22] Ah, I see :) Thank you joal. Anything to do on our side? [07:53:31] I don't think so itamarWMDE [07:53:43] +1 Thank you :) [07:53:47] The new schema should be picked up automatically :) [07:56:11] ok second metastore ready [07:56:25] the last step is to restart the hive server to pick the new metastore [07:57:53] connection encrypted to the DB via TLS, all good [08:01:02] and of course the hive server complains [08:01:06] :( [08:07:47] of course because I am stupid [08:08:00] working now :) [08:13:02] joal: analytics-hive.eqiad.wmnet is not working for both metastore and server, let's see :) [08:13:06] *now working [08:13:15] Yeah! [08:14:47] elukey: How do we move forward? [08:17:44] joal: so my hope is nothing explodes now that webrequest-load is running, if so I'd say that we could just move all oozie jobs to analytics-hive, and keep checking that it works.. eventually we'll just upgrade an-coord1001 too, and possibly test a failover via DNS (that should be transparent for everybody) [08:17:49] what do you think? [08:18:10] works for me elukey - Let's do that :) [08:18:31] elukey: we also need to make other teams aware! [08:19:50] joal: I already followed up with Erik for the discovery jobs, and I think that the other team that uses oozie is product analytics.. the docs on wikitech are updated (say the examples for pyhive etc..) and when we upgrade the hive-site.xml configs all our clients should behave fine [08:20:02] ok :) [08:20:10] we should also eventually move refine to analytics-hive too [08:20:19] For sure [08:20:43] ah no my bad, one wise man from northern france made the script in a way that it reads hive-site.xml [08:23:07] Arf [08:37:31] ok refine completed, all good! [09:02:17] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [09:12:44] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [09:12:54] RECOVERY - Check the last execution of monitor_refine_eventlogging_legacy_failure_flags on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_eventlogging_legacy_failure_flags https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [09:14:30] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [09:14:32] (03PS1) 10Elukey: oozie: Replace all references of an-coord1001 with analytics-hive [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647612 (https://phabricator.wikimedia.org/T268028) [09:15:16] 10Analytics, 10Analytics-Wikistats, 10Performance-Device-Lab: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [09:16:56] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [09:21:11] (03PS2) 10Elukey: oozie: Replace all references of an-coord1001 with analytics-hive [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647612 (https://phabricator.wikimedia.org/T268028) [09:39:59] 10Analytics: Add a presto query logger - https://phabricator.wikimedia.org/T269832 (10elukey) [09:41:31] * elukey bbiab [10:02:32] 10Analytics, 10Product-Analytics, 10Inuka-Team (Kanban): Set up preview counting for KaiOS app - https://phabricator.wikimedia.org/T244548 (10nshahquinn-wmf) >>! In T244548#6680157, @SBisson wrote: > @nshahquinn-wmf did a spot check a few weeks ago and the data was showing up. He should probably check again... [10:04:49] (03CR) 10Awight: Sanitize and keep TemplateDataEditor events (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/646670 (https://phabricator.wikimedia.org/T260343) (owner: 10Awight) [10:06:15] (03CR) 10Awight: Sanitize and keep TemplateDataEditor events (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/646670 (https://phabricator.wikimedia.org/T260343) (owner: 10Awight) [10:07:37] (03PS3) 10Awight: Sanitize and keep TemplateDataEditor events [analytics/refinery] - 10https://gerrit.wikimedia.org/r/646670 (https://phabricator.wikimedia.org/T260343) [10:35:59] itamarWMDE: I think you now have data (from today hour=7) [10:37:34] Oh Hi! my pings from this morning! Thanks joal and elukey! [10:37:56] sorry to have logspammed you all yesterday :| [10:38:19] No worries itamarWMDE, all fixed now :) [10:40:16] FTR, is there a better way to ensure that things end up in hadoop (apart from trying it live)? I'm pretty new to all this EventLogging business [10:40:54] hmmm - I don't exactly what to answer here itamarWMDE :) [10:41:17] yep, indeed [10:41:30] data end up in hadoop if schema is validated by EventLogging [10:41:52] Then in hadoop it is refined if no schema problem is encountered (which is usually the case) [10:42:06] not sure if that helps itamarWMDE --^ [10:43:56] So the chain is kinda this? Client -> Kafka -> EventLogging -> Hadoop -> Refinement -> Hadoop? [10:44:05] yessir [10:44:59] More on the last hadoop steps: Hadoop[raw] -> Refinement -> Hadoop[Hive+spark] -> Sanitization -> Hadoop[Sanitized] [10:46:41] So to refine (awful pun intended) my question, Is there a way for me to test the first part of the chain (up until and including Hadoop[raw]), without sending data to the production beacons? [11:18:45] itamarWMDE: sorry for the lag, needed to drop for a minute [11:19:16] itamarWMDE: There is a test eventlogging cluster IIRC, but I'm readily not knowledgeable about this part of the pipeline :S [11:30:46] (03CR) 10Lucas Werkmeister (WMDE): Add script to collect lexicographical data statistics (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/647284 (owner: 10Lucas Werkmeister (WMDE)) [11:52:52] No worries joal, thanks for your help :) [11:58:27] * elukey lunch! [12:29:35] !log Drop-Recreate-Repair wmf_raw.mediawiki_image table [12:29:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:47:26] (03PS1) 10Joal: Correct mediawiki_image table sqoop and creation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647681 (https://phabricator.wikimedia.org/T266077) [12:48:14] (03CR) 10Joal: "Table change applied in prod with data for commonswiki manually inserted." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647681 (https://phabricator.wikimedia.org/T266077) (owner: 10Joal) [12:58:09] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Add dimensions to editors_daily dataset - https://phabricator.wikimedia.org/T256050 (10JAllemandou) Ping @cchen - Any news on this front? [13:48:58] * elukey afk ~1h, ping me on the phone if needed [14:22:52] (03CR) 10Mforns: [C: 03+1] "LGTM!" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/646670 (https://phabricator.wikimedia.org/T260343) (owner: 10Awight) [14:37:33] 10Analytics-Clusters, 10Product-Analytics: Configure superset cache - https://phabricator.wikimedia.org/T268784 (10Ottomata) > When it says it should only be run "on a build server", is that`deneb.codfw.wmnet`? Currently, yup! [14:38:31] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog, 10Epic: Vertical: Virtualpageview datastream on MEP - https://phabricator.wikimedia.org/T238138 (10Ottomata) > When we migrate this, we need to make sure we include geocoded data Great, thanks for the heads up. C... [14:42:24] Gone for kids, back in 1h20 for meeting, then will meetings [14:42:33] +miss [14:42:45] I'll miss regular meetings tonight, will send escrum [14:50:46] (03CR) 10Ottomata: Move pageview filters to PageviewDefinition; add Webrequest.isWMFHostname (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/646808 (https://phabricator.wikimedia.org/T256674) (owner: 10Ottomata) [14:56:13] (03CR) 10Ottomata: "Hm, yeah I think if your Docker is buster, it should work the same as on deneb." [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [14:56:22] (03CR) 10Ottomata: "+1 if it looks good according to Elukey's instructions." [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [14:56:26] (03CR) 10Ottomata: [C: 03+1] Install pylibmc and update wheels for superset [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [15:03:22] 10Analytics, 10Event-Platform, 10Product-Analytics, 10Product-Infrastructure-Data: MEP: Should stream configurations be written in YAML? - https://phabricator.wikimedia.org/T269774 (10Mholloway) I like this idea quite a bit. My first pass at how I'd set it up is: 1) Create a new repo for stream configs a... [15:06:25] ottomata: o/ when you get a chance I'd like to have your opinion on a schema: https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/647723 [15:07:40] esp the part where I try to reuse some parts of a fragment ($ref: /fragment/common/1.0.0#/properties/meta) I'm not sure I'm on the right track [15:09:54] (03CR) 10Ottomata: Move pageview filters to PageviewDefinition; add Webrequest.isWMFHostname (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/646808 (https://phabricator.wikimedia.org/T256674) (owner: 10Ottomata) [15:14:15] dcausse: ok cool will look, one quick comment, we are working on standardizing the dt fields and making them consistent [15:14:37] the decision we came to was that meta.dt should be server side receive time, and a top level dt should be event time [15:14:52] i think this is not documented well yet because we are working on making them consistent in some analytics related schemas [15:14:57] i'll update guidelines [15:16:43] ottomata: thanks, good to know, is this top-level dt available for mw events? [15:18:39] https://wikitech.wikimedia.org/wiki/Event_Platform/Schemas/Guidelines#Required_fields [15:18:46] dcausse: it isn't in any common schema yet [15:18:52] https://phabricator.wikimedia.org/T267648 [15:18:58] so no [15:19:01] thanks! [15:19:47] so the kafka timestamp will always be meta.dt? [15:19:52] or dt? [15:21:19] sorry ignore my question, just read the ticket [15:29:06] 10Analytics, 10Event-Platform, 10Product-Analytics, 10Product-Infrastructure-Data: MEP: Should stream configurations be written in YAML? - https://phabricator.wikimedia.org/T269774 (10Ottomata) > reads the stream config YAML from the submodule and initializes $wgEventStreams How to handle per-wiki config... [15:32:12] 10Analytics, 10Event-Platform, 10Product-Analytics, 10Product-Infrastructure-Data: MEP: Should stream configurations be written in YAML? - https://phabricator.wikimedia.org/T269774 (10Mholloway) >>! In T269774#6682401, @Ottomata wrote: >> reads the stream config YAML from the submodule and initializes $wgE... [15:33:18] 10Analytics, 10Event-Platform, 10Product-Analytics, 10Product-Infrastructure-Data: MEP: Should stream configurations be written in YAML? - https://phabricator.wikimedia.org/T269774 (10jlinehan) [15:44:43] fdans: i have a meeting conflict for this director of data engineering kickoff meeting [15:44:49] which one should I attend?! [15:44:56] i guess the hiring kickoff is pretty important... [15:44:58] grrr [16:11:16] hadoop | 2.10.1-1 | stretch-wikimedia | thirdparty/bigtop15 | amd64 [16:11:20] * elukey dances [16:11:42] this is the RC of the 1.5 release, going to test it today/tomorrow in the test cluster [16:11:45] :) [16:11:46] \o/ [16:14:46] I'll create another cookbook, the cdh -> bigtop one is not ideal [16:16:09] 10Analytics-Clusters, 10Operations, 10ops-eqiad: an-presto1004 shows only the NIC in the boot list - https://phabricator.wikimedia.org/T268951 (10Cmjohnson) This is scheduled for Monday 14Dec [16:32:43] (03CR) 10Mforns: "reviewing now :]" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647612 (https://phabricator.wikimedia.org/T268028) (owner: 10Elukey) [16:48:29] (03CR) 10Razzi: "> In theory 'deneb' is a host in which we build stuff, but it doesn't support all our use cases for the moment (at least last time that I " [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [16:49:09] (03CR) 10Mforns: [C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/647612 (https://phabricator.wikimedia.org/T268028) (owner: 10Elukey) [16:55:16] (03PS1) 10Andrew-WMDE: [WIP] Process EventLogging events for VisualEditor [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/647742 (https://phabricator.wikimedia.org/T262209) [17:17:59] (03CR) 10Elukey: [C: 03+1] "> Patch Set 1:" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [17:37:39] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-41] - https://phabricator.wikimedia.org/T260445 (10Cmjohnson) @elukey This is what I have currently +2 servers in A2 +2 servers in A4 +2 servers in A7 ** this is new +2 servers in B2 +2 servers in... [17:44:36] 10Analytics, 10Analytics-Kanban: Address refinery security vulnerabilities with jackson and netty - https://phabricator.wikimedia.org/T237774 (10fdans) a:03Ottomata [17:45:45] 10Analytics: AQS pageview default caching is one day - https://phabricator.wikimedia.org/T268809 (10fdans) p:05Triage→03High a:03razzi [17:52:32] 10Analytics, 10Analytics-Kanban: AQS should be more resilient to druid nodes not available - https://phabricator.wikimedia.org/T268811 (10fdans) a:03fdans [17:53:01] (03CR) 10Elukey: [C: 03+1] "> Patch Set 1:" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [17:53:36] (03CR) 10Elukey: [C: 03+1] "> Patch Set 1:" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [17:53:50] 10Analytics, 10Analytics-Kanban, 10Event-Platform, 10EventStreams, and 5 others: Set up internal eventstreams instance exposing all streams declared in stream config (and in kafka jumbo) - https://phabricator.wikimedia.org/T269160 (10fdans) p:05Triage→03Medium [17:55:42] 10Analytics: Convert labsdb1012 from multi-source to multi-instance - https://phabricator.wikimedia.org/T269211 (10fdans) p:05Triage→03High [17:57:38] 10Analytics, 10Analytics-Kanban: Can't use custom conda kernel in Newpyter within PySpark UDFs - https://phabricator.wikimedia.org/T269358 (10fdans) a:03Ottomata [17:58:46] 10Analytics-Clusters: Set yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds - https://phabricator.wikimedia.org/T269616 (10fdans) a:05Ottomata→03razzi [17:59:17] 10Analytics-Clusters, 10Analytics-Kanban: Set yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds - https://phabricator.wikimedia.org/T269616 (10fdans) p:05Triage→03High [18:04:55] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify macOS versions - https://phabricator.wikimedia.org/T269722 (10fdans) p:05Triage→03Low a:03mforns [18:06:33] (03PS1) 10Razzi: Make aqs cache for 4 hours [analytics/aqs] - 10https://gerrit.wikimedia.org/r/647756 (https://phabricator.wikimedia.org/T268809) [18:12:50] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10fdans) p:05Triage→03Medium @gh87 thanks for the awesome report! We'll try to replicate and take a good look at this [18:15:04] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify Windows 7, 8, and 10 versions - https://phabricator.wikimedia.org/T269729 (10fdans) p:05Triage→03Low a:03mforns [18:19:46] 10Analytics, 10Product-Analytics, 10Epic: Readership Retention: New vs. Returning Unique devices - https://phabricator.wikimedia.org/T269815 (10fdans) tagging @JAllemandou here [18:19:51] (03CR) 10Elukey: "Adding more people, let's also find a good way to test this to be sure that we don't mess with caching headers!" [analytics/aqs] - 10https://gerrit.wikimedia.org/r/647756 (https://phabricator.wikimedia.org/T268809) (owner: 10Razzi) [18:24:25] 10Analytics: Add a presto query logger - https://phabricator.wikimedia.org/T269832 (10fdans) [18:24:27] 10Analytics, 10Analytics-Kanban: Analytics Presto improvements - https://phabricator.wikimedia.org/T266639 (10fdans) [18:24:35] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Deprecate the 'researchers' posix group - https://phabricator.wikimedia.org/T268801 (10Legoktm) I don't need any analytics access anymore. [18:24:48] 10Analytics: Add a presto query logger - https://phabricator.wikimedia.org/T269832 (10fdans) p:05Triage→03Medium [18:35:20] (03CR) 10Razzi: "> Sorry we also need to specify a custom docker registry, I don't know the endpoint but there is something in puppet for sure (see "docker" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [18:46:27] * razzi afk for lunch [19:01:24] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify macOS versions - https://phabricator.wikimedia.org/T269722 (10mforns) Thanks for the report @gh87! I think both this task and T269729 refer to the same fact from the implementation perspective: We do not report on O... [19:01:41] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify macOS versions - https://phabricator.wikimedia.org/T269722 (10mforns) [19:01:43] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify Windows 7, 8, and 10 versions - https://phabricator.wikimedia.org/T269729 (10mforns) [19:02:21] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify macOS or Windows versions. - https://phabricator.wikimedia.org/T269722 (10mforns) [19:02:43] 10Analytics, 10Analytics-Dashiki: Chart data from analytics.wikimedia.org do not fully specify macOS or Windows versions. - https://phabricator.wikimedia.org/T269722 (10mforns) [19:02:45] 10Analytics: Browser Reports. Break down "Other" bucket a little more? - https://phabricator.wikimedia.org/T131127 (10mforns) [19:07:26] (03CR) 10Ottomata: [C: 03+1] "That's awesome! Please update the README with new instructions. :)" [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 (owner: 10Razzi) [19:09:33] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-41] - https://phabricator.wikimedia.org/T260445 (10elukey) I reviewed rack settings for hadoop, this is my proposal: >>! In T260445#6682850, @Cmjohnson wrote: > @elukey This is what I have currently... [19:10:07] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data: Client-side error logging should use Elastic Common Schema (ECS) fields when possible - https://phabricator.wikimedia.org/T267602 (10jlinehan) >>! In T267602#6677656, @Ottomata wrote: > B. is hard to do, and requires a lot of coordination. But we... [19:11:06] * elukey afk! [19:15:09] 10Analytics, 10Event-Platform, 10Product-Infrastructure-Data: Client-side error logging should use Elastic Common Schema (ECS) fields when possible - https://phabricator.wikimedia.org/T267602 (10Ottomata) Interesting idea! However, there are some Event Platform specifics that we'd need to handle, mainly `me... [19:19:54] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10gh87) [19:32:02] (03CR) 10BryanDavis: multiinstance: Attempt to make quarry work with multiinstance replicas (036 comments) [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/632804 (https://phabricator.wikimedia.org/T264254) (owner: 10Bstorm) [20:02:25] 10Analytics: Wikistats map's choropleth shows the same color for 0 and minimum nonzero value - https://phabricator.wikimedia.org/T269883 (10fdans) [20:21:58] (03CR) 10Lucas Werkmeister (WMDE): "Another idea to track: number of Lexemes without Senses/Forms." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/647284 (owner: 10Lucas Werkmeister (WMDE)) [21:00:17] (03PS2) 10Razzi: Install pylibmc and update wheels for superset [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/647387 [21:15:10] (03PS1) 10Fdans: Add Active Editors per Country metric to Wikistats [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/647792 (https://phabricator.wikimedia.org/T188859) [21:50:01] 10Analytics-Radar, 10Product-Analytics, 10Product-Infrastructure-Data, 10Structured Data Engineering, and 3 others: Develop a new schema for MediaSearch analytics or adapt an existing one - https://phabricator.wikimedia.org/T263875 (10egardner) Just a quick update on this in case anyone is wondering. @jli... [21:54:25] 10Analytics-Clusters, 10Patch-For-Review: Move Superset and Turnilo to an-tool1010 - https://phabricator.wikimedia.org/T268219 (10Ottomata) Just talked with Razzi, we decided to do this in a slightly different order just to allow him to progress and to decouple memcached from the an-tool1010 move. Razzi has a...