[00:32:40] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Druid access for view on event.editeventattempt - https://phabricator.wikimedia.org/T249945 (10fdans) @kaldari hi! can you confirm that the 90 day limitation works for yall? If you require data before that, the fields required must be part of the whitelist... [00:35:39] (03PS3) 10Fdans: Replace numeral with numbro and fix bytes formatting [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/585725 (https://phabricator.wikimedia.org/T199386) [00:36:17] 10Analytics: Superset: "Error while fetching database list" - https://phabricator.wikimedia.org/T249825 (10srishakatux) Trying to recall..I don't think I have ever logged into superset or turnilo before. Anything else I can do to help reproduce the issue? :-/ [01:29:44] 10Analytics, 10ChangeProp, 10Community-Tech, 10Event-Platform, and 4 others: RFC: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10Krinkle) Re-triaging based on T218812#5096038, to reflect current status in the new system. [02:32:42] (03CR) 10Nuria: Allow pageview titles that include Unicode character values above 0xFFFF like emoji (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/589383 (https://phabricator.wikimedia.org/T245468) (owner: 10Lex Nasser) [03:12:33] 10Analytics: Superset: "Error while fetching database list" - https://phabricator.wikimedia.org/T249825 (10Nuria) @srishakatux then you just need to be added to the ldap group that has access to private data, that's it [04:10:01] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Nuria) @nettrom_WMF was wondering if backfill of these events happened as he found a gap cc @Ottomata [04:17:42] (03PS8) 10Nuria: Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [04:18:53] (03CR) 10Nuria: "Tested last patch with:" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [04:20:28] (03CR) 10Nuria: "If changes look good the other two UDFs can be modified in similar manner" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [06:39:20] 10Analytics: Superset: "Error while fetching database list" - https://phabricator.wikimedia.org/T249825 (10elukey) Nuria is right! Just checked and you are not in any LDAP group (wmf/nda). @srishakatux just added you to the `wmf` LDAP group, you should now be able to access. Please check https://wikitech.wikime... [06:39:30] 10Analytics: Superset: "Error while fetching database list" - https://phabricator.wikimedia.org/T249825 (10elukey) [06:39:32] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Users having issues with presto sqllab on superset - https://phabricator.wikimedia.org/T249923 (10elukey) [06:40:06] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Users having issues with presto sqllab on superset - https://phabricator.wikimedia.org/T249923 (10elukey) [06:49:16] (03PS6) 10Lex Nasser: Allow pageview titles that include Unicode character values above 0xFFFF like emoji [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/589383 (https://phabricator.wikimedia.org/T245468) [07:33:55] addshore: o/ [07:34:05] ohaloo [07:34:20] I am reviewing the puppet config of stat1007, and I noticed a "temp" rsync for /srv/log/mw-log/archive/wmde [07:34:24] is it still needed? [07:34:35] comment says "# WMDE temporary debug logs - T171958" [07:34:37] T171958: Requesting access to mwlog1001.eqiad.wmnet for goransm - https://phabricator.wikimedia.org/T171958 [07:44:36] addshore: --^ [07:54:35] GoranSM: ^^ do you still use these logs? I barely remember what they were for [07:54:43] Something to do with the new editor banners I think [07:57:13] if not I'll remove the puppet code and delete them :) [09:32:21] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10elukey) After a chat with Manuel on IRC/Gerrit we decided to get rid of TokuDB tables (recreating them as InnoDB tables) since TokuDB support i... [09:54:23] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10elukey) Procedure followed: * `sudo mysql <<< "SELECT table_schema,table_name FROM INFORMATION_SCHEMA.TABLES WHERE engine = 'TokuDB'" > tokudb... [10:11:37] I need to run something that would eat a LOT of memory, I run it on stat1005, if it causes issues, I stop it (I'm sorry, I can't find any way to run it in smaller place) [10:15:27] Amir1: thanks for reaching out :) so keep it mind that we have systemd cgroup rules that automatically kill (via OOM) the top most processes eating memory when crossing the 85/90% of overall memory usage [10:15:39] on all stat boxes [10:16:07] so ok to run a heavy job, but keep an eye on it if it gets killed etc.. [10:16:16] good, so hopefully it doesn't ring any alarms [10:17:48] no no with this cgroup kill in place we got rid of all the alerts :) [10:19:10] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10jcrespo) Have you considered using InnoDB compression? That should give you around a 50% in disk savings. [10:21:32] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10jcrespo) Also, `mysql -BN -e "SELECT CONCAT(table_schema, '.', table_name) ..."` should give you the above list without an intermediate step. [10:21:35] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10elukey) >>! In T234826#6065172, @jcrespo wrote: > Have you considered using InnoDB compression? That should give you around a 50% in disk savin... [10:26:01] !log re-created default venv for notebooks on notebook100[3,4] (missed to git pull before re-creaing it the last time) [10:26:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:26:08] Andrew-WMDE: o/ [10:26:28] if you are trying to use jupyter on notebook1003 I just fixed it [10:26:32] should work now :) [10:26:45] * elukey lunch! [10:30:43] ^ that was some wizardry [10:30:56] elukey: Yep, it works now! Thanks! [11:03:28] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10Marostegui) You can include it on the same alter with `ENGINE=INNODB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8` [11:31:44] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10jcrespo) >>! In T234826#6065261, @Marostegui wrote: > You can include it on the same alter with `ENGINE=INNODB ROW_FORMAT=COMPRESSED KEY_BLOCK_... [12:06:31] (03CR) 10Mforns: [C: 03+1] "LGTM! @Joal @Nuria, gave a +1, and Lex tested the coord successfully. I let you guys +2." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/576618 (https://phabricator.wikimedia.org/T244597) (owner: 10Lex Nasser) [12:24:00] (03PS9) 10Joal: Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 [12:37:52] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10elukey) @jcrespo do you mean first move one TokuDB table to InnoDB, measure size, and then apply the compressed alter again? The database is no... [12:48:28] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10Marostegui) >>! In T234826#6065354, @jcrespo wrote: >>>! In T234826#6065261, @Marostegui wrote: >> You can include it on the same alter with `E... [13:07:02] 10Analytics: Add Authentication/Encryption to Kafka Jumbo's clients - https://phabricator.wikimedia.org/T250146 (10elukey) [13:14:28] 10Analytics: Add authentication and encryption to Druid Analytics clients - https://phabricator.wikimedia.org/T250484 (10elukey) [13:14:39] 10Analytics: Add authentication and encryption to Druid Analytics clients - https://phabricator.wikimedia.org/T250484 (10elukey) p:05Triage→03Medium [13:16:24] 10Analytics: Add authentication and encryption to Druid Analytics clients - https://phabricator.wikimedia.org/T250484 (10elukey) [13:16:26] 10Analytics, 10User-Elukey: Upgrade Druid to its latest upstream version (currently 0.17.1) - https://phabricator.wikimedia.org/T244482 (10elukey) [13:16:28] 10Analytics, 10User-Elukey: Only hdfs (or authenticated user) should be able to run Druid indexing jobs - https://phabricator.wikimedia.org/T192959 (10elukey) [13:17:31] 10Analytics: Verify if Turnilo can pull data from Druid using Kerberos/TLS - https://phabricator.wikimedia.org/T250485 (10elukey) [13:20:03] 10Analytics: Verify if Turnilo can pull data from Druid using Kerberos/TLS - https://phabricator.wikimedia.org/T250485 (10elukey) There seems to be no support for Kerberos/TLS in Turnilo, see https://github.com/allegro/turnilo/issues/263 Some code will likely need to created and/or upstream convinced about doin... [13:23:42] (03CR) 10Ottomata: [C: 03+1] Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [13:23:58] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10jcrespo) >>! In T234826#6065446, @elukey wrote: > so I'd say that we could go with something like what Manuel suggested and then tune it furthe... [13:26:33] 10Analytics: Verify if Superset can authenticate to Druid via TLS/Kerberos - https://phabricator.wikimedia.org/T250487 (10elukey) [13:27:32] 10Analytics, 10User-Elukey: Secure hue and other private data access sites with 2FA - https://phabricator.wikimedia.org/T159584 (10elukey) p:05Low→03High a:03elukey [13:28:01] 10Analytics, 10User-Elukey: Secure Hue/Superset/Turnilo with CAS (and possibly 2FA) - https://phabricator.wikimedia.org/T159584 (10elukey) [13:32:12] 10Analytics, 10User-Elukey: Secure Hue/Superset/Turnilo with CAS (and possibly 2FA) - https://phabricator.wikimedia.org/T159584 (10elukey) Reporting some thoughts and adding Moritz to the conversation, since he has been involved in rolling out CAS. The idea would be to: 1) Ideally deprecate Hue with Superset... [13:33:05] 10Analytics: Enable Security (stronger authentication and data encryption) for the Analytics Hadoop cluster and its dependent services - https://phabricator.wikimedia.org/T211836 (10elukey) 05Open→03Resolved This monster can be finally close I believe :) [13:36:06] (03PS10) 10Joal: Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 [13:36:15] 10Analytics: Enable Security (stronger authentication and data encryption) for the Analytics Hadoop cluster and its dependent services - https://phabricator.wikimedia.org/T211836 (10Ottomata) 🎉 [13:37:49] ottomata: o/ [13:37:54] I dumped all my thoughts in https://phabricator.wikimedia.org/T199088 [13:38:01] and open tasks etc.. [13:38:15] some details are missing but the bulk of the work is there [13:38:18] and it is a lot :D [13:38:54] nice! [13:39:40] I mean it may take the full next fiscal [13:39:41] (03CR) 10Joal: "I tested on the cluster using the maxmid-test db we use for refinery-tests:" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [13:39:45] ottomata: --^ [13:39:48] sigh [13:40:13] heh at least! [13:41:13] so many tasks for the new SRE [13:41:14] ahahhaah [13:41:16] joal: you wanted to make the static strings private, ya? [13:41:20] and keep the methods public [13:41:20] hah yes! [13:41:38] ottomata: we need both public [13:41:52] and we need he configure method [13:41:56] :( [13:42:02] Meh [13:42:39] configure sure why do you need static strings public? [13:42:49] can't you call getDefaultDatabasePathPropertyName() from the UDF? [13:42:50] ottomata: How do we access them in UDF? [13:43:04] getDefaultDatabasePathPropertyName is not static [13:43:16] and can't be, as static are not abstract [13:43:19] hm [13:43:40] aye k [13:43:46] makes sense you might want to note that for nuria [13:44:08] ottomata: need to drop for groceries - will talk later (I explained yesterday night, you probably weere gone) [13:44:36] s'ok makes sense to me! [13:45:01] !log lock down /srv/log/mw-log/archive/ on stat1007 to analytics-privatedata-users access only [13:45:03] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:51:15] Starting build #1 for job analytics-refinery-update-jars-docker [13:51:21] Project analytics-refinery-update-jars-docker build #1: 04FAILURE in 5.6 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars-docker/1/ [13:51:34] wow! [13:52:02] this was started by our dear friend hashar [13:52:36] exec /run.sh failed: No such file or directory - missing COPY in the docker file? [13:54:24] o/ [13:54:26] oh [13:54:37] sorry I forgot to comment out the mail /irc notification [13:55:58] I am finally moving the update-jars job for refinery to use Docker containers [13:56:04] nice thanks!! [13:57:00] ottomata: I completely misunderstood the mw-api-logs prune rule, it is already configured to drop data after 30d.. and it is 1.3T of compressed text.. Is there somebody reading it? [13:57:13] I mean, do we have a use case to keep all that storage used? :( [13:57:44] elukey@stat1007:/srv/log/mw-log$ du -hsc [13:57:44] 1.3T . [13:57:44] 1.3T total [13:58:48] elukey: i don't know who uses it, i doubt we need that much, and if it anything we can just stick it in hadoop maybe? i dunno, i think they recently got those logs into logstash so maybe they aren't needed anymore [13:59:43] ottomata: yeah I agree, we might just decom this rsync if people don't need it.. [13:59:50] it would be nice to free 1.3T [14:00:37] ya [14:01:42] hmm elukey also [14:01:46] we have api request logs in hive now [14:01:51] i really think we don't need this at all [14:01:55] the mw-api ones at least [14:02:54] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Ottomata) My past self commented that it did! I have a vague memory of doing this too. More info please @nettrom_WMF ? [14:03:46] elukey: those pre date the addition of logstash/kibana stack. So maybe we don't need them much nowadays [14:03:56] elukey: the api.log iirc has every single api queries [14:04:20] oh that is on statXXXX .. so hmm different usage I guess [14:06:56] hashar: thanks for the info! [14:07:41] ottomata: my understanding is that the data is copied from mwlog1001, so in theory we could drop it and restore if there is a valid use case? [14:08:01] yes [14:08:11] also i'm pretty sure that mediawiki_api_request events cover this 100% [14:08:12] and better in hiv [14:08:13] e [14:08:17] because more data and queryable [14:08:22] super [14:10:20] elukey: though I referred to the raw files on mwlog1001.eqiad.wmnet [14:11:15] elukey: the kafka ca cert pathc landed in helm chart repo [14:11:27] ottomata: \o/ [14:11:30] want to submit a patch (with me?) today and then we can apply next week? [14:11:42] that would be awesome [14:11:48] can I just file the patch for the mw logs? [14:12:13] yes [14:12:38] elukey: how should we do eventgate kafka tls patch, bc? you do it I review? I do it you review? [14:13:38] we can do bc you do and I follow [14:13:43] asking n00b questions [14:14:09] ok [14:14:16] now good? [14:15:09] actually, 5 mins... [14:15:09] gimme 5 that I am finishing the mwlog patch :) [14:15:11] YES! [14:18:06] https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/589600/ [14:18:11] super happy about it [14:20:51] k in bc luca whenver you are aready [14:29:55] (03CR) 10Nuria: ">the configure function is needed for manual setting to take effect in hive" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [14:44:35] (03PS1) 10Fdans: Fix language dropdown for ios devices [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/589606 (https://phabricator.wikimedia.org/T246971) [14:44:58] (03PS2) 10Fdans: Fix language dropdown for ios devices [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/589606 (https://phabricator.wikimedia.org/T246971) [14:50:31] (03PS1) 10Hashar: update-refinery-source-jars: show diff stat in dry-run [analytics/refinery] - 10https://gerrit.wikimedia.org/r/589609 [14:50:33] (03PS1) 10Hashar: update-refinery-source-jars: use curl instead of wget [analytics/refinery] - 10https://gerrit.wikimedia.org/r/589610 [14:58:02] (03CR) 10Hashar: "wget always show a progress indicator which is updated every few kilobytes, that spams the CI console log heavily. We could run wget with" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/589610 (owner: 10Hashar) [15:01:39] 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10hashar) https://gerrit.wikimedia.org/r/#/c/integration/config/+/589589/ is t... [15:09:49] 10Analytics, 10Research: Proposed adjustment to wmf.wikidata_item_page_link to better handle page moves - https://phabricator.wikimedia.org/T249773 (10Isaac) Thanks @Milimetric ! @MGerlach proposed that we build a query that gathers all of the data across all of the snapshots to help temporarily address this... [15:39:05] elukey: https://meet.google.com/kqr-znrt-nrz [15:53:44] (03CR) 10Nuria: [C: 03+1] Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [15:54:17] ottomata: We can merge the UDF :) [15:54:20] \o/ [15:54:25] 10Analytics, 10Patch-For-Review: Enable TLS encryption from Eventgate Analytics to Kafka Jumbo - https://phabricator.wikimedia.org/T250149 (10Ottomata) All staging eventgate services are now using TLS. We will do the production eventgate-main and eventgate-analytics on Monday. [15:54:30] joal: yeehaw merge away! [15:55:01] (03CR) 10Joal: [C: 03+2] "Merging to facilitate new patches using that one." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [15:55:12] ottomata: will be merged by jenkins in a few minutes [15:59:43] (03Merged) 10jenkins-bot: Update hive geocoded-data udf [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/588715 (owner: 10Joal) [16:15:23] joal: something fun [16:15:49] yesterday one day of atskafka logs (snappy compressed) for one cp host is 36G x 12 partitions [16:15:52] ahhahahah [16:16:24] there are a ton of things to filter [16:20:02] wow - this feels a lot indeed :) [16:20:31] I found a lot of things like http_status: 000 [16:21:03] trying to get what they are, maybe pre-fetches or internals, surely things to drop on the atskafka side [16:21:21] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10nettrom_WMF) @Ottomata : Sure thing! Last week @jwang and I identified missing data in several EventBus schemas in the Data Lake. I'm not sure wh... [16:44:43] (03CR) 10Nuria: [C: 03+2] Allow pageview titles that include Unicode character values above 0xFFFF like emoji [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/589383 (https://phabricator.wikimedia.org/T245468) (owner: 10Lex Nasser) [16:49:46] (03Merged) 10jenkins-bot: Allow pageview titles that include Unicode character values above 0xFFFF like emoji [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/589383 (https://phabricator.wikimedia.org/T245468) (owner: 10Lex Nasser) [17:08:51] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Ottomata) 👀 That is not good! I think I was not aware at the time that this issue affected anything except for mediawiki_api_request and mediawik... [17:18:22] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Nuria) >could we use the anomaly detection work you did for webrequest to get alerts for things like this? I think this is a much simpler problem... [17:18:31] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Nuria) 05Resolved→03Open [17:24:49] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10Ottomata) IIRC we have some basic alarms on thresholds in Kafka topics, but not in refined event tables. The anomaly stuff could be nice to autom... [17:25:13] 10Analytics, 10Operations, 10Traffic: Create replacement for Varnishkafka - https://phabricator.wikimedia.org/T237993 (10elukey) There is currently too much data that flows to kafka, for cp3050 we have 36GB * 12 partitions for a single day, definitely too much. I took a look to kafka messages and using `kaf... [17:36:59] * elukey off! [17:39:42] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: High volume mediawiki analytics events camus import is lagging - https://phabricator.wikimedia.org/T233718 (10nettrom_WMF) Yeah, I kinda figured that the data was lost as this point. For the edit-related tables, it's not a huge concern because I think all... [17:58:55] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Develop test environment solution for MEP analytics events - https://phabricator.wikimedia.org/T238837 (10mpopov) a:05mpopov→03None [17:59:08] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics, 10Product-Infrastructure-Team-Backlog: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10mpopov) [21:10:37] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Druid access for view on event.editeventattempt - https://phabricator.wikimedia.org/T249945 (10dr0ptp4kt) @kaldari should weigh in on the 90 day window. @fdans, to clarify, there are two pieces here, right? (1) initial import: If we want anything from mor... [23:59:13] 10Analytics, 10Product-Analytics (Kanban), 10Readers-Web-Backlog (Needs Product Owner Decisions), 10covid-19: Weekly updates on editors - https://phabricator.wikimedia.org/T248427 (10jwang) Have published the weekly editing dashboard at [[ https://analytics.wikimedia.org/published/notebooks/weekly_edits/we... [23:59:54] 10Analytics, 10Product-Analytics (Kanban), 10Readers-Web-Backlog (Needs Product Owner Decisions), 10covid-19: Weekly updates on editors - https://phabricator.wikimedia.org/T248427 (10jwang)