[02:24:21] 10Analytics, 10Event-Platform, 10Growth-Team, 10MediaWiki-Revision-backend, and 7 others: Replace LinksUpdate Revision methods with RevisionRecord - https://phabricator.wikimedia.org/T249397 (10DannyS712) [02:44:12] 10Analytics, 10Event-Platform, 10Growth-Team, 10MediaWiki-Revision-backend, and 8 others: Replace LinksUpdate Revision methods with RevisionRecord - https://phabricator.wikimedia.org/T249397 (10DannyS712) [06:04:23] 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Experiment with Druid and SqlAlchemy - https://phabricator.wikimedia.org/T249681 (10elukey) [06:34:49] 10Analytics, 10Event-Platform, 10Growth-Team, 10MediaWiki-Revision-backend, and 8 others: Replace LinksUpdate Revision methods with RevisionRecord - https://phabricator.wikimedia.org/T249397 (10DannyS712) [06:59:25] 10Analytics, 10Operations, 10ops-eqiad: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10ayounsi) FYI, `kafka-jumbo1008` switch port has been flapping and flooding logs. Please disable the switch port if the host is neither in production nor be... [07:01:14] joal: bonjour! [07:01:16] interesting https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&refresh=5m&var-server=an-launcher1001&var-datasource=eqiad%20prometheus%2Fops&var-cluster=analytics&from=now-24h&to=now [07:02:20] happens every day at around the same time [07:02:40] and I think those are the refinery-import-* timers [07:02:52] not a big deal, some cpu usage is fine, but nice to know [07:05:42] maybe I can ask to the SRE team to have more vcores for that vm [07:06:12] that's fine, just open a Phab task and tag is vm-requests [07:07:57] moritzm: sure :) [07:08:17] I also added to the "hw requirements" for analytics that we'll need 4/5 vms next fiscal like that one [07:08:22] just to account for usage in Ganeti [07:08:45] (I don't know the exact number but it is good to have an idea) [07:43:59] 10Analytics, 10Operations, 10ops-eqiad: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10elukey) >>! In T244506#6038826, @ayounsi wrote: > FYI, `kafka-jumbo1008` switch port has been flapping and flooding logs. > > Please disable the switch por... [08:08:41] 10Analytics, 10Operations, 10ops-eqiad: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10elukey) Nope serial settings are good, but I have powered it down to avoid spamming logs while we work on partman. [08:20:08] 10Analytics, 10Operations, 10ops-eqiad: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10elukey) this is what is displayed before the error msg that Chris pointed out: ` ┌─────────────────────────┤ [!] Partition disks ├───────────────────────... [09:16:06] 10Analytics, 10Performance-Team: Release performance data on a regular schedule - https://phabricator.wikimedia.org/T205342 (10Gilles) a:05Gilles→03None [09:18:04] (03PS1) 10WMDE-Fisch: Only track unique users disabling TwoColConflict [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/587496 (https://phabricator.wikimedia.org/T247944) [09:21:51] (03CR) 10Ladsgroup: [C: 03+2] Only track unique users disabling TwoColConflict [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/587496 (https://phabricator.wikimedia.org/T247944) (owner: 10WMDE-Fisch) [09:22:12] (03Merged) 10jenkins-bot: Only track unique users disabling TwoColConflict [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/587496 (https://phabricator.wikimedia.org/T247944) (owner: 10WMDE-Fisch) [09:24:23] (03CR) 10WMDE-Fisch: "Thanks :-)!" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/587496 (https://phabricator.wikimedia.org/T247944) (owner: 10WMDE-Fisch) [10:04:55] https://databricks.com/session/rocm-and-distributed-deep-learning-on-spark-and-tensorflow [10:09:28] very cool http://numba.pydata.org/ [10:11:33] wow Yarn in hops supports rocm [10:14:36] https://www.logicalclocks.com/blog/welcoming-amd-rocm-to-hopsworks [10:20:38] opened https://issues.apache.org/jira/browse/YARN-10225 just in case :) [10:25:30] they (hops) also use conda as we are planning to [10:34:47] * elukey lunch! [10:36:14] is it that time already? [10:39:10] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-Focus-Sprint): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10santhosh) @MoritzMuehlenhoff Thanks. Just to confirm, I can just build marian(that will use CPU optmized openblas) and do... [10:54:31] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-Focus-Sprint): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10MoritzMuehlenhoff) Exactly, the OpenBLAS available in Debian Buster (which stat1008 uses provides CPU-optimized computatio... [11:25:39] elukey: hi - about to send a disclaimer on the message you sent on sql-on-druid [11:26:01] elukey: do ou wish to read it before I send it ? [11:29:53] No answer, sending :) [12:00:11] joal: ah yes you are always more precise than me, thanks :) [12:00:17] makes sense [12:01:38] what I wanted to say was that we should stop using druid datasources [12:01:46] and use sql-druid-datasources [12:01:55] and see if people come up with complains etc.. [12:02:39] (brb) [12:28:28] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-Focus-Sprint): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10santhosh) |Load | Openblas(stat1008) performance testing | GPU based OpusMT |  10 requests concurrency 1| {P10944} | {... [12:42:44] (03CR) 10Joal: "A bunch of comments :)" (037 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/587305 (https://phabricator.wikimedia.org/T238230) (owner: 10Ottomata) [13:04:12] (03CR) 10Joal: "Detail about comments, otherwise looks good" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/586432 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns) [13:22:42] o/! any reason I can not tweet about the graphs in https://superset.wikimedia.org/superset/dashboard/108/ ? mainly the top 2. [13:23:04] I guess there is nothing stopping me from doing so? I might include them in my next blog post about this whole thing [13:26:54] addshore: I thought there was a proper channel to ask for publishing data, this seems good but it needs a sign-off from somebody in my opinion (since it comes from superset) [13:27:58] * addshore re reads the pages nur_ia sent him last week [13:39:19] (03PS2) 10Mforns: Add new dimensions to druid pageviews_daily [analytics/refinery] - 10https://gerrit.wikimedia.org/r/586432 (https://phabricator.wikimedia.org/T243090) [13:40:16] (03CR) 10Mforns: [V: 03+2] Add new dimensions to druid pageviews_daily (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/586432 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns) [13:41:40] (03CR) 10Joal: [V: 03+2 C: 03+2] "Thanks for the patch - LGTM merging" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/586432 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns) [13:42:38] Thanks mforns for the patch --^ The comment is very usefull IMO as it reminds us that those settings are not that easy ;) [13:43:42] I guess https://wikitech.wikimedia.org/wiki/Analytics/Data_Access_Guidelines#Sharing_data_externally pretty much overs my case which says "If you are unsure whether a type of data constitutes sensitive information, please reach out to your department's privacy contact or the Security team." [13:44:53] I think I was more thinking perhaps wmf wants to write something more substantial about the increase in page views, and have a nice thing about it, in which case I would wait for that to surface and link to that [13:49:09] joal: the comment makes sense! thanks for merging :] [15:09:59] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Technical contributors metrics definition - https://phabricator.wikimedia.org/T247419 (10jwang) Add number for the newly ended quarter in 2020 |time frame |num_submitter |num_changeset| |-----------|-------------|---------------| |2018-01 ~ 2018-03|165 |... [15:14:18] joal: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DevelopYourOwnDevicePlugin.md [15:15:00] Thanks a lot elukey, will read that! [15:19:41] elukey: wanna tardis 5min for kerberos? [15:21:09] mforns: sure! [15:21:12] omw [15:50:06] 10Analytics, 10Better Use Of Data, 10Product-Analytics, 10Epic, 10Product-Infrastructure-Team-Backlog (Kanban): Session Length Metric. Web implementation - https://phabricator.wikimedia.org/T248987 (10jlinehan) p:05Triage→03Medium [15:50:18] 10Analytics, 10Better Use Of Data, 10Wikimedia-Logstash, 10Documentation, and 3 others: Documentation of client side error logging capabilities on mediawiki - https://phabricator.wikimedia.org/T248884 (10jlinehan) p:05Triage→03Low [16:02:00] (03CR) 10Awight: Reject invalid Page titles (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498702 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [16:20:53] (03CR) 10Awight: Reject invalid Page titles (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498702 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [16:53:34] 10Analytics: Rewrite cassandra loading - https://phabricator.wikimedia.org/T249735 (10Nuria) [16:58:05] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Infrastructure-Team-Backlog: Produce an instrumentation event stream using new EPC and EventGate from client side browsers - https://phabricator.wikimedia.org/T241241 (10Ottomata) In meeting today we made a decision. To be clear, here's what t... [16:58:49] 10Analytics, 10Analytics-EventLogging, 10Event-Platform, 10Patch-For-Review: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10Ottomata) In https://phabricator.wikimedia.org/T241241#6040663 we have a decision, legacy eventlogging schemas will be... [17:16:34] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10elukey) @Cmjohnson I powered up again 1008 and I don't see any DHCP ACK in syslog when PXE installing: ` Apr 8 17:14:09 install1003... [17:27:28] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: (Need by: TBD) rack/setup/install kafka-jumbo100[789].eqiad.wmnet - https://phabricator.wikimedia.org/T244506 (10elukey) I am also not able to ssh to `kafka-jumbo1007.mgmt.eqiad.wmnet` :( [17:43:24] * elukey off! [18:05:03] (03CR) 10Nuria: "Sounds like the thing to do here is have lex move this change forward?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498702 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [18:31:33] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Define reduce calculations needed to compute active editors per project family - https://phabricator.wikimedia.org/T249751 (10Nuria) [18:35:40] 10Analytics: Decomission notebook hosts - https://phabricator.wikimedia.org/T249752 (10Nuria) [18:37:09] (03CR) 10Awight: "> Sounds like the thing to do here is have lex move this change" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498702 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [18:38:01] 10Analytics, 10Analytics-Kanban: Unify stat1007 puppet role with the rest of the stats cluster - https://phabricator.wikimedia.org/T249754 (10Nuria) [18:41:17] 10Analytics, 10Cassandra: Cassandra3 migration for Analytics AQS - https://phabricator.wikimedia.org/T249755 (10Nuria) [18:41:43] 10Analytics, 10Cassandra: Cassandra3 migration plan proposal - https://phabricator.wikimedia.org/T249756 (10Nuria) [18:44:47] 10Analytics, 10Analytics-Kanban: Hourly labeling of "automated" traffic before loading of pageviews into pageview_hourly - https://phabricator.wikimedia.org/T238361 (10Nuria) 05Open→03Resolved [18:44:50] 10Analytics: Deploy high volume bot spike detector to hungarian wikipedia - https://phabricator.wikimedia.org/T238358 (10Nuria) [18:48:39] 10Analytics, 10Patch-For-Review: Use spark to split webrequest on tags - https://phabricator.wikimedia.org/T164020 (10Nuria) 05Open→03Declined [18:48:44] 10Analytics: Webrequest tagging and distribution. Measuring non-pageview requests - https://phabricator.wikimedia.org/T164019 (10Nuria) [18:51:40] 10Analytics, 10Analytics-Wikistats: Combine filters and splits on wikistats UI - https://phabricator.wikimedia.org/T249758 (10Nuria) [18:57:04] 10Analytics: Add hourly resolution to data quality outage/censhorship alarms - https://phabricator.wikimedia.org/T249759 (10Nuria) [18:57:29] 10Analytics, 10Analytics-Kanban: Add hourly resolution to data quality outage/censhorship alarms - https://phabricator.wikimedia.org/T249759 (10Nuria) a:03mforns [19:04:34] 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10Nuria) Closing, thanks everyone for prompt responses [19:04:44] 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10Nuria) 05Open→03Resolved [19:06:03] 10Analytics, 10Analytics-Wikistats: Needs Design: combine multiple filters and/or splits - https://phabricator.wikimedia.org/T183316 (10Nuria) [19:06:05] 10Analytics, 10Analytics-Wikistats: Combine filters and splits on wikistats UI - https://phabricator.wikimedia.org/T249758 (10Nuria) [19:06:10] 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10Jdforrester-WMF) 05Resolved→03Open Unfortunately only half of this is do... [19:07:10] 10Analytics, 10Analytics-Kanban: Upgrade jupyterhub-systemdspawner from 0.9.9 to 0.13 to allow the use of systemd custom slices - https://phabricator.wikimedia.org/T247055 (10Nuria) 05Open→03Resolved [19:09:33] 10Analytics, 10Analytics-Kanban, 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Patch-For-Review, and 2 others: Migrate analytics/refinery/source release jobs to Docker - https://phabricator.wikimedia.org/T210271 (10Nuria) Ah , i see, the update-jars is not done [19:12:08] (03CR) 10Nuria: "Since this is merged, please make note of it in the train page: https://etherpad.wikimedia.org/p/analytics-weekly-train" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/586432 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns) [19:12:58] joal: can you update the deployment train page, i think some things amarked as "next deployment" are the ones actually done in teh last one but i might be wrong [19:12:59] https://etherpad.wikimedia.org/p/analytics-weekly-train [19:13:41] correct nuria - I ddin't update the doc - my bad [19:15:01] done nuria [19:35:19] 10Analytics, 10Analytics-SWAP: Spark Scala kernel dying under Jupyter - https://phabricator.wikimedia.org/T249761 (10awight) [20:25:49] 10Analytics, 10Analytics-SWAP: Spark Scala kernel dying under Jupyter - https://phabricator.wikimedia.org/T249761 (10Nuria) Are you running these on stats machines , stat1006/stat1005? [21:00:25] 10Analytics, 10Research: Proposed adjustment to wmf.wikidata_item_page_link to better handle page moves - https://phabricator.wikimedia.org/T249773 (10Isaac) [21:11:59] 10Analytics, 10Analytics-SWAP: Spark Scala kernel dying under Jupyter - https://phabricator.wikimedia.org/T249761 (10awight) >>! In T249761#6041587, @Nuria wrote: > Are you running these on stats machines , stat1006/stat1005? Thanks, I should have specified: I'm running on notebook1003. My pipeline so far is...