[06:02:20] morning! [06:02:56] hi! [06:19:13] (03PS1) 10Fdans: Replace literal "anonymous editor" with null [analytics/aqs] - 10https://gerrit.wikimedia.org/r/468927 (https://phabricator.wikimedia.org/T206968) [06:44:26] druid 0.12.3 seems indexing fine! [09:34:59] and turnilo deployed, seems working fine [10:20:39] 10Analytics, 10Operations, 10ops-eqiad: Degraded RAID on analytics1029 - https://phabricator.wikimedia.org/T207644 (10elukey) p:05Triage>03Normal [10:20:57] 10Analytics, 10Operations, 10ops-eqiad: Degraded RAID on analytics1029 - https://phabricator.wikimedia.org/T207644 (10elukey) This host needs to be decommed soon, so let's not replace the disk. [11:18:29] (03CR) 10Fdans: "I realize this is already merged, I'll correct this in my next patch" (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/467868 (https://phabricator.wikimedia.org/T199693) (owner: 10Milimetric) [12:08:05] 10Analytics, 10Operations, 10ops-eqiad: Degraded RAID on analytics1029 - https://phabricator.wikimedia.org/T207644 (10elukey) @Ottomata one nice thing that I forgot - we can keep running this host without stopping the HDFS daemons due to: ``` dfs.datanode.failed.volumes.tolerated 10Analytics, 10Analytics-Kanban, 10User-Elukey: Upgrade to Druid 0.12.3 - https://phabricator.wikimedia.org/T206839 (10elukey) Deployed turnilo on turnilo.eqiad.wmflabs, seems working fine. To test it: ``` ssh -L 9091:turnilo.eqiad.wmflabs:9091 turnilo.eqiad.wmflabs ``` [12:13:11] 10Analytics, 10Analytics-Kanban, 10User-Elukey: Upgrade to Druid 0.12.3 - https://phabricator.wikimedia.org/T206839 (10elukey) This indexation example worked fine: ``` { "type" : "index_hadoop", "spec" : { "ioConfig" : { "type" : "hadoop", "inputSpec" : { "type" : "static",... [12:38:24] (03PS1) 10Fdans: Handle null name values in top metrics from UI [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/468964 (https://phabricator.wikimedia.org/T206968) [12:58:28] 10Analytics, 10Language-strategy, 10Tool-Pageviews: Have a way to show the most popular pages per country - https://phabricator.wikimedia.org/T207171 (10Amire80) [13:36:02] o/ [13:40:18] 10Analytics, 10EventBus, 10MediaWiki-Watchlist, 10WMF-JobQueue, and 6 others: Clear watchlist on enwiki only removes 50 items at a time - https://phabricator.wikimedia.org/T207329 (10kostajh) [13:48:05] o/ [13:54:21] hey teaaam :] [14:01:41] holaaa [14:02:10] * elukey afk for a bit! [14:14:06] (03CR) 10Mforns: [C: 032] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/463548 (https://phabricator.wikimedia.org/T202490) (owner: 10Joal) [14:14:20] (03CR) 10Mforns: [V: 032 C: 032] Add mediawiki-history-wikitext oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/463548 (https://phabricator.wikimedia.org/T202490) (owner: 10Joal) [14:20:37] a-team, going to deploy source/refinery, I merged a couple small Gerrit changes for refinery that I will include, does anyone need sth from source merged ASAP? If so, will review before merge [14:20:53] not me [14:20:55] *before deploy [14:23:52] ok, deploying, if sth comes I can redeploy [14:24:08] (03PS1) 10Mforns: Update changelog.md for v0.0.79 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/468991 [14:24:44] !log Starting deployment of refinery-source using jenkins [14:24:46] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:32:51] (03CR) 10Gilles: [V: 032] Keep recently added navtiming + survey fields (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/466607 (https://phabricator.wikimedia.org/T187299) (owner: 10Gilles) [14:35:35] (03PS1) 10Gilles: Discard properties that could be combined to deanonymize [analytics/refinery] - 10https://gerrit.wikimedia.org/r/468993 [14:50:32] !log Finished deployment of refinery-source using jenkins [14:50:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:51:19] !log Starting deployment of refinery using scap and refinery-deploy-to-hdfs [14:51:21] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:51:43] a-team: I'm in the airport, boarding in a lil bit, so I won't be at meetings. Will send e-scrum later :) [14:52:19] k fdans, I'm taking a look at the code reviews now, will have something for you soon [14:52:32] milimetric: thank youuu [14:55:17] (03CR) 10Milimetric: Clean up and order table chart properly (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/467868 (https://phabricator.wikimedia.org/T199693) (owner: 10Milimetric) [15:02:30] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Services (watching): Prototype in node intake service - https://phabricator.wikimedia.org/T206815 (10Ottomata) [15:09:37] !log Finished deployment of refinery using scap and refinery-deploy-to-hdfs [15:09:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:11:18] 10Analytics, 10Android-app-Bugs, 10Product-Analytics, 10Reading-analysis, and 3 others: Many errors on ReadingDepth.enable (?) schema - https://phabricator.wikimedia.org/T207423 (10Nuria) See graph of errors: https://grafana.wikimedia.org/dashboard/db/eventlogging?panelId=13&fullscreen&orgId=1&from=now-2d&... [15:11:58] (03CR) 10Mforns: [C: 032] Update changelog.md for v0.0.79 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/468991 (owner: 10Mforns) [15:18:38] 10Analytics, 10Community-Tech, 10Grant-Metrics: Review category queries - https://phabricator.wikimedia.org/T206783 (10Milimetric) p:05Triage>03High [15:22:41] 10Analytics: events_sanitized could drop columns like recvfrom and sequenceId - https://phabricator.wikimedia.org/T207431 (10Milimetric) 05Open>03declined recvfrom is in the capsule, sequenceId may be useful to troubleshoot events in the past [15:24:45] 10Analytics, 10Project-Admins: Create project for SWAP - https://phabricator.wikimedia.org/T207425 (10Milimetric) Seems fine by us, but when phab admins approve it someone should add a herald rule to tag anything with Analytics-SWAP with #Analytics also. And Analytics-SWAP is fine with us - you get to name it! [15:27:58] 10Analytics, 10Android-app-Bugs, 10Product-Analytics, 10Reading-analysis, and 3 others: Many errors on ReadingDepth.enable (?) schema - https://phabricator.wikimedia.org/T207423 (10Milimetric) p:05High>03Unbreak! [15:35:23] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10Milimetric) Just let us know what the domain/name of the site is, when it's up, and we can make you a little tracking code. Also, do you have any estimates of traffic? [15:38:13] 10Analytics: Register refinery-hive UDF functions as stored names, so we don't have to CREATE TEMPORARY FUNCTION every time. - https://phabricator.wikimedia.org/T95455 (10Milimetric) 05Open>03declined [15:38:58] 10Analytics, 10Analytics-Cluster: Import Kafka messages into HDFS authenticating with TLS/SSL - https://phabricator.wikimedia.org/T166832 (10Milimetric) p:05Lowest>03Low [15:41:15] 10Analytics, 10Analytics-Kanban: Order Data Lake Hardware - https://phabricator.wikimedia.org/T198424 (10Milimetric) [15:41:17] 10Analytics: Order hardware labs storage for mediawiki history analytics friendly DB - https://phabricator.wikimedia.org/T175604 (10Milimetric) [15:43:31] 10Analytics, 10Pageviews-API, 10User-Elukey: Improve user management for AQS Cassandra - https://phabricator.wikimedia.org/T142073 (10Milimetric) [15:46:55] 10Analytics, 10Performance-Team (Radar): Possible statsv corruption? - https://phabricator.wikimedia.org/T189530 (10Milimetric) p:05Low>03Normal [15:48:16] 10Analytics, 10Analytics-EventLogging, 10goodfirstbug: Archive and drop the MobileOptionsTracking EventLogging MySQL table - https://phabricator.wikimedia.org/T185339 (10Milimetric) a:03Nuria [15:50:21] 10Analytics: Set up sparksql to try out sql access to hdfs data - https://phabricator.wikimedia.org/T196661 (10Milimetric) If we set up presto on the Hadoop cluster, it would supersede this idea [15:57:36] 10Analytics, 10Operations, 10Traffic, 10User-Elukey: Sort out analytics service dependency issues for cp* cache hosts - https://phabricator.wikimedia.org/T128374 (10Milimetric) The fix for this would be high risk and low gain. So keeping around to just have context in case the problem does manifest. [15:59:58] 10Analytics: Scan npm dependencies for vulnerabilities - https://phabricator.wikimedia.org/T200717 (10Milimetric) [16:02:46] nuria: mforns my spanish friends. did you know: that Andalusia is named after the Vandals, who migrated from scandanavia/poland and made their way to spain the early 400s? [16:02:53] Andalusia is really Vandalusia without the V! [16:03:05] xDDDDD [16:04:24] https://en.wikipedia.org/wiki/Name_of_Andalusia [16:05:57] :) [16:06:28] (ok well at least that what Issac Asimov says in his "Chronology of the World" which is near my toilet... :p ) [16:09:10] heh I remember that book [16:12:45] ottomata: sounds a little made up but hey if isaac asimov says so [16:14:40] ottomata: thanks for looking at slides [16:16:19] (03CR) 10Nuria: Keep recently added navtiming + survey fields (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/466607 (https://phabricator.wikimedia.org/T187299) (owner: 10Gilles) [16:27:25] (03CR) 10Nuria: Handle null name values in top metrics from UI (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/468964 (https://phabricator.wikimedia.org/T206968) (owner: 10Fdans) [16:27:49] 18/10/22 16:20:08 ERROR kafka.CamusJob: failed to create decoder [16:27:49] com.linkedin.camus.coders.MessageDecoderException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.linkedin.camus.etl.kafka.coders.StringMessageD [16:27:53] ecoder not found [16:27:56] ufff [16:28:25] the class should be in the new camus [16:29:12] ahhhh [16:29:13] --jar /srv/deployment/analytics/refinery/artifacts/org/wikimedia/analytics/camus-wmf/camus-wmf-0.1.0-wmf7.jar [16:29:37] but [16:29:37] elukey@an-coord1001:~$ ls /srv/deployment/analytics/refinery/artifacts/org/wikimedia/analytics/camus-wmf/ [16:29:41] camus-wmf-0.1.0-wmf6.jar camus-wmf-0.1.0-wmf7.jar [16:29:52] there should be -wmf9 in there [16:30:05] hmm [16:30:17] maybe it isn't added by the deploy jenkins stuff? [16:30:26] elukey: you can manually add it and deploy refinery (not refinery source) [16:32:13] ottomata: you mean under artifacts/org/wikimedia/analytics/camus-wmf/ right? [16:32:14] (03CR) 10Nuria: [V: 032 C: 032] Discard properties that could be combined to deanonymize [analytics/refinery] - 10https://gerrit.wikimedia.org/r/468993 (owner: 10Gilles) [16:32:16] (in refinery) [16:34:17] yeah looks like what we did the last time https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/276120/ [16:35:52] 10Analytics, 10Readers-Web-Backlog: Popups schema is keeping page/session id longer than 90 days - https://phabricator.wikimedia.org/T207670 (10Nuria) [16:36:00] 10Analytics, 10Readers-Web-Backlog: Popups schema is keeping page/session id longer than 90 days - https://phabricator.wikimedia.org/T207670 (10Nuria) a:03Jdlrobson [16:36:34] elukey: yup [16:38:45] ottomata: is https://archiva.wikimedia.org/repository/snapshots/org/wikimedia/analytics/camus-wmf/ correct? [16:39:02] IIRC we had wmf9 in there the last time [16:39:05] elukey: you are looking in snapshots [16:39:10] https://archiva.wikimedia.org/repository/releases/org/wikimedia/analytics/camus-wmf/ [16:39:16] ahhhhhh [16:39:19] wrong link [16:39:21] bad luca [16:39:33] thanks :) [16:42:14] 10Analytics-Tech-community-metrics, 10Code-Health, 10Release-Engineering-Team (Kanban): Develop canonical/single record of origin, machine readable list of all repos deployed to WMF sites. - https://phabricator.wikimedia.org/T190891 (10Jrbranaa) [16:57:54] ottomata: sorry for the dumb question but is there a specific way to add the jar via git-fat? [16:58:01] never done it [17:01:38] elukey: if your repo is git fat init-ed [17:01:40] just git add is all you need [17:02:11] it is probably not fat inited then [17:06:33] elukey: [17:06:34] https://github.com/wikimedia/analytics-refinery [17:06:49] and [17:06:49] https://wikitech.wikimedia.org/wiki/Archiva#Setting_up_git-fat_for_your_project [17:07:14] If the steps below have already been run by someone for your repository (i.e. .gitfat and .gitattributes exist with proper content), then all you have to do is install git-fat, cd into your clone, and then run git-fat init [17:08:47] yeah I am trying.. basically git fat init, download the jar from archiva, git add + commit, git review [17:08:56] ls [17:08:59] argh [17:09:23] ah yes now it works [17:09:52] (03PS1) 10Elukey: Add camus-wmf-0.1.0-wmf9.jar to refinery [analytics/refinery] - 10https://gerrit.wikimedia.org/r/469033 [17:09:56] there it si [17:10:11] ottomata: ---^ [17:10:29] gr8! [17:10:35] if it is ok for you I'll +2 and deploy [17:10:38] (03CR) 10Ottomata: [C: 031] Add camus-wmf-0.1.0-wmf9.jar to refinery [analytics/refinery] - 10https://gerrit.wikimedia.org/r/469033 (owner: 10Elukey) [17:10:48] (03CR) 10Elukey: [V: 032 C: 032] Add camus-wmf-0.1.0-wmf9.jar to refinery [analytics/refinery] - 10https://gerrit.wikimedia.org/r/469033 (owner: 10Elukey) [17:12:55] !log deploy refinery (new version of camus) [17:12:57] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:21:40] !log deploy refinery to hdfs (via stat1005) [17:21:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:24:24] !log upgraded camus jar version in an-coordq1001's crontab (via puppet) [17:24:25] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:24:36] all done! [17:25:05] coooo [17:26:42] ottomata, you already +1'd, can you merge if apropriate? https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/468588/ Thaaaanks! [17:31:14] mforns: done and puppet ran on an-coord1001 [17:31:25] whoooo! [17:31:27] thanks! [17:35:39] milimetric: wanna cave? [17:35:47] intellij tests and then node eventbus? [17:36:09] I’m out at lunch but def. After [17:37:04] k [17:37:09] ottomata: ^ [17:37:30] (sorry, slow network) [17:54:04] 10Quarry, 10Cloud-Services, 10cloud-services-team (Kanban): Migrate 'Quarry' project to eqiad1 - https://phabricator.wikimedia.org/T207677 (10Andrew) p:05Triage>03Normal [17:59:04] going off! Will check camus later on :) [17:59:57] laters! [18:09:01] 10Analytics, 10Project-Admins: Create project for SWAP - https://phabricator.wikimedia.org/T207425 (10Aklapper) (General note: Given that our [[ https://phabricator.wikimedia.org/T108586#4602901 | Herald performance gets worse ]] I'd generally would like to see teams consider [[ https://www.mediawiki.org/wiki/... [18:19:46] 10Analytics, 10Analytics-EventLogging, 10Growth-Team, 10MediaWiki-ContentHandler, and 4 others: Multiple MediaWiki hooks are not documented on mediawiki.org - https://phabricator.wikimedia.org/T157757 (10Anomie) [18:34:15] (camus looks good!) [18:39:03] yeehaw [18:39:54] ottomata: ok, ready when you are [18:40:58] ok! [18:41:15] milimetric: going to bc [18:43:48] I've been working with spark in Yarn for my reading time analysis. There is a pre-processing step that works on a sample, but I'm having trouble scaling to the full dataset. Would it be OK for me to use a custom Yarn kernels to increase RAM or number of workers? [18:44:06] https://wikitech.wikimedia.org/wiki/SWAP#Custom_Spark_Kernels [18:44:13] Or do you think I should stick with Hive? [18:48:42] groceryheist: yes! you can do that [18:48:57] if you end up using too much we'll get grumpy and let you know [18:49:01] so be aware of that [18:49:06] but you should def try it out [18:53:43] ottomata: Alright! [18:59:53] ottomata, finally EL data looks good on Turnilo! thanks for allll! [19:02:45] 10Analytics, 10Android-app-Bugs, 10Product-Analytics, 10Reading-analysis, and 5 others: Many errors on ReadingDepth.enable (?) schema - https://phabricator.wikimedia.org/T207423 (10Jdlrobson) Change is live in 1.32.0-wmf.26. I'm still seeing errors when I run kafkacat but this is likely to be the case beca... [19:12:37] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10atgo) As far as traffic estimates, industry standards x our view targets for the videos says about 100k clickthroughs during the campaign. [19:19:49] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10ayounsi) We currently have the following relevant vlans: - private: 10/8 IPs, not reachable from cloud instances and the... [19:20:37] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10ayounsi) a:05ayounsi>03Ottomata [19:24:02] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10faidon) How many servers are we talking about both right now, as well as in the mid-term e.g. in the next year or two? H... [19:35:25] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10Ottomata) Right now, 8: 3 ganeti instances and 5 bare metal worker nodes. We wouldn't be adding more nodes in within thi... [19:36:34] 10Analytics, 10Analytics-Kanban, 10Operations, 10netops: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster - https://phabricator.wikimedia.org/T207321 (10Ottomata) Where are the labsdb hosts going to live if they are being moved out of the labs-support VLAN? Likely these cl... [20:06:26] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10Nuria) @atgo: 100K clickthrough over what period of time? What is the top level site? [20:07:42] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10atgo) ~1 month. This is on es.wiki - I'm not sure if that's what you mean by top level site. [20:20:40] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10Nuria) @atgo, let me understand, we thought this was a microsite, like http://transparency.wikimedia.org or http://reserach.wikimedia.org which is what we normally measure with piwik, is that not the case?... [20:21:44] milimetric: trying to implement the async event error produce thing inside like you said [20:21:46] not really sure how to do that... [20:23:13] well, not sure what the best way would be but you could always make a new promise and resolve().then... [20:23:19] that seems wrong... lemme see [20:23:29] setTimeout also seems wrong [20:27:34] ottomata: right, generators: https://medium.com/javascript-scene/the-hidden-power-of-es6-generators-observable-async-flow-control-cfa4c7f31435 [20:27:55] (working since Node 6) [20:32:23] milimetric: skimming/reading but its a bit deep [20:32:40] ottomata: yeah, was just thinking maybe just process.nextTick or setTimeout [20:33:07] process.nextTick sticks something at the top of the event queue, so I'm not sure how it affects performance [20:33:17] setImmediate seems to be new in Node 10 [20:33:32] but yeah, process.nextTick(() => do stuff) [20:33:39] works fine [20:33:58] hmm [20:34:15] so basically if given createEventError function [20:34:23] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10atgo) Yes, it's supposed to be a page from es.wiki. I believe the URL we'd settled on was http://es.wikipedia.org/bienvenida [20:34:36] inside of the promise i'm about to return, i schedule the event error produce, but still return the same promise [20:34:43] (since inside the promise is where i have the results [20:39:08] milimetric: from reading https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/ that seems like it would execute the error produce stuff immediately? [20:39:43] " Instead, the nextTickQueue will be processed after the current operation completes, regardless of the current phase of the event loop. [20:39:43] Looking back at our diagram, any time you call process.nextTick() in a given phase, all callbacks passed to process.nextTick() will be resolved before the event loop continues. [20:39:43] " [20:40:57] oh maybe setTimeout 0 hm [20:40:59] yeah, but that means after the end of the function, so after the response goes back [20:41:13] and setTimeout would add it to the end of the event queue, more gentle [20:42:13] yeah, and you can change it when async/await are available. [20:42:30] I think the way the generator thing works is you would define [20:42:37] function* something ()... and do stuff in the function [20:42:59] doesn't have to yield anything if you're not awaiting the results [20:43:19] but it's all really syntactic sugar, the event loop is the same, maybe basic setTimeout is the best for now [20:44:31] OH milimetric i think i can just do this now that i'm writing it. even without setTimeout. [20:44:52] i can just call this.process(eventErrors); inside of the process() funciton...hmm [20:44:52] no [20:45:00] infinite loop unless i protect against it [20:45:00] right [20:45:07] set timeout would do infinite loop too... [20:45:21] infinite recursion [20:45:30] i'd have to have something to stop errors from errors from being produced [20:47:06] hm [20:47:22] right milimetric? [20:47:32] (one sec) [20:48:20] k, was distracted [20:49:05] you can always have a boolean on the process that says "processErrors" and default it to true, set it to false on the setTimeout version, simple [20:49:26] hm yeah ok could do, i think i could also have another function layer here [20:49:30] this.process does [20:49:47] processEvents.then(processErrors), and processErrors is slightly different [20:50:39] yeah, processEvents returns the result dictionary, processErrors doesn't return anything [20:51:02] you could just factor out validate/produce from process and call that [20:51:05] wait sorry [20:51:06] yeah [20:51:08] that's already there [20:51:09] could do that [20:51:21] still need the boolean on that though [20:51:26] loop would still be there [20:51:34] oh no [20:51:47] loop's after those you call process again, right [20:52:15] so results = validateAndProduce(events) / async validateAndProduce(errors) / return results [20:53:09] processResultsPromise = new Promise(resolve, reject) { [20:53:09] const results = this._processEvents(events); [20:53:09] if errors; convert to eventErrors, this._processEvents(errors); [20:53:09] resolve(results); [20:53:09] } [20:53:35] PROBLEM - Hadoop NodeManager on analytics1068 is CRITICAL: PROCS CRITICAL: 0 processes with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager [20:54:34] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10Nuria) I see, I think there is a missunderstanding, piwik is not used on wikis, it is designed for low traffic sites like http://transparency.wikimedia.org , what we call "microsites". If the page you are c... [20:55:19] ah no. [20:55:19] hm [20:55:33] results is a promise no [20:55:34] nopers [20:55:37] ok will try some stuff [20:58:52] 10Analytics, 10New-Readers: Instrument the landing page - https://phabricator.wikimedia.org/T202592 (10atgo) >>! In T202592#4687177, @Nuria wrote: > I see, I think there is a missunderstanding, piwik is not used on wikis, it is designed for low traffic sites like http://transparency.wikimedia.org , what we ca... [21:13:39] 10Analytics, 10Product-Analytics, 10Readers-Web-Backlog: Popups schema is keeping page/session id longer than 90 days - https://phabricator.wikimedia.org/T207670 (10Tbayer) a:05Jdlrobson>03Tbayer [21:14:37] 10Analytics, 10Product-Analytics, 10Readers-Web-Backlog: Popups schema is keeping page/session id longer than 90 days - https://phabricator.wikimedia.org/T207670 (10Tbayer) Thanks for flagging this. I will check with @ovasileva about which of the two are more important from a product perspective, but I gue... [21:31:11] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): Explore NavigationTiming by faceted properties - EventLogging refine - https://phabricator.wikimedia.org/T166414 (10Nuria) Data is back in druid , please be so kind as to take a look: https://turnilo.wikimedia.org/#event_navig... [21:32:58] 10Analytics, 10Analytics-Kanban, 10Page-Issue-Warnings, 10Product-Analytics, and 3 others: Ingest data from PageIssues EventLogging schema into Druid - https://phabricator.wikimedia.org/T202751 (10Nuria) Data is in druid, please let us know if this ticket can be closed: https://turnilo.wikimedia.org/#event... [21:33:59] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Splits on top metrics when selections are present on url - https://phabricator.wikimedia.org/T206822 (10Nuria) [21:35:03] milimetric: if this is OK https://gerrit.wikimedia.org/r/#/c/analytics/wikistats2/+/468205/ (let me know if it is not) we can merge and i can deploy the latest way of bugfixes to wikistats [21:35:26] nuria: sorry I fell behind on reviews, just finishing up fran's then I'll do yours, but might take a break for dinner [21:35:45] milimetric: np, can wait [21:37:42] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking): Ingest data into druid for readingDepth schema - https://phabricator.wikimedia.org/T205562 (10Nuria) Data is back up, let us know if anything is missing: https://turnilo.wikimedia.org/#event_readingdepth [21:42:36] 10Analytics, 10Analytics-Kanban: turnilo x axis improperly labeled - https://phabricator.wikimedia.org/T197276 (10Nuria) a:05Ottomata>03elukey [21:42:53] 10Analytics, 10Analytics-Kanban: turnilo x axis improperly labeled - https://phabricator.wikimedia.org/T197276 (10Nuria) Assigning to luca as he is testing new version of turnilo in labs [21:43:40] 10Analytics, 10Analytics-Kanban: Deprecate reportcard: https://analytics.wikimedia.org/dashboards/reportcard/ - https://phabricator.wikimedia.org/T203128 (10Nuria) a:03fdans [21:44:21] 10Analytics, 10Analytics-Kanban: Update datasets to have explicit timestamp for druid indexation facilitation - https://phabricator.wikimedia.org/T205617 (10Nuria) a:05fdans>03None [21:46:48] 10Analytics, 10Community-Tech, 10Grant-Metrics: Review category queries - https://phabricator.wikimedia.org/T206783 (10MusikAnimal) [21:47:32] milimetric: i removed fdans from https://phabricator.wikimedia.org/T205617 which will be a long running task and assigned him to: https://phabricator.wikimedia.org/T203128 which is something we need to do for wikistats beta, it is small set of code chnages but probably a bunch of changes in wikis that point to the old reportcard that should now point to wikistats. [21:48:07] ok, sounds good [21:51:49] nuria: I'm almost done with the code change for having timestamps in pageview_hourly, maybe I can create a subtask of T206783 and push that? [21:51:50] T206783: Review category queries - https://phabricator.wikimedia.org/T206783 [22:01:20] fdans: sounds good, we will need to test and update datasets 1 by 1, sorry , I shoudl have asked you before [22:02:27] nuria: nono it's just that it's what I've been working on today and I don't want to drop it, i should have sent an escrum before boarding [22:16:37] fdans: also one thing, the patch for anonymous users requires also changes on aqs, makes sense? as the service now needs to return "null" when it is returning astring. [22:24:13] milimetric: got it here, thanks https://github.com/ottomata/eventbus/pull/1/commits/da49795319b245c8e2d299093ee669ed15980f69#diff-ad3e9bdb3d6f01144dbe5dc0bcf62d90L149 [22:29:16] nuria: oh sorry i didn't add you as reviewer, I pushed it earlier https://gerrit.wikimedia.org/r/#/c/analytics/aqs/+/468927/ [22:30:28] fdans: no worries, looks good [22:35:00] jdlrobson: the https://grafana.wikimedia.org/dashboard/db/reading-web-dashboard?panelId=15&fullscreen&orgId=1 minerva errors are averaged here per minute? [22:59:29] 10Quarry, 10Cloud-Services, 10cloud-services-team (Kanban): Migrate 'Quarry' project to eqiad1 - https://phabricator.wikimedia.org/T207677 (10zhuyifei1999) The main thing would be copying over the db and creating the config file (containing the secrets). Everything else should be automatic. However, I think... [23:08:35] 10Analytics, 10Analytics-Data-Quality, 10Product-Analytics: mediawiki_history datasets have null user_text for IP edits - https://phabricator.wikimedia.org/T206883 (10Tbayer) >>! In T206883#4665399, @JAllemandou wrote: > The `event_user_text` field in `wmf.mediawiki_history` is what can be called the "curren... [23:30:46] nuria: yup i believe so [23:31:51] nuria: it's what ever graphite pumps out by default [23:36:22] ottomata: k, I’ll look tomorrow