[05:58:29] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4185608 (10Psychoslave) >>! In T193728#4182296, @Ivanhercaz wrote: >> by removing them from Wikidata, or by some other solution yet to identify. > > Items with statements that only has... [06:26:25] 10Analytics-Legal, 10WMF-Legal, 10Wikidata: Solve legal uncertainty of Wikidata - https://phabricator.wikimedia.org/T193728#4185658 (10Psychoslave) >>! In T193728#4182350, @Denny wrote: > @Psychoslave, I am not sure I entirely follow. > > You said "there are contributors of Wikidata that do make massive im... [07:22:00] Hi elukey - reminder: I'm on holidays today ;) [07:22:18] elukey: I have tested druid indexation on Friday - It failed because of a java7 error :( [07:24:02] elukey: on d-1: grep -C 20 'Cannot run program "/usr/lib/jvm/java-7-openjdk-amd64/bin/java' /var/log/druid/middlemanager.log | less [07:24:09] Gone again ;) [07:42:44] joal: o/ [07:42:54] ack checking! [07:42:59] * elukey will miss joal [07:44:10] ahhh wait I think that after the last puppet merge for prod I haven't updated hiera in labs for java8 [07:44:13] uffff [07:44:21] will make java8 the default [07:52:22] mmmm no it is already the default [07:55:58] ahhhh [07:55:59] /etc/druid/middlemanager/runtime.properties:3:druid.indexer.runner.javaCommand=/usr/lib/jvm/java-7-openjdk-amd64/bin/java [08:15:48] (03PS1) 10Addshore: Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431524 (https://phabricator.wikimedia.org/T194008) [08:16:09] (03PS1) 10Addshore: Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431525 (https://phabricator.wikimedia.org/T194008) [08:16:42] (03CR) 10Addshore: [C: 032] Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431524 (https://phabricator.wikimedia.org/T194008) (owner: 10Addshore) [08:16:46] (03CR) 10Addshore: [C: 032] Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431525 (https://phabricator.wikimedia.org/T194008) (owner: 10Addshore) [08:16:50] (03Merged) 10jenkins-bot: Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431524 (https://phabricator.wikimedia.org/T194008) (owner: 10Addshore) [08:16:54] (03Merged) 10jenkins-bot: Don't call outputMessage statically in bf/counts.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/431525 (https://phabricator.wikimedia.org/T194008) (owner: 10Addshore) [08:19:16] elukey: if you have corrected labs, I can quickly launch a test indexation [08:20:32] joal: still working on it, but I have your gist in gh so probably I can do it later on [08:20:48] elukey: Great :) [08:27:30] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade Druid clusters to 0.11 - https://phabricator.wikimedia.org/T193712#4185795 (10elukey) [08:30:42] all right labs should work now, middlemanagers running with the new config [08:56:22] drain + reimage of analytics1037/8 [09:12:11] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Reimage the Debian Jessie Analytics worker nodes to Stretch. - https://phabricator.wikimedia.org/T192557#4185848 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by elukey on neodymium.eqiad.wmnet for hosts: ``` ['analytics1037.eqiad.wmnet', 'an... [09:18:08] !log re-run webrequest-load-wf-text-2018-5-7-7 - failed due to reimages [09:18:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:28:05] so I got something like Caused by: java.lang.VerifyError: class com.fasterxml.jackson.datatype.guava.deser.HostAndPortDeserializer overrides final method deserialize. [09:28:16] in /var/lib/druid/indexing-logs/index_hadoop_webrequest_2018-05-07T08:35:54.401Z.log [09:28:42] I basically changed joseph's gist to include a new time window [09:28:50] but I am probably doing something wrong [09:28:57] I'll double check with him tomorrow :) [10:14:06] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Reimage the Debian Jessie Analytics worker nodes to Stretch. - https://phabricator.wikimedia.org/T192557#4185952 (10ops-monitoring-bot) Completed auto-reimage of hosts: ``` ['analytics1037.eqiad.wmnet', 'analytics1038.eqiad.wmnet'] ``` and were **ALL** su... [10:17:51] PROBLEM - HDFS corrupt blocks on analytics1001 is CRITICAL: 8 ge 5 https://grafana.wikimedia.org/dashboard/db/analytics-hadoop?orgId=1&panelId=39&fullscreen [10:18:35] yeah these are the new nodes --^ [10:35:07] going afk for ~2h, ttl! [10:35:10] * elukey afk! [11:56:36] elukey: realtime indexation started, working :) [12:05:44] !log Rerun mediawiki-history-reduced-wf-2018-04 [12:05:45] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:15:43] hellooo milimetric. yt? [12:16:02] * lzia waves to Analytics folks after the return from a week of vacation. ;) [12:21:59] hi lzia welcome back :) [12:28:10] Thanks, milimetric. :) Re T191343, do you think it makes sense to ask the owner of the Python code to compare the results? (I have no idea who had written that code;) [12:28:10] T191343: Vet new geo wiki data - https://phabricator.wikimedia.org/T191343 [12:29:33] lzia: oh you don’t have to vet that, I already did, results are on wikitech [12:30:08] The short version is that new numbers are more accurate but bigger [12:30:15] milimetric: Ok, that's already a good news. I did one pass over the results on wiki, and it makes sense to me. [12:30:29] The biggest factor being that old data didn’t geolocate very well [12:30:59] milimetric: what made me wonder was that the statements were not very deterministic. For example, the statement is that we think a lot of the different can be explained by IP addresses that would be invalid in the old system but not in the new system. Do we feel confident about this? [12:31:21] milimetric: do you know how much of the difference will be explained by that factor? [12:33:05] It’s hard to say without really thorough analysis, and we didn’t think there was a lot of value in spending that kind of time on it, what do you think lzia? [12:34:28] I mean what I did was compare higher level numbers, if I wanted exact comparisons I’d have to step through the old python code labeling with both systems and comparing. Is there need for that kind of precision? [12:35:11] milimetric: I see what you say, milimetric. the value of it is basically how much we can trust the new data. If we can say that we're very confident that is accurate, that's great. Given that there was an old system in place, one way to say that is to be able to explain the differences with the old system. With Global Innovation Index using this data as part of their reports, for example, mistakes can be costly for countries. [12:35:48] milimetric: we already received at least one note last year from a country that had employed a consulting group to help them understand how to improve their numbers. People are making country-level decisions based on this data now. [12:36:50] milimetric: what do you mean by higher level numbers? if you mean, for example, aggregate edit counts by country, that's good enough. [12:37:40] milimetric: but if there are big differences that the invalid IP address doesn't explain, we should dig deeper. that's why I asked what percentage of the differences are explained by the invalid IP address counts. [12:38:24] (one sec) [12:39:45] joal: ah ok! I think I am not super familiar with where to check for indexation, I thought it wasn't working on d-1 :( [12:40:00] (hourly webrequest indexation I mean) [12:42:06] ok, lzia at my computer now, with some cookies to help me cope :) [12:43:22] milimetric: you don't have to do it now, and I'm not sure if /you/ should do it. but I'm telling you since you're the owner. :) [12:43:23] lzia: so the differences come from several factors that combine, as far as I can tell [12:43:54] lzia: yeah, no, it's ok, I had given the GII some thought, and figured they'd all be happy since the numbers are basically always higher per country [12:44:07] ok, so here are the differences as I know of them: [12:44:28] old: only namespace 0 edits, bad geolocation [12:45:21] new: no namespace filter, good geolocation, only edits (no admin activity) [12:46:37] so the only way in which the new data counts less than the old data is that it discounts administrative activity (page moves, etc.) [12:47:47] but this decrease is always overcome by the increase due to the other changes - like counting all namespaces and better geolocation [12:47:52] make sense lzia? [12:48:43] there are also some small differences with bot filtering, where the old system was making some mistakes, but those I believe always mean higher counts for new data as well [12:49:28] so we have a task to port the GII code to a recurring oozie job, and when I did that I had in mind to report back to the GII folks: https://phabricator.wikimedia.org/T190535 [12:53:56] milimetric: I agree with you that it's safe to assume the count should be higher with the new code. The question is: are the differences explained by the 3 factors (almost completely) or there can be mistakes in the new code. If you are super sure there are no mistakes in the new code, nothing to check further. If you're not, you can take a sample of the data, run the two codes on it, do some extra computations and see if you resolve invalid IPs and [12:54:40] oh lzia I see what you mean, no I am as confident as I get in the new code, it's very basic [12:56:05] milimetric: re GII specifically, great to see the Oozie job becoming a reality. I'd say once the system is vetted, let's talk with them to see what metric makes sense for them to include in 2018+ years. They currently only include Wikipedia main namespace edits. (and I'm not sure in their current model including all namespace edits makes sense but if you have a way to filter data, that's fine.) [12:56:16] milimetric: ok, re confidence in code. [12:57:28] lzia: hm, yeah, if you think they're only interested in main namespace, then I can add a flag for that. So with the new data there's also a flag for anonymous, so they can get that data as well. But you think I should make it possible to look at the numbers for just namespace 0? [13:32:10] RECOVERY - HDFS corrupt blocks on analytics1001 is OK: (C)5 ge (W)2 ge 0 https://grafana.wikimedia.org/dashboard/db/analytics-hadoop?orgId=1&panelId=39&fullscreen [13:33:13] gooooood [13:46:26] heya team :] [13:56:11] o/ [13:56:17] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4186766 (10Ottomata) [14:14:54] 10Analytics, 10Cassandra, 10Maps-Sprint, 10Operations, and 4 others: Upgrade prometheus-jmx-exporter on all services using it - https://phabricator.wikimedia.org/T192948#4154846 (10herron) The puppetdb servers have been upgraded to prometheus-jmx-exporter 0.3.0-1 [14:15:13] 10Analytics, 10Cassandra, 10Maps-Sprint, 10Operations, and 4 others: Upgrade prometheus-jmx-exporter on all services using it - https://phabricator.wikimedia.org/T192948#4186804 (10herron) [14:42:38] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4186917 (10Ottomata) [14:50:08] (03PS1) 10Milimetric: Change print to logger [analytics/refinery] - 10https://gerrit.wikimedia.org/r/431579 [14:50:22] (03CR) 10Milimetric: [V: 032 C: 032] "self-merging basic change" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/431579 (owner: 10Milimetric) [14:54:57] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade Druid clusters to 0.11 - https://phabricator.wikimedia.org/T193712#4186948 (10elukey) I found an interesting thing in the druid logs today: ``` Error: com.google.inject.CreationException: Unable to create injector, see the followi... [14:59:29] (03PS4) 10Mforns: Add source page fields to wmf.virtualpageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/430889 (https://phabricator.wikimedia.org/T186728) [15:01:31] ottomata, fdans joal [15:01:34] stahdduppp [15:15:38] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Update ua-parser package. Both uap-java and uap-core - https://phabricator.wikimedia.org/T192464#4187004 (10Nuria) [15:26:43] 10Analytics, 10EventBus: SSL and inter broker encryption for Kafka main - https://phabricator.wikimedia.org/T193778#4179461 (10mforns) p:05Triage>03Normal [15:29:19] 10Analytics: Tests clone of pivot - https://phabricator.wikimedia.org/T194054#4187065 (10Nuria) [15:31:56] 10Analytics: Tests clone of pivot - https://phabricator.wikimedia.org/T194054#4187065 (10mforns) p:05Triage>03High [15:33:23] 10Analytics: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4187092 (10Nuria) [15:33:33] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Wikimedia-Logstash, and 2 others: EventBus HTTP Proxy service does not report errors to logstash - https://phabricator.wikimedia.org/T193230#4187101 (10mforns) [15:33:45] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Update version of ua-parser in eventlogging - https://phabricator.wikimedia.org/T192529#4187106 (10mforns) [15:33:48] 10Analytics: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4187108 (10Nuria) [15:34:11] 10Analytics, 10User-Elukey: Tests clone of pivot - https://phabricator.wikimedia.org/T194054#4187110 (10elukey) [15:34:31] 10Analytics, 10User-Elukey: Add maxmind ip info to webrequest dataset on druid - https://phabricator.wikimedia.org/T194055#4187119 (10elukey) [15:36:11] PROBLEM - Number of segments reported as unavailable by the Druid Coordinators -Public cluster- on einsteinium is CRITICAL: 4992 gt 10 https://grafana.wikimedia.org/dashboard/db/druid?refresh=1m&panelId=46&fullscreen&orgId=1&var-cluster=druid_public&var-druid_datasource=All [15:37:29] 10Analytics, 10Product-Analytics, 10Reading-analysis: Assess impact of ua-parser update on core metrics - https://phabricator.wikimedia.org/T193578#4187130 (10fdans) a:03fdans [15:37:51] ah that one is probably due to the new segments loaded! [15:38:05] ok I def need to review that alarm [15:38:27] 10Analytics-Kanban, 10Patch-For-Review, 10Puppet: Puppetize job that saves old versions of Maxmind geoIP database - https://phabricator.wikimedia.org/T136732#4187139 (10mforns) [15:39:18] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4187142 (10Ottomata) [15:43:07] 10Analytics, 10Analytics-Kanban, 10Wikidata, 10Patch-For-Review, 10Wikidata-Ministry-Of-Magic: Add Wikidata website extract oozie job - https://phabricator.wikimedia.org/T191022#4187158 (10mforns) [15:46:44] 10Analytics, 10Analytics-Kanban: Update version of ua-parser in refinery source - https://phabricator.wikimedia.org/T192463#4187169 (10mforns) [15:48:02] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Reimage the Debian Jessie Analytics worker nodes to Stretch. - https://phabricator.wikimedia.org/T192557#4187171 (10mforns) [15:48:19] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Reimage the Debian Jessie Analytics worker nodes to Stretch. - https://phabricator.wikimedia.org/T192557#4187172 (10mforns) [15:49:10] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Productionize EventLogging sanitization - https://phabricator.wikimedia.org/T193176#4187176 (10mforns) [15:49:37] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade Druid clusters to 0.11 - https://phabricator.wikimedia.org/T193712#4187179 (10mforns) [15:51:56] 10Analytics-Kanban, 10Product-Analytics, 10Reading-analysis: Final Vetting of Family Wide unique devices data - https://phabricator.wikimedia.org/T169550#3401448 (10mforns) @Tbayer Is there any work to be done here before we can close this task? [15:53:33] 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Patch-For-Review, 10Software-Licensing: Add a license file to wikimetrics - https://phabricator.wikimedia.org/T60753#4187198 (10mforns) [15:56:37] 10Analytics: Publishing project anomaly data for censorship researchers. Evaluate privacy threats - https://phabricator.wikimedia.org/T183990#4187214 (10mforns) [15:58:48] 10Analytics: Handle long-term retention of ChangesListFilters and ChangesListFilterGrouping schemas - https://phabricator.wikimedia.org/T185009#4187220 (10mforns) [16:00:23] 10Analytics: Final steps to expose project family unique devices data - https://phabricator.wikimedia.org/T167539#4187240 (10mforns) [16:01:30] 10Analytics: Final steps to expose project family unique devices data - https://phabricator.wikimedia.org/T167539#3336133 (10mforns) p:05Triage>03Low [16:01:36] 10Analytics, 10Discovery, 10Discovery-Analysis, 10Product-Analytics: Private data access for non-person user that calculates metrics - https://phabricator.wikimedia.org/T174110#4187249 (10mforns) [16:02:05] 10Analytics, 10Patch-For-Review, 10User-Elukey: Refactor puppet code for the Hadoop Analytics cluster to roles/profiles - https://phabricator.wikimedia.org/T167790#4187250 (10mforns) [16:03:29] 10Analytics, 10Operations, 10Patch-For-Review: Puppet admin module should support adding system users to managed groups - https://phabricator.wikimedia.org/T174465#4187253 (10mforns) [16:03:32] 10Analytics, 10Operations, 10Patch-For-Review: Puppet admin module should support adding system users to managed groups - https://phabricator.wikimedia.org/T174465#3562875 (10mforns) p:05Normal>03Low [16:03:38] 10Analytics, 10Operations, 10Patch-For-Review: Puppet admin module should support adding system users to managed groups - https://phabricator.wikimedia.org/T174465#3562875 (10mforns) p:05Low>03Normal [16:04:29] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Refresh zookeeper nodes in eqiad - https://phabricator.wikimedia.org/T182924#4187261 (10mforns) [16:05:08] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Move away from jmxtrans in favor of prometheus jmx_exporter - https://phabricator.wikimedia.org/T175344#4187263 (10mforns) [16:05:24] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 Backend: Resiliency, Rollback and Deployment of Data - https://phabricator.wikimedia.org/T177965#4187264 (10mforns) [16:08:04] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Intervals/buckets for data arround pageviews per country in wikistats maps - https://phabricator.wikimedia.org/T188928#4187269 (10mforns) [16:13:28] 10Analytics, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015#4187274 (10mforns) [16:13:50] 10Analytics, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015#4187278 (10mforns) p:05Triage>03Unbreak! [16:14:00] 10Analytics, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015#4187279 (10mforns) p:05Unbreak!>03Triage [16:22:41] 10Analytics, 10Analytics-Kanban: Archive old geowiki data (editors per country) and make it easily available at WMF - https://phabricator.wikimedia.org/T190856#4187308 (10mforns) [16:25:08] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: The popup in the line chart is obstructive - https://phabricator.wikimedia.org/T191985#4187311 (10mforns) 05Open>03Resolved This was resolved by T188277. [16:26:27] elukey: a thought about log.message.format.version [16:26:30] https://kafka.apache.org/documentation/#upgrade_10_performance_impact [16:26:42] we might want to do the first two steps of the upgrade [16:26:51] and THEN update api.version stuff for clients [16:27:05] before we do the log.message.format.version rolling restart [16:27:29] 10Analytics, 10Analytics-Wikistats, 10Google-Summer-of-Code: When cursor is out of graph overlay should not display - https://phabricator.wikimedia.org/T192416#4187319 (10mforns) [16:28:20] ottomata: yes makes a lot of sense [16:28:29] (need to read carefully but I got the picture) [16:28:55] but ya once we set log.message.format.version, i think we cannot really rollback [16:34:05] wow https://github.com/ggerganov/wave-share#wave-share haha [16:34:10] check out that video [16:34:16] file sharing over sound waves [16:38:44] 10Analytics, 10Analytics-Kanban: Sesssion reconstruction - privacy breach - https://phabricator.wikimedia.org/T194058#4187360 (10Nuria) [16:40:38] 10Analytics, 10Analytics-Kanban: Sesssion reconstruction - evaluate privacy breach - https://phabricator.wikimedia.org/T194058#4187376 (10Nuria) [16:44:08] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4187392 (10Nuria) I can see how the addition of source-page for wikis with low traffic will make this agreggation not be such... [17:23:53] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4187556 (10Ottomata) [17:25:33] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4187563 (10Pchelolo) [17:32:47] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4187578 (10Ottomata) [17:39:15] * elukey off! [17:50:38] 10Analytics, 10Analytics-Kanban: Sesssion reconstruction - evaluate privacy threat - https://phabricator.wikimedia.org/T194058#4187662 (10Nuria) [17:50:43] (03CR) 10Nuria: "From what i can see errors with naming have been corrected. I filed https://phabricator.wikimedia.org/T194058 to follow up on this." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/430889 (https://phabricator.wikimedia.org/T186728) (owner: 10Mforns) [17:50:45] (03CR) 10Nuria: [V: 032 C: 032] Add source page fields to wmf.virtualpageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/430889 (https://phabricator.wikimedia.org/T186728) (owner: 10Mforns) [17:57:37] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4187672 (10Ottomata) [17:59:42] gotta get lunch and run an errand, bbl [19:12:11] 10Analytics: 2018-03 snapshot still broken - https://phabricator.wikimedia.org/T194075#4187918 (10Milimetric) [19:12:27] 10Analytics, 10Analytics-Kanban: 2018-03 snapshot still broken - https://phabricator.wikimedia.org/T194075#4187929 (10Milimetric) p:05Triage>03Unbreak! [19:12:45] 10Analytics, 10Analytics-Kanban: 2018-03 snapshot still broken - https://phabricator.wikimedia.org/T194075#4187918 (10Milimetric) p:05Unbreak!>03High [19:18:42] 10Analytics, 10Analytics-Kanban: 2018-03 snapshot still broken - https://phabricator.wikimedia.org/T194075#4187918 (10Nuria) @milimetric: we did not redo the snapshot at all, so yes, it is broken. We reindexed data 2018-02 on wikistats [19:18:53] milimetric: just updated ticket but snapshot is still broken yes, we did not redo 2018-03 at all [19:21:35] ottomata: shall we merge this? I forgot we had it open [19:21:35] https://gerrit.wikimedia.org/r/#/c/430067/ [19:22:10] nuria_: oh! I thought we did... my fault. Neil was looking at the numbers and found them off, so we should probably delete the snapshot if it's still bad [19:22:21] or mark it invalid somehow [19:23:36] I talked to neilpquinn about this last 1 on 1 in that data was bad regarding anonymous editors, there might be also other things i did not know of [19:24:04] milimetric: we shoudl not delete snapshot but rather use it to test our quality testing with [19:24:11] Question: Is this the right graph to read if I’m trying to understand the impact of an app making a heavy request load from wmflabs to production enwiki? https://grafana.wikimedia.org/dashboard/db/api-summary?orgId=1&from=1524938400000&to=1525730400000&var-percentile=p50&var-dc=eqiad [19:24:37] I’m very surprised to not see any impact at all (May 4, c. 20:00-21:24) [19:24:46] awight: what api is the app hiting? [19:24:53] awight: there are many [19:25:08] awight: php api? [19:25:25] awight: restbase wikipedia /restbase analytics? [19:26:28] 10Analytics, 10Analytics-Kanban: 2018-03 snapshot still broken - https://phabricator.wikimedia.org/T194075#4187982 (10Milimetric) [19:30:09] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4187999 (10mforns) [19:30:13] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Change '--' to something more helpful in Wikistats page views by country table view - https://phabricator.wikimedia.org/T187427#4187998 (10mforns) 05Open>03Resolved [19:31:00] nuria_: GET requests against the PHP API. In this case, they’re for randomly distributed revisions so shouldn’t be in any recent caches. [19:32:54] awight: i see, you are expecting noticeable volume increase? [19:42:24] nuria_: I proved to myself that I was only 0.5% of the load, so it was egocentric of me to expect to appear in the graphs :) [19:42:48] Thanks for confirmation that I’m looking at the right graph, I assume that “ction API GET requests” is [19:42:56] grr *”action API” is the PHP API [19:43:05] awight: i cannot think of a better graph of the ones i know of [19:43:14] wonderful [19:43:29] awight: as always ops might know best [19:43:53] +1 sorry to spam -analytics [19:44:16] * awight contributes to 0.5% of load on #wikimedia-operations as well [19:50:14] oh fdans yes! [19:56:07] 10Analytics, 10Analytics-Wikistats, 10Accessibility, 10Easy, 10Patch-For-Review: Wikistats Beta: Fix accessibility/markup issues of Wikistats 2.0 - https://phabricator.wikimedia.org/T185533#4188062 (10mforns) @Volker_E Even if you merged this patch, I understand that this task is not done yet, because th... [20:00:00] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4188072 (10Ottomata) [20:05:41] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Display of radio buttons in Wikistats 2 is somewhat confusing - https://phabricator.wikimedia.org/T183185#4188094 (10Milimetric) @Amitjoki yeah that looks good. Only detail is don't forget to set the cursor: pointer style when you hover over those. Th... [20:07:41] 10Analytics, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015#4188106 (10Milimetric) p:05Triage>03Normal [20:07:43] (03PS1) 10Ottomata: Make Kafka api_version configurable [analytics/statsv] - 10https://gerrit.wikimedia.org/r/431646 (https://phabricator.wikimedia.org/T167039) [20:07:48] 10Analytics, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015#3677997 (10Milimetric) p:05Normal>03High [20:08:36] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add data-quality check on mediawiki-history-reduced before druid indexation - https://phabricator.wikimedia.org/T192483#4188114 (10Milimetric) p:05Triage>03High [20:09:50] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add Mediawiki-History data-quality check stage in oozie using statistics - https://phabricator.wikimedia.org/T192481#4188122 (10Milimetric) p:05Triage>03High [20:10:33] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Add a tooltip to all non-obvious concepts like split categories, abbreviations - https://phabricator.wikimedia.org/T177950#4188125 (10Milimetric) p:05Triage>03High [20:11:30] 10Analytics: Under construction page in wikistats to take site down - https://phabricator.wikimedia.org/T192847#4188131 (10Milimetric) p:05High>03Normal [20:12:12] (03CR) 10Ottomata: "The problem with not using puppet for this is that statsv runs in mulitple datacenters, and subscribes to different Kafka clusters. We ar" [analytics/statsv] - 10https://gerrit.wikimedia.org/r/429432 (https://phabricator.wikimedia.org/T193238) (owner: 10Imarlier) [20:12:27] 10Analytics, 10Analytics-Wikistats: Audit Wikistats unit testing - https://phabricator.wikimedia.org/T192836#4188138 (10Milimetric) p:05High>03Unbreak! [20:12:29] 10Analytics, 10Analytics-Wikistats: Audit Wikistats unit testing - https://phabricator.wikimedia.org/T192836#4188141 (10Milimetric) p:05Unbreak!>03High [20:12:59] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Intervals/buckets for data arround pageviews per country in wikistats maps - https://phabricator.wikimedia.org/T188928#4188147 (10Milimetric) a:05Milimetric>03None [20:13:59] 10Analytics, 10Analytics-Wikistats: Beta: Y-axis units and rounding issues - https://phabricator.wikimedia.org/T187429#4188151 (10mforns) I think #2 is still an untackled issue: {F18061453} I will adapt title and description of this task, and keep it open for GSoC. [20:14:51] 10Analytics, 10Analytics-Wikistats: Reindex mediawiki_history_reduced with lookups - https://phabricator.wikimedia.org/T193650#4188152 (10Milimetric) p:05Normal>03Low [20:15:13] 10Analytics, 10Analytics-Wikistats: roadmap of migration to Wikistats 2 - https://phabricator.wikimedia.org/T183180#4188154 (10Milimetric) p:05Normal>03High [20:15:35] 10Analytics: stats.wikimedia.org home page should link to wikistats 2 - https://phabricator.wikimedia.org/T191555#4188157 (10Milimetric) p:05Normal>03High [20:15:37] 10Analytics: stats.wikimedia.org home page should link to wikistats 2 - https://phabricator.wikimedia.org/T191555#4109929 (10Milimetric) p:05High>03Normal [20:17:10] 10Analytics, 10Analytics-Wikistats: Wikistats2 line chart and map displacement bugs in Chrome+Ubuntu - https://phabricator.wikimedia.org/T189197#4188183 (10mforns) Line chart problem is already solved, but map problem is still present. [20:17:29] 10Analytics: Map tooltip and line graph guide are misaligned in Ubuntu Chrome - https://phabricator.wikimedia.org/T187453#4188185 (10Milimetric) 05Open>03Resolved a:03Milimetric [20:17:49] (03PS2) 10Ottomata: Make Kafka api_version configurable [analytics/statsv] - 10https://gerrit.wikimedia.org/r/431646 (https://phabricator.wikimedia.org/T167039) [20:19:17] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Limit pan in Wikistats2 maps - https://phabricator.wikimedia.org/T189195#4188191 (10mforns) [20:19:22] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: The alert message about adblocker is not fully shown on smaller screens - https://phabricator.wikimedia.org/T188208#4188192 (10mforns) [20:19:26] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Change '--' to something more helpful in Wikistats page views by country table view - https://phabricator.wikimedia.org/T187427#4188193 (10mforns) [20:19:31] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Wikistat Beta: expand topic explorer by default - https://phabricator.wikimedia.org/T186335#4188197 (10mforns) [20:19:40] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Display of radio buttons in Wikistats 2 is somewhat confusing - https://phabricator.wikimedia.org/T183185#4188198 (10mforns) [20:19:47] 10Analytics-Kanban, 10Analytics-Wikistats, 10Easy, 10Patch-For-Review: [Wikistats2] The detail page for tops and maps metrics does not indicate time range - https://phabricator.wikimedia.org/T182990#4188199 (10mforns) [20:19:49] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4188190 (10mforns) [20:26:11] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Upgrading Wikistats 2.0 footer UI/design - https://phabricator.wikimedia.org/T191672#4188244 (10mforns) [20:26:13] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4188243 (10mforns) [20:27:30] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Hide "Load more rows..." once all the data is visible in Table Chart - https://phabricator.wikimedia.org/T192407#4188257 (10mforns) [20:27:32] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4035243 (10mforns) [20:29:29] 10Analytics, 10Analytics-Wikistats: Add wikistats metric about "pagecounts" - https://phabricator.wikimedia.org/T189619#4188260 (10mforns) [20:29:32] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4188259 (10mforns) [20:30:05] (03CR) 10Ottomata: [V: 032 C: 032] Make Kafka api_version configurable [analytics/statsv] - 10https://gerrit.wikimedia.org/r/431646 (https://phabricator.wikimedia.org/T167039) (owner: 10Ottomata) [20:31:37] 10Analytics, 10Analytics-Wikistats: Add wikistats metric about "pagecounts" - https://phabricator.wikimedia.org/T189619#4188268 (10Nuria) Note that this is a bit tricky as pagecounts are on a different time scale than rest of metrics and thus cannot be visible on dashboard yet they might be findable with navi... [20:33:02] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Operations, and 2 others: Kafka API negotiation errors on kafka main brokers - https://phabricator.wikimedia.org/T193238#4188269 (10Ottomata) FYI, had to do https://gerrit.wikimedia.org/r/#/c/431646/ to do Kafka upgrade. [20:34:04] 10Analytics, 10Analytics-Wikistats, 10Google-Summer-of-Code: When cursor is out of graph overlay should not display - https://phabricator.wikimedia.org/T192416#4188275 (10mforns) [20:34:06] 10Analytics-Kanban, 10Google-Summer-of-Code (2018): [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189210#4188274 (10mforns) [20:34:46] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Change 'NaN' & 'Infinite' to something more helpful in metrics % change over the selected time range - https://phabricator.wikimedia.org/T192028#4188281 (10mforns) 05Open>03Resolved [20:36:34] 10Analytics, 10Analytics-Wikistats: When cursor is out of graph overlay should not display - https://phabricator.wikimedia.org/T192416#4188282 (10sahil505) [20:37:28] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4188288 (10Ottomata) [20:38:05] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Some metrics don't work in the topic selector - https://phabricator.wikimedia.org/T188268#4188289 (10mforns) 05Open>03Resolved [20:47:57] (03PS1) 10Framawiki: Fix UnicodeDecodeError in output._stringfy() [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) [21:11:57] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4188425 (10Ottomata) [21:23:02] (03PS6) 10Nuria: [WIP] UA parser specification changes [analytics/ua-parser/uap-java] (wmf) - 10https://gerrit.wikimedia.org/r/429527 (https://phabricator.wikimedia.org/T189230) [21:31:53] (03CR) 10Sturmkrahe: Fix accessibility/markup issues (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425925 (https://phabricator.wikimedia.org/T185533) (owner: 10Sturmkrahe) [21:36:15] (03CR) 10VolkerE: [C: 04-1] "@Sturmkrahe Yes, there is. I've already posted above. See code for this mixin: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/co" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425925 (https://phabricator.wikimedia.org/T185533) (owner: 10Sturmkrahe) [21:37:07] (03CR) 10VolkerE: [C: 031] "Millimetric, over to you." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/419958 (https://phabricator.wikimedia.org/T185533) (owner: 10Sturmkrahe) [21:37:55] (03CR) 10VolkerE: [C: 031] "I prefer to write you with two 'l' obviously :}" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/419958 (https://phabricator.wikimedia.org/T185533) (owner: 10Sturmkrahe) [21:41:41] 10Analytics, 10Analytics-Wikistats, 10Accessibility, 10Easy, 10Patch-For-Review: Wikistats Beta: Fix accessibility/markup issues of Wikistats 2.0 - https://phabricator.wikimedia.org/T185533#4188468 (10Volker_E) @mforns Don't follow which patch you refer to, this one https://gerrit.wikimedia.org/r/#/c/419... [21:42:09] (03PS2) 10Framawiki: Fix UnicodeDecodeError in output's escape usage [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) [21:42:28] 10Analytics, 10Analytics-Wikistats, 10Accessibility, 10Easy, 10Patch-For-Review: Wikistats Beta: Fix accessibility/markup issues of Wikistats 2.0 - https://phabricator.wikimedia.org/T185533#4188470 (10Volker_E) [21:45:04] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4188475 (10Pchelolo) [22:12:42] (03CR) 10Zhuyifei1999: [C: 031] Fix UnicodeDecodeError in output's escape usage [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) (owner: 10Framawiki) [22:14:30] (03CR) 10Zhuyifei1999: [C: 031] "On a side note, we should really move on to py3 so we don’t run into such issues." [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) (owner: 10Framawiki) [22:35:37] (03CR) 10Framawiki: [C: 032] Fix UnicodeDecodeError in output's escape usage [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) (owner: 10Framawiki) [22:35:54] (03Merged) 10jenkins-bot: Fix UnicodeDecodeError in output's escape usage [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/431654 (https://phabricator.wikimedia.org/T117644) (owner: 10Framawiki) [22:55:17] 10Analytics, 10EventBus, 10Services (doing), 10User-Elukey: Kafka sometimes misses to rebalance topics properly - https://phabricator.wikimedia.org/T179684#4188627 (10Pchelolo) This happened again today with `on_transclusions_update` group - it just stopped being consumed completely without a visible reaso...