[07:01:22] joal: o/ [07:01:23] elukey@analytics1030:~$ dpkg -l | egrep 'python3-pandas|python3-matplotlib' [07:01:26] ii python3-matplotlib 2.0.0+dfsg1-2 amd64 Python based plotting system in a style similar to Matlab (Python 3) [07:01:30] ii python3-pandas 0.19.2-5.1 all data structures for "relational" or "labeled" data - Python 3 [07:01:33] ii python3-pandas-lib 0.19.2-5.1 amd64 low-level implementations and bindings for pandas - Python 3 [07:01:41] those are the ones that you needed right? [07:02:30] elukey: I didn't need them right now, but they'll be usefull for sure anytime soon [07:02:33] :) [07:02:41] And, hi, elukey :) [07:03:29] joal: ah yes, this is part of the package refactoring that me and Andrew did.. so now we have a set of packages installed on stat/notebook/hadoop nodes that are the same [07:03:36] elukey: I ided [07:03:36]  [07:03:36] 11:04 PM [07:03:42] oops [07:03:44] scuse me [07:03:45] :D [07:04:34] elukey: anything I can help today, or shall I take advantage of having found a very serious lead towards data-resiliency? [07:10:15] nothing relevant, please fight data :) [11:19:51] (03PS1) 10WMDE-Fisch: Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) [11:27:22] PROBLEM - Kafka MirrorMaker main-eqiad_to_eqiad max lag in last 10 minutes on einsteinium is CRITICAL: 9.407e+05 gt 1e+05 https://grafana.wikimedia.org/dashboard/db/kafka-mirrormaker?var-datasource=eqiad+prometheus/ops&var-lag_datasource=eqiad+prometheus/ops&var-mirror_name=main-eqiad_to_eqiad [11:28:42] PROBLEM - Kafka MirrorMaker main-eqiad_to_jumbo-eqiad max lag in last 10 minutes on einsteinium is CRITICAL: 9.422e+05 gt 1e+05 https://grafana.wikimedia.org/dashboard/db/kafka-mirrormaker?var-datasource=eqiad+prometheus/ops&var-lag_datasource=eqiad+prometheus/ops&var-mirror_name=main-eqiad_to_jumbo-eqiad [11:32:51] lag is fine, I am pretty sure that they rebooted kafkamon [11:32:51] RECOVERY - Kafka MirrorMaker main-eqiad_to_eqiad max lag in last 10 minutes on einsteinium is OK: (C)1e+05 gt (W)1e+04 gt 0 https://grafana.wikimedia.org/dashboard/db/kafka-mirrormaker?var-datasource=eqiad+prometheus/ops&var-lag_datasource=eqiad+prometheus/ops&var-mirror_name=main-eqiad_to_eqiad [11:33:11] elukey@kafkamon1001:~$ uptime 11:33:07 up 12 min, 1 user, load average: 0.33, 0.59, 0.51 [11:34:12] RECOVERY - Kafka MirrorMaker main-eqiad_to_jumbo-eqiad max lag in last 10 minutes on einsteinium is OK: (C)1e+05 gt (W)1e+04 gt 0 https://grafana.wikimedia.org/dashboard/db/kafka-mirrormaker?var-datasource=eqiad+prometheus/ops&var-lag_datasource=eqiad+prometheus/ops&var-mirror_name=main-eqiad_to_jumbo-eqiad [11:42:06] * elukey lunch + errand! [11:49:09] (03CR) 10Thiemo Kreuz (WMDE): [C: 031] "Should this patch depend on I6fd5ae2d1a65952b67baa25cf3dd13d9bd5b6209, or can it be merged before that other one?" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [11:49:15] (03CR) 10Addshore: [C: 032] Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [11:49:21] (03PS1) 10Addshore: Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437971 (https://phabricator.wikimedia.org/T195370) [11:49:24] (03Merged) 10jenkins-bot: Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [11:49:27] (03CR) 10Addshore: [C: 032] Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437971 (https://phabricator.wikimedia.org/T195370) (owner: 10Addshore) [11:49:35] (03Merged) 10jenkins-bot: Add FileExporter to beta features count script [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437971 (https://phabricator.wikimedia.org/T195370) (owner: 10Addshore) [11:53:18] (03CR) 10WMDE-Fisch: "Seems Addshore answered that question by merging this change :-)" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [12:02:30] (03CR) 10Addshore: [C: 032] "Yes, it can be merged before :)" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [12:02:52] (03CR) 10Addshore: [C: 032] "Note this doesn't get deployed until the patch is also merged into the production branch, (which I also did)" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/437961 (https://phabricator.wikimedia.org/T195370) (owner: 10WMDE-Fisch) [12:03:20] spam spam spam [13:56:51] 10Analytics, 10Analytics-Kanban, 10Operations, 10Patch-For-Review, 10User-Elukey: Import some Analytics git puppet submodules to operations/puppet - https://phabricator.wikimedia.org/T188377#4264029 (10elukey) Ready to merge the code reviews for varnishkafka, kafkatee and jmxtrans, but it seems from the... [14:00:55] 10Analytics, 10Analytics-Cluster, 10Services (doing): Support connection/rate limiting in EventStreams - https://phabricator.wikimedia.org/T196553#4264046 (10Ottomata) The varnish option could be complimentary, but it won't really be sufficient. We aren't really worried about the service itself becoming ove... [14:07:01] 10Analytics, 10Recommendation-API, 10Research: PySpark 2 cannot find numpy - https://phabricator.wikimedia.org/T196592#4264063 (10bmansurov) Installing `numpy` locally seems to have solved the problem for now. [14:24:21] 10Analytics, 10Analytics-Kanban, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Huge messages in eqiad.mediawiki.job.cirrusSearchElasticaWrite (and other?) topics - https://phabricator.wikimedia.org/T196032#4264106 (10Ottomata) Ah! Yup, we do use compression = snappy. Just noticed that you re commented.... [14:25:21] 10Analytics, 10Recommendation-API, 10Research: PySpark 2 cannot find numpy - https://phabricator.wikimedia.org/T196592#4262641 (10elukey) Hi! We have deployed python3-numpy and python-numpy on the analytics worker nodes and on the stat boxes (I just tested `import numpy` in python2/3 and it seems to work).... [14:28:04] 10Analytics, 10Recommendation-API, 10Research: PySpark 2 cannot find numpy - https://phabricator.wikimedia.org/T196592#4264116 (10elukey) Just tried your example in the description on stat1004 and works fine to me! [14:33:21] 10Analytics, 10Recommendation-API, 10Research: PySpark 2 cannot find numpy - https://phabricator.wikimedia.org/T196592#4264137 (10bmansurov) @elukey, thanks for the reply. I have a virtual environment under `/home/bmansurov/venv/2/` for python2 in stat1005. So the problem was happening before I installed `nu... [15:00:55] hey team [15:01:48] milimetric, yt? you wanted to chat about routing no? [15:02:47] hi mforns, yeah I was just scratching my head about it [15:02:50] you wanna scratch together? [15:02:56] sure :] [15:02:58] omw [15:16:19] (03Draft2) 10Reedy: Add new wikis [analytics/refinery] - 10https://gerrit.wikimedia.org/r/438024 (https://phabricator.wikimedia.org/T183706) [15:16:28] ^ new wikis [15:27:52] (03PS1) 10Milimetric: [WIP] Trying to improve routing logic [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/438030 [15:45:06] hey fdans guess you are not around? [15:45:11] would love a review of https://gerrit.wikimedia.org/r/#/c/437479/4/eventlogging/utils.py [15:45:18] https://gerrit.wikimedia.org/r/#/c/437479/4/eventlogging/utils.py [15:53:34] (03PS1) 10EBernhardson: Fix date parsing [analytics/refinery] - 10https://gerrit.wikimedia.org/r/438034 [15:55:53] (03CR) 10Ottomata: [V: 032 C: 032] Fix date parsing [analytics/refinery] - 10https://gerrit.wikimedia.org/r/438034 (owner: 10EBernhardson) [15:57:43] @srwell that's awfully quick :) [15:57:57] hah [15:59:17] (03CR) 10Nuria: [V: 032 C: 032] Add new wikis [analytics/refinery] - 10https://gerrit.wikimedia.org/r/438024 (https://phabricator.wikimedia.org/T183706) (owner: 10Reedy) [16:00:02] joal: not sure if you saw but there's new wikis ^ [16:00:33] a-team: in an interview, will join standup with a bit of delay :( [16:02:03] ping fdans [16:02:03] not so late elukey :) [16:13:18] 10Analytics, 10Analytics-Kanban, 10Operations, 10Patch-For-Review: Puppet admin module should support adding system users to managed groups - https://phabricator.wikimedia.org/T174465#4264640 (10Ottomata) [16:14:07] (03CR) 10Milimetric: [C: 032] "Wikimedia Statistics" was not shown on title for "all projects" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/437878 (https://phabricator.wikimedia.org/T194224) (owner: 10Nuria) [16:26:56] 10Analytics: Access request for Superset: - https://phabricator.wikimedia.org/T196458#4257350 (10Nuria) Do all these users are either employees or have an NDA on file? If so they should have access to https://turnilo.wikimedia.org/ Can you please verify? [17:14:36] 10Analytics, 10Recommendation-API, 10Research: PySpark 2 cannot find numpy - https://phabricator.wikimedia.org/T196592#4264893 (10elukey) I tested pyspark even on stat1005 and it seems to work fine, I suppose that this means that your venv was preventing pyspark to pick up the numpy packages installed on sta... [17:19:00] 10Analytics: Set up sparksql to try out sql access to hdfs data - https://phabricator.wikimedia.org/T196661#4264937 (10Nuria) [17:27:47] hey milimetric, I think the problem with Wikistats router is that we're not updating the detail store from router/index.js [17:27:48] sorry joal I have another meeting so can't chat about the data stuff [17:28:02] mforns: I saw you saying that and was thinking about it [17:28:25] milimetric, if you have time later today or tomorrow, ping me and we can head-scratch [17:28:34] milimetric: no bother- nuria_ wants to hear about it as well, so let's postpone later today (in 1.5h) [17:28:36] mforns: if you think you have a fix, feel free to push a new patchset, I have a meeting and reviews and I'm off tomorrow so I won't be able to work on it till monday [17:29:01] joal: same, I think I only have time today for finishing reviews since I'm off tomorrow [17:29:06] milimetric, ok, but today I have to finish peer reviews, I'll prioritize that and see if I can throw a patch [17:29:20] mforns: oh, no worries, I can get back to it Monday [17:29:47] milimetric, no prob, let's talk then [17:33:41] np milimetric [17:52:16] FYI that I modified whitelist of wikis with Reedy new additions [17:54:15] * elukey off! [21:08:53] 10Analytics, 10Analytics-Kanban, 10Discovery, 10Discovery-Analysis, 10Product-Analytics: Private data access for non-person user that calculates metrics - https://phabricator.wikimedia.org/T174110#4265899 (10Ottomata) [21:09:07] 10Analytics, 10Analytics-Kanban, 10Operations, 10Patch-For-Review: Puppet admin module should support adding system users to managed groups - https://phabricator.wikimedia.org/T174465#4265904 (10Ottomata) [21:11:55] 10Analytics, 10Analytics-Kanban, 10Discovery, 10Discovery-Analysis, 10Product-Analytics: Private data access for non-person user that calculates metrics - https://phabricator.wikimedia.org/T174110#4265908 (10Ottomata) Hey doods, this is done. `analytics-search` user is now in the `analytics-privatedata-... [23:16:14] 10Analytics, 10Analytics-Kanban, 10Discovery, 10Discovery-Analysis, 10Product-Analytics: Private data access for non-person user that calculates metrics - https://phabricator.wikimedia.org/T174110#3551186 (10mpopov) 05Open>03Resolved Awesome!!! Thank you so much, @Ottomata! :D [23:28:48] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Move statistics::discovery jobs from stat1002 -> stat1005 - https://phabricator.wikimedia.org/T170471#4266252 (10mpopov)