[00:28:52] <grrrit-wm>	 (PS2) Neil P. Quinn-WMF: Fix calculation of time-to-read [analytics/limn-ee-data] - https://gerrit.wikimedia.org/r/280371 (https://phabricator.wikimedia.org/T131206)
[01:15:05] <wikibugs>	 Analytics-EventLogging, Performance-Team, Patch-For-Review: Support kafka in eventlogging client on terbium - https://phabricator.wikimedia.org/T112660#2163970 (Dzahn) Open>Resolved
[05:49:09] <elukey>	 madhuvishy: woooooooooooo
[07:55:01] <wikibugs>	 Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2164194 (elukey) a:elukey
[08:15:15] <wikibugs>	 Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2164245 (elukey) Open>Resolved As stated in the commit message:  """ We added a graphite metric (using a diamond script) to track nf_conntrack_count and observed the k...
[08:27:41] <wikibugs>	 Analytics-Kanban, Operations, Traffic, Patch-For-Review: varnishkafka integration with Varnish 4  for analytics - https://phabricator.wikimedia.org/T124278#2164270 (elukey) Summary of last updates:  1) varnish-kafka code changes in https://gerrit.wikimedia.org/r/#/c/276439/ and https://gerrit.wikim...
[08:29:54] <joal>	 Yay for uniques !
[09:02:31] <elukey>	 yep a really big result!
[09:02:45] <joal>	 Indeed !
[09:58:58] <wikibugs>	 Analytics-Tech-community-metrics, DevRel-March-2016: For some people, top-contributors.html displays three-digit numbers instead of names - https://phabricator.wikimedia.org/T128171#2164522 (Aklapper) Open>Resolved This bug looks fixed! (Probably by the two changes linked in T129837?)  Yay, thanks!...
[09:59:00] <wikibugs>	 Analytics-Tech-community-metrics, DevRel-March-2016: Key performance indicator: Top contributors: Find good Ranking algorithm fix bugs on page - https://phabricator.wikimedia.org/T64221#2164526 (Aklapper)
[10:01:03] <wikibugs>	 Analytics-Tech-community-metrics, Developer-Relations, DevRel-April-2016: Who are the top 50 independent contributors and what do they need from the WMF? - https://phabricator.wikimedia.org/T85600#2164533 (Aklapper)
[10:01:05] <wikibugs>	 Analytics-Tech-community-metrics, DevRel-March-2016: Key performance indicator: Top contributors: Find good Ranking algorithm fix bugs on page - https://phabricator.wikimedia.org/T64221#2164531 (Aklapper) stalled>Resolved **All blocking tasks are fixed, hence closing as resolved.**  If someone does...
[12:28:18] <elukey>	 joal: o/
[12:28:21] <joal>	 Hi !
[12:28:26] <elukey>	 do you have a minute for a unique question?
[12:28:28] <joal>	 wasup?
[12:28:30] <joal>	 sure
[12:28:45] <joal>	 unique question, as of only one? ;-P
[12:29:08] <elukey>	 O_O
[12:29:11] <elukey>	 :D
[12:29:18] <elukey>	 I am on stat1002 executing
[12:29:19] <elukey>	 kafkacat -C -b kafka1012.eqiad.wmnet:9092 -t webrequest_maps -c 20 | grep --color x_analytics
[12:29:41] <elukey>	 basically cp1043 is running varnish/vk 4, meanwhile cp1044 v3
[12:30:00] <elukey>	 we applied a patch to re-enable the x_analytics header
[12:30:15] <elukey>	 and I can now see WMF-Last-Access sometimes
[12:30:23] <elukey>	 (on cp1043)
[12:30:25] <joal>	 k, following so far
[12:30:59] <elukey>	 most of the times the header is only "x_analytics":"https=1;nocookies=1", that should be correct but I wanted to double check with you
[12:33:23] <joal>	 elukey: easy check is to ask hive :)
[12:35:03] <joal>	 elukey: currently running SELECT x_analytics, COUNT(1) as c from wmf.webrequest where webrequest_source = 'maps' and year = 2016 and month = 3 and day = 31 group by x_analytics order by c desc limit 20;
[12:35:24] <elukey>	 joal: ahhh nice!
[12:35:40] <elukey>	 writing it down :)
[12:36:15] <joal>	 elukey: top row is 'https=1;nocookies=1', second is '-'
[12:36:26] <joal>	 So even more 'https=1;nocookies=1' than empty headers :)
[12:36:32] <joal>	 looks ok :)
[12:37:05] <joal>	 When I say even more, it about 3 'https=1;nocookies=1' for 2 empty
[12:37:09] <joal>	 elukey: --^
[12:38:27] <elukey>	 all right, seems good :)
[13:00:43] <grrrit-wm>	 (PS1) Joal: Add monitoring to every endpoint [analytics/aqs] - https://gerrit.wikimedia.org/r/280661
[13:02:49] <grrrit-wm>	 (PS1) Joal: Add script inserting monitoring fake data in cass [analytics/aqs/deploy] - https://gerrit.wikimedia.org/r/280662
[13:04:26] <grrrit-wm>	 (PS2) Joal: Add script inserting monitoring fake data in cass [analytics/aqs/deploy] - https://gerrit.wikimedia.org/r/280662
[13:05:40] <joal>	 a-team, I'm AFK for a bit
[13:29:34] <elukey>	 ottomata: Maps cluster running Varnish 4 and VK patched to support the X-Analytics header
[13:29:43] <ottomata>	 good morning! oh awesome
[13:29:48] <ottomata>	 link to patch?  just wanna see
[13:29:49] <elukey>	 morning! :)
[13:29:59] <ottomata>	 wait im' sure i have it
[13:30:04] <ottomata>	 just haven't gotten that far in email yet :)
[13:30:21] <elukey>	 https://gerrit.wikimedia.org/r/#/c/280612/
[13:30:23] <ottomata>	 danke
[13:30:31] <elukey>	 :)
[13:32:10] <ottomata>	 nice :)
[13:33:34] <ottomata>	 yeehaw
[13:33:35] <ottomata>	   "x_analytics": "https=1;nocookies=1",
[13:50:26] <ottomata>	 elukey:  you got time for a couple of puppet reviews?
[13:50:56] <elukey>	 ottomata: sure!  but usually I always come up with wrong ideas and you correct me, not the other way around :P
[13:51:23] <ottomata>	 haha, maybe maybe not, but your questions make me think differently too!
[13:51:30] <ottomata>	 these two:
[13:51:33] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/280486/
[13:51:33] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/280497/
[13:52:14] <ottomata>	 the first one is just a little rearrangement
[13:52:16] <ottomata>	 previously
[13:52:30] <ottomata>	 both eventlogging and eventlogging::package had package dependencies in them
[13:52:33] <ottomata>	 which was dumb
[13:52:43] <ottomata>	 some places need to be able to just install and run eventlogging from the CLI
[13:52:57] <ottomata>	 others, like eventlog1001, need upstart configs ready to manage eventlogging daemons
[13:53:39] <ottomata>	 the worst part about this whole deal is that eventlogging::service::service doesn't currently  work on trusty because of dependencies, and eventlogging is deployed from a different source on tin for it for eventbus
[13:53:43] <ottomata>	 using scap
[13:53:52] <ottomata>	 i'm trying to fix all that
[13:53:57] <ottomata>	 these two patches are the first steps
[13:54:06] <elukey>	 coffee && review :)
[13:54:26] <ottomata>	 1. consolidate and organize some stuff
[13:54:26] <ottomata>	 2. run EL from deployed dirs rather than globally installed via pip
[13:54:44] <ottomata>	 once we no longer use pip to install, we can fix up the dependencies, because pip won't be checking them
[13:55:29] <ottomata>	 ok ! :)
[14:01:21] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: Make sure ALL eventlogging dependencies are available in both jessie and trusty via .deb packages - https://phabricator.wikimedia.org/T131354#2164782 (Ottomata)
[14:12:15] <elukey>	 ottomata: 1. +1, will check 2. in a bit :)
[14:12:30] <elukey>	 I have a question for you about vk rsyslog
[14:12:40] <ottomata>	 great
[14:12:44] <ottomata>	 ok ask me!
[14:12:50] <elukey>	 do you remember that we decided to put a ensure file inside the vk module?
[14:13:27] <grrrit-wm>	 (PS3) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[14:13:34] <elukey>	 Should I stick the complete name (70_blabla) as the rsyslog config would do or does it feel wrong?
[14:16:11] <grrrit-wm>	 (PS4) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[14:29:02] <ottomata>	 elukey:  sorry
[14:29:05] <ottomata>	 got distracted
[14:29:16] <ottomata>	 hm
[14:29:30] <ottomata>	 elukey:  probably so
[14:29:42] <ottomata>	 are those prefixes just added by the rsyslog::conf define?
[14:31:36] <elukey>	 ottomata: yep
[14:33:17] <ottomata>	 elukey:  hmm elukey i doubt it really matters
[14:33:20] <ottomata>	 and i don't have a preference i think
[14:33:22] <ottomata>	 so whatever you prefer
[14:34:54] <elukey>	 the alternative is to add a ryslog dependency
[14:35:20] <elukey>	 or just to do it elsewhere
[14:42:22] <grrrit-wm>	 (PS5) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[14:42:50] <milimetric>	 :( irccloud failed me
[14:44:19] <ottomata>	 naw, don't add the dep
[14:44:24] <ottomata>	 in the module is fine with just a file resource
[14:44:32] <ottomata>	 it will work as long as something in puppet declares a file
[14:44:37] <ottomata>	 it doesn't have to be in rsyslog module
[14:48:22] <elukey>	 all right
[14:48:28] <elukey>	 finished the code reviews, they look sound!
[14:49:22] <ottomata>	 great!
[14:49:23] <ottomata>	 thanks!
[14:49:29] <ottomata>	 appreciate it
[15:01:10] <grrrit-wm>	 (PS6) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[15:06:36] <milimetric>	 joal: I'm assuming these UNSET cassandra load things are from your uniques job?
[15:06:54] <milimetric>	 hm they say pageviews tho
[15:24:37] <joal>	 milimetric: cassandra has been a bit overhelmed lately
[15:27:04] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Move EventLogging service to scap3 - https://phabricator.wikimedia.org/T118772#2164923 (Ottomata)
[15:27:28] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Move EventLogging service to scap3 - https://phabricator.wikimedia.org/T118772#1808930 (Ottomata) a:Ottomata
[15:27:48] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Move EventLogging to scap3 - https://phabricator.wikimedia.org/T118772#1808930 (Ottomata)
[15:28:05] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#1808930 (Ottomata)
[15:29:13] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Integrate  new browser visualization into wikistats - https://phabricator.wikimedia.org/T129101#2164944 (Nuria) Open>Resolved
[15:29:15] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Stop using global pip install for eventlogging deploy - https://phabricator.wikimedia.org/T131263#2164945 (Ottomata)
[15:30:43] <madhuvishy>	 A-team: running couples minutes late. Had to restart router due to connectivity issues
[15:39:14] <wikibugs>	 Analytics-Cluster, Operations, hardware-requests: setup/deploy server analytics1003/WMF4541 - https://phabricator.wikimedia.org/T130840#2164979 (Ottomata) @Robh, who should we ask?  @faidon?
[15:47:02] <wikibugs>	 Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#2165022 (mark) a:mark>RobH Approved from the pool of new spare systems.
[15:47:49] <wikibugs>	 Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#2165029 (Ottomata) @Robh, are the SSDs for this already ordered too?
[15:49:06] <ottomata>	 elukey:  error is
[15:49:22] <ottomata>	 Could not find resource 'Rsyslog::Conf[varnishkafka]' for relationship on 'Varnishkafka::Instance[statsv]'
[15:49:37] <ottomata>	 looks like there is probably some require or supbscribe left in the cache roles that you need to remove
[15:50:01] <elukey>	 ottomata: yes yes I wanted to talk about it with you :)
[15:50:04] <grrrit-wm>	 (PS7) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[15:50:43] <wikibugs>	 Analytics-Kanban, Release-Engineering-Team: [Spike] Figure out how to automate releases with jenkins {hawk} - https://phabricator.wikimedia.org/T130576#2165035 (madhuvishy)
[15:50:58] <ottomata>	 elukey:  i think probably role::cache::kafka line 18
[15:51:01] <ottomata>	     Rsyslog::Conf['varnishkafka'] -> Varnishkafka::Instance <|  |>
[15:51:02] <elukey>	 soooo I'd need to remove Rsyslog::Conf['varnishkafka'] -> Varnishkafka::Instance <|  |>
[15:51:28] <ottomata>	 ja, there is no more Rsyslog::Conf['varnishkafka'], you removed it
[15:51:31] <elukey>	 yes but there is a message on top of it saying "this needs to happen otherwise you'll be sad"
[15:51:51] <elukey>	 but why was it enforced before?
[15:51:55] <ottomata>	 elukey:  you could move that to varnishkafka init class
[15:52:03] <ottomata>	 and change it to the File
[15:52:24] <ottomata>	 File['.../rsyslog.d/varnishkafka'] -> Varnishkafka::Instance  <| |>
[15:52:39] <ottomata>	 hm
[15:52:41] <ottomata>	 oh
[15:52:43] <ottomata>	 might not be necessary
[15:52:44] <wikibugs>	 Analytics-Kanban, Release-Engineering-Team: [Spike] Figure out how to automate releases with jenkins {hawk} - https://phabricator.wikimedia.org/T130576#2165037 (madhuvishy) Tagging releng team here. I'm happy to do the work for this task - and it would be great if someone from releng could help - since thi...
[15:52:53] <ottomata>	 ja, not necessary
[15:52:58] <ottomata>	 elukey: because varnishkafka::instance
[15:52:58] <ottomata>	 does
[15:52:59] <ottomata>	     require ::varnishkafka
[15:53:05] <ottomata>	 so, everything in varnishkafka init class has to be set up before the instance anyway
[15:53:22] <ottomata>	 this was just needed before because the rsyslog::conf was not in the module (as it should be :) )
[15:54:24] <elukey>	 sigh
[15:55:43] <ottomata>	 makes sense?
[15:55:46] <ottomata>	 no its better this way!
[15:55:47] <ottomata>	 just remove it
[15:55:52] <ottomata>	 then the dependency is hanlded totally by the module
[15:56:44] <ottomata>	 that explicit dependency was a hack, because the rsyslog conf file should be in place by the module OR by the package, whichever it is it would be managed by the module (since the module installs the package). we only need to declare the file at all because puppet's rsyslog module purges files it doesn't know about
[15:56:46] <elukey>	 yep yep I am running puppet compiler
[15:56:48] <ottomata>	 cool
[15:58:45] <elukey>	 ottomata: https://puppet-compiler.wmflabs.org/2246/cp1044.eqiad.wmnet/ better now, the only thing that still remains if the missing notify to rsyslog
[15:59:01] <elukey>	 not sure if it matters or not
[16:00:18] <grrrit-wm>	 (PS8) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144)
[16:02:09] <ottomata>	 elukey:  that probably does matter
[16:02:16] <ottomata>	 can you add that in the vk module?
[16:02:22] <ottomata>	 HMMM
[16:02:24] <ottomata>	 actually
[16:03:01] <ottomata>	 yeah it does matter
[16:03:03] <elukey>	 we are trying to avoid a dependency to rsyslog but it keeps biting us
[16:03:06] <elukey>	 :D
[16:03:08] <ottomata>	 its ok
[16:03:15] <ottomata>	 we are trying to avoid a dependency on the rsyslog module
[16:03:28] <ottomata>	 we're already assuming rsyslog is available and instaleld
[16:03:33] <ottomata>	 just by putting something in /etc/rsyslog.d
[16:03:40] <ottomata>	 and by doing the invoke-rc.d rsyslog rotate
[16:03:42] <ottomata>	 so
[16:03:50] <ottomata>	 you can make the file notify
[16:03:55] <ottomata>	 notify => Service['rsyslog']
[16:04:04] <nuria>	  ottomata: come to taskinggg please
[16:04:08] <ottomata>	 OH sorry
[16:04:08] <ottomata>	 k
[16:22:53] <wikibugs>	 Analytics: Pageviews API doesn't work for titles with % symbol. - https://phabricator.wikimedia.org/T131369#2165157 (Pchelolo)
[17:23:03] <elukey>	 didn't want to keep all of you in the batcave but as you might have guessed by my face I have understood 30% of what you were talking about :D Is there any merciful soul that will have the patience to answer my questions during the next days?
[17:23:46] <elukey>	 I'd like to spend a bit of time reading to get the basics
[17:30:05] <joal>	 elukey: sure :)
[17:30:42] * elukey hugs joal
[17:32:47] <ottomata>	 :_
[17:32:48] <ottomata>	 :)
[17:43:53] <elukey>	 ottomata: https://gerrit.wikimedia.org/r/#/c/280690/1 good to merge?
[17:44:43] <ottomata>	 elukey:  just merged! :)
[17:44:51] <elukey>	 yeeaaahhh
[17:49:17] <ottomata>	 elukey:  oook, feel free to merge that through ops/puppet, +2 from me
[17:49:22] <ottomata>	 i gotta run out for a bit, back in like 45 mins prob
[17:49:54] <elukey>	 ottomata: running puppet compiler, I'll probably merge it with ema tomorrow
[17:49:57] <elukey>	 thankssssss
[17:50:48] <ottomata>	 k laters
[18:00:44] <joal>	 milimetric: Do you mind having a look at the two CR I sent on aqs and aqs-deploy?
[18:00:55] <milimetric>	 no, I don't mind
[18:00:59] <joal>	 :)
[18:01:02] <joal>	 Thanks mate
[18:01:03] <milimetric>	 but is it ok if I do it after metrics?
[18:01:08] <milimetric>	 no rush, right?
[18:01:09] <joal>	 For sure !
[18:01:14] <joal>	 no rush at all
[18:01:20] <milimetric>	 k, will do after
[18:02:21] <wikibugs>	 Analytics-Kanban: Bug in pageviews - https://phabricator.wikimedia.org/T131386#2165578 (Milimetric)
[18:02:58] <elukey>	 going offline team!
[18:02:59] <elukey>	 byeeeee
[18:27:35] <ottomata>	 (back)
[19:07:54] <milimetric>	 nuria: I'm gonna step out soon unless you wanna talk about the dashiki changes
[19:08:09] <milimetric>	 brb 10 minutes, let me know if you've reviewed it and want to talk / merge
[19:11:42] <wikibugs>	 Analytics-Cluster, Operations, hardware-requests: setup/deploy server analytics1003/WMF4541 - https://phabricator.wikimedia.org/T130840#2165810 (Cmjohnson)
[19:11:44] <wikibugs>	 Analytics-Cluster, Operations, hardware-requests: update label on analytics1003/WMF4541 - https://phabricator.wikimedia.org/T130845#2165808 (Cmjohnson) Open>Resolved updated label and racktables
[19:29:10] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Add monitoring to every endpoint [analytics/aqs] - https://gerrit.wikimedia.org/r/280661 (owner: Joal)
[19:29:35] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Add script inserting monitoring fake data in cass [analytics/aqs/deploy] - https://gerrit.wikimedia.org/r/280662 (owner: Joal)
[19:30:14] <grrrit-wm>	 (PS2) Milimetric: Add manual test scripts in new test folder [analytics/aqs/deploy] - https://gerrit.wikimedia.org/r/280465 (owner: Joal)
[19:30:52] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] "this is fine for now, we can certainly improve it, add scap commands and make it non-zero exit when there's a problem, etc. But it's in m" [analytics/aqs/deploy] - https://gerrit.wikimedia.org/r/280465 (owner: Joal)
[19:31:47] <wikibugs>	 Analytics-Kanban: Bug in pageviews - https://phabricator.wikimedia.org/T131386#2165854 (Milimetric)
[19:31:49] <wikibugs>	 Analytics: Pageviews API doesn't work for titles with % symbol. - https://phabricator.wikimedia.org/T131369#2165855 (Milimetric)
[19:32:01] <wikibugs>	 Analytics-Kanban: Pageviews API doesn't work for titles with % symbol. - https://phabricator.wikimedia.org/T131369#2165157 (Milimetric) a:Milimetric
[19:32:49] <Pchelolo>	 milimetric: do you have time to chat about https://phabricator.wikimedia.org/T131369 a bit?
[19:33:51] <milimetric>	 Pchelolo: sure
[19:34:10] <Pchelolo>	 So, the relevant normalisation code in frontend RESTBase: https://github.com/wikimedia/restbase/blob/master/lib/normalize_title_filter.js
[19:34:11] <milimetric>	 let me read it slowly, one minute
[19:36:41] <Pchelolo>	 I think we can add a special-case for pageviews there, but the question is how are the titles store in cassandra in aqs. The normalization returns localized namespace name + article name in db-key format, respecting MW configs. The question is whether AQS uses the same format.
[19:36:43] <milimetric>	 Pchelolo: hm...
[19:36:58] <milimetric>	 so they're stored using our own normalization
[19:37:04] <milimetric>	 one sec
[19:37:48] <milimetric>	 the normalization code is: https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/PageviewDefinition.java#L459 or https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/PageviewDefinition.java#L501
[19:37:48] <milimetric>	 depending how we get the page
[19:38:11] <milimetric>	 the other thing is, does your new restbase normalization resolve redirects?
[19:38:17] <milimetric>	 (because we serve view counts for redirects too)
[19:39:55] <milimetric>	 Pchelolo: ^
[19:40:22] <Pchelolo>	 hm.. I'm not sure normalization you do will provide the same result as our `mediawiki-title` lib..
[19:41:25] <milimetric>	  yeah, it won't
[19:41:49] <milimetric>	 but the normalization in mediawiki-title does seem to include redirect resolution (https://github.com/wikimedia/restbase/blob/master/lib/normalize_title_filter.js#L117)
[19:42:00] <milimetric>	 so that won't work because we need to serve the view count for redirects too
[19:42:28] <Pchelolo>	 milimetric: this is a special case for files in commons made for mobile apps
[19:42:47] <milimetric>	 I see... I saw other redirect stuff above
[19:42:52] <milimetric>	 so in general it won't resolve them?
[19:42:56] <Pchelolo>	 in general we just send a 301 redirect to a normalized version of a title if non-normalized version is encountered
[19:43:08] <milimetric>	 ok, I see
[19:43:45] <milimetric>	 then yeah, we need to change our normalization code, backfill all the data, and then maybe move to this... seems rough
[19:44:10] <Pchelolo>	 So, as you're making your own normalisation prior to saving, I think it's better for you to just do the same normalisation in AQS
[19:44:30] <Pchelolo>	 easier than backfilling all data one more time
[19:45:13] <milimetric>	 yeah, but we still have to uriDecode, right?
[19:45:38] <milimetric>	 'cause people are sending in encoded data, and in my opinion sending an un-encoded % is malformed
[19:45:46] <milimetric>	 we should catch it and return a proper error
[19:45:47] <ottomata>	 !log merging puppet change to run eventlogging code out of deploy repo
[19:45:50] <analytics-logbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master
[19:45:50] <milimetric>	 but I'm not sure we should handle it
[19:45:56] <Pchelolo>	 milimetric: no, uriDecode is done transparently before the parameters get to the handler.
[19:46:09] <milimetric>	 Pchelolo: is that something new?
[19:46:20] <Pchelolo>	 milimetric: no, that was happening all the time
[19:46:34] <milimetric>	 Pchelolo: then why was it failing when people sent encoded titles and it worked after our patch?
[19:46:50] <Pchelolo>	 lemme check
[19:48:03] <milimetric>	 (I tested that personally)
[19:48:16] <Pchelolo>	 milimetric: are you referring to 'Speciaal:MyPage/zeusmodepreferences.js' bug?
[19:48:39] <milimetric>	 no, that wasn't encoded, I meant the one Yuri filed, one sec
[19:48:57] <milimetric>	 https://phabricator.wikimedia.org/T127034
[19:49:31] <Pchelolo>	 milimetric: Ah, because you've added not only 'decodeURIComponent', but also space to underscore replacement I think
[19:49:44] <milimetric>	 ok, so doing both of those at the same time confused the matter
[19:49:52] <Pchelolo>	 so 'decodeURIComponent' in that case should be a no-op
[19:49:58] <milimetric>	 ok, I guess all there is to do then is to remove the decode
[19:50:37] <milimetric>	 btw, in case you didn't see, usage of the pageview API just went up a lot
[19:50:48] <Pchelolo>	 milimetric: yep, that should work.
[19:51:22] <milimetric>	 ok, I'll do that before this next deploy, but I gotta head out now
[19:51:24] <gwicke>	 milimetric: might be related to "Magnus labs tools"
[19:51:26] <milimetric>	 thanks Pchelolo!
[19:51:27] <grrrit-wm>	 (PS1) Joal: Correct special character in cassandra load job [analytics/refinery] - https://gerrit.wikimedia.org/r/280712
[19:51:42] <Pchelolo>	 milimetric: but that could be a cool feature to integrate proper normalisation, because there's really a lot of variants how the same article can be requested.
[19:51:49] <grrrit-wm>	 (CR) Joal: [C: 2 V: 2] "Self merging bug." [analytics/refinery] - https://gerrit.wikimedia.org/r/280712 (owner: Joal)
[19:51:59] <milimetric>	 thanks gwicke, was that announced on wikitech?
[19:52:38] <milimetric>	 Pchelolo: I agree, it's awful, we need maybe a service that handles all page metadata retrieval
[19:53:01] <milimetric>	 title normalization, id, renames, namespace changes, category changes, etc.
[19:53:01] <gwicke>	 milimetric: no, I just happened to look into recent UA trends, and noticed that Magnus trended up
[19:53:02] <nuria>	 milimetric: looking at layouts now
[19:53:12] <milimetric>	 aha, cool
[19:53:28] <milimetric>	 nuria: ok, I'm gonna head out but I'll stay on IRC so we can chat
[19:53:52] <gwicke>	 milimetric: although overall pageview volume is still quite low
[19:54:08] <gwicke>	 32/s
[19:54:27] <gwicke>	 https://grafana-admin.wikimedia.org/dashboard/db/restbase?panelId=15&fullscreen
[19:56:22] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Stop using global python setup.py install for eventlogging deploy - https://phabricator.wikimedia.org/T131263#2165903 (Ottomata)
[19:56:33] <milimetric>	 low
[19:57:28] <milimetric>	 But high for cassandra apparently
[19:57:44] <milimetric>	 Because load jobs died left and right :)
[19:58:18] <milimetric>	 Time to consider Eric's offline load and replicate solution
[19:58:33] <gwicke>	 milimetric: you are limited by rotating disks
[19:59:00] <milimetric>	 Yeah and sadly there are some delays getting ssds
[20:01:52] <ottomata>	 !log stopping eventlogging, uninstalling globally installed eventlogging python code, running puppet, restarting eventlogging from /srv/deployment/eventlogging/eventlogging
[20:01:54] <analytics-logbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master
[20:07:12] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Stop using global python setup.py install for eventlogging deploy - https://phabricator.wikimedia.org/T131263#2165935 (Ottomata) Did the following! And it works!  ``` eventloggingctl status eventloggingctl s...
[20:12:31] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Stop using global python setup.py install for eventlogging deploy - https://phabricator.wikimedia.org/T131263#2165971 (Ottomata) Docs updated here:  https://wikitech.wikimedia.org/wiki/Analytics/EventLogging...
[20:16:48] <Pchelolo>	 ottomata: small question: do we have eventlogging-service in labs?
[20:17:51] <icinga-wm>	 PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 26.67% of data above the critical threshold [30.0]
[20:17:59] <ottomata>	 Pchelolo:  jaaaa
[20:18:03] <ottomata>	 in deployment-prep
[20:18:08] <ottomata>	 deployment-eventlogging04 instance
[20:18:16] <ottomata>	 you testing the resource_change?
[20:18:22] <ottomata>	 it should jsut work if you get the thing merged in beta mw
[20:19:44] <Pchelolo>	 ottomata: not testing yet, but thinking about it :)
[20:21:21] <Pchelolo>	 thank you for the pointer
[20:21:24] <ottomata>	 yup!
[20:28:30] <icinga-wm>	 RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 20.00% above the threshold [20.0]
[20:40:26] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#2166201 (Ottomata)
[20:48:52] <ottomata>	 HM, madhuvishy maybe we should move the eventlogging role classes into modules/role
[20:49:13] <madhuvishy>	 ottomata: ahhh - yeah might as well - will you change the name?
[20:49:27] <madhuvishy>	 rather, do we need to rename them?
[20:49:54] <ottomata>	 ja some
[20:50:10] <ottomata>	 i'm actually tryign to figure out how to make this all work with scap
[20:50:24] <ottomata>	 and...ha, the differentiation between eventlogging and eventlogging::server I just made might not make sense anymore! :o
[20:50:40] <ottomata>	 might get rid of eventlogging module init class, and just call it eventlogging:;dependencies
[20:50:49] <ottomata>	 and have a define that configures an eventlogging scap target
[20:54:07] <madhuvishy>	 ottomata: I didn't finish reviewing your patch because I thought luca did
[20:56:08] <ottomata>	 he did :)
[20:56:13] <ottomata>	 already merged :)
[20:56:29] <ottomata>	 madhuvishy:  help me figure this out though...
[20:56:31] <ottomata>	 for scap to work
[20:56:34] <madhuvishy>	 ottomata: sure
[20:56:37] <ottomata>	 we need a scap/ directory
[20:56:40] <ottomata>	 with scap.cfg and other stuff
[20:56:45] <madhuvishy>	 okay..
[20:56:52] <ottomata>	 in the repo on tin
[20:57:00] <ottomata>	 i don't want to commit this to eventlogging repo
[20:57:05] <ottomata>	 because there will be multiple sources and targets
[20:57:18] <ottomata>	 e.g. eventlogging/eventbus, eventlogging/analytics (my proposed new name for our eventlog1001 target)
[20:57:42] <ottomata>	 in other repos
[20:57:43] <madhuvishy>	 right - we can do eventlogging-scap or something like that - and git clone them into an /srv/eventlogging/config folder from puppet?
[20:57:45] <ottomata>	 this is done with deploy repos
[20:57:52] <madhuvishy>	 right
[20:58:04] <madhuvishy>	 yeah - ores does this, wikimetrics too
[20:58:09] <ottomata>	 well, the scap/ needs to be next to the .git dir it will deploy
[20:58:10] <madhuvishy>	 fabric is the deployer ther
[20:58:11] <ottomata>	 right
[20:58:14] <ottomata>	 so, what
[20:58:16] <ottomata>	 new repos?
[20:58:23] <ottomata>	 eventlogging/deploy/analytics
[20:58:23] <ottomata>	 eventlogging/deploy/eventbus
[20:58:30] <ottomata>	 each with eventlogging source as a submodule?
[20:58:34] <ottomata>	 and their own scap/ dirs?
[20:58:45] <madhuvishy>	 why do we need eventlogging source to be a submodule?
[20:59:01] <ottomata>	 how else woudl you do it?
[20:59:21] <madhuvishy>	 we can just clone both where we need them?
[20:59:33] <ottomata>	 ?
[20:59:49] <madhuvishy>	 uhhh may be i don't know enough about what setup scap needs
[21:00:08] <ottomata>	 ok so scap/ dir has to live right next to .git for repo that it will be deploying
[21:00:09] <ottomata>	 so
[21:00:21] <ottomata>	 myrepo/
[21:00:21] <ottomata>	   scap/
[21:00:21] <ottomata>	   .git
[21:00:21] <ottomata>	    src/
[21:00:21] <ottomata>	    ...
[21:00:48] <ottomata>	 if your source repo only ever goes to one set of targets (and you don't expect anyone else to ever deploy it for something else)
[21:00:54] <ottomata>	 then you can commit the scap/ dir to your source repo
[21:00:59] <madhuvishy>	 right
[21:01:03] <ottomata>	 but, eventlogging isn't like that, its more generic
[21:01:11] <ottomata>	 we don't want to put wmf scap deploy configs in the source repo
[21:01:18] <ottomata>	 so, we need to put it somewhere else
[21:01:26] <ottomata>	 but we still need it to deploy eventlogging source code
[21:01:40] <madhuvishy>	 my idea was - we have the scap repo in eventlogging/deploy/eventbus - and just clone it inside the repo/ dir
[21:01:47] <madhuvishy>	 using Git::clone
[21:01:54] <ottomata>	 hm
[21:01:59] <ottomata>	 manually on tin?  or i guess puppetized?
[21:02:00] <ottomata>	 hm
[21:02:04] <madhuvishy>	 puppet
[21:02:07] <ottomata>	 hm
[21:02:10] <ottomata>	 hm
[21:02:39] <madhuvishy>	 we can ensure updated so it would automatically be updated?
[21:02:40] <ottomata>	 hmm, i think the .git dirs will conflict
[21:02:48] <ottomata>	 you can't clone two repos into the same dir
[21:03:15] <madhuvishy>	 but isn't myrepo at a higher level?
[21:03:46] <ottomata>	 hm oh right clone it at scap/
[21:04:02] <ottomata>	 myrepo/.git
[21:04:02] <ottomata>	  myrepo/scap/.git
[21:04:09] <madhuvishy>	 clone myrepo, clone scap inside myrepo
[21:04:09] <madhuvishy>	 yes
[21:04:22] <ottomata>	 hm, ja, if we did that, id' call the repo  eventlogging/scap/eventbus
[21:04:22] <ottomata>	 hm
[21:04:26] <ottomata>	 not a bad idea
[21:04:36] <ottomata>	 lemme try to make that work with eventbus as is
[21:04:42] <ottomata>	 right now the scap/ dir only exists on tin manually
[21:04:45] <ottomata>	 not in git anywhere
[21:05:07] <madhuvishy>	 cool
[21:05:34] <madhuvishy>	 it sounds like it'll work - and i don't like dealing with submodule commits so kinda like this way
[21:07:18] <ottomata>	 yeah i like
[21:07:21] <ottomata>	 good idea!
[21:09:08] * madhuvishy goes to look at how scap works