[16:17:18] <Reedy>	 Krinkle: https://twitter.com/w00teh/status/826826401722544129 any ideas? :P
[17:46:38] <Krinkle>	 ostriches: https://gist.github.com/Krinkle/b59e559d59f5013ca38c442319bf3d1d
[17:47:02] <Krinkle>	 11 reqs in Nov 2016
[17:47:44] <ostriches>	 Yeah I ran some stats the other day
[17:47:50] <ostriches>	 For December, I think
[17:48:37] <Krinkle>	 Reedy: Yes actually, easy one :) Will reply (here) later
[17:49:33] <ostriches>	 11 is low enough that I don't mind breaking them
[17:49:48] <ostriches>	 Considering some are probably us (meaning devs/wmf/etc) testing them and such
[17:49:52] <ostriches>	 And/or wondering wtf they are
[17:51:35] <Krinkle>	 Yeah, I shared one on irc in that month
[17:51:58] <Krinkle>	 ostriches: I'm trying to get a reasonable gauge of how many req/s an apache server has in practice for us. 
[17:52:04] <Krinkle>	 Tried various approaches to get at it, but failed.
[17:52:30] <ostriches>	 Hmmm 
[17:52:46] <Krinkle>	 varnish has it in its request response (Server: ..) but I'd have to get them from all varnishes. Looking at kafkacat/wmfrequest we have cache-status:miss, but no server field (hostname= varnish)
[17:53:00] <Krinkle>	 e.g. 10/s, 100/s, 500/s, 1000/s
[17:53:40] <Krinkle>	 I guess I could count all misses for a period and divide by app server count
[17:53:47] <ostriches>	 I mean it'd be super crude but you could tail access.log on an apache and then extrapolate 
[17:54:00] <ostriches>	 That would give you an order of magnitude at least 
[17:56:22] <Krinkle>	 We enable those files?
[17:56:39] <ostriches>	 I think so
[17:57:04] <ostriches>	 I can't check tho, not root
[17:57:11] <Krinkle>	 interesting
[17:57:13] <Krinkle>	 Yep
[17:57:18] <Krinkle>	 other_vhosts_access.log in var/log/apache
[17:57:19] <Krinkle>	 cool
[18:00:32] <Krinkle>	 kafkacat capture:
[18:00:32] <Krinkle>	 - count: 35744 entries
[18:00:32] <Krinkle>	 - first-dt: 2017-02-01T17:54:16
[18:00:32] <Krinkle>	 - last-dt: 2017-02-01T17:54:35
[18:00:32] <Krinkle>	 - diff: 19s
[18:00:33] <Krinkle>	 - g-rate: 35744/19 = 1,881 per second (globally)
[18:00:33] <Krinkle>	 - servers: 118 (conftool-data/nodes/eqiad# api + appserver)
[18:00:34] <Krinkle>	 - s-rate: 1881/118 = 15 per second (per server)
[18:00:47] <Krinkle>	 I'll compare to a real apache in a minute
[18:01:04] <Krinkle>	 (varnish hits with cache miss)
[18:03:25] <Krinkle>	 root@mw1252:/var/log/apache2# cat other_vhosts_access.log | grep '2017-02-01T17:4' | wc -l
[18:03:25] <Krinkle>	 - count: 27079 entries
[18:03:25] <Krinkle>	 - duration: 10 minutes
[18:03:25] <Krinkle>	 - rate: 27079/10/60 = 45 per second
[18:03:28] <Krinkle>	 ostriches: 
[18:03:47] <ostriches>	 meeting I'll check scrollback
[18:12:06] <Krinkle>	 more captures - https://gist.github.com/Krinkle/c410707a3b92fe37c815d3c78b820f30