[00:48:11] <wikibugs>	 07HTTPS, 10Traffic, 06Operations, 07Browser-Support-Firefox: Secure connection failed when attempting to send POST request - https://phabricator.wikimedia.org/T134869#2280304 (10Elvey) Seeing this error too. For me too, "it occurs when saving almost any edit", (or at least most edits) since a couple days a...
[06:54:55] <wikibugs>	 10Traffic, 06Operations: Ashburn servers almost unreachable from part of Europe - https://phabricator.wikimedia.org/T136067#2321971 (10Nemo_bis)
[07:07:03] <wikibugs>	 10Traffic, 06Operations: Ashburn servers almost unreachable from part of Europe - https://phabricator.wikimedia.org/T136067#2321998 (10Nemo_bis)
[07:09:50] <wikibugs>	 10Traffic, 06Operations: Ashburn servers almost unreachable from part of Europe - https://phabricator.wikimedia.org/T136067#2321999 (10Nemo_bis) 05Open>03Resolved a:03Nemo_bis Looks like a temporary capacity problem now solved by either Telia or the ISPs...  ``` $ mtr -w -c 50 wikitech.wikimedia.org Star...
[09:31:58] <wikibugs>	 10Traffic, 06Discovery, 06Operations, 10Wikidata, and 2 others: Wikidata Query Service REST endpoint returns truncated results - https://phabricator.wikimedia.org/T133490#2322153 (10ema)
[09:32:01] <wikibugs>	 10Traffic, 10Varnish, 06Operations: Upgrade all cache clusters to Varnish 4 - https://phabricator.wikimedia.org/T131499#2322154 (10ema)
[09:32:04] <wikibugs>	 10Traffic, 10Varnish, 06Operations, 13Patch-For-Review: cache_misc's misc_fetch_large_objects has issues - https://phabricator.wikimedia.org/T128813#2322155 (10ema)
[09:32:07] <wikibugs>	 10Traffic, 10Varnish, 06Operations, 13Patch-For-Review: Convert misc cluster to Varnish 4 - https://phabricator.wikimedia.org/T131501#2322151 (10ema) 05Open>03Resolved It's now been a week after our re-upgrade of cache_misc to Varnish 4 and I'm not aware of any new issues being discovered. Closing.
[10:36:49] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 06Labs, 06Operations, 13Patch-For-Review: Varnishlog doesn't properly rotates logs, varnish.log is empty since forever (was: deployment-cache-upload04 (m1.medium) / is almost full) - https://phabricator.wikimedia.org/T135700#2322313 (10hashar) Puppet is broken on...
[11:29:24] <ema>	 https://ripe72.ripe.net/wp-content/uploads/presentations/21-Ripe_2016v5_final.pdf
[11:29:43] <ema>	 4GB over avian carriers
[12:08:41] <paravoid>	 https://grafana.wikimedia.org/dashboard/db/tls-ciphers seems broken
[12:08:43] <paravoid>	 is there a replacement?
[12:09:21] <paravoid>	 broken since May 19th it seems
[12:24:18] <paravoid>	 ema: ^
[12:25:13] <ema>	 paravoid: you're right
[12:25:26] <ema>	 perhaps something went wrong with https://gerrit.wikimedia.org/r/#/c/289588/?
[12:25:39] <ema>	 that's from May 19th
[12:28:08] <ema>	 or d6ad60ab010e4a15738cd4fbf9bf93aebc9b982d
[12:28:11] <ema>	 mmh
[12:40:45] <ema>	 on cp1008 d6ad60ab010e4a15738cd4fbf9bf93aebc9b982d seems to be doing the right thing...
[12:46:50] <ema>	 paravoid: the data seems to be there, I've just created this one as a test https://grafana.wikimedia.org/dashboard/db/ssl_ciphers
[13:00:02] <bblack>	 hi :)
[13:00:21] <ema>	 hey :)
[13:01:43] <bblack>	 what seems broken to both of you?
[13:01:55] <bblack>	 with tls-ciphers I mean
[13:02:26] <bblack>	 oh... probably non-default time range
[13:02:34] <bblack>	 try zooming out to a months-long view
[13:03:00] <bblack>	 I set the data to weekly averages to get the weekday/weekend blip-noise out of it, to see long-term trends better
[13:03:20] <bblack>	 (and set the default time window to "when we deployed cipher logging -> now", which is 10 months and growing
[13:15:30] <ema>	 bblack: right! Staring at the view for this month made it look like no data has been collected after the 19th 
[13:16:11] <bblack>	 there's probably a "right" way to set it up
[13:16:31] <bblack>	 where the summarize resolution is a variable dependent on the time range being viewed or something
[13:53:05] <paravoid>	 do we collect stats about SSL/TLS versions?
[13:53:43] <paravoid>	 I think we used to, no?
[14:07:52] <ema>	 paravoid: https://grafana.wikimedia.org/dashboard/db/client-connections
[14:08:12] <paravoid>	 ah
[14:08:50] <paravoid>	 thanks :)
[14:08:54] <ema>	 np :)
[16:18:39] <bblack>	 so, in addition to the new dual-cert code, nginx master also added:
[16:18:49] <bblack>	 http://hg.nginx.org/nginx/rev/ce94f07d5082
[16:20:10] <bblack>	 (which is the HTTP/2 pre-read stuff to get around various POST regressions/perf-issues on HTTP/2, may affect the FF 46.0.1 bug too)
[16:20:26] <bblack>	 and they cut a 1.11.0 release with all of that in it
[16:21:01] <bblack>	 so, sometime soon, we should figure out on cp1008 how we move to 1.11.0 in our package (mostly, it's a matter of figuring out the new stapling stuff)