[10:10:11] is there anyone around who can point me to where the nginx configuration for our varnish TLS termination is defined? [10:11:02] TLS config? [10:11:20] a lot is determined by ssl_ciphersuite() [10:11:44] https://github.com/wikimedia/puppet/blob/production/modules/wmflib/lib/puppet/parser/functions/ssl_ciphersuite.rb [10:15:03] no, the nginx config itself [10:17:05] nginx is configured by localssl::instance AFAIK [10:17:09] sorry [10:17:12] tlsproxy::instance [10:17:59] thanks [10:24:04] 10Traffic, 10DNS, 10Operations: Use DNS discovery record for deployment CNAME - https://phabricator.wikimedia.org/T164460 (10fgiunchedi) a:05fgiunchedi→03None [11:46:41] mark: https://gerrit.wikimedia.org/r/#/c/operations/debs/pybal/+/478203/ is a WIP? [12:02:17] 10Traffic, 10DNS, 10Operations, 10Core Platform Team Backlog (Watching / External), 10Services (watching): icinga alerts on nodejs services when a recdns server is depooled - https://phabricator.wikimedia.org/T162818 (10mobrovac) [12:02:44] 10Traffic, 10Operations, 10Core Platform Team Backlog (Watching / External), 10Services (watching), and 2 others: Figure out an etcd deploy strategy that includes multi DC failure scenarios. - https://phabricator.wikimedia.org/T98165 (10mobrovac) [12:17:55] 10Traffic, 10HyperSwitch, 10Operations, 10RESTBase-API, and 2 others: Respect host header in RESTBase, and redirect /rest_v1 to /rest_v1/ - https://phabricator.wikimedia.org/T167972 (10mobrovac) [12:45:49] 10Traffic, 10HyperSwitch, 10Operations, 10RESTBase-API, and 2 others: Respect host header in RESTBase, and redirect /rest_v1 to /rest_v1/ - https://phabricator.wikimedia.org/T167972 (10BBlack) I thought this was in a different ticket somewhere at one point, but in any case I just noticed it during someone... [12:46:50] 10Traffic, 10CX-cxserver, 10Citoid, 10Operations, and 5 others: Decom legacy ex-parsoidcache cxserver, citoid, and restbase service hostnames - https://phabricator.wikimedia.org/T133001 (10mobrovac) [12:52:04] 10Traffic, 10HyperSwitch, 10Operations, 10RESTBase-API, and 2 others: Respect host header in RESTBase, and redirect /rest_v1 to /rest_v1/ - https://phabricator.wikimedia.org/T167972 (10mobrovac) >>! In T167972#4837630, @BBlack wrote: > I thought this was in a different ticket somewhere at one point, but i... [12:55:48] 10Traffic, 10Operations, 10TechCom-RFC, 10Core Platform Team Backlog (Designing), 10Services (designing): Make API usage limits easier to understand, implement, and more adaptive to varying request costs / concurrency limiting - https://phabricator.wikimedia.org/T167906 (10mobrovac) [12:55:56] 10Traffic, 10Multimedia, 10Operations, 10RESTBase-API, and 4 others: Thumb API: Varnish / CDN questions - https://phabricator.wikimedia.org/T150673 (10mobrovac) [12:56:31] 10Traffic, 10Operations, 10TechCom-RFC, 10Wikipedia-Android-App-Backlog, and 3 others: RFC: API-driven web front-end - https://phabricator.wikimedia.org/T111588 (10mobrovac) [15:05:06] 10Traffic, 10Analytics, 10Operations, 10Performance-Team: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) Plain nginx config has the ability to remove the headers, but it can't do so conditionally... [15:20:28] 10Traffic, 10HyperSwitch, 10Operations, 10RESTBase-API, and 2 others: Respect host header in RESTBase, and redirect /rest_v1 to /rest_v1/ - https://phabricator.wikimedia.org/T167972 (10BBlack) >>! In T167972#4837684, @mobrovac wrote: >>>! In T167972#4837630, @BBlack wrote: >> I thought this was in a diffe... [15:24:37] 10Traffic, 10Analytics, 10Operations, 10Performance-Team: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) @BBlack would you miss x-cache, x-cache-status and x-varnish if those were completely removed at the TLS termination level? Some o... [15:38:33] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10BBlack) I don't know off-hand if we can live without them all for manual debugging and such, or if nginx is the best... [15:39:37] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) Is there an nginx "site" or config specific to varnish termination? [15:43:12] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) Could be a puppet variable too, to make the filtering block conditional. [15:52:12] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10BBlack) `localssl.erb` would probably be more appropriate and is the site file, but it's a generic TLS reverse proxy... [16:03:14] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) I wasn't aware that the latest plan was to use ATS for TLS termination. There might be a way to do this in... [17:06:38] 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): Renumber cloud-instance-transport1-b-eqiad to public IPs - https://phabricator.wikimedia.org/T207663 (10aborrero) `lang=shell-session root@cloudcontrol1004:~# neutron subnet-create --gateway 208.80.155.89 --name cloud-instances-tra... [17:09:48] 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): Renumber cloud-instance-transport1-b-eqiad to public IPs - https://phabricator.wikimedia.org/T207663 (10aborrero) `lang=shell-session root@cloudcontrol1004:~# neutron router-gateway-set --fixed-ip subnet_id=cloud-instances-transpo... [17:40:40] 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): Renumber cloud-instance-transport1-b-eqiad to public IPs - https://phabricator.wikimedia.org/T207663 (10aborrero) 05Open→03Resolved All was fine. Thanks @ayounsi . Closing task. [17:50:14] 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): Renumber cloud-instance-transport1-b-eqiad to public IPs - https://phabricator.wikimedia.org/T207663 (10ayounsi) 05Resolved→03Open a:05aborrero→03ayounsi Keeping it open for the cleanup part after the break. [18:06:37] !log doc1001 - meged gerrit:480881 and then manually moved the entire /srv/org/wikimedia/doc/ structure into /srv/docroot/srv/org/wikimedia/ and deleted the old dirs T137890 [18:06:39] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:06:40] T137890: Relocate CI generated docs and coverage reports - https://phabricator.wikimedia.org/T137890 [18:06:55] wrong channel but it logged :) [19:43:42] 10Traffic, 10Analytics, 10Operations, 10Performance-Team, 10Patch-For-Review: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Krinkle) I'm unfamiliar with the complexity needed in VCL to make this work, but if at all feasible, I think we shou... [20:50:42] 10Traffic, 10DNS, 10Operations, 10Operations-Software-Development, 10Patch-For-Review: DNS repo: add CI checks for obvious configuration errors - https://phabricator.wikimedia.org/T182028 (10BBlack) 05Open→03Resolved We've done all this and gone way past it at this point. We might tag some future im... [20:52:56] 10Traffic, 10DNS, 10Operations: AuthDNS CM/CI refactor - https://phabricator.wikimedia.org/T161148 (10BBlack) 05Open→03Resolved a:03BBlack Resolving this, as recent work has fixed a lot of it (other than discovery issues specifically), and at this point all the text above is woefully outdated and point... [20:53:29] 10Traffic, 10DNS, 10Operations, 10Patch-For-Review: adding new languages to DNS langs.tmpl doesn't work until zone template is edited as well - https://phabricator.wikimedia.org/T97051 (10BBlack) 05Open→03Resolved a:03BBlack This is fixed now, no workarounds should be needed. [21:00:18] 10Domains, 10Traffic, 10Operations: SOA serial numbers returned by authoritative nameservers differ - https://phabricator.wikimedia.org/T206688 (10BBlack) Update for the record: with recent changes to authdns CI and deployment scripts, this scenario should no longer be possible and workarounds shouldn't be n... [23:21:15] 10Domains, 10Traffic, 10Operations: SOA serial numbers returned by authoritative nameservers differ - https://phabricator.wikimedia.org/T206688 (10Dzahn) all these cool little updates before year end, nice!