[10:31:46] vgutierrez: what's an host I can test https://gerrit.wikimedia.org/r/c/operations/puppet/+/977696 on ? [10:32:11] godog: 127.0.0.1? :) [10:32:40] it should be able to dump MSS values for any reachable TCP endpoint [10:32:52] ncredir4001 already hast the required deps manually installed though [10:33:06] ack, thank you [10:43:27] 10netops, 10Infrastructure-Foundations, 10SRE: Add 4x10G breakout cable to cr2-esams - https://phabricator.wikimedia.org/T347323 (10ayounsi) 05Open→03Resolved Ports freed up in T347403 [10:43:51] godog: BTW, given that tcp-mss-clamper already has a prometheus exporter it makes sense to expose the configured MSS values in there, right? [10:44:37] instead of writing another dummy node exporter with harcoded values provided by puppet [10:44:42] *hardcoded [10:45:27] vgutierrez: indeed, might as well do it there [10:52:22] vgutierrez: that consideration doesn't affect reviewing https://gerrit.wikimedia.org/r/c/operations/puppet/+/977696 ? [10:52:43] godog: nope [10:53:02] that provides monitoring for MSS [10:53:09] another CR will provide the configured MSS [10:53:26] and then we can create an alert comparing both values [10:54:39] sgtm [11:22:33] apparently we can't set several reviewers on gitlab [11:22:43] godog, fabfur https://gitlab.wikimedia.org/repos/sre/tcp-mss-clamper/-/merge_requests/8 that's for both of you [11:24:48] ok so, for some context, that's for comparing what tcp-mss-clamper has configured and what it's the actual value (with the prom exporter file you merged above), correct? [11:25:38] fabfur: that's exactly what it says on the MR description :_) [11:49:05] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10daniel) 05Resolved→03Open Re-opening, since we still see CI for restbasefai... [11:56:16] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10Vgutierrez) p:05Unbreak!→03High @daniel what's time outing is `parsoid-exte... [13:47:04] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10Jdforrester-WMF) Looks like https://github.com/wikimedia/restbase/commit/2d9006... [13:50:18] 10Acme-chief, 10Traffic: Provide second acmechief server configured for Puppet 7 in eqiad - https://phabricator.wikimedia.org/T352242 (10KOfori) a:03BCornwall Hi @BCornwall, can you take care of this? [13:53:47] 10Traffic, 10DNS, 10Patch-For-Review: Update DNS records for 1Password - https://phabricator.wikimedia.org/T352579 (10KOfori) [14:05:27] 10Traffic, 10Data-Engineering, 10Observability-Logging: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117 (10Fabfur) Hi @Milimetric sorry for the late reply, I'll try to answer to your question but consider we're still investigating about all pro and cons of this "migrati... [14:35:23] 10Traffic, 10DNS, 10Patch-For-Review: Update DNS records for 1Password - https://phabricator.wikimedia.org/T352579 (10ssingh) 05Open→03Resolved a:03ssingh > wikimedia.org. 594 IN TXT "1password-site-verification=ZAZP5U62WJFMDDZKWJG7TPKFI4" > [15:43:23] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10Vgutierrez) A quick check on deployment-parsoid12 tells that ferm rules on that... [16:09:46] XioNoX: implementing a workaround for a scapy bug I discovered https://pypi.org/project/pyroute2/ [16:09:55] XioNoX: pretty cool [16:20:54] 10Traffic, 10SRE, 10Patch-For-Review: Enable IPIP encapsulation for ncredir - https://phabricator.wikimedia.org/T351069 (10Vgutierrez) [16:47:52] godog: how can I compare metrics from different exporters? [16:48:07] so the instance label isn't the same [16:50:05] vgutierrez: would be nice to be able to use that during debian-installer :) [16:50:23] XioNoX: it's available as a debian package too [16:51:49] vgutierrez: I'm going shortly though right off the bat I'd try with making instance the same by stripping the port with label_replace [16:59:25] label_replace(lvs_realserver_mss_value, "hostname", "$1", "instance", "(.*):.*") [16:59:32] godog: thanks for the pointer [18:20:57] 10Traffic, 10GitLab (Project Migration), 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [21:25:31] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10daniel) Ah right, that's {T350353}. I failed to realize that PRs aren't automa... [22:15:32] 10Acme-chief, 10Traffic: Provide second acmechief server configured for Puppet 7 in eqiad - https://phabricator.wikimedia.org/T352242 (10BCornwall) @KOfori I can but want to point out that, unless I'm mistaken, the hosts that actually use acme-chief are much smaller than the numbers put forth: ` $ sudo -i cum... [22:31:26] 10Traffic, 10Abstract Wikipedia team, 10Beta-Cluster-Infrastructure, 10WikiLambda, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10daniel) 05Open→03Resolved CI works after rebase. Sorry for the noise.