[08:15:42] 10Traffic, 10MediaWiki-General-or-Unknown, 06Operations: Failure to save recent changes - https://phabricator.wikimedia.org/T150503#2791741 (10ema) @Marshallsumter we've deployed a configuration change on Friday evening that should have fixed the problem. Can you please confirm whether that is the case? Thanks! [11:36:39] 10Traffic, 10ArchCom-RfC, 06Commons, 10MediaWiki-File-management, and 14 others: Define an official thumb API - https://phabricator.wikimedia.org/T66214#2792118 (10Gilles) Passing context like the original dimensions seems like a hack. The thumbnailing server doesn't need it. The API should be agnostic of... [11:41:28] 10Traffic, 10ArchCom-RfC, 06Commons, 10MediaWiki-File-management, and 14 others: Define an official thumb API - https://phabricator.wikimedia.org/T66214#2792125 (10Gilles) As for face recognition to guide focused cropping, this is a functionality Thumbor provides out of the box. It even supports doing the... [12:57:57] ema, bblack: when installing pillow security updates on cp3043 I noticed that cp3043 has an inconsistent package state: [12:58:11] The following packages have unmet dependencies: [12:58:12] varnish-dbg : Depends: varnish (= 4.1.3-1wm3) but 4.1.3-1wm4 is to be installed [13:01:17] I think ema made a new package to test a new extrachance patch, I'm not sure if it was uploaded to carbon or just manual on some hosts [13:02:10] if 3043 is the only one, probably manual [13:02:51] didn't see it anywhere else so far [13:04:29] grrrrrrrrrr at the whole "differing concepts of upstream original" thing between git repos and tarballs :) [13:52:01] moritzm, bblack: yup, that was a manual and on 3043 only [13:52:14] we should probably downgrade to wm3 there for consistency [13:54:30] ema: we should probably make a separate ticket for un-varnish3-ing all our puppetization (and probably wait at least another week before we starting merging that stuff) [13:54:45] +1 [13:55:14] I'm looking forward to one fewer dimensions of complexity! :) [13:55:20] indeed! [13:56:07] i always enjoyed that step [13:56:20] taking a machete to puppet [13:59:07] i think one of the last things I did before I became director, was removing all last remaining bits of squid in our config management [14:02:41] We can title it something like: Reducing varnish puppetization complexity to be describable in dS₇ [14:02:50] (that one's for _joe_ lol) [14:03:02] https://en.wikipedia.org/wiki/De_Sitter_space [14:12:25] moritzm: fixed the inconsistency on cp3043, thanks for reminding me of doing that :) [14:50:43] <_joe_> bblack: ahah [15:15:01] ema: what do you think about the extrachance situation in general? should we stick with wm3? [15:16:26] (and if so, I guess we need to get it installed wherever it isn't) [15:18:04] and then probably promoting whatever is our current varnish4 packaging from experimental to normal repo, is probably one of the last steps in the post-varnish4 cleanup, so we can get the experimental repo back out of apt config on them all [15:18:40] bblack: re: experimental, I'll add that to the v3 cleanup ticket that I'm about to write [15:19:29] not sure about extrachance, wm3 is working fine but it might be good to add a simple (perhaps boolean?) option to toggle the behavior [15:19:51] ok [15:20:02] right now we have wm3 on text+maps+misc, and wm2 on upload [15:20:38] I guess I'll go back and review the wm4 change more, to make sure of whatever I was thinking the other day about the approach maybe backfiring (the whole error 11 thing) [15:21:10] assuming it works as intended, wm4 would be ok to move everything to? [15:21:49] if it does indeed work as advertised I'd say it would :) [15:53:54] ema: do you have a link? I've lost it [15:56:08] bblack: oh I think I didn't open a CR for wm4 [15:56:37] bblack: https://gerrit.wikimedia.org/r/321384 [15:58:51] ema: did you file anything upstream yet? I don't think I did? [15:59:01] bblack: not yet [15:59:29] maybe mention the possible connection with https://github.com/varnishcache/varnish-cache/issues/2126 and related, too [15:59:39] there may be a better overall fix for the whole mess that upstream can think of [15:59:56] 10Traffic, 10DNS, 06Operations, 13Patch-For-Review: Set SPF (... -all) for toolserver.org - https://phabricator.wikimedia.org/T131930#2183433 (10Dzahn) bump, there is a patch waiting in Gerrit for this since many months [16:00:22] 10Traffic, 10DNS, 06Operations, 13Patch-For-Review: Set SPF (... -all) for toolserver.org - https://phabricator.wikimedia.org/T131930#2792581 (10Dzahn) a:05Mschon>03None [16:00:34] 10Traffic, 10DNS, 06Labs, 10Labs-Infrastructure, and 2 others: Set SPF (... -all) for toolserver.org - https://phabricator.wikimedia.org/T131930#2792584 (10Dzahn) [16:00:53] 10Traffic, 10DNS, 06Labs, 10Labs-Infrastructure, and 3 others: Set SPF (... -all) for toolserver.org - https://phabricator.wikimedia.org/T131930#2792585 (10Dzahn) [16:07:47] 10Traffic, 10Varnish, 06Operations: Post Varnish 4 migration cleanup - https://phabricator.wikimedia.org/T150660#2792598 (10ema) [16:13:25] ema: I think the patch is actually fine. with the default of "1", it's probably what was originally intended. It will double the user-facing ultimate timeout in that broken parsoid/rb case, but that case should be fixed anyways. [16:13:48] (should be fixed at the applayer, I mean) [16:23:01] 10Traffic, 06Operations: Extra RTT on TLS handshakes - https://phabricator.wikimedia.org/T150561#2792649 (10BBlack) Above SAL entries deployed the workaround (setting default libssl buffer size to 8K at compile-time), which solves the immediate issue and restores normal handshake performance. Leaving this ope... [16:57:08] bblack: if that's the intended behavior, yeah :) [17:49:39] 10Traffic, 06Operations: Extra RTT on TLS handshakes - https://phabricator.wikimedia.org/T150561#2792910 (10GWicke) My latest round of benchmarks confirm the fix as well. @bblack, thank you for investigating & addressing this regression so quickly! [18:05:09] 10Traffic, 06Operations: Extra RTT on TLS handshakes - https://phabricator.wikimedia.org/T150561#2792961 (10BBlack) [18:29:07] 10Traffic, 10ArchCom-RfC, 06Commons, 10MediaWiki-File-management, and 14 others: Thumb API: Varnish / CDN questions - https://phabricator.wikimedia.org/T150673#2793046 (10GWicke)