[11:27:45] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations, 10Release-Engineering-Team-TODO: Puppet disabled in beta cluster varnish deployment-cache-text06 - https://phabricator.wikimedia.org/T267578 (10hashar) 05Open→03Declined In T267561 , the varnish related packages have been upgraded to the ones for...
[11:27:49] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar)
[11:46:03] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar) modules/profile/manifests/cache/varnish/frontend.pp line 87 invokes `confd::file` which initalizes the confd module. It has: ` name=modules/confd/manifests/ini...
[12:02:56] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar) I have moved the `confd::srv_dns: deployment-prep.eqiad.wmflabs` setting to a new puppet prefix `deployment-cache` at https://horizon.wikimedia.org/project/pre...
[13:09:52] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10ArielGlenn) I was able to get further along by doing things manually as root on one of the instances,  deployment-cache-text06.  I looked at the systemd unit file, /li...
[14:28:48] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10AlexisJazz)
[14:30:19] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar) Ariel pointed /usr/lib/x86_64-linux-gnu/varnish/vmods/libvmod_re2.so    is the wrong version.  One this task or another related one, I remembered someone marke...
[14:31:36] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar) ` name="$ sudo systemctl status varnish-frontend" ● varnish-frontend.service - varnish-frontend (Varnish HTTP Accelerator)    Loaded: loaded (/lib/systemd/syst...
[14:32:46] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10AlexisJazz) Hey it works now!
[14:45:05] <wikibugs>	 10Traffic, 10Beta-Cluster-Infrastructure, 10Operations, 10User-Ryasmeen: Beta needs to be upgraded to Varnish 6 - https://phabricator.wikimedia.org/T267561 (10hashar) 05Open→03Resolved I have enabled puppet again and ran it. Something I spotted: ` --- /lib/systemd/system/confd.service 2020-11-06 20:04:...
[15:58:46] <wikibugs>	 10Traffic, 10Operations: Varnish 503 errors on page with large number of flag icons. - https://phabricator.wikimedia.org/T267804 (10Bugreporter)
[16:55:03] <wikibugs>	 10Traffic, 10Operations: Varnish 503 errors on page with large number of flag icons. - https://phabricator.wikimedia.org/T267804 (10Joe) p:05Triage→03High
[16:58:43] <wikibugs>	 10Traffic, 10Operations: Varnish 503 errors on page with large number of flag icons. - https://phabricator.wikimedia.org/T267804 (10Joe) I strongly doubt the problem happens at the traffic layer. This seems to be a different kind of problem - maybe those pages once edited overflow some specific limit.  A null...