[00:47:45] volans_: just got another connection refused fwiw, [00:48:23] paravoid: ack, thanks for the data point, looking at logs. P.S. you should not be awake ;) [00:48:40] request did not show up in access.log [00:48:53] timestamp is ~:46, can't get more exact than that [00:49:22] ack, and no html error, just the browser saying conn refused? [00:49:27] correct [00:58:53] if you're sure it doesn't show in access log it's weird as cannot be something at netbox or uwsgi level, and probably not even apache but network directly [00:59:17] I'm sure it didn't show up in the access log [00:59:25] as there are other requests of yours around that time that were successful, so getting your word on that ;) [00:59:53] puppet runs at :45 but has no changes, is a noop [01:00:18] s/45/44/ [01:09:46] fyi, from --debug [01:09:46] Debug: Exec[remove_uwsgi_initd](provider=posix): Executing check '/usr/sbin/update-rc.d -n -f uwsgi remove | /bin/grep -Pq rc..d' [01:09:50] Debug: Executing: '/usr/sbin/update-rc.d -n -f uwsgi remove | /bin/grep -Pq rc..d' [01:09:53] Debug: /Stage[main]/Uwsgi/Exec[remove_uwsgi_initd]/onlyif: update-rc.d: error: unknown option [01:09:56] probably unrelated [01:12:09] I think so, but thanks [01:12:38] so far it seems that the best way would be to start a packet capture when you start surfing on netbox [01:12:42] :) [01:16:08] and just happened to me again [01:16:15] 2 minutes after the puppet run, so [01:16:21] I think we have our culprit [01:17:19] I'll try to repro [01:17:48] an uwsgi error should show on apache side no matter what [01:56:37] sent https://gerrit.wikimedia.org/r/c/operations/puppet/+/570489 to fix the uwsgi issue [01:59:03] jbond42_: I'm getting the same "undefined method `key?' for nil:NilClass" error that was mentioned before regarding beta on the compiler [01:59:08] https://puppet-compiler.wmflabs.org/compiler1003/20638/netbox1001.wikimedia.org/change.netbox1001.wikimedia.org.err [02:05:15] nevermind, it was a typo in a os_version() check, I mistyped debian. Maybe that can help though ;)