[07:08:45] _joe_: there are a bunch of appserver warnings about etcd last index [07:09:11] a handful in both eqiad and codfw [07:09:21] I pushed a dbctl change a few minutes ago [07:09:22] <_joe_> cdanis: not completley uncommon, I'm in the middle of doing something [07:09:23] Maybe that? [07:09:26] <_joe_> that yes [07:09:26] ah okay [07:09:29] sure sure [07:09:35] wait [07:09:44] these have lasted 22 minutes [07:09:47] is the recheck interval long? [07:09:54] <_joe_> cdanis: there is some defect in the check, yes [07:10:01] <_joe_> etcd last index (475598) is newer than the master one (475296) [07:10:05] The last check was done at 06:45 [07:10:10] <_joe_> which if anything means we're in better shape [07:10:18] <_joe_> than what icinga thinks [07:10:21] yeah ok [07:10:29] We have observed that sometimes, indeed (that warning I mean) [07:11:52] recheck has solved it [07:35:16] <_joe_> so when what mediawiki is readin is newer than what icinga knows, we shouldn't even have a warning IMHO [07:35:28] <_joe_> blame a certain SRE with extreme ocd for it [07:35:46] 🤔 [09:32:50] cdanis: was referring to me ;) [09:34:14] the update of the base IDs on the icinga host to compare with is done every 30s [09:35:01] and the checks are every minute with retry every minute [09:35:08] so if that lasted longer should be investigated [11:43:20] out of curiosity, what will theemin host be for? that is a name I have never heard before [11:44:11] for testing partman recipes etc. [11:44:23] on baremetal [11:44:39] we should call it partman101 (not a typo) :-P [11:45:14] elukey this is for you ^^ :-P [13:28:40] I want to make sure scap gets ported to Python3 (eventually, but not while I'm on train duty), and to help with that: what version of Python3 can I assume is available everwhere scap needs to run? [13:30:05] (https://phabricator.wikimedia.org/T246025 is the task I have for this) [13:35:51] right now we still have scap deployed on some jessie hosts (so Python 3.4), but those will vanish in 3-6 months, so depending on when you'll do the port that or 3.5 (mw* is stretch for quite a while) [13:36:30] check, thanks; I'll aim at Python 3.4 for now then [13:37:50] volans: Moritz and Filippo removed all the complexity so no more fun with partman! :D [13:38:27] ehehe [14:18:42] volans: just one nit, 🤔 is more often than not being used ironically ;) [14:19:19] hard to tell for me ;) [23:31:58] has anyone seen puppet-master logging *everything* to /var/log/debug before? [23:34:51] and also /var/log/syslog and /var/log/daemon.log [23:35:23] not debug, but that stuff is both in daemon.log and syslog, yes, i have seen that [23:40:04] what kind of configuration can cause that to happen mutante ? [23:47:26] Aha! [23:47:35] /usr/share/puppet/rack/puppet-master/config.ru [23:47:35] some rsyslog::conf snippet? [23:47:43] Someone has uncommented the debug line [23:47:55] i was going to guess j.bond was testing puppetmaster upgrade or so [23:48:09] Hey I'm not saying it wasn't me :P [23:48:14] ah :)