[06:01:32] hmm it looks like compiler1003 lacks some puppet facts... [08:22:51] BTW, any "easy" way of checking the uptime of a TCP connection? [08:22:56] from the SO point of view [08:23:01] s/SO/OS/ [08:45:22] vgutierrez: is it a matter of running https://wikitech.wikimedia.org/wiki/Nova_Resource:Puppet-diffs/Documentation#How_to_update_the_compiler's_facts?_(e.g._INFO:_Unable_to_find_facts_for_host_conf2001.codfw.wmnet,_skipping) ? [08:45:32] (about the puppet facts) [08:47:50] I was about to suggest that [08:48:09] also I think there was some work ongoing regarding the pupet compiler, may had some effect [08:49:15] yeah there is an email "New PCC hosts compiler1003" [09:35:41] vgutierrez: you first need to find the PID and FD (for instance with lsof -iTCP) and then stat /proc/$pid/fd/$fd [09:35:51] yeah [09:35:55] I've been doing that [09:36:04] but maybe I was missing some ss flag or something like that :) [09:36:21] vgutierrez: could you send me the pcc that failed i saw a simlar issue yesterday but thouoght i fixed it https://phabricator.wikimedia.org/T236717 [09:36:35] vgutierrez: yeah I also thought ss would show this info, but couldn't find it in the man page [09:36:59] jbond42: so it was complaining about the facts of cp5001 & cp5007 [09:38:57] seems to work from the cli, do you still have the link to the jenkins job [09:39:13] jbond42: let me check my terminal buffer... [09:39:25] jbond42: https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/19115 [09:39:41] cheers [09:53:22] shold be working now vgutierrez was an error with permissions [09:59:52] jbond42: thx <3 [10:00:24] np [11:21:40] effie: so in short, the CI job that generates mediawiki code coverage uses PHPUnit which relies on php-xdebug to track what part of the code is run [11:21:43] and thus build the coverage [11:22:06] since I have moved the job from jessie to stretch VMCS instances it times out (runtime went from 2hours to timing out at 5hours) [11:22:28] so I have dig into it, there are some unknowns but one thing I found is that php-xdebug keeps invoking the system call getpid() [11:23:03] and perf says that is a good share of the running time (5% if I got it right) [11:23:36] the fix is in php-xdebug 2.7.1 but but we build from upstream debian packaging repository which is still at 2.7.0 and thus lack the fix ;] [11:24:09] right now I see we have 2.5.0-1 installed [11:24:14] from debmonitor [11:24:50] debmonitor is for production, that is unrelated [11:24:57] and I have send puppet patches to get rid of it [11:25:08] the CI jobs use docker containers that install php-xdebug from component/php7.2 [11:26:23] (and those releases* hosts should be switch to php7.2 but that is a different story) [11:28:16] ok I think we sould add on the task desc as well that this is for the CI Docker containers running php7.2 [11:28:28] volans|off: cdanis: these are indeed the calico kubernetes CNI plugin interface names. They are very ephemeral so it probably makes sense to not populate them as facts [11:29:27] hashar: I will aim for this week, but today is highly unlikely [11:29:51] effie: ok ok :] [11:29:55] and the best would be xdebug 2.7.1 yes? [11:30:05] 2.7.1 has the fix at least :] [11:30:44] ok good [11:30:56] but I guess we can benefit from 2.7.2 as well. I will add the changelog to the task [11:31:25] thank you [11:33:33] thx :) [12:32:16] as announced yesterday on sre meeting, now alex and me are running the backup migration [12:32:32] new backups will not run right now [14:26:43] anyone have advice on testing / running manually custom facter definitions? [14:28:36] see e.g. modules/wmflib/lib/facter/kernel_details.rb, that's probably quite similar to what you need for the firmware tracking [14:29:30] thanks! but is there an easy way to run a modified definition and see what it would output? [14:31:20] cdanis: you can place a the ruby file in /var/lib/puppet/lib/facter/ and run `sudo facter -p your_fact` [14:33:50] thanks! [14:40:04] I hope people are keeping track of the Machine Vision extension work going on, because that could mean a lot more "depicts" statements on commons (= a lot more revisions) in our future [14:40:18] especially but not only serviceops folks [14:51:59] jbond42: apologies in advance for this code, I don't really know the language [15:00:03] no probs :) [15:48:50] hi all, just a heads up i have currrent;ly switch puppetboard2001 to use the new puppetdb's. further both puppetmaster1003 and icinga2001 are using the new buster puppetdb [19:56:34] hey mutante [19:56:36] https://phabricator.wikimedia.org/T236829 [19:56:46] are there any checks before I create a mailing list? [19:56:49] or do we just do it if someone asks? [20:24:59] ottomata: we just do it if it sounds reasonable (wikimedia related in some way) [20:25:26] k sounds fine to me will do [20:25:46] kind of like what would get a labs instance. yea, sounds fine to me too [20:27:50] ottomata: thanks for merging the new deployment group [20:28:35] looking at that follow-up now [20:29:34] oh yea, we need to create new keys and add them to keyholder, i'll get into that [21:42:32] thanks mutante [21:42:38] out for the day will follow upa nd see where we are tomorrow [21:47:33] yep, created new deployment keypair. added in private repo. and fake keys in labs/private. Mukunda's change compiles now. +1ed for now [22:59:10] another jessie machine (and public IP) removed - RT is now on buster