[11:01:25] godog: you around? [11:03:46] went to lunch 4 minutes ago arturo [11:03:57] ack. no problem [11:04:13] (just saw the other channel) [11:54:56] arturo: yup I'm back now [11:55:04] godog: nevermind :-P [12:00:13] lol [13:42:08] any prometheus/grafan experts around? [13:42:56] jbond42: I think you well get more succesful by asking for someone that may help with those :-D [13:43:04] *will *Success [13:43:22] *any one who can help with prometheus/grafan :) [13:43:42] it seems https://grafana.wikimedia.org/d/CVfSPeqiz/iptables is no longer updating [13:44:10] I'm taking a look, but I have an interview in 15 [13:44:13] i see the mtail metrics from http://localhost:3903/metrics on lithium and im not sure how to further debug [13:44:38] ack thanks godog, not sure if the other mtail metrics are affected as i can find any graphs with them :) [13:47:08] data stops around when logrotate runs, my bet is mtail didn't get the memo [13:47:33] "stops" as in, goes flat lien [13:48:32] simlar things happened yesterday [13:48:53] perhaps we should add a postrotate to restart mtail? [13:51:36] so if other attempts fail, yes definitely [13:52:05] it should be relatively easy to test an upgrade to busters' mtail version [13:52:40] last famous words heh [13:53:21] ack i can create a package/component for that [13:55:11] jbond42: if you have time that'd be awesome, if not the postrotate will do too [13:56:00] godog: no probs [13:56:15] does mtail export any meta-metrics about number of lines read, or similar? [13:58:36] it dosn't look like it [15:28:51] https://github.com/numirias/security/blob/master/doc/2019-06-04_ace-vim-neovim.md [15:31:32] you can't even trust curl | vim - these days [15:31:43] XD [15:32:01] soon we won't be able to trust curl | sudo bash [15:33:50] hehe [15:39:22] vim as packaged in Debian disables modelines by default in /usr/share/vim/vim80/debian.vim [15:39:55] ah, TIL [15:40:31] "soon" [15:41:56] docker run vim would give you plenty of opportunities [15:41:58] :P [15:51:57] godog: are you fine for me to install mtail_3.0.0~rc19-2.1+deb9u1_amd64.deb on wezen? [15:54:54] jbond42: +1 [15:55:09] cool, ill do it after the meeting just in case [15:55:35] kk, thanks jbond42 for taking a look! btw a stretch (or buster?) upgrade of centrallog hosts is also imminent [15:56:12] oh i hadn't noticed it was jessie will need to rebuild the package [15:56:35] but that fine [18:58:05] jijiki (if around), do you know if it will cause trouble for me to move deployment-imagescaler03? It would be down for maybe 30 minutes. [22:13:43] https://gerrit.wikimedia.org is timing out [22:13:50] gerrit seems down [22:13:55] yes [22:13:56] What bd808 said. [22:13:59] cobalt is ok [22:14:04] like it's running at least [22:14:22] chaomodus: Restart gerrit? `sudo service gerrit restart`? [22:14:44] java.lang.OutOfMemoryError: Java heap space [22:14:46] oic [22:15:41] Didn't we just bump the heap space a bunch? Or was that unrelated? [22:15:47] java's favorite thing to do, OOM [22:15:55] restarting in progress [22:22:59] chaomodus: Looks like it's back. Thanks! [22:23:07] Sure thing :)