[00:01:35] puppet debug [00:02:00] uh, wrong window [00:58:55] bd808: how do you keep a deploy server up to date? [00:59:09] it does not seem to make any git pulls on its own [01:00:03] up to date with which parts? the trebuchet repos? [01:00:38] They purposefully have to be updated on the deploy server by a human [01:00:56] the puppet repos should update automagically [01:01:53] sadly I've never found a way to reliably automate trebuchet even for the beta cluster [01:11:19] bd808: that makes sense I guess [01:11:31] so how do you keep the minions up to date? :) [01:12:06] they sync with whatever the latest git-deploy on the master made even if they are created after [01:12:16] puppet tells salt to do that [01:12:29] does not seem to be happening [01:12:42] it should be part of the puppet run, right? [01:13:37] It should be, yes. [01:14:29] have you signed the salt keys for the minions on your salt master? [01:14:56] if the puppet run keeps saying that it is starting the minion that is often the problem [01:15:25] I don't think you need to do that [01:15:44] it gets autosigned when puppet is run on the minion [01:16:05] anyway, I could install sentry/sentry, I just can't update it [01:16:32] weird. want me to log in and poke around? [01:17:07] let me try some debug logging first [01:17:38] git deploy report sync --detailed might help [01:17:56] that will dump the state of a repo from the redis server [01:18:42] trebuchet is ridiculously complicated for the way we use it [01:18:58] lots of features that generally aren't needed [01:21:38] now that we have basically officially decided that nobody will maintain it we should probably start replacing it with simpler things like git::clone in most of the places we are using it [01:23:42] is it ok to just use git::clone in a role? [01:23:52] that would certainly make my life easier [01:24:24] It should be fine, yeah. I think there are several that do that [01:25:59] tgr: https://github.com/wikimedia/operations-puppet/blob/76a5132487a79456a2618f8047f1a12e88d14efa/manifests/role/performance.pp#L17-L23 [01:26:40] I should do that with kibana acutally [01:28:37] the lame thing about using git::clone is that is makes testing new changes harder [01:29:09] why? [01:29:59] because as soon as you merge to master the change shows up everwhere [01:30:06] *everywhere [01:30:23] well ~20 minutes later it does anyway [01:30:27] not if you use ensure=>present [01:30:48] or specify a tag which seems to be the nice way to go about it [01:31:24] yeah you could do it with a tag that you bump manually [01:32:04] just ensure=>present doesn't scale very well to larger clusters [01:32:18] since you then have to run around and update manually [01:32:57] most of the things we sling around with trebuchet today are on pretty small clusters though [01:35:53] git::install + version tags seems like a good way for both small and large clusters [01:38:26] except that you can't push tags to Gerrit, or so it seems [01:40:13] a branch will probably do [01:45:41] which is also not permitted [01:45:43] hm [01:46:31] I guess if you can just create/change tags in an ops repo that's a security hole, and gerrit does not seem to have a review process for refs changes [01:46:43] I could just use commit ids, but ew [01:48:04] I'll just go with ensure=>latest and hope someone figures it out by the time it's actually needed for Sentry [01:53:40] it's easy enough to add permissions to gerrit to allow tags and branches for a given repo [01:53:51] mediawiki/core has both [04:03:22] Fwiw I used git::install in this way to track a tag with puppet and a mechanism for gradual deploy w/ a lock file [04:04:28] With phabricator that is. Dunno if I should have done that but it's very straight forward and easy and I had easy and first class citizen labs testing in mind [04:04:37] And I was ignorant [17:40:35] bd808: ... https://en.wikipedia.org/wiki/Inemuri [17:40:54] lol [17:42:07] o.O [17:42:21] anomie: there is still time for you to put in to speak at SeaGL -- http://seagl.org/news/2015/07/28/CFP-extended.html ;) [17:43:03] No thanks [18:27:26] bd808: what owns /srv/org/wikimedia (in https://github.com/wikimedia/operations-puppet/blob/76a5132487a79456a2618f8047f1a12e88d14efa/manifests/role/performance.pp#L17-L23)? [18:54:41] tgr: crazily it seems to come from this -- https://github.com/wikimedia/operations-puppet/blob/1bdecc141603a88b1573e10b2f755f3c7d3f1559/modules/bugzilla_static/manifests/init.pp [18:55:23] I would guess it was built by hand at some point [18:56:24] here's a dup declare for it -- https://github.com/wikimedia/operations-puppet/blob/acacf97e2df962fef83487a461f3559fa07e4d6f/modules/releases/manifests/init.pp [18:56:47] and another -- https://github.com/wikimedia/operations-puppet/blob/acacf97e2df962fef83487a461f3559fa07e4d6f/modules/contint/manifests/website.pp [18:57:01] hey! I remember there was a tool that allows to see on the wiki page which part of the text belongs to which revision. Anybody remembers what that tool is? [18:57:11] and another -- https://github.com/wikimedia/operations-puppet/blob/61841b150b8e37cbefceb5ae0983d259a4a9082b/modules/publichtml/manifests/init.pp [18:57:44] SMalyshev: maybe http://wikipedia.ramselehof.de/wikiblame.php ? [18:59:13] bd808: thanks, will check out! [19:35:04] ookay [19:35:19] I'll stay away from that directory then [19:35:58] SMalyshev: wikitrust, but it's very complicated to set up and has been unmaintained for years [19:39:44] tgr: thanks! I remember seeing something that worked... but can't remember what it was. From the looks of wikitrust, probably a different one [19:43:09] SMalyshev: https://de.wikipedia.org/wiki/Benutzer:Schnark/js/artikel-statistik ? [19:43:22] SMalyshev: TimStarling is interested in the general topic of wikiblame and may have other good links [19:43:44] tgr: that looks more like it, thanks, I'll check it out [19:46:33] https://de.wikipedia.org/wiki/Benutzer:Atlasowa/edit_history_visualization has a nice collection of blame-like tools [19:53:00] SMalyshev: I typically use http://wikiwho.net/ which is forked from halfak's stuff [19:53:15] there's a GUI for it somewhere too... [19:54:43] legoktm: thanks but that one seems to give me 404 now... [20:01:48] :/ [20:40:21] if ( $dbr->tableExists( 'page' ) ) { [20:40:27] legoktm: man spamblacklist is old [20:40:38] yeaaaaaaaah :( [20:40:57] it won't work on MW that old anyway for dozen other reasons [20:43:57] and that selectDB() code actually makes me laugh [20:44:34] AaronSchulz: sometime can you take a look at https://gerrit.wikimedia.org/r/#/c/227491/ (adding a job for usage tracking) [20:44:56] * aude wants this in before we further deploy usage tracking to more wikis [20:53:42] haha, and $revision->getText(); would use the wrong table [20:57:43] thanks AaronSchulz [20:59:30] bd808: I messed up my ssh config for labs. While I debug that, could you possibly run 'apt-get update ; apt-get install nutcracker' on the two mediawikis on the beta cluster? [21:00:02] ori: yeah I can do that. [21:00:21] thanks sir [21:06:02] ori: {{done}} [21:07:39] csteipp: do you think you'll have time to do https://phabricator.wikimedia.org/T99086 soon? it's ~600 lines of php now [21:08:09] legoktm: Yeah... maybe Thursday? [21:08:30] that would be awesome :) [21:47:06] bd808: thanks again [22:21:13] tgr: I abandoned my wfTrack() patch so I guess you can abandon the usage patches you made now [22:25:00] tgr: is the statsd upgrade patch ready to be merged? [22:25:25] legoktm: yes. I was just waiting for the branch cut [22:25:41] ok, I'll leave it to you then :) [22:27:40] legoktm: what's the order for that? Force mw/vendor and then let jenkins do the right thing for the core patch? [22:27:47] bd808: done [22:27:48] yep