[15:37:48] <K4-713>	 the-wub: What up? Everything going okay?
[15:38:31] <the-wub>	 yep, all fine thanks K4-713. tests are done for the day
[15:38:54] <K4-713>	 Eeeexcellent. Nothing seems to have fallen over from my end. :)
[15:39:17] <the-wub>	 that's good to know :)
[15:39:32] <K4-713>	 In that case: I'm going to suspend and drink some coffee. I'm out today.
[15:39:47] <K4-713>	 Just wanted to make sure, as most of the exciting things happened while I was driving this morning.
[15:40:18] * K4-713 waves
[15:41:19] <the-wub>	 ah, have a nice day :)
[16:58:58] <atgo>	 ccogdill are you in office today?
[16:59:11] <ccogdill>	 I’m not atgo. I relapsed :(
[16:59:13] <ccogdill>	 so wfh for now
[16:59:20] <atgo>	 ok! i'll give up the roon
[16:59:25] <ccogdill>	 mmkay
[17:13:25] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] Synchronous logging to stdout [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/141751 (owner: Mwalker)
[17:16:32] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] "Or u could change the default" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/141753 (owner: Mwalker)
[17:16:51] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] it helps to page on a column that exists... [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/141756 (owner: Mwalker)
[17:27:00] <mwalker>	 awight, I'm going to deploy those ^
[17:27:21] <grrrit-wm>	 (CR) Awight: [C: -1] "nitpicks, inline" (3 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/141826 (https://bugzilla.wikimedia.org/56657) (owner: Ejegg)
[17:27:56] <awight>	 mwalker: ok!
[17:28:11] <grrrit-wm>	 (PS1) Mwalker: Merge remote-tracking branch 'gerrit/master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/142006
[17:28:31] <grrrit-wm>	 (CR) Mwalker: [C: 2] Merge remote-tracking branch 'gerrit/master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/142006 (owner: Mwalker)
[17:29:40] <mwalker>	 !log updating fundraising tools from 5f3a7316b636c0723ce3fa353186d4041b662872 to cdc4b73bd59d27c8d386b6df629b1c574cfed85f
[17:29:45] <morebots>	 Logged the message, Master
[17:30:10] <grrrit-wm>	 (CR) Awight: Show friendlier error message for zero donation amount (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/141826 (https://bugzilla.wikimedia.org/56657) (owner: Ejegg)
[17:31:22] <atgo>	 awight is there a card(s) for the civi case work?
[17:33:11] <awight>	 atgo: funny that you mention that
[17:33:21] <awight>	 atgo: I was about to start a Trello board.  n'est-ce pas?
[17:34:12] <atgo>	 yeah
[17:34:20] <atgo>	 i wish one tool worked for everything. some day
[17:35:09] <mwalker>	 down that path lies Lotus Notes
[17:35:19] <mwalker>	 I can assure you... you dont want to go down that road
[17:35:24] <atgo>	 ooohh no
[17:35:27] <atgo>	 lotus notes is awufl
[17:35:58] <mwalker>	 unspeakably so; it does everything... badly
[17:36:25] <awight>	 mwalker: aww, you sound like you've never used... or administered... Outlook.
[17:36:36] <awight>	 lotus notes--
[17:36:38] <mwalker>	 I've done both; I love outlook
[17:36:42] <awight>	 omg
[17:36:49] <awight>	 you have been trolled.
[17:37:02] <mwalker>	 hehe; nah; it does everything I want in an email client
[17:37:15] <awight>	 yeah I love that the web client has no ability to star or otherwise mark a message
[17:37:33] <awight>	 and the contacts thing is awesome, how you can't just start typing a name and have it autocomplete
[17:37:41] <mwalker>	 you can tag in the web interface; same as you can in outlook itself
[17:37:53] <mwalker>	 your exchange server doesn't have LDAP set up in that case
[17:37:58] <mwalker>	 or you're using the wrong default contact book
[17:38:03] <awight>	 mwalker: wanna see?
[17:51:04] <awight>	 Jeff_Green: can I ask... Do you remember where we left the trigger privilege vs binary logging?  mwalker says we are using mixed logging, which might give us the granular permissions on trigger stuff?
[17:51:25] <Jeff_Green>	 i added comments to the RT ticket and that's the last I heard
[17:51:38] <awight>	 Erm I'll go look.  I just read that we would need to enable "trust-function-creators"
[17:51:54] <Jeff_Green>	 basically, it seemed feasible to use that ^^^ if a wee bit risky
[17:52:06] <Jeff_Green>	 main risk being that it's concievable to break replication
[17:52:14] <awight>	 yeah no joke.
[17:52:20] <awight>	 Jeff_Green: sorry, I cannot search for the ticket
[17:52:26] <Jeff_Green>	 orly? looking
[17:52:37] <awight>	 aha
[17:52:48] <awight>	 Jeff_Green: #7561
[17:52:53] <Jeff_Green>	 i closed it right?
[17:53:31] <awight>	 yep.  Well... there is a big reason coming down the tube, to enable this.
[17:53:48] <awight>	 we need to upgrade Civi again, and they're becoming more dependent on triggers.
[17:54:12] <awight>	 unfortunately, they do not manage triggers in a sane way however.  It's all on-the-fly.
[17:56:15] <Jeff_Green>	 yeah, that's just horrible
[17:56:36] <Jeff_Green>	 pretty well thwarts QA
[17:57:11] <Jeff_Green>	 I'm in the mysql puppet module anyway, will add the config knob
[18:02:19] <grrrit-wm>	 (CR) Ejegg: "Justified the regex zero check" (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/141826 (https://bugzilla.wikimedia.org/56657) (owner: Ejegg)
[18:03:04] <grrrit-wm>	 (PS3) Ejegg: Show friendlier error message for zero donation amount [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/141826 (https://bugzilla.wikimedia.org/56657)
[18:04:13] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] Show friendlier error message for zero donation amount [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/141826 (https://bugzilla.wikimedia.org/56657) (owner: Ejegg)
[18:05:06] <awight>	 mwalker: you... want to review https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/1637
[18:05:16] <mwalker>	 sure
[18:05:39] <mwalker>	 well; this isn't a huge patch at all
[18:05:40] <awight>	 I will get the fake fruit squares if that helps
[18:05:47] * awight rrrghs
[18:05:52] <mwalker>	 heh
[18:05:58] <mwalker>	 we're doing lunch after this
[18:06:02] <mwalker>	 because I will be hungry
[18:06:04] <awight>	 ok deal
[18:06:11] <awight>	 or you may lose yr appetite
[18:07:12] <awight>	 OH wow.  I've been wondering about this for a while: you can add entire groups to CR in gerrit
[18:07:19] * awight goes to make a useful one for fr-tech
[18:35:07] <awight>	 atgo: so we never finished that conversation about CiviCase
[18:35:14] <awight>	 Can I go ahead and make a Trello board?
[18:35:28] <awight>	 There is a lot of stuff for MG to do and we are not about to put them into Mingle, so...
[18:35:37] <atgo>	 yeah that sounds like a great idea
[18:35:55] <awight>	 hehe fox watching the turkey hutch, I see
[18:36:58] <awight>	 atgo: https://trello.com/b/HmPIQDhc/civi-major-gifts
[18:38:16] <grrrit-wm>	 (PS1) Mwalker: Create working path if it doesn't exist [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/142028
[19:01:04] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] Create working path if it doesn't exist (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/142028 (owner: Mwalker)
[19:01:54] <mwalker>	 awight, the exports are large; ideally we would keep them somewhere -- but I don't know if jeff wants to use logrotate or something on them
[19:02:05] <mwalker>	 I could also work directly in an archive folder
[19:03:23] <awight>	 gzip will give us huge savings there...
[19:03:48] <mwalker>	 aye
[19:03:48] <awight>	 mwalker: probably working in an archive folder is not ideal, cos the mirror scripts do not delete temporary files
[19:03:56] <awight>	 maybe you don't have those?
[19:04:06] <mwalker>	 mirror scripts?
[19:04:14] <mwalker>	 *shrugs* I really want logrotate; it'll gzip them for me
[19:06:52] <awight>	 Jeff_Green: ^^
[19:07:01] <awight>	 fairy dogmother
[19:37:19] <Jeff_Green>	 logrotate good!
[19:37:45] <Jeff_Green>	 i just finished overhauling the logrotate crap in puppet so it should be easy (unless this turns out to be a corner case that breaks everything I assumed)
[20:55:12] <grrrit-wm>	 (CR) Mwalker: [C: 1] "Interesting. This looks pretty solid; I don't have any major blockers, I think you've documented them all for future TODOs in this branch." (21 comments) [wikimedia/fundraising/crm] (dedupe) - https://gerrit.wikimedia.org/r/134998 (owner: Awight)
[21:08:25] <grrrit-wm>	 (PS1) Mwalker: Merge remote-tracking branch 'origin/master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/142134
[21:08:39] <grrrit-wm>	 (CR) Mwalker: [C: 2 V: 2] Merge remote-tracking branch 'origin/master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/142134 (owner: Mwalker)
[21:13:01] <awight>	 Jeff_Green: wrt slow query diagnostics...
[21:22:11] <awight>	 Jeff_Green: fwiw it's very odd but the slow query log has lots of queries << 0.5sec
[21:32:27] <Jeff_Green>	 it's set at 0.47 iirc
[21:32:36] <Jeff_Green>	 i have no idea how we arrived at that number
[21:33:25] <awight>	 yes but a lot of shorter queries are showing up...
[21:33:59] <Jeff_Green>	 dunno
[21:34:11] <Jeff_Green>	 check_fundraising_jobs is whining
[21:36:00] <greg-g>	 17:35 <+icinga-wm> PROBLEM - check_fundraising_jobs on db1025 is CRITICAL: CRITICAL
[21:36:04] <greg-g>	 oh, you just said that
[21:36:07] <greg-g>	 :)
[21:36:17] <Jeff_Green>	 missing_thank_yous=786
[21:36:35] <Jeff_Green>	 we're not being thankful enough
[21:38:51] <awight>	 Jeff_Green: can you add the civi hostnames to their own /etc/hosts, or is that a security thing?
[21:39:35] <Jeff_Green>	 ?
[21:40:06] <awight>	 I realized there are some features failing because the backend connects to itself :-/
[21:40:12] <Jeff_Green>	 ah
[21:41:54] <Jeff_Green>	 imo the fix should be with civi, not DNS
[21:41:57] <awight>	 argh
[21:42:19] <awight>	 yes it's a horrible thing to do, but not something I'm going to be able to upstream
[21:42:28] <Jeff_Green>	 putting it in /etc/hosts affects other things
[21:42:47] <awight>	 okay.
[21:43:02] <Jeff_Green>	 anybody putting civi behind a proxy must run into this
[21:50:20] <Jeff_Green>	 i wonder if we should be configuring it differently somehow
[21:51:29] <awight>	 I'm reading about it...  Seems like a lot of people have the same symptoms
[21:51:42] <Jeff_Green>	 what if we set $base_url to localhost?
[21:51:55] <awight>	 that will kill the UI
[21:52:05] <Jeff_Green>	 you say that like it's a bad thing...
[21:52:13] <awight>	 I had the same thought :p
[21:53:11] <Jeff_Green>	 another option would be some horrible iptables routing
[21:53:23] <Jeff_Green>	 maybe...I'm not sure that would work.
[21:54:09] <awight>	 Jeff_Green: can you (sorry) look at waiting sockets for me?
[21:54:10] <Jeff_Green>	 what port does it try to hit when it does calls to itself?
[21:54:17] <Jeff_Green>	 sure. where?
[21:54:27] <awight>	 lut... one moment...
[21:54:42] <awight>	 Jeff_Green: okay it should fail now, for about a minute.
[21:55:19] <Jeff_Green>	 see lutetium:/tmp/netstat
[21:56:45] <Jeff_Green>	 awight: I've gotta head out. dinner time!
[21:56:57] <awight>	 Jeff_Green: ok thx
[21:57:06] <Jeff_Green>	 i'm supposed to be PTO tomorrow/fri but I'll probably be around a bit in the AM
[21:57:18] <awight>	 Jeff_Green: btw, the slow query log totally caught my perp!
[21:57:23] <Jeff_Green>	 oh good!
[21:57:42] <Jeff_Green>	 i can't believe we've gone this long with out dev access to slow log on lutetium
[21:57:46] <Jeff_Green>	 :-(
[23:35:39] <grrrit-wm>	 (PS1) Awight: knock out a 24-second delay in loading a contact page [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/142156
[23:36:05] <awight>	 ejegg_: mwalker: I would love to deploy that today ^^.  CR?
[23:38:11] <grrrit-wm>	 (CR) Mwalker: [C: 2 V: 2] knock out a 24-second delay in loading a contact page [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/142156 (owner: Awight)
[23:38:15] <mwalker>	 awight, nice
[23:38:25] <awight>	 thx!
[23:39:05] <grrrit-wm>	 (PS1) Awight: update civicrm submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/142158
[23:39:16] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] update civicrm submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/142158 (owner: Awight)
[23:39:25] <mwalker>	 awight, the union is a bit sketchy; but hopefully the upstream keeps the two table definitions aligned
[23:40:03] <awight>	 mwalker: what is sketchy?  the union is only performed on the one column in each select
[23:40:26] <awight>	 the table schemas are already totally different...
[23:41:20] <mwalker>	 oh right; I slightly misread this
[23:41:22] <mwalker>	 but its still good
[23:42:46] <grrrit-wm>	 (PS1) Awight: update civicrm submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/142160
[23:42:56] <grrrit-wm>	 (CR) Awight: [C: 2 V: 2] update civicrm submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/142160 (owner: Awight)
[23:43:28] <awight>	 !log updated crm from f3389daa94e9ad924175bdf0d5bc09c4a26aeb8c to e2fe03a9cd51e30206d9a1114d62dfbd6960816b
[23:43:32] <morebots>	 Logged the message, Master
[23:48:45] <awight>	 mwalker: hah.  upstream already had the same fix.
[23:49:09] <mwalker>	 *facepalm*
[23:49:13] <mwalker>	 apparently we need to update
[23:49:22] <awight>	 ooh yeah