[00:15:48] (PS8) Katie Horn: Generic audit logic and worldpay audit job [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/159761 [00:15:54] awight: ^^ [00:16:03] where is the... [00:16:07] Note the part where it no longer says "don't do it". [00:16:09] * awight looks for loose WIPs [00:16:18] whhhpshhh! [00:16:25] ok then [00:16:36] There are exactly 3 beers left. [00:16:44] Only 2001 lines. [00:16:48] :p [00:16:54] that gives me just enough time to get away [00:16:55] * K4-713 waits 10 minutes [00:16:57] bahaha [00:59:00] (PS1) Awight: WIP autoload test classes [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/163787 [02:14:45] (CR) Awight: [C: 2] "I really like how it's broken down into small functions, and" (27 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/159761 (owner: Katie Horn) [03:43:15] (CR) Ejegg: "Just a couple more possible concerns noted inline-ish." (2 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/139452 (owner: Awight) [03:53:26] (CR) Ejegg: [C: 2 V: 2] add PHPMailer [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/163615 (owner: Awight) [04:04:51] (CR) Ejegg: [C: 2] Use Composer-managed PHPMailer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163635 (owner: Awight) [04:14:24] (CR) Ejegg: "Looks good - just one phpdoc comment in queue2civicrm.module (line 231) referring to the old Stomp_Frame class you might want to update." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163637 (owner: Awight) [04:22:03] (CR) Ejegg: [C: 2] Differentiate rapidhtml failure cases [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/95874 (owner: Awight) [04:24:45] (CR) Ejegg: [C: 2] update civicrm submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/158540 (owner: Awight) [04:30:44] (PS2) Ejegg: clean up variable usage after Mailer API change [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/158542 (owner: Awight) [04:31:10] (CR) Ejegg: [C: 2] clean up variable usage after Mailer API change [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/158542 (owner: Awight) [14:27:35] (PS1) Ejegg: Fix 'Edit' link for custom data tab [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/163856 [16:33:42] anyone else having trouble accessing Filesrv1 [16:41:12] SVentura_: yeah, I'm having trouble too [16:42:44] Trouble at the mill? [16:42:55] ...don't mind me. [16:43:17] the ol' filesrv up and quit! [16:43:36] ejegg: Consarnit! [16:43:49] (Now I don't know how to stop) [16:44:05] Huh. I can still get in. [16:44:11] But... this happened yesterday, too. [16:44:15] awight couldn't get there. [16:44:34] ejegg: I had a question for you, though. [16:44:51] ah, and i have one for you! [16:44:54] you ask first [16:45:02] Yay, question trade. [16:45:08] So, this composer stuff in civi. [16:45:15] yah? [16:45:20] If I deploy that now, do I need to change settings anywhere? [16:45:41] i think just drush cc all [16:45:48] oh, right. [16:45:50] That thing. [16:45:59] whee, registries! [16:46:14] Do you think I should stop the mail-related jobs when I do that? [16:46:24] ok. thanks! there were 2 of us trying to get in ... wasn't sure if it was low-cafeine or server [16:46:42] if you're planning to deploy crm code, there's a trivial template fix for that 'can't edit prospect' card [16:47:05] I was just trying to figure out if I can just take everything in master. [16:47:08] K4- I stopped all the jobs the one time I deployed crm [16:47:41] Ha. That's usually not so necessary, unless we're messing with certain things. [16:47:55] But, it shouldn't ever hurt anything. [16:48:02] Link fix: https://gerrit.wikimedia.org/r/163856 [16:48:31] O_O [16:49:20] (CR) Katie Horn: [C: 2] "Argh!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/163856 (owner: Ejegg) [16:50:16] ejegg: When was the last time you did a civi deploy? [16:51:41] Oh, a few weeks ago [16:51:49] Want to do one now? [16:52:03] I'd say stop the mail jobs since we're updating to use the PHPMailer from composer [16:52:05] I just remembered I also have to deploy something we probably haven't touched in... half a year or something. [16:52:09] I can handle the jobs. [16:52:15] Um, sure. What's the other deploy? [16:52:28] And actually, I'd like to stop most of them, because we might lose failmail, too. [16:52:36] ooh, good point [16:52:37] The other deploy, is fr-tools. [16:53:03] oh, exciting. Audit! [16:53:27] Aye. [16:53:43] Oh, I was going to poke GC again. [16:53:59] OK, let me get set up for a crm deploy [16:54:00] Gah. [16:54:32] after that i'll quiz you on just how many places we need to recalculate non-USD donations since 6/20 [16:54:41] And, suddenly, the file download job works again. [16:54:44] I changed nothing. [16:54:51] Their server took a walk last night or something. [16:54:52] ehh? [16:55:02] Oh, those 3 fails [16:55:11] Yeah. Two were me running the job again. [16:57:23] (PS1) Ejegg: Submodule bump for civicrm link fix [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163876 [16:57:50] (CR) Ejegg: [C: 2] Submodule bump for civicrm link fix [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163876 (owner: Ejegg) [16:58:37] gah [16:59:03] Tools repo is being... kind of a tool. [16:59:07] :| [16:59:11] heh [17:01:02] Wait, I'm an idiot. Smashpig. [17:01:04] Not tools. [17:02:30] Which I guess we deploy from master. [17:02:39] * K4-713 blinks several times [17:07:18] so, how exactly does one merge into deployment and get that nice message listing all the merged commits? [17:08:56] Is that the thing that we tried to to last time that would have worked if we didn't cherry-pick like... farmers? [17:09:17] I think we're good on merges for crm [17:09:41] I just see awight's last merge message had a nice list of merged commits [17:10:09] I'm not entirely sure what he does. I'm usually the jerk cherry-picking and screwing it up for everybody else. [17:13:42] (PS1) Ejegg: Merge remote-tracking branch 'origin/master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163877 [17:13:48] ejegg: Let me know before you try to deploy anything. I'm waiting for this to empty out: http://ganglia.wikimedia.org/latest/graph.php?r=hour&z=xlarge&c=Fundraising+eqiad&h=silicon.frack.eqiad.wmnet&jr=&js=&v=10&m=ActiveMQ+QueueSize+donations&vl=Messages [17:14:04] OK, will definitely let you know [17:14:37] Man, we're going to stack up some messages. [17:14:50] (CR) Ejegg: [C: 2] Merge remote-tracking branch 'origin/master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163877 (owner: Ejegg) [17:15:27] ok, heading up to the deploy server to get stuff ready to sync [17:16:06] Turning off qc jobs! [17:18:01] ejegg: You're clear. [17:18:22] ok. just realized I should try it on staging first! [17:18:26] har [17:18:34] I *suppose*... [17:18:36] :p [17:19:10] !log disabled qc jobs and TY mail send for pending civi deploy [17:19:15] Logged the message, Master [17:21:10] ejegg: I wonder if I would confuse the issue if I tried to deploy smashpig at the same time... [17:21:14] aww dang, I seem to have just enough permissions on staging to mess it up, but not update it all the way [17:21:20] d'oh. [17:21:29] Maybe Jeff_Green can... do a thing? [17:21:40] * K4-713 waves at Jeff_Green [17:21:46] Good aftermath to you, sir. [17:21:49] We're making a mess. [17:22:38] for civi there's no staging project on the deploy server, so I thought I'd just head over to staging and git checkout deployment etc [17:23:12] but permissions are preventing me from doing that [17:23:17] thanks to whoever changed the civi logo in the top of the tab! that’s helpful to me :) [17:24:12] oh hi [17:24:12] ccogdill: I think it was awight yesterday. [17:24:19] mess whut [17:24:32] heh [17:24:36] I think civi staging is in between git revisions right now [17:24:37] ahh! I’ll thank him tomorrow. but thank you, K4-713, for passing along that request! [17:25:06] ccogdill: No problem! Glad it got fixed so fast. [17:26:37] Jeff_Green: I think ejegg is trying to do something with crm staging before we make a mess in prod. [17:27:35] ok [17:27:57] Also... blah. [17:28:29] Just noticed something else, but... [17:28:36] * K4-713 hides [17:30:15] Also: Look at my drawing. [17:30:15] http://ganglia.wikimedia.org/latest/graph.php?r=hour&z=xlarge&c=Fundraising+eqiad&h=silicon.frack.eqiad.wmnet&jr=&js=&v=10&m=ActiveMQ+QueueSize+donations&vl=Messages [17:30:31] It's a portside metropolis. [17:33:31] * K4-713 takes screenshot [17:56:42] !log updated civicrm from e83c999f39e6ae847d9b48e38c8c825fc10d1635 to b6c350f620c8dc1f3410de179c19cbcbdeb62270 [17:56:49] Logged the message, Master [17:58:14] cache cleared, no updates to run [17:58:21] I can log in... [17:58:44] AndyRussG: Ah, something tells me that we're going to miss the new standup time. [17:58:49] Just deployed things. [17:58:50] edit prospect link works... [17:58:55] Turning jobs back on. [17:59:23] I'll, ah... uh-oh. [17:59:34] oh? [17:59:35] Whitescreen going to the q2c admin settings. [17:59:39] test ty works [17:59:49] ooh, whitescreen is no good... [17:59:55] Usually, no. [18:00:13] Then again, it could just be a minsconfigured menu. [18:00:20] Also misconfigured. [18:00:34] also, mincemeat [18:00:46] Exactly. [18:00:51] Okay... poking job. [18:01:00] Changed duty cycle from 90 seconds, to 2. [18:01:22] brb, moving from library's quiet room for standup [18:01:52] Yeah, I can't really do standup until we restore... [18:01:59] ...working jobs. :/ [18:02:18] I think it's just you, me, and AndyRussG today anyway. [18:03:06] ejegg: Ran TY job... and that didn't complain about anything. [18:03:12] Okay, going back up all the way. [18:03:18] Here I go. :) [18:04:36] !log re-enabled all queue consumers [18:04:42] Logged the message, Master [18:05:25] oops network [18:05:27] network? [18:05:53] hm? [18:06:02] K4-713: hmm hmmm? [18:06:03] We have really bad timing today. [18:06:08] K4-713: np [18:06:22] I killed my network by mistake [18:06:39] And by that I mean: I decided that deploying all the things was the most fun we could have right now. [18:08:04] And, we appear to be back to normal. [18:08:14] woohoo! [18:08:23] ejegg: http://ganglia.wikimedia.org/latest/graph.php?r=hour&z=xlarge&c=Fundraising+eqiad&h=silicon.frack.eqiad.wmnet&jr=&js=&v=10&m=ActiveMQ+QueueSize+donations&vl=Messages [18:08:39] I know I just keep sending the same link and that you can just refresh, but whatever. [18:08:56] wow, that drops off good & quick [18:09:04] Seriously. [18:09:16] I'm still impressed when I get to see that. [18:09:33] It didn't do that for the first year I was here. [18:09:45] Then awight kicked it in the head a few times. [18:10:33] heh [18:10:43] Precision foot-brain surgery to a machine [18:10:52] not bad [18:10:52] standup? [18:11:13] NEtime on this end [18:11:29] ejegg: what language is that on the glass behind you? [18:11:37] all of them [18:11:47] wow [18:11:50] Ah, yeah... I'm still trying to push some things through... [18:11:54] That's a lot of languages for one sheet of glass [18:12:00] Hang on a second. [18:12:06] cool [18:12:58] i'll try the exchange rate update, just for kicks [18:25:35] ejegg: You should move some cards around. [18:26:11] yep! [18:26:51] oh, I need to make a nightly jenkins job to finish the exchange rate deployment [18:27:06] just updated via UI and it worked, though! [18:27:35] Rates look reasonable similar to historical rates [18:27:59] Aw yeah. :) [18:28:05] Phew! [18:28:38] hmm, euro's down 10 cents against USD since 6/19 - hope that doesn't hurt our revised totals too badly! [18:28:48] darn strong dollar [18:31:14] ejegg: I was going to say: You should, ah... grab some totals before and after the cleanup. [18:33:36] yah [18:37:34] OK, new job 'Update Exchange Rates' under 'CiviCRM Cron' category is set to run @daily. [18:38:36] Which of your audit cards count as deployed now? [18:38:43] Weird... I just enabled my new module in prod, but there's no... UI. [18:38:52] * K4-713 copies link from localhost [18:38:52] hrm? [18:39:10] ejegg: I don't know if I want to move those quite yet. [18:39:15] ok [18:39:24] I will definitely do that. [18:39:28] ...soon. I hope. [18:40:47] ACCESS DENIED?!? [18:40:53] I seriously disbelieve. [18:40:59] oh, did you make a new permission? [18:41:04] No? [18:41:10] huh... [18:43:11] Well, this is awkward. [18:43:21] There's no new permissions in the user admin area either, and... [18:43:22] ...wait. [18:43:27] want me to see if it tells me the same thing? [18:43:27] maybe another drush cc. [18:43:45] maybe, but enabling a module usually does that for you [18:44:50] ejegg: ...can we not run drush cc for prod anymore? [18:45:06] Or, no. Wait. Dumb. [18:45:06] we definitely can [18:45:42] Yeah, I was in the wrong place. Or possibly in the past. [18:46:24] Gah, wth. [18:46:32] This is going to ruin my whole day. [18:50:20] Well... maybe I can do all the config sneakily, from the command line. [18:50:40] drush set-variable to the rescue! [18:50:50] exactly. [18:50:51] Rar [18:51:19] That is significantly more lame than what I had in mind, but it'll work. [18:53:52] i listed some tables I thought might need updating for the recalculation on card #2011. Let me know if you get a chance to look at it some time. [19:22:20] ejegg: Funny story. [19:22:41] I just configured the new audit script at the command line, which took a crazy long time. [19:22:46] Ran it a bunch. [19:23:00] And... everything in the nightly files, is already in civi. [19:23:12] So, it just did a bunch of stuff, found nothing to do, and quit. [19:23:21] I... [19:23:53] Well, sheesh. [19:24:42] aww, shucks [19:24:58] I DON'T EVEN KNOW WHAT TO DO NOW. [19:25:20] so, gotta turn off the nightly processing job for tongiht so it'll have something to test tomorrow? [19:25:22] * K4-713 looks really confused [19:25:31] * K4-713 looks around for guidence [19:25:37] * K4-713 goes for lunch [19:25:47] ooh, good plan. [19:26:15] I guess I could delete some things from civi. [19:26:23] * K4|food goes crazy [20:55:35] K4-713: do you have a sec to look at card 2011 and let me know if I listed all the tables to update? [20:56:00] Interesting. Looking now. [20:56:15] Oh, ha. [20:56:23] Actually, you're overshooting by a wide margin. [20:56:35] oh, i like that answer! [20:56:35] fredge and contribution tracking don't have conversions. [20:56:49] Pretty much just the civi tables. [20:57:00] oh, cool [20:57:10] I know there are columns for the USD equivalent in contribution_tracking, but those are currently unpopulated. [20:57:18] ...which is sad, but there you go. [20:57:43] Hey, so, I'm about to do something weird. [20:57:48] well, i guss you can just join, and no harm done. [20:58:02] what's the weirdness you're preparing to perpetrate? [20:58:39] Trying to find something to damage in prod, so the audit script can rebuild. [20:58:45] Worldpay transactions. :) [20:59:07] Hopefully just, you know, us. [20:59:18] ohh, fun. Did we finally pass the threshold where they tell us stuff happened? [20:59:21] Nobody for whom it would be awkward if they suddenly got a new Thank You email. Ha. [20:59:38] ah, right [21:11:28] Well, I just uncovered a bug that only testers like me would ever run in to. :D [21:11:54] what's that? [21:12:01] Null utm data. [21:12:08] Nothing to match against. [21:12:23] ...let me just fix this real... quick... [21:12:27] ah, tricky [21:12:43] I forgot I existed. [21:14:05] At some point, I should try to figure out what's wrong with the permissions and stuff. [21:14:10] Don't worry, Descarte's pretty sure you exist [21:33:28] (PS1) Katie Horn: Small cleanup round for the WP audit script * Forgot to take into account that testers are going to do strange things with the UTM data * Cleaned up a couple calls to functions that don't actually exist [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163959 [21:35:10] ejegg: ^^ [21:35:19] I did actually find a few things to clean up. :) [21:35:24] on it! [21:36:43] Thank you! [21:46:50] Hee, it found me again in test mode. [21:51:35] (CR) Ejegg: [C: 2] "These changes look useful! Now to zoom out and try to understand the whole shebang..." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163959 (owner: Katie Horn) [21:51:45] heh. Thanks. :) [21:52:00] It should, at least, be well-commented. [21:55:06] (PS1) Katie Horn: Small cleanup round for the WP audit script * Forgot to take into account that testers are going to do strange things with the UTM data * Cleaned up a couple calls to functions that don't actually exist [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163965 [21:55:40] (CR) Katie Horn: [C: 2] "self- +2 for deploy" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163965 (owner: Katie Horn) [21:57:31] !log updated prod civicrm to 477a5107a0c93ceac5214 [21:57:41] Logged the message, Master [21:58:28] d'oh. At least one more. [22:03:42] OMG STUPIDEST BUG. [22:03:54] I've been staring at this block for way too long not to have noticed. [22:04:01] hey, bugs don't have a lot of neurons to work with [22:10:09] (PS1) Katie Horn: WP audit script: Small issue with the contribution tracking data [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163974 [22:10:26] ejegg: ^^ [22:10:29] That's the facepalm [22:11:12] full-on Picard [22:11:18] No kidding! [22:11:27] I looked right at it! Over and over! [22:11:34] Sad. :) [22:11:37] (CR) Ejegg: [C: 2] WP audit script: Small issue with the contribution tracking data [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163974 (owner: Katie Horn) [22:21:11] (PS1) Katie Horn: WP audit script: Small issue with the contribution tracking data [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163977 [22:22:30] (CR) Katie Horn: [C: 2] "deploy" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163977 (owner: Katie Horn) [23:11:40] howdy :) [23:11:45] hello!