[05:22:38] Fundraising Sprint Rocket Surgery 2016, Fundraising Sprint Stirring The Pot, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, and 3 others: Banner not showing up on site - https://phabricator.wikimedia.org/T144952#2704815 (AndyRussG) I've been able to trigger this error a few times on the... [10:25:29] Fundraising-Backlog, Continuous-Integration-Config, Patch-For-Review: wikimedia/fundraising/tools should pass python lint checks (flake8) - https://phabricator.wikimedia.org/T114218#2705178 (hashar) That is apparently also tracked in private task {T145012} https://gerrit.wikimedia.org/r/#/c/314481/... [15:49:26] fundraising-tech-ops: upgrade all frack servers to debian/jessie - https://phabricator.wikimedia.org/T146479#2705930 (Jgreen) [16:37:37] (CR) Ejegg: [C: 2 V: 2] "Looks good! I get an error with dummy test DjangoBannerStats/fundraiser/analytics/tests - looks like it can be zapped. Also, I notice the " [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314475 (https://phabricator.wikimedia.org/T145012) (owner: Awight) [16:38:26] gotta run for a bit but will join TT upon return [16:39:27] (CR) Ejegg: [C: 2 V: 2] "*Snoopy and Woodstock, dancing on a grave*" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314484 (owner: Awight) [16:47:10] (CR) Ejegg: [C: 2 V: 2] "Mindlessly merge formatting fixes" (3 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314485 (owner: Awight) [17:00:56] fr-tech: Don't abandon hope. [17:00:56] Your Captain Midnight decoder ring arrives tomorrow. [17:00:56] -- discuss. [17:01:59] (CR) Ejegg: [C: -1] "Deck needs a little more shuffling" (3 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314486 (owner: Awight) [17:03:02] slander: hahaha [17:03:46] Going to TT in a sec, just getting some snax [17:18:24] cwd: hi! [17:19:23] hiya! [17:21:09] cwd: back at the keyboard now :) ejegg is here too! [17:22:07] Right, tech talk. Let me jump in there too. [18:09:08] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface: Missing ct_id on fredge messages from orphan rectifier - https://phabricator.wikimedia.org/T147873#2706582 (Ejegg) [18:29:51] (PS2) Ejegg: Add source fields to Redis audit messages [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314629 (https://bugzilla.wikimedia.org/147341) [18:29:57] XenoRyet: do you have time for some CR? ^^^ [18:32:14] Sure, let me take a look. [18:34:36] (CR) XenoRyet: [C: 2] "Yep, this will do for now." [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314629 (https://bugzilla.wikimedia.org/147341) (owner: Ejegg) [18:34:42] (Merged) jenkins-bot: Add source fields to Redis audit messages [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/314629 (https://bugzilla.wikimedia.org/147341) (owner: Ejegg) [18:35:35] awight|half: I CR'ed a little way through your tools overhaul [19:00:17] (PS6) Cdentinger: Use redis for Adyen jobs [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/313130 (owner: Ejegg) [19:02:24] (CR) Cdentinger: [C: 2] "Let's do it!" (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/313130 (owner: Ejegg) [19:03:52] (Merged) jenkins-bot: Use redis for Adyen jobs [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/313130 (owner: Ejegg) [19:09:03] (CR) Cdentinger: [C: 2] "Can't be a bad idea with the lucky ID" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/314777 (https://phabricator.wikimedia.org/T147679) (owner: Ejegg) [19:10:44] (Merged) jenkins-bot: Add ct_id to error forms as reference number. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/314777 (https://phabricator.wikimedia.org/T147679) (owner: Ejegg) [19:15:36] thanks cwd! [19:17:07] for sure. looking forward to the big decommission [19:23:33] Wow, do I hear someone singing "Daisy Bell"? [19:24:04] hehe [19:35:57] (Abandoned) Cdentinger: Add some sparse logging to PayPal listener. [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/312273 (owner: Cdentinger) [19:46:06] (PS1) Cdentinger: Some logging in PP listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/315336 [19:50:59] (CR) Ejegg: "Good start! Maybe different conditions to choose the log id?" (2 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/315336 (owner: Cdentinger) [19:58:45] Fundraising-Backlog, fundraising-tech-ops, FR-Smashpig: Figure out a sane way to deploy SmashPig.yaml configuration - https://phabricator.wikimedia.org/T147503#2707078 (awight) [20:05:42] (CR) Cdentinger: Some logging in PP listener (2 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/315336 (owner: Cdentinger) [20:45:07] (CR) Ejegg: Some logging in PP listener (2 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/315336 (owner: Cdentinger) [20:49:17] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315396 [20:52:23] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315396 (owner: Ejegg) [20:52:39] hey I'm going to join the backlog grooming hangout on chat only [20:55:44] !log updated fundraising tools from 6e36fd547b97c426bfb6810d5bc2c9fd4b66efa5 to cc7628355e25422bda2b479a009757acbb702072 [20:55:49] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [20:57:29] !log enabled PayPal audit parsing job [20:57:34] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:01:23] !log replaced pay-lvs1001 [21:01:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:03:19] awight: AndyRussG ejegg meeting? [21:03:46] eileen: isn't is super late where you are? You don't have to call in. [21:04:50] awight: I've got distraction factors to the max so will have to miss pretty much everything for the near future [21:05:42] awight: ok.. are you attempting to disengage? [21:06:08] :) I sent a properly vague email about this [21:11:48] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-2016-17-Q2-Bugs: Generic Individuals Import Error - https://phabricator.wikimedia.org/T147521#2695203 (DStrine) [21:12:20] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, FR-2016-17-Q2-Bugs: Missing ct_id on fredge messages from orphan rectifier - https://phabricator.wikimedia.org/T147873#2707359 (DStrine) [21:12:26] dstrine: ok - well flick me any questions you have - I'm still on my laptop [21:12:57] (although I might go to bed soon) [21:14:24] Fundraising Sprint Stirring The Pot, Fundraising-Backlog, Patch-For-Review, Unplanned-Sprint-Work: New json encoding is breaking recurring and refund consumers - https://phabricator.wikimedia.org/T147532#2695626 (Ejegg) Open>Resolved a:awight [21:14:45] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint Rocket Surgery 2016, Fundraising Sprint Stirring The Pot, Fundraising-Backlog, and 2 others: Move recurring queue consumer off ActiveMQ - https://phabricator.wikimedia.org/T145233#2707379 (Ejegg) Open>Resolved [21:14:55] Fundraising Sprint Stirring The Pot, Fundraising-Backlog, Patch-For-Review, Unplanned-Sprint-Work: Job runners are segfaulting - https://phabricator.wikimedia.org/T146987#2707380 (Ejegg) Open>Resolved [21:15:31] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-2016-17-Q2-Bugs: Wrong date encoding fails some donation messages - https://phabricator.wikimedia.org/T147603#2707383 (DStrine) [21:20:19] Fundraising-Backlog: Reconsider Redis message expiry - https://phabricator.wikimedia.org/T147531#2707419 (DStrine) @Ejegg will check in our config to see if this can wait until after big english [21:21:50] Fundraising-Backlog, fundraising-tech-ops, FR-Smashpig, FR-2016-17-Q2-Ops: Figure out a sane way to deploy SmashPig.yaml configuration - https://phabricator.wikimedia.org/T147503#2707421 (DStrine) [21:23:45] Fundraising Sprint Stirring The Pot, Fundraising-Backlog, FR-Adyen, Unplanned-Sprint-Work: Adyen GB form not loading correctly - https://phabricator.wikimedia.org/T147475#2707427 (DStrine) [21:28:27] Fundraising Sprint Stirring The Pot, Fundraising-Backlog, FR-Adyen, Unplanned-Sprint-Work: Adyen GB form not loading correctly - https://phabricator.wikimedia.org/T147475#2707450 (XenoRyet) a:XenoRyet [21:44:47] !log disabled paypal audit parser [21:44:52] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:48:51] (PS1) Ejegg: Add missing import statements [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315431 [21:49:07] fr-tech sorry for that failmail flood. ^^^ should fix it, I think [21:49:18] ty ejegg [21:50:04] (CR) XenoRyet: [C: 2] Add missing import statements [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315431 (owner: Ejegg) [21:50:09] thanks XenoRyet [21:50:10] (Merged) jenkins-bot: Add missing import statements [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315431 (owner: Ejegg) [21:50:13] No worries [21:51:33] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315434 [21:52:05] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315434 (owner: Ejegg) [21:52:05] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315434 (owner: Ejegg) [21:53:15] !log updated fundraising tool from cc7628355e25422bda2b479a009757acbb702072 to 112b3fa8cf097c8f702d3824ff169d7a1059f723 [21:53:21] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:53:30] ok, this time for sure [21:53:43] !enabled paypal audit parser [21:54:15] looking better [21:56:08] bah [21:56:19] one more import... [21:58:51] my inbox just exploded [22:01:38] (PS1) Ejegg: One more missing import :S [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315435 [22:01:47] sorry again fr-tech [22:01:57] XenoRyet: this time I actually smoke tested locally ^^ [22:02:07] ;-) [22:02:23] (CR) XenoRyet: [C: 2] One more missing import :S [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315435 (owner: Ejegg) [22:02:28] (Merged) jenkins-bot: One more missing import :S [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315435 (owner: Ejegg) [22:03:00] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315436 [22:03:17] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315436 (owner: Ejegg) [22:03:24] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315436 (owner: Ejegg) [22:04:25] !log updated fundraising tools from 112b3fa8cf097c8f702d3824ff169d7a1059f723 to 206799da4d006e6e1e2c747a28039950f47bf2e6 [22:04:31] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:06:33] ugh, now the date format errors [22:06:50] ok, guess we really need to pull in that task [22:07:18] !log disabled donations queue consumer [22:07:23] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:07:34] dang [22:07:44] is the PP audit the primary source of bad dates? [22:07:46] or... wait, I think that's the patch we decided not to deploy friday. [22:08:05] cwd no, that's ok cupid [22:08:09] *rimshot* [22:08:19] hehehe [22:08:27] i was looking for the picture from indiana jones [22:08:36] when the monkey eats the date and dies [22:08:48] aw man [22:09:49] k, I think I just need to deploy an update to the SmashPig lib in CRM [22:10:03] but first, moving to an actual table [22:22:06] ejegg: is there a ticket for those missing predecessor ones? [22:22:50] cwd err, I think they'll be retried successfully as soon as i get the crm lib updated [22:23:16] it's normal to have them processed before the subscription sometimes, but the requeue logic is busted right now [22:23:37] should they be fail mailing if there's not a real problem? [22:24:17] oh right, that was the other thing we changed last week - logging at a lower level for retry date [22:26:27] (PS1) Ejegg: Update SmashPig lib for better re-queue handling [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/315438 [22:27:21] (CR) Ejegg: [C: 2] Update SmashPig lib for better re-queue handling [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/315438 (owner: Ejegg) [22:28:47] https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/master/Core/Logging/Logger.php#L45 :| [22:30:47] (PS1) Ejegg: Update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/315439 [22:30:56] (CR) Ejegg: [C: 2] Update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/315439 (owner: Ejegg) [22:32:28] cwd well ain't that a pretty situation [22:33:21] cwd oops, I should DRY up that requeue date logic in CRM [22:33:25] (CR) jenkins-bot: [V: -1] Update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/315439 (owner: Ejegg) [22:34:26] :P phplint55 and httpfoundation again [22:35:08] (CR) Ejegg: [V: 2] Update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/315439 (owner: Ejegg) [22:38:31] cwd oops, SmashPig library function getOriginalDateOrNow isn't actually what I want... I want getOriginalDateOrZero. I'll stick with the nearly-duplicated logic [22:39:02] (Merged) jenkins-bot: Update SmashPig lib for better re-queue handling [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/315438 (owner: Ejegg) [22:39:16] ty zuul [22:39:16] heh, will 0 make the date parser happy? [22:39:46] cwd nah, but it'll make sure something with no date fields gets rejected instead of infinitely retried [22:39:59] ah, good stuff [22:40:59] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/315442 [22:41:08] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/315442 (owner: Ejegg) [22:41:18] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/315442 (owner: Ejegg) [22:42:03] k, let's see if that fixes it [22:43:54] !log updated CiviCRM from 17fab4ded647bad51b30ce65157c88a87e1f7e40 to 8682821cb591b2861aee1fd167157a1b7aa27abd [22:43:59] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:44:25] !log enabled donation queue consumer [22:44:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:44:55] ugh, no order ids... [22:45:05] !log disabled donation queue consumer [22:45:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:45:22] ok, I'll clear out the audit messages and just turn the parser off [22:48:28] whew [22:48:42] Fundraising-Backlog, fundraising-tech-ops, FR-Smashpig, FR-2016-17-Q2-Ops: Figure out a sane way to deploy SmashPig.yaml configuration - https://phabricator.wikimedia.org/T147503#2707678 (awight) In IRC, @Jgreen made the point that everything in the config overrides is the sort of thing that belo... [22:49:00] wtf, i still don't see source_ fields on them [22:54:43] (PS1) Ejegg: Add order_id to paypal audit donations [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315444 [22:55:32] fr-tech one more tiny fix for some of the paypal audit donations ^^ [22:56:25] hmm, though I'm still trying to figure out why those source fields don't appear [22:57:17] guh, i see why [22:58:40] Fundraising Sprint Stirring The Pot, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Smashpig, Unplanned-Sprint-Work: Missing "gross" and "currency"; positive amount; failmails from donations queue consumer - https://phabricator.wikimedia.org/T147491#2707691 (MBeat33) Not sure, @awight... [22:58:47] (PS1) Ejegg: Add source_ fields BEFORE encoding to json [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315445 (https://phabricator.wikimedia.org/T147341) [22:58:57] ok, that was a really silly one fr-tech ^^^ [22:59:20] doh, I should've seen that too. [22:59:55] (CR) XenoRyet: [C: 2] Add source_ fields BEFORE encoding to json [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315445 (https://phabricator.wikimedia.org/T147341) (owner: Ejegg) [23:00:21] thanks XenoRyet ! parent patch is begging for attention too [23:00:42] Yea, about to +2 but gerrit is messing with me. [23:01:03] ah k [23:01:22] back soon [23:14:48] (CR) XenoRyet: [C: 2] Add order_id to paypal audit donations [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315444 (owner: Ejegg) [23:18:49] (Merged) jenkins-bot: Add order_id to paypal audit donations [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315444 (owner: Ejegg) [23:18:51] (Merged) jenkins-bot: Add source_ fields BEFORE encoding to json [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/315445 (https://phabricator.wikimedia.org/T147341) (owner: Ejegg) [23:21:29] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315448 [23:22:00] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315448 (owner: Ejegg) [23:22:06] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/315448 (owner: Ejegg) [23:23:54] !log updated fundraising tools from 206799da4d006e6e1e2c747a28039950f47bf2e6 to cbf4dcd734956f95a7485ce1f420d05222c09e5a [23:24:00] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [23:27:16] woohoo, we have source fields! [23:28:57] phew, I'm going to head out to NOLA hack night. [23:29:02] ejegg: sooo less fail mail [23:29:04] ? [23:29:05] see y'all tomorrow [23:29:08] yep dstrine [23:29:12] contrats!!! [23:29:16] or [23:29:17] have fun! [23:29:18] congrats [23:29:21] thanks [23:29:30] * dstrine waves [23:30:18] ugh... what is this new batch? [23:30:49] guh, decimal dates again :( [23:31:14] well, I'll just leave the audit parser off and fix that tomorrow [23:31:22] sorry for the false hope [23:32:08] !disabled paypal audit parser