[00:31:48] !log rollback civicrm to a30fc0aa2c2dc3a7a9a0b09bef19d112bdf5f98e [00:31:53] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [00:45:59] fr-tech: aww looks like I missed the moment [00:46:15] I left a patch out of the latest deployment merge... [00:47:53] (PS9) Awight: Update composer libs and resolve global namespace conflict [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/301521 [01:09:17] Hi all, Japan pre-test is now over. [15:44:33] Fundraising-Backlog: Skeuomorphic interface for entering CC details - https://phabricator.wikimedia.org/T143426#2567792 (spatton) [15:48:19] Fundraising-Backlog: Better CC Validation and Error Messages - https://phabricator.wikimedia.org/T143427#2567809 (spatton) [15:48:38] Fundraising-Backlog: Better CC Validation and Error Messages - https://phabricator.wikimedia.org/T143427#2567827 (spatton) [15:51:38] Fundraising-Backlog: Banners on Portal - https://phabricator.wikimedia.org/T143428#2567842 (spatton) [15:59:27] Fundraising-Backlog: Recurring upsell for donors - https://phabricator.wikimedia.org/T143429#2567867 (spatton) [16:02:11] Fundraising-Backlog: Duplicate iframe bug - https://phabricator.wikimedia.org/T143430#2567887 (spatton) [16:02:31] Fundraising-Backlog: Duplicate iframe bug - https://phabricator.wikimedia.org/T143430#2567899 (spatton) [16:03:19] Fundraising-Backlog: Duplicate Ingenico iframe bug - https://phabricator.wikimedia.org/T143430#2567887 (spatton) [16:28:21] fundraising-tech-ops, Operations, Security-General: use granularity (g=) restrictions for wikimedia.org fundraising DKIM records - https://phabricator.wikimedia.org/T142205#2567940 (CCogdill_WMF) I've been reviewing this request with IBM, and haven't gotten good news as to their ability to integrate... [17:06:49] Fundraising-Backlog: Duplicate Ingenico iframe bug - https://phabricator.wikimedia.org/T143430#2567887 (XenoRyet) This looks like it might be a duplicate of T142059 which had a fix deployed late Wednesday the 17th. @MeganHernandez_WMF can you check whether or not this is still happening for you? [17:14:15] fr-tech anyone for tech talk? [17:14:44] i have been on the phone with comcast for 3 hours [17:15:07] oof, sorry to hear it [17:15:20] AUGH [17:15:37] freaking purgatory [17:36:08] omg that was awful [17:36:16] but i'm out of the "contract" [17:36:25] for $50 [17:36:50] whew [17:48:26] fundraising-tech-ops, Operations, Security-General: use granularity (g=) restrictions for wikimedia.org fundraising DKIM records - https://phabricator.wikimedia.org/T142205#2568249 (faidon) Could you explain a little bit what happens when you send using e.g. caitlin@wikimedia.org? Do you get random p... [17:59:34] Any luck with the python/redis stuff XenoRyet ? [18:00:48] Tests are pretty tricky with that repo - I'm trying to make the database wrapper work with sqlite but there's a lot of mysql-specific syntax sprinkled around [18:02:39] Yea, little bit. Mostly set up and reading so far. I've not actually done python before. It became a thing after I was entrenched in Microsoft land. Pretty cool so far though. [18:03:22] I'm not really blocked on waiting for the tests, so no rush there. [18:03:32] k [18:04:23] They'll definitely be useful, but don't push too hard on my account. I've got stuff in the mean time. [18:05:21] I am interested to try this approach overall. The thing that's kept me from TDD before was that I didn't like being the one writing the tests and then writing the code. Be interesting to see how splitting that up works out. [18:06:18] right... I'm just trying to get it to parse a file without touching real databases or queues at this point [18:06:37] but I guess we'll want to test that it sends the same thing to both queues etc [18:07:14] seems like I'm going to have to mess with a bunch of the supporting libs to make any of that testable [18:08:24] Parsing does seem like a reasonable place to start though. I suppose we could manually check the queues if we had to. [18:08:56] yeah, i was just hoping to be able to run unit tests without a real redis instance running [18:09:16] Be nice [18:11:06] not that i don't strongly prefer python to php but is it worth keeping it in python if it makes it hard to test/integrate? [18:13:45] cwd i'm starting to wonder [18:14:38] It is a valid quesiton. I can switch gears if that would be better. [18:15:26] cwd / XenoRyet the TrrFile looks like it would be dead simple to implement as a SmashPig / wmf_audit file parser [18:15:34] I guess Adam was saying the file format might be a pain/impossible in PHP, but I don't know that we got a solid answer on that. [18:16:17] ohh, he thought the encoding would break [18:16:37] If we can get around that though, does seem a lot cleaner to put it in SmashPig with everything else. [18:16:47] yeah [18:17:09] Let me see if I can figure something out there. [18:17:25] ok, let me know if you want to do anything in parallel [18:17:25] Like you said, it's just bytes, there's gotta be something we can do. [18:17:38] Yea, will do. [18:18:11] so, the wmf_audit code currently doesn't handle any recurring contributions - that would be new logic [18:18:27] what encoding problems would python handle that php can't? [18:18:49] I mean, if we didn't care about making it more testable, it probably would be easier to just tack on some redis lib calls in the existing code [18:19:04] maybe that should be the short term focus? [18:20:29] Yea, that's the balance. [18:20:42] Seems right on the tipping point either way though. [18:20:58] is the redis stuff that bad? [18:21:07] would that mean depending on working redis for integration tests? [18:21:14] and...is that a thing right now? [18:21:22] no tests now [18:21:45] there's a lot that assumes a live civicrm db to connect to [18:22:06] oh right no tests now [18:22:07] and some mysql-specific syntax that's making it hard to allow sqlite [18:22:11] dang [18:22:12] No, modifying the current stuff doesn't seem that bad. Just kind of wrong place and have to not care about tests. [18:22:43] well, it looks pretty self contained [18:23:11] both SarFile.py and TrrFile.py have a 'send' function that just pushes a message to a stomp queue [18:23:11] I suppose we are in a short-term mode right now as well. [18:23:37] it looks like it would be pretty simple to add a push to redis right after that [18:23:48] Yea, that's the plan, and it shouldn't be tough. [18:24:21] The flip side, as always, is that if we don't move it into SmashPig now, we may never touch it again. [18:24:44] But I guess David would want us to stay focused. [18:24:49] better that than not getting the queue stuff done before October! [18:24:56] Indeed [18:25:21] Ok, yea. Let's stick with python then. [18:26:47] and the only redis command we need is rpush, I think [18:30:25] Fundraising-Backlog, FR-ActiveMQ: Install redis-py on CRM server - https://phabricator.wikimedia.org/T143443#2568350 (Ejegg) [18:43:52] (PS1) Ejegg: DO NOT MERGE: try to make paypal audit testable [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/305672 [19:00:20] anyone got time to review this SmashPig thing to make testing pending/damaged db stuff easier? : https://gerrit.wikimedia.org/r/305568 [19:00:26] fr-tech ^^ [19:02:18] I think I better stick on my current tasks for now, but maybe I can take a look later if nobody else gets to it. [19:04:39] also in the middle of stuff but i'll try to make some time [19:05:32] thanks! [19:32:04] (PS1) Ejegg: Mirror messages to new queues in GC audit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305674 (https://phabricator.wikimedia.org/T141484) [19:44:05] (PS1) Ejegg: Mirror queue messages from wmf_audit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305677 (https://phabricator.wikimedia.org/T141484) [19:52:49] (PS2) Ejegg: Mirror queue messages from wmf_audit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305677 (https://phabricator.wikimedia.org/T141484) [20:05:08] (PS1) Ejegg: Always look for and delete pending entries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305679 (https://phabricator.wikimedia.org/T141477) [20:07:42] (CR) jenkins-bot: [V: -1] Always look for and delete pending entries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305679 (https://phabricator.wikimedia.org/T141477) (owner: Ejegg) [20:41:07] (CR) Ejegg: [C: 2] Update composer libs and resolve global namespace conflict [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/301521 (owner: Awight) [20:43:07] ejegg: thanks for getting that one, i knew it was urgent but also huge [20:43:38] yeah, i missed the fact that te earlier lib update included donation interface! [20:44:31] (Merged) jenkins-bot: Update composer libs and resolve global namespace conflict [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/301521 (owner: Awight) [20:52:14] (CR) Ejegg: [C: -1] "I think we need one more code change to let the orphan rectifier run antifraud hooks, but avoid running them in the recurring job's pre_pr" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/300708 (https://phabricator.wikimedia.org/T141487) (owner: Awight) [21:24:31] fundraising-tech-ops, Operations, Security-General: use granularity (g=) restrictions for wikimedia.org fundraising DKIM records - https://phabricator.wikimedia.org/T142205#2568708 (CCogdill_WMF) > Could you explain a little bit what happens when you send using e.g. caitlin@wikimedia.org? Do you get... [21:26:34] fr-tech that damaged message datastore should probably split out error message and stack trace into different columns, huh? [21:26:56] Yea, that seems like a good idea. [21:27:23] is the plan to store the stack trace as a text blob? [21:28:17] (PS2) Ejegg: Base class for dbs, they can create tables [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305568 [21:40:46] cwd yeah, text blob [21:46:14] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, FR-ActiveMQ: Store damaged and delayed messages in a database not a queue - https://phabricator.wikimedia.org/T142028#2568801 (Ejegg) Resolved>Open [21:46:27] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising-Backlog, FR-ActiveMQ: Store damaged and delayed messages in a database not a queue - https://phabricator.wikimedia.org/T142028#2520228 (Ejegg) [21:46:36] (PS1) Ejegg: Damaged messages: store trace separate from message [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305742 (https://phabricator.wikimedia.org/T142028) [22:13:59] (PS1) Ejegg: WIP damaged message table UI [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305744 (https://phabricator.wikimedia.org/T142058)