[00:31:46] <wikibugs>	 (03PS1) 10Eileen: Add step to delete mailing_job rows once orphaned [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964608 (https://phabricator.wikimedia.org/T335070)
[03:41:15] <wikibugs>	 (03PS1) 10Eileen: Insert step (with special before magic) to remove old queue items [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964621 (https://phabricator.wikimedia.org/T335070)
[03:44:14] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Clean up old TY CiviMail records - https://phabricator.wikimedia.org/T335070 (10Eileenmcnaughton) I have queued up a string of updates to  1) remove civicrm_queue records older than a year -...
[03:52:16] <wikibugs>	 (03PS2) 10Eileen: Insert step (with special before magic) to remove old queue items [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964621 (https://phabricator.wikimedia.org/T335070)
[06:08:35] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/964699 (owner: 10L10n-bot)
[09:00:43] <wikibugs>	 (03PS3) 10Damilare Adedoyin: Individual Engage File Import Duplicates [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964569 (https://phabricator.wikimedia.org/T347362)
[09:31:25] <wikibugs>	 (03CR) 10Damilare Adedoyin: [C: 03+2] "Looks good to me, thanks for the work on this." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/962239 (https://phabricator.wikimedia.org/T347771) (owner: 10Wfan)
[09:33:21] <wikibugs>	 (03Merged) 10jenkins-bot: Pass gateway_session_id from playload paymentContextId [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/962239 (https://phabricator.wikimedia.org/T347771) (owner: 10Wfan)
[11:17:12] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 10273 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 11 keys, up 59 days 18 hours - memory use is 7.84M (peak 28.06M, 0.18% of max, fragmentation 1.88%), connected_slaves is 3, donations is 30, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 8, payments-antifraud is 0, payments-init is 0, pending is 0, refund is 0, unsubscribe is 2 htt
[11:17:12] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:22:16] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 10807 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 18 hours - memory use is 8.07M (peak 28.06M, 0.18% of max, fragmentation 1.86%), connected_slaves is 3, donations is 59, jobs is 0, jobs-adyen is 17, jobs-dlocal is 0, jobs-paypal is 7, payments-antifraud is 3, payments-init is 14, pending is 0, refund is 0, unsubscribe is 2 h
[11:22:16] <icinga-wm>	 inga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:27:16] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 11588 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 18 hours - memory use is 8.57M (peak 28.06M, 0.19% of max, fragmentation 1.82%), connected_slaves is 3, donations is 37, jobs is 0, jobs-adyen is 9, jobs-dlocal is 0, jobs-paypal is 6, payments-antifraud is 8, payments-init is 1, pending is 0, refund is 0, unsubscribe is 3 htt
[11:27:16] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:32:12] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 12186 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 11 keys, up 59 days 18 hours - memory use is 8.79M (peak 28.06M, 0.19% of max, fragmentation 1.81%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 11, payments-antifraud is 0, payments-init is 9, pending is 0, refund is 0, unsubscribe is 3 htt
[11:32:12] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:37:16] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13009 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 13 keys, up 59 days 18 hours - memory use is 9.29M (peak 28.06M, 0.20% of max, fragmentation 1.76%), connected_slaves is 3, donations is 59, jobs is 0, jobs-adyen is 33, jobs-dlocal is 0, jobs-paypal is 6, payments-antifraud is 1, payments-init is 0, pending is 0, refund is 0, unsubscribe is 3 ht
[11:37:16] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:42:16] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13578 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 19 hours - memory use is 9.68M (peak 28.06M, 0.20% of max, fragmentation 1.72%), connected_slaves is 3, donations is 52, jobs is 0, jobs-adyen is 4, jobs-dlocal is 0, jobs-paypal is 6, payments-antifraud is 4, payments-init is 15, pending is 0, refund is 0, unsubscribe is 5 ht
[11:42:16] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:47:14] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 14051 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 10 keys, up 59 days 19 hours - memory use is 9.89M (peak 28.06M, 0.20% of max, fragmentation 1.67%), connected_slaves is 3, donations is 53, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 0, payments-init is 0, pending is 0, refund is 0, unsubscribe is 5 htt
[11:47:14] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:52:12] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13798 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 16 keys, up 59 days 19 hours - memory use is 9.79M (peak 28.06M, 0.19% of max, fragmentation 1.64%), connected_slaves is 3, donations is 68, jobs is 0, jobs-adyen is 25, jobs-dlocal is 0, jobs-paypal is 1, payments-antifraud is 3, payments-init is 21, pending is 0, refund is 1, unsubscribe is 6 h
[11:52:12] <icinga-wm>	 inga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[11:57:16] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13799 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 19 hours - memory use is 9.72M (peak 28.06M, 0.19% of max, fragmentation 1.63%), connected_slaves is 3, donations is 41, jobs is 0, jobs-adyen is 7, jobs-dlocal is 0, jobs-paypal is 2, payments-antifraud is 2, payments-init is 0, pending is 0, refund is 1, unsubscribe is 7 htt
[11:57:16] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:02:15] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13544 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 13 keys, up 59 days 19 hours - memory use is 9.64M (peak 28.06M, 0.19% of max, fragmentation 1.65%), connected_slaves is 3, donations is 75, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 2, payments-init is 17, pending is 0, refund is 0, unsubscribe is 1 ht
[12:02:15] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:07:13] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13545 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 12 keys, up 59 days 19 hours - memory use is 9.57M (peak 28.06M, 0.19% of max, fragmentation 1.65%), connected_slaves is 3, donations is 63, jobs is 0, jobs-adyen is 12, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 1, payments-init is 0, pending is 0, refund is 0, unsubscribe is 1 ht
[12:07:13] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:12:15] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13321 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 19 hours - memory use is 9.49M (peak 28.06M, 0.19% of max, fragmentation 1.65%), connected_slaves is 3, donations is 39, jobs is 0, jobs-adyen is 15, jobs-dlocal is 0, jobs-paypal is 1, payments-antifraud is 5, payments-init is 12, pending is 0, refund is 0, unsubscribe is 3 h
[12:12:15] <icinga-wm>	 inga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:15:41] <jgleeson>	 oh man
[12:17:15] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13323 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 19 hours - memory use is 9.57M (peak 28.06M, 0.19% of max, fragmentation 1.64%), connected_slaves is 3, donations is 55, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 1, payments-antifraud is 3, payments-init is 1, pending is 1, refund is 0, unsubscribe is 3 htt
[12:17:15] <icinga-wm>	 ga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:22:15] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13086 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 13 keys, up 59 days 19 hours - memory use is 9.33M (peak 28.06M, 0.19% of max, fragmentation 1.68%), connected_slaves is 3, donations is 59, jobs is 0, jobs-adyen is 19, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 2, payments-init is 10, pending is 0, refund is 0, unsubscribe is 5 h
[12:22:15] <icinga-wm>	 inga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:27:13] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 13089 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 12 keys, up 59 days 19 hours - memory use is 9.35M (peak 28.06M, 0.19% of max, fragmentation 1.68%), connected_slaves is 3, donations is 45, jobs is 0, jobs-adyen is 25, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 1, payments-init is 1, pending is 0, refund is 0, unsubscribe is 7 ht
[12:27:14] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:32:19] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 12838 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 19 hours - memory use is 9.36M (peak 28.06M, 0.19% of max, fragmentation 1.66%), connected_slaves is 3, donations is 81, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 1, payments-antifraud is 4, payments-init is 10, pending is 0, refund is 0, unsubscribe is 8 ht
[12:32:19] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:37:13] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 12841 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 12 keys, up 59 days 19 hours - memory use is 9.21M (peak 28.06M, 0.19% of max, fragmentation 1.67%), connected_slaves is 3, donations is 62, jobs is 0, jobs-adyen is 32, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 2, payments-init is 2, pending is 0, refund is 0, unsubscribe is 8 ht
[12:37:13] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:38:37] <jgleeson>	 howdy Jeff_Green 
[12:38:53] <Jeff_Green>	 hey jgleeson, how's it going?
[12:40:02] <jgleeson>	 good good. we've been getting a fair bit of noise from the icinga alerts set up to track the recurring queue count. I tried acknowledging the alert yesterday in an attempt to quiet it down but I didn't have perms. Would you be able to do that pls?
[12:40:23] <jgleeson>	 hope you have a nice long weekend!
[12:40:27] <Jeff_Green>	 yeah, that's on my radar too because we keep getting paged at 7:30AM
[12:42:13] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 12628 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 20 hours - memory use is 9.12M (peak 28.06M, 0.19% of max, fragmentation 1.69%), connected_slaves is 3, donations is 44, jobs is 0, jobs-adyen is 28, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 5, payments-init is 15, pending is 2, refund is 0, unsubscribe is 8 h
[12:42:14] <icinga-wm>	 inga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:42:37] <wikibugs>	 10Fundraising-Backlog: Remove the recurring queue threshold alerts - https://phabricator.wikimedia.org/T348533 (10jgleeson)
[12:44:20] <jgleeson>	 I can't remember why we monitor the queue count for recurring, I feel like there was a good reason but at this point it feels like it probably isn't a concern unless it goes over some crazy number 
[12:44:32] <jgleeson>	 it might actually be to do with timing of cron jobs
[12:45:25] <jgleeson>	 I created T348533 so we can hash it out
[12:45:27] <stashbot>	 T348533: Remove the recurring queue threshold alerts - https://phabricator.wikimedia.org/T348533
[12:46:28] <Jeff_Green>	 I guess it can be helpful to know if we're going wildly out of pattern
[12:47:13] <icinga-wm>	 PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: recurring is 12632 9500 - REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 11 keys, up 59 days 20 hours - memory use is 9.18M (peak 28.06M, 0.19% of max, fragmentation 1.69%), connected_slaves is 3, donations is 80, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 0, payments-antifraud is 1, payments-init is 0, pending is 0, refund is 0, unsubscribe is 10 ht
[12:47:13] <icinga-wm>	 nga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:48:25] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Silverpop/Acoustic export timing out after 10 hours - https://phabricator.wikimedia.org/T347997 (10jgleeson) 05Open→03Resolved @Eileenmcnaughton pushed up https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/963146 as a potential fix for thi...
[12:49:09] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Silverpop/Acoustic export timing out after 10 hours - https://phabricator.wikimedia.org/T347997 (10jgleeson) 05Resolved→03In progress
[12:52:13] <icinga-wm>	 RECOVERY - check_redis on frqueue1004 is OK: OK: REDIS 6.0.16 on 127.0.0.1:6379 has 1 databases (db0) with 14 keys, up 59 days 20 hours - memory use is 8.98M (peak 28.06M, 0.19% of max, fragmentation 1.70%), connected_slaves is 3, donations is 73, jobs is 0, jobs-adyen is 30, jobs-dlocal is 0, jobs-paypal is 2, payments-antifraud is 0, payments-init is 14, pending is 0, recurring is 12403, refund is 0, unsubscribe is 11 https://icinga.wi
[12:52:13] <icinga-wm>	 rg/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis
[12:52:20] <wikibugs>	 10Fundraising-Backlog: Remove the recurring queue threshold alerts - https://phabricator.wikimedia.org/T348533 (10Jgreen) I raised the alert thresholds to 15K/warning, 20K/critical for now.
[13:00:27] <jgleeson>	 ty Jeff_Green 
[13:03:38] <Jeff_Green>	 jgleeson: are you able to ack alerts for anything in Icinga? I don't see anything relevant for that particular alert (both fr-tech and fr-tech-ops are listed as contact_groups)
[13:04:21] <jgleeson>	 I'll check
[13:09:43] <jgleeson>	 Jeff_Green: I just tried to add a service comment to https://icinga.wikimedia.org/cgi-bin/icinga/cmd.cgi?cmd_typ=3&host=frpm1002&service=check_timesync and I got not authorized 
[13:10:48] <Jeff_Green>	 ok
[13:13:05] <jgleeson>	 looks like this version of icinga doesn't have a place to view permissions
[13:13:35] <Jeff_Green>	 yeah, I'm trying to figure out whether this is something that's in the UI vs in config files
[13:13:42] <jgleeson>	 I just asked chatgpt https://chat.openai.com/share/bd692031-4a3f-4964-af05-1bb953e38525
[13:13:49] <Jeff_Green>	 we only deal with a very specific part of the config
[13:14:21] <Jeff_Green>	 ha! let's see if it hallucinates some features or conflates it with some other software
[13:14:31] <jgleeson>	 lol
[13:16:07] <ejegg>	 hi fr-tech
[13:16:12] <jgleeson>	 hey ejegg 
[13:16:17] <Jeff_Green>	 hey ejegg
[13:16:42] <ejegg>	 oh hey, it's more recurring queue alerts
[13:16:51] <Jeff_Green>	 lol yes
[13:17:15] <ejegg>	 so instead of just raising and raising the thresholds, we could do this: T240581
[13:17:16] <stashbot>	 T240581: Send recurring payment notifications to the donations queue - https://phabricator.wikimedia.org/T240581
[13:17:44] <ejegg>	 pretty small task, I think
[13:17:50] <ejegg>	 and should maybe help with deadlocks too
[13:20:08] <jgleeson>	 hmm ejegg without looking at the code, does the donation QC just need to know how to update the contribution_recur row for that to work?
[13:20:51] <ejegg>	 jgleeson: i forget exactly what it was that's missing from the donation one
[13:21:01] <ejegg>	 maybe some normalization for some paypal ipns?
[13:21:19] <ejegg>	 i think it was on the normalization side
[13:21:25] <jgleeson>	 gotcha
[13:21:36] <jgleeson>	 yeah that sounds like a good one to pull in.
[13:21:48] <ejegg>	 would be nice to add that to the ticket when we figure it out, if we're not doing that right now
[13:22:03] <ejegg>	 i'll try to jog my memory
[13:27:54] <wikibugs>	 (03PS6) 10Ejegg: Drop WmfFramework::debugLog [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/961172 (https://phabricator.wikimedia.org/T347412)
[13:28:17] <wikibugs>	 (03CR) 10Ejegg: "Thanks for catching that, Damilare! Fixed in PS6" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/961172 (https://phabricator.wikimedia.org/T347412) (owner: 10Ejegg)
[13:31:20] <damilare>	 np
[13:35:12] <ejegg>	 grr, window manager is very crashy lately.
[13:35:19] <ejegg>	 well, like 2-3 crashes a week
[13:35:33] <ejegg>	 but disruptive to lose all the window state
[13:36:25] <ejegg>	 hmm, so I wonder how this one passed the main test build but failed gate-and-submit (with what looks like a legit test fail) https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/959796
[13:37:14] <ejegg>	 i can see the line in the first test output where it passes that same test:
[13:37:17] <ejegg>	 11:44:58 Test that we correctly calculate the number of active recurrings and the latest ID ... ok
[13:37:33] <ejegg>	 but in gate-and-submit:
[13:37:34] <ejegg>	 15:50:08 Test that we correctly calculate the number of active recurrings and the latest ID ... FAIL
[13:37:40] <ejegg>	 could it be time-related?
[13:37:51] <jgleeson>	 and space
[13:37:54] <ejegg>	 lol
[13:38:02] <jgleeson>	 ¯\_(ツ)_/¯
[13:38:20] <ejegg>	 the server and the planet it's on have moved quite a distance since the last test
[13:38:36] <jgleeson>	 ha
[13:51:54] <ejegg>	 passes each time on my box, and I can't see anything that should be time-dependent in the code
[13:52:05] <ejegg>	 ah, lemme rebase over eileen's patch
[13:52:14] <ejegg>	 that stops forcing that index
[13:53:15] <ejegg>	 still passes
[13:53:24] <wikibugs>	 (03PS2) 10Ejegg: Include pending in has_active_recurring [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/959796 (https://phabricator.wikimedia.org/T346819)
[13:53:46] <ejegg>	 let's see if gerrit likes it better rebased
[13:53:57] <damilare>	 hey fr-tech, bumping this up as we need to merge this in, in order to progress with the fundraiseup pilot: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/959346
[13:54:15] <ejegg>	 thanks for the reminder damilare 
[13:54:34] <damilare>	 np
[13:56:47] <ejegg>	 aha, there IS a way to search your command history in irssi (like ctrl-r in bash)
[13:56:56] <ejegg>	 it just needs an add-on script:
[13:57:26] <ejegg>	 https://scripts.irssi.org/scripts/history_search.pl
[14:00:56] <ejegg>	 script looks legit
[14:04:48] <ejegg>	 hmm, seems to not quite function as expected
[14:06:35] <wikibugs>	 (03PS1) 10Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/964920
[14:06:40] <wikibugs>	 (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/964920 (owner: 10Ejegg)
[14:07:19] <wikibugs>	 (03Merged) 10jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/964920 (owner: 10Ejegg)
[14:10:43] <ejegg>	 oops, I just realized that table whose join I switch is actually not used at all in the select!
[14:10:52] <ejegg>	 It was only there for the where conditions
[14:11:00] <ejegg>	 and now that it's a left join, those don't matter
[14:11:14] <ejegg>	 follow-up cleanup patch incoming
[14:15:49] <ejegg>	 ah, i have an idea to make it more precise, keeping those existing where conditions with an OR start_date is in the future
[14:32:22] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Successful Adyen transactions not making it into CiviCRM - https://phabricator.wikimedia.org/T345736 (10AKanji-WMF) 05Resolved→03Open Based on a call with DR this morning, seems like this is still an issue - should I open a new task?
[14:38:36] <wikibugs>	 10Fundraising-Backlog, 10Wikipedia-iOS-App-Backlog, 10iOS Release FY2023-24: As a donor I want to be able to make an easy and convenient donation via Apple Pay in the Wikipedia app - https://phabricator.wikimedia.org/T288285 (10Tsevener)
[14:40:45] <ejegg>	 hmm, not quite passing
[14:42:38] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Successful Adyen transactions not making it into CiviCRM - https://phabricator.wikimedia.org/T345736 (10Ejegg) >>! In T345736#9239095, @AKanji-WMF wrote: > Based on a call with DR this morning, seems like this is still an issue - should I open a new task?...
[14:42:52] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Successful Adyen transactions not making it into CiviCRM - https://phabricator.wikimedia.org/T345736 (10Ejegg) Did DR give any specific transaction IDs?
[14:43:35] <ejegg>	 well, let me put that attempt at the tools cleanup up for someone else to help spot the error
[14:47:57] <wikibugs>	 (03PS1) 10Ejegg: Tighten which recurrings we count as active [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/964924 (https://phabricator.wikimedia.org/T346819)
[14:48:31] <ejegg>	 so is that just a bad test? Or bad code? ^^^
[14:49:05] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Tighten which recurrings we count as active [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/964924 (https://phabricator.wikimedia.org/T346819) (owner: 10Ejegg)
[14:59:13] <jgleeson>	 ejegg: oh rly
[14:59:33] <jgleeson>	 review landmines
[14:59:40] <ejegg>	 well, i'm not sure how much those conditions really mattered
[14:59:57] <ejegg>	 if DR is refunding all the donations in a recurring they're probably cancelling it too
[15:00:14] <ejegg>	 and we don't actually offer recurring endowment donations yet
[15:00:19] <jgleeson>	 I thought it only pulled in pendings
[15:01:29] <ejegg>	 so it got rid of the restriction that required the recurring to be associated with a donation that was a) completed and b) not endowment and c) greater than 0 amount
[15:02:01] <ejegg>	 c) is pretty obsolete since we don't make negative contribution rows for refunds any more
[15:02:19] <ejegg>	 and a) and b) I just mentioned above
[15:02:38] <ejegg>	 so functionally, yeah, it SHOULD only pull in pending recurrings
[15:03:14] <jgleeson>	 I'm struggling today with the random noises. I've been sat here trying to ignore a faint ringing sound thinking it was the new tinnitus feature my ears activated this week but then I closed the bedroom window and the noise went away...
[15:03:15] <ejegg>	 worst case, removing those restrictions will list a few extra ppl as having active recurrings when all donations have been cancelled
[15:03:38] <ejegg>	 oh man, tinnitus. sorry to hear that
[15:03:45] <ejegg>	 or rather, sorry that YOU'RE hearing that
[15:03:46] <jgleeson>	 good pun
[15:03:52] <jgleeson>	 ha
[15:04:31] <ejegg>	 ehh, that new patch feels needlessly complicated anyway
[15:04:42] <jgleeson>	 tbh it could be construction work in the distance but last night I kept hearing faint vibrations and it was drying me nuts. I walked all around the house and it wouldn't go so I figured it's early signs
[15:04:47] <ejegg>	 I'll change to just removing the join completely
[15:04:50] <jgleeson>	 ok back to work stuff
[15:06:59] <ejegg>	 oh silly me
[15:07:03] <ejegg>	 we DO use that table
[15:07:11] <jgleeson>	 I'm gonna go back down the matching gifts check import rabbit hole. 
[15:07:13] <ejegg>	 we just don't prefix all the columns in the select
[15:07:21] <ejegg>	 OK, it's fine, I'mma deploy as-is
[15:08:35] <ejegg>	 side note: it's really annoying that nosetests doesn't print the actual value on an expectation failure
[15:08:59] <jgleeson>	 hmm
[15:09:03] <ejegg>	 oh right, that's because I'm just using built-in assert
[15:09:12] <jgleeson>	 no -vvvv...
[15:09:47] <ejegg>	 derp, there are totally nose functions to do assertions
[15:10:25] <ejegg>	 https://nose.readthedocs.io/en/latest/testing_tools.html
[15:14:51] <wikibugs>	 10Fundraising-Backlog: Upgrade recur amount when donor upgrades their recurring in Fundraiseup - https://phabricator.wikimedia.org/T348544 (10Damilare)
[15:21:17] <jgleeson>	 ejegg: databases... https://phabricator.wikimedia.org/F38190033
[15:21:36] <ejegg>	 lolyup
[15:22:15] <ejegg>	 algorithms on the other hand...
[15:22:23] <jgleeson>	 exactly
[15:22:51] <jgleeson>	 I still have to google binary tree every time
[15:23:27] <ejegg>	 i pine for the days of optimizing the big-O to a factor of log(n)
[15:23:38] <jgleeson>	 yes indeed
[15:24:11] <jgleeson>	 who was the guy who invented the big-0 name, I used to feel proud that I remembered that
[15:24:49] <jgleeson>	 The concept of "Big O" notation in computer science and mathematics was introduced by Paul Bachmann in 1894, but the term "Big O" and its popular usage were made prominent by Donald Knuth in the 1970s in his seminal work "The Art of Computer Programming."
[15:25:46] <ejegg>	 sadly most of those algo-intensive jobs are game engines, weapons, and big tech surveillance capitalists these days
[15:26:10] <ejegg>	 or high frequency traders
[15:26:39] <jgleeson>	 I imagine we'd never need or see anything related to sorting algos this high up at 30 thousand feet in PHP 
[15:26:48] <ejegg>	 ahh, Paul Bachmann? I should read up
[15:26:54] <jgleeson>	 I think usort lets you play with 'em 
[15:27:22] <ejegg>	 just sending in your own comparison fn, right?
[15:27:41] <ejegg>	 not actually saying to go depth-first or whatever
[15:27:51] <jgleeson>	 no idea tbh
[15:28:02] <jgleeson>	 probably 
[15:30:01] <jgleeson>	 yeah it's entirely different
[15:30:11] <ejegg>	 offtopic, repeating these music recommendations for the next 3 yrs: https://1001albumsgenerator.com/ (one album a day for 1001 days, not including weekends)
[15:30:40] <jgleeson>	 that's a pretty big commitment 
[15:31:03] <ejegg>	 hehe, if you miss some days there's a history tab to go back and find em
[15:31:21] <ejegg>	 I'm listening through a few missed days now
[15:31:52] <jgleeson>	 php has a uksort()... I presume that results in an orderly sort full of polite queue items 
[15:32:32] <jgleeson>	 although I do wonder where the myth that we're the only country that queue orderly came from 
[15:33:37] <jgleeson>	 arghh brain. back to work
[15:51:59] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10FR-dlocal, 10Patch-For-Review: dlocal refund and script to batch refund csv files - https://phabricator.wikimedia.org/T334311 (10jgleeson)
[15:58:36] <wikibugs>	 10Fundraising-Backlog: All Ingenico/Worldline active tokens will be migrated to Adyen - https://phabricator.wikimedia.org/T344645 (10EMartin) @XenoRyet @AKanji-WMF Adyen has now imported all of the Ingenico tokens to their platform.  As above, I would like to confirm that everything we need to bill in the next 3...
[16:13:44] <wikibugs>	 10Wikimedia-Fundraising-Banners, 10Wikipedia-Android-App-Backlog (Android Release - FY2023-24), 10Wikipedia-iOS-App-Backlog (Design), 10iOS Release FY2023-24: JSON for new app fundraising banners (multilingual, Italy and Netherlands) - https://phabricator.wikimedia.org/T348278 (10JTannerWMF) p:05Triage→...
[16:24:11] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Add CSP for Fundraiseup on DonateWiki - https://phabricator.wikimedia.org/T345379 (10sbassett) @MSuijkerbuijk_WMF - I've left a comment on the change set: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/957983/4#messag...
[16:30:32] <wikibugs>	 (03CR) 10Jgleeson: [C: 04-1] "Nice job, this works... but then the script tells you it never. It lies! Jokes aside, I think the response mapper needs a tweak, as it's c" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[16:34:19] <wikibugs>	 10Fundraising-Backlog: Remove the recurring queue threshold alerts - https://phabricator.wikimedia.org/T348533 (10jgleeson) @Ejegg pointed out that we should probably do this instead {T240581}
[16:52:30] <wikibugs>	 (03CR) 10Damilare Adedoyin: [C: 03+2] Add fetch customer info graphql for venmo [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963268 (https://phabricator.wikimedia.org/T347771) (owner: 10Wfan)
[16:53:01] <wikibugs>	 (03Merged) 10jenkins-bot: Add fetch customer info graphql for venmo [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963268 (https://phabricator.wikimedia.org/T347771) (owner: 10Wfan)
[16:53:09] <wfan>	 Thanks Dami. I will add test for decline patch
[16:53:16] <damilare>	 np
[16:54:37] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Individual Engage File Import Duplicates - https://phabricator.wikimedia.org/T347362 (10MDemosWMF) Thanks @Damilare ! I wonder why this is happening all of a sudden when we have never had iss...
[17:11:54] <wikibugs>	 (03PS1) 10Ejegg: Use nose assertions in tests [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/964951
[17:14:18] <wikibugs>	 10Fundraising-Backlog, 10SRE, 10SRE Observability: Simplify and fix icinga fr-tech user configuration - https://phabricator.wikimedia.org/T348559 (10Jgreen)
[17:15:27] <ejegg>	 Uni thesis on chatbots to get ppl up and out of their chairs: https://www.humane-ai.eu/project/evidence-based-chatbot-interaction-aimed-at-reducing-sedentary-behavior/
[17:15:54] <ejegg>	 I figure an easy IRC script could remind us every half hour to get up and move
[17:16:19] <ejegg>	 maybe using a different little factoid each time
[17:16:45] <ejegg>	 k, now focusing on code review
[17:18:56] <dwisehaupt>	 all meetings 25 min max. :)
[17:19:57] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Add CSP for Fundraiseup on DonateWiki - https://phabricator.wikimedia.org/T345379 (10MSuijkerbuijk_WMF) @sbassett Thanks for your prompt answer, I have a follow up question:  when does the production config deploy happen? Is this als...
[17:20:09] <wikibugs>	 10Fundraising-Backlog: Remove the recurring queue threshold alerts - https://phabricator.wikimedia.org/T348533 (10Ejegg) > @Elliott Eggleston what was the reason we set up this alert again? The main reason for the queue alerts is to remind us if we turn the consumers off and forget to turn them on again.
[17:20:18] <wikibugs>	 (03PS3) 10Cstone: Add dlocal refund script [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311)
[17:20:56] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Add dlocal refund script [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[17:21:45] <wikibugs>	 (03PS4) 10Cstone: Add dlocal refund script [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311)
[17:28:22] <wikibugs>	 (03CR) 10Damilare Adedoyin: [C: 03+2] "Thanks for the work on this, I was able to see the payments log entries in the mediawiki logfile. LGTM!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/961172 (https://phabricator.wikimedia.org/T347412) (owner: 10Ejegg)
[17:30:14] <wikibugs>	 (03Merged) 10jenkins-bot: Drop WmfFramework::debugLog [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/961172 (https://phabricator.wikimedia.org/T347412) (owner: 10Ejegg)
[17:36:21] <wikibugs>	 (03CR) 10Ejegg: "Seeming really good so far! I still need to smoke test. For now, a few observations inline." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620) (owner: 10Damilare Adedoyin)
[17:36:54] <wikibugs>	 10Fundraising-Backlog, 10SRE, 10SRE Observability: Simplify and fix icinga fr-tech user configuration - https://phabricator.wikimedia.org/T348559 (10Jgreen)
[17:40:16] <ejegg>	 thanks for the review on that logging patch, damilare !
[17:40:26] <damilare>	 np
[17:40:49] <wikibugs>	 (03CR) 10Cstone: Add dlocal refund script (031 comment) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[17:42:22] <ejegg>	 today I learned about `git whatchanged --stat`
[17:42:41] <ejegg>	 for a quick list of changed files by amount
[17:43:17] <ejegg>	 there must be a similar set of options to 'git log' that achieve that though, right?
[17:44:44] <ejegg>	 oh, 'git log --raw' has the files but is a bit messier
[18:01:25] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Add CSP for Fundraiseup on DonateWiki - https://phabricator.wikimedia.org/T345379 (10Ejegg) @MSuijkerbuijk_WMF it sounds like @sbassett wants @greg and @Lgruwell-WMF to acknowledge that the security department has assessed FundraiseU...
[18:04:02] <jgleeson>	 ejegg: is that like git show --name-only
[18:04:06] <jgleeson>	 lemme try it
[18:04:31] <ejegg>	 oh yeah, git show is a thing too
[18:04:56] <jgleeson>	 ooo pretty colours
[18:06:19] <ejegg>	 lol fundraising_code_update just asked me to press the any key:
[18:06:19] <ejegg>	 Enter 'any' to continue, or <q> to quit. any
[18:07:01] <jgleeson>	 ha
[18:11:17] <ejegg>	 !log fundraising python tools upgraded from 2e19cd39 to 0c17296c
[18:11:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[18:13:35] <wikibugs>	 (03Abandoned) 10Ejegg: Tighten which recurrings we count as active [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/964924 (https://phabricator.wikimedia.org/T346819) (owner: 10Ejegg)
[18:13:56] <wikibugs>	 (03PS31) 10Damilare Adedoyin: Create Fundraiseup Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620)
[18:14:58] <cstone>	 jgleeson: thanks for that earlier review for the refunds, looks like both statuses could come back I added the success one
[18:15:01] <wikibugs>	 (03PS32) 10Damilare Adedoyin: Create Fundraiseup Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620)
[18:15:23] <wikibugs>	 (03CR) 10Damilare Adedoyin: "Thanks ejegg for the CR, made the changes." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620) (owner: 10Damilare Adedoyin)
[18:15:51] <jgleeson>	 just revisting it cstone thanks
[18:20:52] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Post-payment monthly convert donors need appropriate flag in Civi - https://phabricator.wikimedia.org/T346819 (10Ejegg) @KHaggard and @krobinson We just deployed the code change to export pending monthly convert donors as having an a...
[18:23:09] <ejegg>	 speaking of that recurring->donations queue ticket, we have gotten a dozen deadlock failmails from the recurring queue consumer today
[18:33:37] <jgleeson>	 argh
[18:43:03] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Post-payment monthly convert donors need appropriate flag in Civi - https://phabricator.wikimedia.org/T346819 (10KHaggard) Thanks @Ejegg . I'll leave it up to @krobinson and her team to determine the urgency around updating this now vs. 24 days.
[18:47:04] <wikibugs>	 (03PS6) 10Wfan: Remove Astropay from DI [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/960675 (https://phabricator.wikimedia.org/T326594)
[18:47:45] <wikibugs>	 (03PS33) 10Ejegg: Create Fundraiseup Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620) (owner: 10Damilare Adedoyin)
[18:48:01] <wikibugs>	 (03CR) 10Ejegg: [C: 03+2] "This patch looks good, and running the parser locally then consuming the queues yielded a bunch of correct-seeming donations and recur row" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620) (owner: 10Damilare Adedoyin)
[18:53:42] <damilare>	 thanks for the CR ejegg, I already added the patch to add venmo. Just gonna have to push a patch to update civi composer with the new version
[18:54:01] <ejegg>	 oh i see, wonderful
[19:00:02] <wikibugs>	 (03CR) 10Jgleeson: [C: 04-1] "It still doesn't like it for credit cards. I think you need to add a case for 'SUCCESS' to https://github.com/wikimedia/wikimedia-fundrais" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[19:00:21] <wikibugs>	 (03PS3) 10Wfan: Add api to delete recurring and customer [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606)
[19:00:50] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Add api to delete recurring and customer [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606) (owner: 10Wfan)
[19:03:17] <wikibugs>	 (03PS4) 10Wfan: Add api to delete recurring and customer [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606)
[19:04:34] <wikibugs>	 (03Merged) 10jenkins-bot: Create Fundraiseup Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/959346 (https://phabricator.wikimedia.org/T346620) (owner: 10Damilare Adedoyin)
[19:06:14] <wikibugs>	 10Fundraising-Backlog, 10Epic: India Recurring - Post launch issues epic - https://phabricator.wikimedia.org/T339180 (10XenoRyet)
[19:06:18] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: India Recurring - Handle Wallet disabled status in IPNs - https://phabricator.wikimedia.org/T341300 (10XenoRyet) 05Open→03Resolved
[19:06:34] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Implement Auto-Rescue retryable failure detection - https://phabricator.wikimedia.org/T340163 (10XenoRyet) 05Open→03Resolved
[19:06:37] <wikibugs>	 10Fundraising-Backlog, 10Recurring-Donations, 10Epic: Implement Adyen Auto-Rescue feature to improve donor recurring conversion - https://phabricator.wikimedia.org/T331920 (10XenoRyet)
[19:07:39] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10IP Masking, 10MediaWiki-extensions-CentralNotice, 10MW-1.41-notes (1.41.0-wmf.28; 2023-09-26): Update Fundraising Tech-owned products that may be affected by IP Masking - https://phabricator.wikimedia.org/T326943 (10XenoRyet) 05Open→03Re...
[19:07:47] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Patch-For-Review: Create FundraiseUp Audit module in CiviCRM - https://phabricator.wikimedia.org/T346620 (10XenoRyet)
[19:07:49] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Create SmashPig FundraiseupAudit Class - https://phabricator.wikimedia.org/T346621 (10XenoRyet) 05Open→03Resolved
[19:08:07] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Add fundraiseup to the list of processors in CiviCRM buildkit - https://phabricator.wikimedia.org/T347359 (10XenoRyet) 05Open→03Resolved
[19:08:11] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Patch-For-Review: Create FundraiseUp Audit module in CiviCRM - https://phabricator.wikimedia.org/T346620 (10XenoRyet)
[19:08:18] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾: Add dynamic descriptor for venmo transaction - https://phabricator.wikimedia.org/T347537 (10XenoRyet) 05Open→03Resolved
[19:08:36] <wikibugs>	 10Fundraising-Backlog, 10Recurring-Donations, 10Epic: Implement Adyen Auto-Rescue feature to improve donor recurring conversion - https://phabricator.wikimedia.org/T331920 (10XenoRyet)
[19:08:49] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Implement Auto-Rescue Webhook handling - https://phabricator.wikimedia.org/T340164 (10XenoRyet) 05Open→03Resolved
[19:09:13] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Create Fundraiseup PaymentProcessor in CiviCRM - https://phabricator.wikimedia.org/T346622 (10XenoRyet) 05Open→03Resolved
[19:09:18] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Patch-For-Review: Create FundraiseUp Audit module in CiviCRM - https://phabricator.wikimedia.org/T346620 (10XenoRyet)
[19:09:20] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Establishing gateways for Fundraise Up and Braintree for new donation processing in Civi CRM - https://phabricator.wikimedia.org/T346392 (10XenoRyet)
[19:09:54] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: `coworker` internal maintenance mode - https://phabricator.wikimedia.org/T344395 (10XenoRyet) 05Open→03Resolved
[19:09:58] <wikibugs>	 (03CR) 10Eileen: [C: 03+2] Individual Engage File Import Duplicates [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964569 (https://phabricator.wikimedia.org/T347362) (owner: 10Damilare Adedoyin)
[19:11:04] <wikibugs>	 10Fundraising Sprint Rasberry Pi Beret 👩🏽‍🎨, 10Fundraising Sprint Spacebar Oddity 👾, 10MW-1.41-notes (1.41.0-wmf.28; 2023-09-26), 10Patch-For-Review: Add Fundraiseup JS Script to DonateWiki Head - https://phabricator.wikimedia.org/T346975 (10XenoRyet) 05Open→03Resolved
[19:11:19] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Establishing gateways for Fundraise Up and Braintree for new donation processing in Civi CRM - https://phabricator.wikimedia.org/T346392 (10XenoRyet) 05Open→03Resolved
[19:11:36] <wikibugs>	 10Fundraising-Backlog, 10Wikipedia-iOS-App-Backlog, 10iOS Release FY2023-24: As a donor I want to be able to make an easy and convenient donation via Apple Pay in the Wikipedia app - https://phabricator.wikimedia.org/T288285 (10XenoRyet)
[19:12:00] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog, 10Wikipedia-iOS-App-Backlog, 10MW-1.41-notes (1.41.0-wmf.29; 2023-10-03), and 3 others: fr-tech task: Create getAppPaymentMethods donation API method - https://phabricator.wikimedia.org/T344860 (10XenoRyet) 05Open→03Resolved
[19:12:18] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Post-payment monthly convert donors need appropriate flag in Civi - https://phabricator.wikimedia.org/T346819 (10XenoRyet) 05Open→03Resolved
[19:12:33] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10MW-1.41-notes (1.41.0-wmf.30; 2023-10-10): Venmo donation with no customer info in civi - https://phabricator.wikimedia.org/T345965 (10XenoRyet) 05Open→03Resolved
[19:13:43] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Silverpop/Acoustic export timing out after 10 hours - https://phabricator.wikimedia.org/T347997 (10XenoRyet) 05In progress→03Resolved
[19:15:12] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Smashpig: Explicitly set manualCapture in case Adyen capture settings are not as expected - https://phabricator.wikimedia.org/T345955 (10Ejegg)
[19:15:25] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10FR-Tech-Analytics: Recent Email Sends Have No Clicks - https://phabricator.wikimedia.org/T347406 (10XenoRyet) 05Open→03Resolved
[19:16:14] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10FR-Tech-Analytics: Recent Email Sends Have No Clicks - https://phabricator.wikimedia.org/T347406 (10AKanji-WMF) This is done from our perspective, @nisrael  - let us know if there's anything else.
[19:17:50] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job Status Failed - https://phabricator.wikimedia.org/T347462 (10XenoRyet)
[19:18:17] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job Status Failed - https://phabricator.wikimedia.org/T347965 (10XenoRyet)
[19:18:23] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job Status Failed - https://phabricator.wikimedia.org/T347462 (10XenoRyet)
[19:53:55] <wikibugs>	 (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/964971
[19:54:13] <wikibugs>	 (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/964971 (owner: 10Eileen)
[19:55:21] <eileen>	 damilare: I think a couple of your patches in Dami -https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/964971 - anything to be aware of with deploy
[19:55:47] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices, 10Patch-For-Review: Indian donor charged twice for second installment of recurring donation - https://phabricator.wikimedia.org/T348258 (10AKanji-WMF)
[20:00:26] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising-Backlog: Venmo report miss refund transactions at that day of generate report - https://phabricator.wikimedia.org/T348025 (10XenoRyet) 05Open→03Resolved
[20:09:16] <wikibugs>	 10Fundraising-Backlog: Check the Adyen audit files August 25 to September 15 - https://phabricator.wikimedia.org/T343171 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF
[20:13:50] <damilare>	 thanks eileen, that looks good. The audit won't run yet as the job hasn't been created.
[20:14:43] <wikibugs>	 (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/964576 (owner: 10Eileen)
[20:15:28] <wikibugs>	 (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/964576 (owner: 10Eileen)
[20:15:56] <wikibugs>	 (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/964971 (owner: 10Eileen)
[20:17:15] <eileen>	 !log civicrm upgraded from 4329014b to f2f1e23e
[20:17:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[20:46:37] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Individual Engage File Import Duplicates - https://phabricator.wikimedia.org/T347362 (10Damilare) Hi @MDemosWMF I guess we had unique check numbers for a while before the duplicates started popping up. The fix has...
[20:49:18] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Engage DAF import- duplicate issue - https://phabricator.wikimedia.org/T345773 (10Damilare) @MDemosWMF, I believe this is also related to T347362, which has been fixed and deployed. Please rerun this import and let us know if you still come across the dup...
[20:50:17] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Matching import - duplicate contribution error - https://phabricator.wikimedia.org/T345665 (10Damilare) @MDemosWMF, I believe this is also related to T347362, which has been fixed and deployed. Please rerun this import and let us know if you still come ac...
[20:51:14] <jgleeson>	 damilare: I was just looking at that earlier
[20:51:53] <jgleeson>	 I wasn't sure if you check number fix was related
[20:51:54] <damilare>	 yh looks like it's related to the one I was working on as the check numbers were duplicated across the files.
[20:52:17] <jgleeson>	 hmm
[20:52:46] <jgleeson>	 I didn't get deep into it but I didn't think that additional code you added to add '' would fix the scenario in that ticket
[20:53:14] <jgleeson>	 is something removing the checknumber from a stack?
[20:53:34] <damilare>	 nah we were hashing on the checking number before, so I added other parameters to make it more unique even for conflicting check numbers
[20:54:16] <damilare>	 so instead of hashing just the check no., we are hashing, the name, and the row index together.
[20:54:34] <jgleeson>	 ohhh I see
[20:54:46] <jgleeson>	 I didn't realise that. I just say the ternary check
[20:54:54] <jgleeson>	 ok that makes sense
[20:54:56] <jgleeson>	 thanks!
[20:55:41] <damilare>	 yh but let's see how it goes with the test
[20:55:52] <jgleeson>	 I got to the point of looking for matching gifts tests but I couldn't find 'em
[20:56:09] <wikibugs>	 (03PS5) 10Cstone: Add dlocal refund script [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311)
[20:57:13] <damilare>	 yh I used this test: `Civi\Wmf\ImportTest` that eileen pointed me towards
[20:57:38] <wikibugs>	 10Fundraising Sprint Spacebar Oddity 👾, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Individual Engage File Import Duplicates - https://phabricator.wikimedia.org/T347362 (10MDemosWMF) Thanks @Damilare - I will run the Engage imports tomorrow and see how it goes! The DAF and matching will likely co...
[20:57:52] <wikibugs>	 (03CR) 10Jgleeson: [C: 03+2] "Working great! thanks" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[20:58:04] <eileen>	 I think the short version is that our efforts to ensure uniqueness basically failed - so we just have to rely on people not importing the same s/sheet twice
[20:58:21] <wikibugs>	 (03Merged) 10jenkins-bot: Add dlocal refund script [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/964078 (https://phabricator.wikimedia.org/T334311) (owner: 10Cstone)
[20:59:17] <jgleeson>	 thanks damilare 
[20:59:17] <damilare>	 yes exactly, that'd be a limitation 
[20:59:27] <damilare>	 np
[21:00:54] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Matching import - duplicate contribution error - https://phabricator.wikimedia.org/T345665 (10jgleeson) a:05jgleeson→03None
[21:01:12] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Engage DAF import- duplicate issue - https://phabricator.wikimedia.org/T345773 (10jgleeson) a:05jgleeson→03None
[21:01:59] <jgleeson>	 I forgot what it was like not being an agent of chaos
[21:18:50] <wikibugs>	 (03PS6) 10Eileen: Remove temporary mailing update, replace with job to run through. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/963401 (https://phabricator.wikimedia.org/T346194)
[21:18:52] <wikibugs>	 (03PS2) 10Eileen: Add step to delete mailing_job rows once orphaned [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964608 (https://phabricator.wikimedia.org/T335070)
[21:18:54] <wikibugs>	 (03PS3) 10Eileen: Insert step (with special before magic) to remove old queue items [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964621 (https://phabricator.wikimedia.org/T335070)
[21:18:56] <wikibugs>	 (03PS1) 10Eileen: Update recur update to fix mis-type recurring ones. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964982 (https://phabricator.wikimedia.org/T345919)
[21:21:22] <jgleeson>	 catch you all tomorrow
[21:42:07] <wikibugs>	 (03PS4) 10Eileen: Insert step (with special before magic) to remove old queue items [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964621 (https://phabricator.wikimedia.org/T335070)
[22:33:47] <wikibugs>	 (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/964995
[22:36:35] <wikibugs>	 (03CR) 10Cstone: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/964995 (owner: 10Cstone)
[22:37:01] <wikibugs>	 (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/964995 (owner: 10Cstone)
[22:40:10] <cstone>	 !log SmashPig upgraded from a78a91d9 to 211284b9
[22:40:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[22:49:54] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices, 10Patch-For-Review: Indian donor charged twice for second installment of recurring donation - https://phabricator.wikimedia.org/T348258 (10Cstone) @krobinson I just deployed the refund script and ran specifically the donor mentione...
[22:54:27] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices, 10Patch-For-Review: Indian donor charged twice for second installment of recurring donation - https://phabricator.wikimedia.org/T348258 (10KHancock99) Thank you @Cstone.  Kristie is out for the rest of the week but I will confirm t...
[22:56:00] <wikibugs>	 (03Abandoned) 10Eileen: Update recur update to fix mis-type recurring ones. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/964606 (https://phabricator.wikimedia.org/T345919) (owner: 10Eileen)
[23:00:31] <wikibugs>	 10Fundraising Sprint Quill Pencil, 10Fundraising Sprint RadioActivewear, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Deadlock should result in requeueing the message - https://phabricator.wikimedia.org/T118487 (10Ejegg) We're getting another wave of deadlock failmails, mostly in the recurring q...
[23:09:53] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Fundraising Sprint T - Priorities - https://phabricator.wikimedia.org/T348408 (10AKanji-WMF)
[23:09:55] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Handle contact update from Fundraiseup - https://phabricator.wikimedia.org/T347882 (10AKanji-WMF)
[23:09:57] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Frontend Import of Fundraiseup Chargeback - https://phabricator.wikimedia.org/T347881 (10AKanji-WMF)
[23:09:59] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Removing some WMF donor fields - https://phabricator.wikimedia.org/T347724 (10AKanji-WMF)
[23:10:01] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10MW-1.42-notes (1.42.0-wmf.1; 2023-10-17): Implement payment content if no client data return from client side for Venmo - https://phabricator.wikimedia.org/T347771 (10AKanji-WMF)
[23:10:04] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Make sure we don't send thank you emails for Fundraise Up donors - https://phabricator.wikimedia.org/T347616 (10AKanji-WMF)
[23:10:06] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Create FundraiseUp Audit module in CiviCRM - https://phabricator.wikimedia.org/T346620 (10AKanji-WMF)
[23:10:08] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Recurring Gift Coding - https://phabricator.wikimedia.org/T345920 (10AKanji-WMF)
[23:10:10] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Patch-For-Review: Implement deletePaymentMethodFromVault to clean venmo vault - https://phabricator.wikimedia.org/T347606 (10AKanji-WMF)
[23:10:12] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Bug with new Recurring Financial Types - https://phabricator.wikimedia.org/T345919 (10AKanji-WMF)
[23:10:15] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Rationalise our mailing table data & related code so we can safely run civimail without insane  queries - https://phabricator.wikimedia.org/T346194 (10AKanji-WMF)
[23:10:17] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-donorservices: Deduper and legacy merging overwriting issue - https://phabricator.wikimedia.org/T345206 (10AKanji-WMF)
[23:10:19] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Data Axle Project - Re-import to Civi - https://phabricator.wikimedia.org/T341902 (10AKanji-WMF)
[23:10:20] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-tech-data-integrity: Making Gift Coding Changes to gifts in July - https://phabricator.wikimedia.org/T344012 (10AKanji-WMF)
[23:10:23] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: All Ingenico/Worldline active tokens will be migrated to Adyen - https://phabricator.wikimedia.org/T344645 (10AKanji-WMF)
[23:10:25] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10MW-1.41-notes (1.41.0-wmf.30; 2023-10-10): Change transaction descriptor to omit phone number reference - https://phabricator.wikimedia.org/T336779 (10AKanji-WMF)
[23:10:27] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Data Axle project - New fields in Civi - https://phabricator.wikimedia.org/T336890 (10AKanji-WMF)
[23:10:29] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Civi - Increase limit on email sends to 2000 / enable CiviMail - https://phabricator.wikimedia.org/T335296 (10AKanji-WMF)
[23:10:32] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Bad query on civicrm mailing dash board - https://phabricator.wikimedia.org/T335071 (10AKanji-WMF)
[23:10:34] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-dlocal: dlocal refund and script to batch refund csv files - https://phabricator.wikimedia.org/T334311 (10AKanji-WMF)
[23:10:36] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikipedia-iOS-App-Backlog, 10Patch-For-Review, 10iOS Release FY2023-24: fr-tech task: Create appDonate donation API method - https://phabricator.wikimedia.org/T344774 (10AKanji-WMF)
[23:10:39] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Patch-For-Review: Auto-Rescue Audit processing updates - https://phabricator.wikimedia.org/T340159 (10AKanji-WMF)
[23:10:43] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Clean up old TY CiviMail records - https://phabricator.wikimedia.org/T335070 (10AKanji-WMF)
[23:10:47] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Can we speed up deduper query? - https://phabricator.wikimedia.org/T333909 (10AKanji-WMF)
[23:10:51] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-donorservices: Unable to transfer donations from one CID to another after accidental CID merge. - https://phabricator.wikimedia.org/T330539 (10AKanji-WMF)
[23:10:55] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-Smashpig, 10FR-dlocal: Remove old AstroPay code from SmashPig - https://phabricator.wikimedia.org/T326218 (10AKanji-WMF)
[23:10:59] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: Issue with Paypal donations made on GiveLively for events - https://phabricator.wikimedia.org/T295726 (10AKanji-WMF)
[23:11:05] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10MediaWiki-extensions-FundraiserLandingPage, 10donate.wikimedia.org: donate.wikimedia.org landing pages don't have descriptions in search engine results - https://phabricator.wikimedia.org/T254808 (10AKanji-WMF)
[23:11:09] <wikibugs>	 10Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: investigate: Record user agent in contribution_tracking - https://phabricator.wikimedia.org/T182039 (10AKanji-WMF)
[23:11:13] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Individual Engage File Import Duplicates - https://phabricator.wikimedia.org/T347362 (10AKanji-WMF)
[23:11:19] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10FR-dlocal, 10MediaWiki-extensions-DonationInterface, and 2 others: Remove old AstroPay code from DonationInterface - https://phabricator.wikimedia.org/T326594 (10AKanji-WMF)
[23:11:23] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog: AMEX rejected transactions through Ingenico are to be ported to Adyen - https://phabricator.wikimedia.org/T338523 (10AKanji-WMF)
[23:11:27] <wikibugs>	 10Fundraising Sprint This LAN is your LAN , 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations, 10Patch-For-Review: Send recurring payment notifications to the donations queue - https://phabricator.wikimedia.org/T240581 (10AKanji-WMF)
[23:20:06] <wikibugs>	 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices, 10Patch-For-Review: Indian donor charged twice for second installment of recurring donation - https://phabricator.wikimedia.org/T348258 (10Cstone) Okay I ran the script to refund 452 more. I left the most recent charge alone and re...