[00:00:40] (03PS3) 10Eileen: Avoid api call in possible pre-boot function call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177520 [00:00:41] (03CR) 10Ejegg: [C:03+2] Avoid api call in possible pre-boot function call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177520 (owner: 10Eileen) [00:00:44] (03CR) 10Ejegg: [V:03+2 C:03+2] Avoid api call in possible pre-boot function call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177520 (owner: 10Eileen) [00:01:56] (03PS1) 10Eileen: Use Redis save handler [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1184184 [00:03:12] (03Abandoned) 10Eileen: Remove old vendor include [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1184180 (owner: 10Eileen) [00:12:45] (03CR) 10CI reject: [V:04-1] Send verification email for confirm set as primary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178692 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [00:15:50] (03CR) 10CI reject: [V:04-1] Adding email template for email confirmation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184181 (https://phabricator.wikimedia.org/T396521) (owner: 10Wfan) [00:18:14] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [00:33:14] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 1224525 Threads: 5 Questions: 121291727 Slow queries: 637 Opens: 2019 Open tables: 1300 Queries per second avg: 99.052 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [00:35:30] (03PS12) 10Eileen: CiviCRM 6.6 rc [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1175975 [00:35:30] (03PS11) 10Eileen: Update composer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177559 [00:35:30] (03PS11) 10Eileen: Add currency to tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177574 [00:35:31] (03PS14) 10Eileen: Update test mapping to Contribution.note [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178720 [00:35:32] (03PS14) 10Eileen: Update test to reflect upstream class change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178711 [00:35:33] (03PS15) 10Eileen: Do exchange rate conversion in Import hook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178715 [00:35:37] (03PS15) 10Eileen: Do not set address_primary.id [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178710 [00:35:41] (03PS17) 10Eileen: Adjust for upstream format change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177575 [00:35:45] (03PS8) 10Eileen: Update test for the add-multiple-to-group functionality now upstreamed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1179763 [00:35:50] (03PS1) 10Eileen: Try uncommenting better check [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184189 [00:36:37] (03PS1) 10Eileen: Handle Standalone sessions using Redis session handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184190 [00:43:13] (03PS2) 10Eileen: Remove data axle fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184165 (https://phabricator.wikimedia.org/T395961) [00:43:14] (03PS1) 10Eileen: Remove fields from install [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184191 (https://phabricator.wikimedia.org/T395961) [00:45:04] (03CR) 10Ejegg: "This makes it load! But I think we still want to pass $session_max_lifetime through getSessionHandler to the constructor" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184190 (owner: 10Eileen) [00:46:49] (03CR) 10Eileen: "well - I think this just gets us back to where we were on master - with prod & staging not using this code but some of our dev installs st" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184190 (owner: 10Eileen) [00:51:26] (03PS1) 10Eileen: my attempt at beating max_execution_time [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1184192 [00:52:03] (03CR) 10CI reject: [V:04-1] CiviCRM 6.6 rc [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1175975 (owner: 10Eileen) [00:56:12] (03CR) 10CI reject: [V:04-1] Update test mapping to Contribution.note [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178720 (owner: 10Eileen) [00:56:42] (03CR) 10CI reject: [V:04-1] Update test to reflect upstream class change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178711 (owner: 10Eileen) [00:57:12] (03CR) 10CI reject: [V:04-1] Do not set address_primary.id [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178710 (owner: 10Eileen) [00:57:31] (03CR) 10CI reject: [V:04-1] Add currency to tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177574 (owner: 10Eileen) [00:58:18] (03CR) 10CI reject: [V:04-1] Adjust for upstream format change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177575 (owner: 10Eileen) [00:58:20] (03PS6) 10Wfan: instead of update primary email via EPC add email as emailPreference [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) [00:59:14] (03CR) 10CI reject: [V:04-1] Do exchange rate conversion in Import hook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178715 (owner: 10Eileen) [00:59:25] (03CR) 10CI reject: [V:04-1] Update composer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177559 (owner: 10Eileen) [01:06:25] (03CR) 10Eileen: [C:03+2] add log detail for the epc message error [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184183 (owner: 10Wfan) [01:07:19] (03CR) 10Eileen: "actually removing until the upgrade is merged - will put back" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184183 (owner: 10Wfan) [01:13:10] (03CR) 10Ejegg: [V:03+2 C:03+2] CiviCRM 6.6 rc [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1175975 (owner: 10Eileen) [01:13:18] (03CR) 10Ejegg: [V:03+2 C:03+2] Update composer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177559 (owner: 10Eileen) [01:13:26] (03CR) 10Ejegg: [V:03+2 C:03+2] Add currency to tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177574 (owner: 10Eileen) [01:13:46] (03CR) 10Ejegg: [V:03+2 C:03+2] Update test mapping to Contribution.note [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178720 (owner: 10Eileen) [01:14:25] (03CR) 10Ejegg: [V:03+2 C:03+2] Update test to reflect upstream class change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178711 (owner: 10Eileen) [01:15:06] (03CR) 10CI reject: [V:04-1] instead of update primary email via EPC add email as emailPreference [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [01:17:36] (03PS1) 10Eileen: Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184193 [01:17:39] (03CR) 10Ejegg: [C:04-1] "If we save the new email anywhere (even to epc location type) without being primary and they wait to confirm, we run the risk of exporting" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [01:17:45] (03CR) 10CI reject: [V:04-1] Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184193 (owner: 10Eileen) [01:18:17] huh, merge failed? [01:19:24] (03PS1) 10Eileen: Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184194 [01:19:34] (03Abandoned) 10Eileen: Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184193 (owner: 10Eileen) [01:19:48] (03CR) 10Eileen: [C:03+2] Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184194 (owner: 10Eileen) [01:20:18] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184195 [01:20:47] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184195 (owner: 10Eileen) [01:21:56] (03CR) 10Eileen: [V:03+2] Do exchange rate conversion in Import hook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178715 (owner: 10Eileen) [01:22:51] (03CR) 10Ejegg: [C:04-1] instead of update primary email via EPC add email as emailPreference (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [01:23:55] (03CR) 10Ejegg: [V:03+2 C:03+2] Do exchange rate conversion in Import hook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178715 (owner: 10Eileen) [01:25:20] (03CR) 10Ejegg: [V:03+2 C:03+2] Do not set address_primary.id [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178710 (owner: 10Eileen) [01:25:43] (03CR) 10Ejegg: [V:03+2 C:03+2] Adjust for upstream format change [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1177575 (owner: 10Eileen) [01:26:02] (03CR) 10Ejegg: [C:03+2] Update test for the add-multiple-to-group functionality now upstreamed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1179763 (owner: 10Eileen) [01:26:54] eileen: so you want to go with the 'try uncommenting better check' ? [01:27:04] (03CR) 10Eileen: [V:03+2 C:03+2] Composer update [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1184194 (owner: 10Eileen) [01:27:10] or try to upgrade at the point of tests passing, then add the next two if necessary? [01:27:25] hmm, I guess we will need the next two locally at any rate [01:27:48] ejegg: well we ony need them locally if we don't sort redis locally - it works for me locally with the patch [01:28:43] !log config revision changed from 01b64dec to 58dce716 [01:28:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:29:43] hmm - I hit it cli but NOT web UI on prod - upgrade running [01:29:59] will see post upgrade [01:35:44] (03PS1) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184196 [01:37:09] (03PS2) 10Eileen: Update Contact Layout editor to 3.0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1174545 [01:38:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: pending is 2038 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 267.63M (peak 296.50M, 3.41% of max, fragmentation 1.04%), connected_slaves is 3, donations is 735, jobs is 0, jobs-adyen is 246, jobs-dlocal is 0, jobs-paypal is 603, payments-antifraud is 2692, payments-init is 1336, recurring is 20, refund is 4, unsubsc [01:38:14] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [01:39:41] (03CR) 10Ejegg: [C:03+2] "Shiny!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1174545 (owner: 10Eileen) [01:42:41] (03PS2) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184196 [01:42:57] (03CR) 10Ejegg: [C:03+2] "Nice to see some slimming down of those big tables!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184196 (owner: 10Eileen) [01:43:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: pending is 2127 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 268.04M (peak 296.50M, 3.41% of max, fragmentation 1.04%), connected_slaves is 3, donations is 760, jobs is 0, jobs-adyen is 249, jobs-dlocal is 0, jobs-paypal is 623, payments-antifraud is 2793, payments-init is 1380, recurring is 21, refund is 4, unsubsc [01:43:14] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [01:43:51] (03PS3) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184196 [01:43:54] (03CR) 10Eileen: [V:03+2] Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184196 (owner: 10Eileen) [01:44:14] (03PS3) 10Eileen: Update Contact Layout editor to 3.0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1174545 [01:45:10] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184197 [01:45:29] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184197 (owner: 10Eileen) [01:47:49] !log civicrm upgraded from 0cf1ef03 to 60b06928 [01:47:50] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:48:16] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: pending is 2223 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 268.34M (peak 296.50M, 3.42% of max, fragmentation 1.04%), connected_slaves is 3, donations is 786, jobs is 0, jobs-adyen is 251, jobs-dlocal is 0, jobs-paypal is 640, payments-antifraud is 2901, payments-init is 1428, recurring is 21, refund is 4, unsubsc [01:48:16] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [01:51:01] (03CR) 10Ejegg: [C:03+2] Use Redis save handler [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1184184 (owner: 10Eileen) [01:51:59] (03Abandoned) 10Eileen: Use Redis save handler [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1184184 (owner: 10Eileen) [01:53:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: pending is 2320 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 268.70M (peak 296.50M, 3.42% of max, fragmentation 1.04%), connected_slaves is 3, donations is 811, jobs is 0, jobs-adyen is 259, jobs-dlocal is 0, jobs-paypal is 664, payments-antifraud is 3013, payments-init is 1480, recurring is 21, refund is 5, unsubsc [01:53:14] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [01:56:38] (03CR) 10Eileen: [V:03+2] Update Contact Layout editor to 3.0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1174545 (owner: 10Eileen) [01:56:46] oh oops [01:56:56] sorry, thought i'd done that [01:57:28] (03PS3) 10Eileen: Remove data axle fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184165 (https://phabricator.wikimedia.org/T395961) [01:57:34] (03CR) 10Ejegg: [V:03+2 C:03+2] Remove data axle fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184165 (https://phabricator.wikimedia.org/T395961) (owner: 10Eileen) [01:58:14] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [01:58:18] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: payments-init is 1531 1500, pending is 2422 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 269.12M (peak 296.50M, 3.43% of max, fragmentation 1.04%), connected_slaves is 3, donations is 845, jobs is 0, jobs-adyen is 274, jobs-dlocal is 0, jobs-paypal is 683, payments-antifraud is 3127, recurring is 22, refund is 9, un [01:58:18] is 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [01:59:14] (03PS2) 10Eileen: Remove fields from install [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184191 (https://phabricator.wikimedia.org/T395961) [01:59:16] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184198 [01:59:18] (03CR) 10Ejegg: [V:03+2 C:03+2] Remove fields from install [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184191 (https://phabricator.wikimedia.org/T395961) (owner: 10Eileen) [01:59:27] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1184198 (owner: 10Eileen) [02:01:01] !log civicrm upgraded from 60b06928 to a8f49cc4 [02:01:02] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:03:12] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:03:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: payments-init is 1598 1500, pending is 2502 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 269.48M (peak 296.50M, 3.43% of max, fragmentation 1.04%), connected_slaves is 3, donations is 868, jobs is 0, jobs-adyen is 280, jobs-dlocal is 0, jobs-paypal is 705, payments-antifraud is 3250, recurring is 23, refund is 9, un [02:03:14] is 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [02:08:14] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:08:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: payments-init is 1697 1500, pending is 2587 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 269.48M (peak 296.50M, 3.43% of max, fragmentation 1.04%), connected_slaves is 3, donations is 214, jobs is 0, jobs-adyen is 287, jobs-dlocal is 0, jobs-paypal is 718, payments-antifraud is 3406, recurring is 23, refund is 9, un [02:08:14] is 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [02:10:55] (03PS1) 10Ejegg: Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 [02:10:58] eileen: ^^^ [02:13:16] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 1230525 Threads: 5 Questions: 121345259 Slow queries: 637 Opens: 3437 Open tables: 1133 Queries per second avg: 98.612 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:13:18] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: payments-init is 1745 1500, pending is 2671 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 21 keys, up 98 days 9 hours - memory use is 269.63M (peak 296.50M, 3.43% of max, fragmentation 1.04%), connected_slaves is 3, donations is 18, jobs is 0, jobs-adyen is 293, jobs-dlocal is 0, jobs-paypal is 736, payments-antifraud is 3509, recurring is 25, refund is 10, un [02:13:18] is 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [02:15:51] !log config revision changed from 58dce716 to 544c968d [02:15:52] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:18:14] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: pending is 2765 2000 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 19 keys, up 98 days 10 hours - memory use is 261.74M (peak 296.50M, 3.34% of max, fragmentation 1.05%), connected_slaves is 3, donations is 9, jobs is 0, jobs-adyen is 2, jobs-dlocal is 0, jobs-paypal is 10, payments-antifraud is 4, payments-init is 16, recurring is 49, refund is 0, unsubscribe is 0 [02:18:14] icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [02:23:14] RECOVERY - check_redis on frqueue1003 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 16 keys, up 98 days 10 hours - memory use is 259.05M (peak 296.50M, 3.31% of max, fragmentation 1.05%), connected_slaves is 3, donations is 57, jobs is 0, jobs-adyen is 0, jobs-dlocal is 0, jobs-paypal is 10, payments-antifraud is 3, payments-init is 1, pending is 0, recurring is 3, refund is 0, unsubscribe is 0 https://icinga.wikim [02:23:14] cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis [02:26:58] (03CR) 10CI reject: [V:04-1] Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 (owner: 10Ejegg) [02:29:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Rejected transactions still getting recurring queue messages - https://phabricator.wikimedia.org/T403024#11142181 (10Cstone) The recurring message in this situation was a recurring convert, it was for 1.75 which is the suggestion from 5.35 [02:35:11] (03CR) 10Eileen: [C:04-1] "I think we are not doing this one - only the other as a stop--gap" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184189 (owner: 10Eileen) [02:35:58] (03PS2) 10Ejegg: Handle Standalone sessions using Redis session handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184190 (owner: 10Eileen) [02:39:49] 03Fundraising Sprint: Queuesadilla, 06Fundraising-Backlog, 07fr-acoustic, 07fr-current-sprint: July 30th: Removing Data Axle CE fields from Civi - License Expiring - https://phabricator.wikimedia.org/T395961#11142185 (10Eileenmcnaughton) a:03Eileenmcnaughton [02:53:33] (03CR) 10CI reject: [V:04-1] Handle Standalone sessions using Redis session handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184190 (owner: 10Eileen) [02:54:00] (03PS2) 10Eileen: Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 (owner: 10Ejegg) [02:59:17] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Set up Redis as our local session handler - https://phabricator.wikimedia.org/T403570 (10Eileenmcnaughton) 03NEW [03:09:55] (03CR) 10CI reject: [V:04-1] Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 (owner: 10Ejegg) [03:11:37] (03PS3) 10Eileen: Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 (owner: 10Ejegg) [03:17:53] (03CR) 10Cstone: [C:03+2] Update SmashPig to v0.9.7.13 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184199 (owner: 10Ejegg) [03:23:08] (03CR) 10Eileen: [C:03+2] add log detail for the epc message error [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184183 (owner: 10Wfan) [03:35:27] (03PS5) 10Eileen: Hush snooze complaints [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1172516 [03:41:55] (03PS7) 10Eileen: Consolidation of retrieval of existing parent contribution [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) [03:41:55] (03PS4) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [03:52:07] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Omni activity load throwing exception when required file is missing - https://phabricator.wikimedia.org/T395457#11142220 (10Cstone) It's back! [03:52:11] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Omni activity load throwing exception when required file is missing - https://phabricator.wikimedia.org/T395457#11142221 (10Cstone) a:03Cstone [03:58:19] (03CR) 10CI reject: [V:04-1] Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) (owner: 10Eileen) [04:11:35] (03PS5) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [04:28:55] (03CR) 10CI reject: [V:04-1] Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) (owner: 10Eileen) [04:59:43] (03PS6) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [05:06:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Omni activity load throwing exception when required file is missing - https://phabricator.wikimedia.org/T395457#11142252 (10Eileenmcnaughton) I did this on prod - lessee `wmf-cv api4 OmnimailJobProgress.update +w job=omnimail_omniactivity_load +v progres... [05:19:07] (03CR) 10CI reject: [V:04-1] Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) (owner: 10Eileen) [05:30:31] (03PS7) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [06:24:00] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_44) - 10https://gerrit.wikimedia.org/r/1184261 (owner: 10L10n-bot) [08:00:21] 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Customized Communications Preference field is not working correctly in Civi - https://phabricator.wikimedia.org/T146096#11142476 (10RLewis) @Lars - thanks for the ping on this. I don't think i've run into this issue... [08:14:51] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1183841 (owner: 10L10n-bot) [08:37:48] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_44) - 10https://gerrit.wikimedia.org/r/1184261 (owner: 10L10n-bot) [10:28:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Silverpop Build export files job failed due to SQL error - https://phabricator.wikimedia.org/T403604 (10jgleeson) 03NEW [10:28:56] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Silverpop Build export files job failed due to Data Axle SQL error - https://phabricator.wikimedia.org/T403604#11143236 (10jgleeson) [10:29:50] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605 (10ppenloglou) 03NEW [10:34:01] (03PS1) 10Eileen: Handle removed dataaxle fields [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T395961) [10:34:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Silverpop Build export files job failed due to Data Axle SQL error - https://phabricator.wikimedia.org/T403604#11143271 (10jgleeson) Looks like this was impacted by the changes in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1184165 [10:38:57] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11143292 (10krobinson) Thanks @ppenloglou Example: CID 34214637 (country has already been manually corrected back to Canada b... [10:55:36] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM: Splitting Organization Contacts in Civi that are Workplace Giving and Restricted Funds - https://phabricator.wikimedia.org/T364978#11143338 (10RLewis) @Lars and @DBautista-WMF - thanks for working on this. I think it would be... [10:56:23] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: New fields for last donation amount in Civi - https://phabricator.wikimedia.org/T324371#11143339 (10RLewis) sounds good to me [11:03:07] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Process to enter and track multi-year gifts - https://phabricator.wikimedia.org/T114001#11143357 (10RLewis) @AFowler-WMF and @Lars - I think we use pledges now to track the multi year gifts. [11:04:20] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Updating multiple contribution in Civi - can we increase the limit? - https://phabricator.wikimedia.org/T253293#11143360 (10RLewis) @Lars - i think we can close this now. [11:04:51] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Endowment related Dashlet in Civi - https://phabricator.wikimedia.org/T247508#11143361 (10RLewis) @Lars - we can close, thank you. [11:45:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: New Gravy JS Errors (2nd Sept) - https://phabricator.wikimedia.org/T403611 (10jgleeson) 03NEW [12:08:23] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: New Gravy JS Error not triggering our client side error API calls - https://phabricator.wikimedia.org/T403538#11143532 (10jgleeson) [12:08:25] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: New Gravy JS Errors (2nd Sept) - https://phabricator.wikimedia.org/T403611#11143530 (10jgleeson) →14Duplicate dup:03T403538 [12:10:24] (03PS2) 10Jgleeson: WIP: Handle removed dataaxle fields [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T403604) (owner: 10Eileen) [12:56:37] (03PS3) 10Jgleeson: Handle removed dataaxle fields [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T403604) (owner: 10Eileen) [12:56:59] (03PS4) 10Jgleeson: Hotfix: Add default for removed dataaxle field. [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T403604) (owner: 10Eileen) [12:59:59] (03CR) 10Jgleeson: [C:03+2] "self-merging fix to unblock silverpop export" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T403604) (owner: 10Eileen) [13:01:19] (03Merged) 10jenkins-bot: Hotfix: Add default for removed dataaxle field. [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1184496 (https://phabricator.wikimedia.org/T403604) (owner: 10Eileen) [13:05:17] (03PS1) 10Jgleeson: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1184521 [13:06:08] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1184521 (owner: 10Jgleeson) [13:06:39] (03Merged) 10jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1184521 (owner: 10Jgleeson) [13:09:29] !log fundraising-tools upgraded from 284579a9 to 3fba9888 [13:09:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [13:14:32] (03CR) 10Damilare Adedoyin: [C:03+2] "Thanks for all the work on this. LGTM" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1182509 (owner: 10Jgleeson) [13:16:33] thanks damilare ! [13:28:06] 03Fundraising Sprint: Queuesadilla, 06Fundraising-Backlog: Pre-renewal notice to Active Annual recurring donors - https://phabricator.wikimedia.org/T393432#11143923 (10MSuijkerbuijk_WMF) Hi team, do you have timelines for this? Please bear in mind that we need a QA step so I would suggest September 30 as deadl... [13:29:42] oops jgleeson, thanks for handling that! [13:29:50] np! [13:29:55] We should get that removed in acoustic and the import mapping, I guess [13:31:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11143940 (10Lars) [13:31:58] (03Merged) 10jenkins-bot: Refactor: Tidy up GravyGateway. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1182509 (owner: 10Jgleeson) [13:35:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Request to Review Civi Pop-Up Behavior and Search Indicator - https://phabricator.wikimedia.org/T403133#11143977 (10Lars) This is now done with the CiviCRM upgrade. @S... [13:42:03] np [13:42:21] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM: Civi user account deleted when deduping INTO linked contact_id - https://phabricator.wikimedia.org/T400065#11144003 (10Lars) [13:42:47] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM: Civi user account deleted when deduping INTO linked contact_id - https://phabricator.wikimedia.org/T400065#11144006 (10Lars) p:05Triage→03Medium [13:43:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM: Civi user account deleted when deduping INTO linked contact_id - https://phabricator.wikimedia.org/T400065#11144007 (10Lars) With the Civi upgrade, this is now done and tested. @Cstone's tab... [13:46:57] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Endowment related Dashlet in Civi - https://phabricator.wikimedia.org/T247508#11144026 (10Lars) 05Open→03Declined [13:47:12] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Updating multiple contribution in Civi - can we increase the limit? - https://phabricator.wikimedia.org/T253293#11144029 (10Lars) 05Open→03Declined [13:47:43] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Process to enter and track multi-year gifts - https://phabricator.wikimedia.org/T114001#11144031 (10Lars) 05Open→03Declined [13:48:22] 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Customized Communications Preference field is not working correctly in Civi - https://phabricator.wikimedia.org/T146096#11144034 (10Lars) 05Open→03Resolved a:03Lars [13:49:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Request to Review Civi Pop-Up Behavior and Search Indicator - https://phabricator.wikimedia.org/T403133#11144036 (10SHust) @Lars, confirming that it is working, thank... [14:22:11] (03PS1) 10Ejegg: Hide frequency change option for annual [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184539 (https://phabricator.wikimedia.org/T402852) [14:24:08] (03PS1) 10Damilare Adedoyin: Update image in Login screen [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184541 [14:33:12] (03CR) 10Ejegg: "Just taken a quick look, but one suggestion for the i18n" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184541 (owner: 10Damilare Adedoyin) [14:35:47] 03Fundraising Sprint: Queuesadilla, 06Fundraising-Backlog, 10FR-Donor-portal, 10Recurring-Donations: Donor Portal cancel page: Remove 'other' freetext box, ensure save to cancel_reason - https://phabricator.wikimedia.org/T402849#11144151 (10Ejegg) a:03Ejegg [14:44:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11144181 (10Lars) Scoping this out: Roughly 1k donors affected, including ~60 with new recur... [14:51:05] (03PS1) 10Ejegg: Disable cancel submit until reason selected [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 [15:06:24] (03PS2) 10Damilare Adedoyin: Update image in Login screen [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184541 [15:07:08] (03CR) 10Damilare Adedoyin: "Good call, thanks for the CR ejegg." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184541 (owner: 10Damilare Adedoyin) [15:07:25] 03Fundraising Sprint: Queuesadilla, 06Fundraising-Backlog, 07fr-acoustic, 07fr-current-sprint, 13Patch-For-Review: Develop process for tracking & handling SMS consent records - https://phabricator.wikimedia.org/T379702#11144240 (10spatton) hey @Eileenmcnaughton and @AKanji-WMF, could y'all provide a quic... [15:07:49] 06Fundraising-Backlog, 06FR-donorrelations: Civi Merging Screen Feature request - https://phabricator.wikimedia.org/T403626 (10SHust) 03NEW [15:12:25] (03PS2) 10Ejegg: Remove free-text 'other' cancel reason. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 (https://phabricator.wikimedia.org/T402849) [15:12:47] oh there's a test to update for that one [15:15:48] (03CR) 10CI reject: [V:04-1] Remove free-text 'other' cancel reason. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 (https://phabricator.wikimedia.org/T402849) (owner: 10Ejegg) [15:17:20] (03PS3) 10Ejegg: Remove free-text 'other' cancel reason. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 (https://phabricator.wikimedia.org/T402849) [15:17:49] ok, now we need to figure out why the Civi side of that wasn't saving the reasons [15:18:58] oh dang, meeting [15:19:50] (03CR) 10CI reject: [V:04-1] Remove free-text 'other' cancel reason. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 (https://phabricator.wikimedia.org/T402849) (owner: 10Ejegg) [15:38:09] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11144417 (10Jhancock.wm) it didn't save the password change for whatever reason. you should be all set for real this time =) [15:45:54] (03PS4) 10Ejegg: Remove free-text 'other' cancel reason. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184547 (https://phabricator.wikimedia.org/T402849) [15:45:54] (03PS1) 10Ejegg: Fix cancel_reason in message [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184558 (https://phabricator.wikimedia.org/T402849) [15:50:29] (03CR) 10CI reject: [V:04-1] Fix cancel_reason in message [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184558 (https://phabricator.wikimedia.org/T402849) (owner: 10Ejegg) [16:05:36] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi Merging Screen Feature request - https://phabricator.wikimedia.org/T403626#11144536 (10Lars) [16:15:29] (03CR) 10Wfan: "Thanks for the explanation, since we will add any non primary email to suppression list and not get them back if we switch those email bac" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [16:21:35] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 07Transaction-missing-in-CiviCRM: recent Gravy / PayPal donations missing PII in Civi - https://phabricator.wikimedia.org/T402841#11144614 (10MBeat33) Seilo found a card donation from August 14th that did not reach Civi. 8b79cf90-... [16:25:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11144661 (10Ejegg) Even for the one-time donors the trigger will change it back to the last... [16:30:41] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11144715 (10Lars) @Ejegg, That's worse than I thought. Have we dealt with this before and wo... [16:39:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11144756 (10Lars) p:05Triage→03Medium [16:42:54] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review, 07payments-orchestration: Handle null buyer return for paypal in gravy - https://phabricator.wikimedia.org/T402609#11144777 (10Cstone) a:03Cstone [16:55:09] 06Fundraising-Backlog, 06FR-donorrelations: Endowment donations not included in "highest donation" totals? - https://phabricator.wikimedia.org/T403641 (10AMJohnson) 03NEW [16:58:39] 06Fundraising-Backlog, 06FR-donorrelations: Endowment donations not included in "highest donation" totals? - https://phabricator.wikimedia.org/T403641#11144841 (10AMJohnson) [16:58:45] 06Fundraising-Backlog, 06FR-donorrelations: Endowment donations not included in "highest donation" totals? - https://phabricator.wikimedia.org/T403641#11144844 (10AMJohnson) [16:59:18] 10fundraising-tech-ops: Fundraising systems Debian Trixie support - https://phabricator.wikimedia.org/T401847#11144850 (10Jgreen) [17:15:43] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11144888 (10Lars) @Cstone Here are a couple SKs: https://civicrm.wikimedia.org/civicrm/admin... [17:17:05] 10fundraising-tech-ops, 06DC-Ops, 10ops-codfw, 06SRE: SSD firmware update for frbackup2002 - https://phabricator.wikimedia.org/T396649#11144892 (10Jhancock.wm) [17:18:48] 10fundraising-tech-ops: Fundraising access request for spatel - https://phabricator.wikimedia.org/T397535#11144900 (10Dwisehaupt) [17:19:05] 10fundraising-tech-ops: Fundraising access request for spatel - https://phabricator.wikimedia.org/T397535#11144918 (10Dwisehaupt) 05Open→03Resolved Received verification that setup is complete. [17:24:08] (03Abandoned) 10Wfan: WIP tmp overpass civi core error [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1182969 (owner: 10Wfan) [17:35:07] 03Fundraising Sprint: Queuesadilla, 06Fundraising-Backlog: Pre-renewal notice to Active Annual recurring donors - https://phabricator.wikimedia.org/T393432#11144993 (10MSuijkerbuijk_WMF) Content [[ https://docs.google.com/document/d/1x6fgBIopjMUepdjeUfeQn2ZxqRfL9eZWoyuesMyMivA/edit?tab=t.0#heading=h.f7880kycdy... [17:39:07] 06Fundraising-Backlog: 2025 Update - Post Payment Annual Recurring - https://phabricator.wikimedia.org/T403078#11145018 (10Ejegg) @AKanji-WMF Maybe we can pull this in to next sprint? [18:03:51] 06Fundraising-Backlog, 10FR-Donor-portal: Do we want to allow donors to type their own cancel reason in Other - https://phabricator.wikimedia.org/T402343#11145151 (10Ejegg) We've decided we don't want to leave a free-text box, but there are still options in the current Donor Portal list that don't occur in the... [18:05:56] 06Fundraising-Backlog, 10FR-Donor-portal: Align cancel_reason list in donor portal to Civi reasons - https://phabricator.wikimedia.org/T402343#11145159 (10Ejegg) [18:08:36] 10Wikimedia-Fundraising-CiviCRM: Civi - Event Activity Update - https://phabricator.wikimedia.org/T329800#11145173 (10JSimmons) Hi @Lars! Thank you for reaching out about this task. Yes, we are still interested in adding these fields to Civi activities. There are a few updates: The following are no longer neede... [18:08:50] 06Fundraising-Backlog, 10FR-Donor-portal: Align cancel_reason list in donor portal to Civi reasons - https://phabricator.wikimedia.org/T402343#11145175 (10Ejegg) [18:22:10] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145215 (10Jgreen) >>! In T400275#11144417, @Jhancock.wm wrote: > it didn't save the password change for whatever reason. you should be all set for real this time =) All good now... [18:31:09] 06Fundraising-Backlog, 10FR-Donor-portal: Merge donor records after verified email change - https://phabricator.wikimedia.org/T403652 (10Ejegg) 03NEW [18:38:47] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10MediaWiki-extensions-CentralNotice, 06SRE, 06Traffic: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097#11145300 (10Ejegg) Hi @ssingh , I'd be hesitant to start using a different cookie for CentralNotice - we do sometimes ne... [19:00:21] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10MediaWiki-extensions-CentralNotice, 06SRE, 06Traffic: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097#11145362 (10ssingh) >>! In T122097#11145300, @Ejegg wrote: > Hi @ssingh , I'd be hesitant to start using a different coo... [19:01:04] 10fundraising-tech-ops: Fundraising access request for ovasileva - https://phabricator.wikimedia.org/T397537#11145371 (10Dwisehaupt) 05Open→03Resolved Closing this out for now. They haven't logged into metabase yet but we can assist if they have any issues when they do go to login. [19:01:13] 10fundraising-tech-ops: Fundraising access request for jdrewniak - https://phabricator.wikimedia.org/T397536#11145374 (10Dwisehaupt) 05Open→03Resolved Closing this out for now. They haven't logged into metabase yet but we can assist if they have any issues when they do go to login. [19:25:28] (03CR) 10Lars SG: [C:03+2] Update email-amender [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184156 (https://phabricator.wikimedia.org/T401600) (owner: 10Eileen) [19:36:04] (03PS1) 10Umherirrender: Use PHP8 constructor property promotion syntax for dependency injection [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1184582 [19:44:15] (03Merged) 10jenkins-bot: Update email-amender [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184156 (https://phabricator.wikimedia.org/T401600) (owner: 10Eileen) [19:48:53] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10MediaWiki-extensions-CentralNotice, 06SRE, 06Traffic: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097#11145538 (10Ejegg) So as long as we switch CentralNotice to use the new cookie at the same time analytics switches the W... [19:58:37] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1183754 is wanting review [20:01:31] 06Fundraising-Backlog, 10dev-images: Fundraising-dev images should apply our own local php.ini files from a couple folders - https://phabricator.wikimedia.org/T403662 (10Ejegg) 03NEW [20:15:44] (03PS2) 10Ejegg: Fix cancel_reason in message [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1184558 (https://phabricator.wikimedia.org/T402849) [20:17:03] (03CR) 10Jgleeson: [C:04-1] "Looks good! I spotted one block that got pasted twice but other than that it looks like a pretty good refactor job! Thanks" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [20:29:41] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145682 (10Jgreen) [20:43:00] 10Wikimedia-Fundraising-CiviCRM: Civi - Event Activity Update - https://phabricator.wikimedia.org/T329800#11145754 (10Lars) Thanks @JSimmons. I've created the requested activity types. Regarding the event type, I'm wondering if this basically replicates the info that is already inherent in the activity type (in... [20:45:01] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145758 (10Jgreen) @Jhancock.wm sorry, we didn't manage to note which vlan these go in earlier. Could you switch the ports for both hosts into frack-fundraising-codfw? I think I... [20:48:43] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145771 (10Papaul) @Jgreen it is already in that vlan https://netbox.wikimedia.org/dcim/interfaces/44404/edit/ [20:49:45] (03PS8) 10Eileen: Consolidation of retrieval of existing parent contribution [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) [20:50:40] (03CR) 10Eileen: "Ah yeah - I remember that now - I actually wrote what I thought was needed & then realised it was the same as the above & went off & dug i" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [20:50:56] (03PS8) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [20:50:58] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145775 (10Jgreen) >>! In T400275#11145771, @Papaul wrote: > @Jgreen it is already in that vlan > https://netbox.wikimedia.org/dcim/interfaces/44404/edit/ Yeahhhh, typing is har... [20:51:11] thanks jgleeson - I fixed it [20:52:28] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145776 (10Papaul) @Jgreen ok let me take care of it [20:53:49] (03CR) 10Jgleeson: [C:03+2] "LGTM!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [21:10:08] (03PS1) 10Jgleeson: Add extra logging to the pending transaction resolver. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184596 [21:12:46] (03Merged) 10jenkins-bot: Consolidation of retrieval of existing parent contribution [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1183754 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [21:15:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11145845 (10Cstone) I don't think we have dealt with a mass send like this in the past @Lars... [21:32:51] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Endowment donations not included in "highest donation" totals? - https://phabricator.wikimedia.org/T403641#11145877 (10Lars) a:03Lars [21:35:04] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi Merging Screen Feature request - https://phabricator.wikimedia.org/T403626#11145883 (10Lars) [21:40:38] 10fundraising-tech-ops: Fundraising systems Debian Trixie support - https://phabricator.wikimedia.org/T401847#11145898 (10Dwisehaupt) [21:42:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Identify and revert wrongly updated country/currency in Civi - https://phabricator.wikimedia.org/T403605#11145900 (10Cstone) Found it, here is where it's happened in the past, looks like they did s... [21:53:15] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: "Engage TYL Pull: WMF Individual and Org Gifts" Searchkit isn't working after Civi upgrade - https://phabricator.wikimedia.org/T403672 (10Lars) 03NEW [21:54:02] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145932 (10Jgreen) Hosts are up and talking to the puppetservers, all good thank you! [21:57:35] 10fundraising-tech-ops: frmx2002.frack.codfw.wmnet final setup - https://phabricator.wikimedia.org/T403673 (10Jgreen) 03NEW [21:59:03] 10fundraising-tech-ops: frdata2002.frack.codfw.wmnet final setup - https://phabricator.wikimedia.org/T403674 (10Jgreen) 03NEW [22:11:03] 10fundraising-tech-ops, 06DC-Ops, 06SRE: Q1:rack/setup/install frdata2002 & frmx2002 - https://phabricator.wikimedia.org/T400275#11145970 (10Papaul) 05Open→03Resolved [22:21:37] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: "Engage TYL Pull: WMF Individual and Org Gifts" Searchkit isn't working after Civi upgrade - https://phabricator.wikimedia.org/T403672#11146003 (10MDemosWMF) Thanks @Lars we will use that workaround in the meantime to continue to se... [22:28:01] 06Fundraising-Backlog, 10fundraising-tech-ops, 06FR-donorrelations: Fundraising Civi access request for Doris Morgan - https://phabricator.wikimedia.org/T403279#11146013 (10Dwisehaupt) [22:28:31] 06Fundraising-Backlog: Receipt Sent - new field in Civi being populated with import date - https://phabricator.wikimedia.org/T403676 (10MDemosWMF) 03NEW [22:28:48] 06Fundraising-Backlog, 10fundraising-tech-ops, 06FR-donorrelations: Fundraising Civi access request for Doris Morgan - https://phabricator.wikimedia.org/T403279#11146026 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt Verified dmorgan's account has been re-activated and logins are successful. Closing. [22:31:27] 06Fundraising-Backlog: Imports not running automatically - https://phabricator.wikimedia.org/T403677 (10MDemosWMF) 03NEW [22:38:19] 06Fundraising-Backlog: Autocomplete specifications - Across Civi - https://phabricator.wikimedia.org/T403678 (10MDemosWMF) 03NEW [22:42:04] 06Fundraising-Backlog: Unable to save searchkit updates - batch data entry - https://phabricator.wikimedia.org/T403679 (10MDemosWMF) 03NEW [23:00:10] (03PS9) 10Eileen: Declare some more fields, translate to preferred ones [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [23:06:07] (03PS7) 10Wfan: Instead of update primary email via EPC [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) [23:10:40] (03PS10) 10Eileen: Declare some more fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184168 (https://phabricator.wikimedia.org/T152249) [23:11:47] this should be reviewable now - although only useful at the margins after larssandergreen got that scheduled job in place [23:11:47] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1172516 [23:24:35] (03CR) 10CI reject: [V:04-1] Instead of update primary email via EPC [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [23:25:15] (03PS9) 10Wfan: Send verification email for confirm set as primary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178692 (https://phabricator.wikimedia.org/T396523) [23:26:01] (03PS8) 10Wfan: Instead of update primary email via EPC [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) [23:46:26] (03CR) 10CI reject: [V:04-1] Send verification email for confirm set as primary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1178692 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [23:46:40] (03CR) 10CI reject: [V:04-1] Instead of update primary email via EPC [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1164497 (https://phabricator.wikimedia.org/T396523) (owner: 10Wfan) [23:53:11] (03PS1) 10Eileen: Remove unused field from test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184610