[12:41:43] (03CR) 10Damilare Adedoyin: [C:03+2] "Thanks for the update all the work on this, LGTM." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1114989 (https://phabricator.wikimedia.org/T385008) (owner: 10Jgleeson) [12:44:08] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 07payments-orchestration: Gravy dLocal Bank Transfers (one-time) - https://phabricator.wikimedia.org/T385012#10516582 (10Damilare) a:03Damilare [12:54:31] thanks damilare ! [12:58:12] (03PS1) 10Jgleeson: Add Cash Payment Provider config to gravy config [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116786 (https://phabricator.wikimedia.org/T385014) [12:58:26] np! [13:00:50] (03PS2) 10Jgleeson: Add Cash Payment Provider config to gravy config [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116786 (https://phabricator.wikimedia.org/T385014) [13:01:15] (03Merged) 10jenkins-bot: Update Gravy supported payment submethods [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1114989 (https://phabricator.wikimedia.org/T385008) (owner: 10Jgleeson) [13:01:17] (03Merged) 10jenkins-bot: Update Gravy transformers [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115030 (https://phabricator.wikimedia.org/T385008) (owner: 10Jgleeson) [13:01:19] (03Merged) 10jenkins-bot: Move PlaceholderFiscalNumber gateway_common [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115353 (https://phabricator.wikimedia.org/T385008) (owner: 10Jgleeson) [13:30:06] (03PS1) 10Jgleeson: WIP: Map Cash/Wallet/QR dLocal payments in Gravy API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116799 (https://phabricator.wikimedia.org/T385014) [13:59:23] (03CR) 10Jforrester: "I think a larger back-port will be needed." [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116231 (https://phabricator.wikimedia.org/T385061) (owner: 10Reedy) [14:06:35] (03PS8) 10Jgleeson: WIP: deal with country-specific field validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1115372 (https://phabricator.wikimedia.org/T385008) [14:06:35] (03PS6) 10Jgleeson: WIP: deal with country-specific personal identifiers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1115487 (https://phabricator.wikimedia.org/T385008) [14:06:35] (03PS3) 10Jgleeson: Add Cash Payment Provider config to gravy config [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116786 (https://phabricator.wikimedia.org/T385014) [14:06:35] (03PS2) 10Jgleeson: WIP: Map Cash/Wallet/QR dLocal payments in Gravy API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116799 (https://phabricator.wikimedia.org/T385014) [14:06:36] (03PS1) 10Jgleeson: Refactor: small tweaks to RequestMapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116805 (https://phabricator.wikimedia.org/T385014) [14:07:21] (03PS1) 10Reedy: Track bundlesizes for modules loaded on page load [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116806 (https://phabricator.wikimedia.org/T373017) [14:08:02] (03CR) 10Reedy: "https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/1116806 does the non phan failure.." [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116231 (https://phabricator.wikimedia.org/T385061) (owner: 10Reedy) [14:08:53] (03PS2) 10Reedy: CNCampaignPager: Protect against some more null columns [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116231 (https://phabricator.wikimedia.org/T385061) [14:10:06] (03CR) 10CI reject: [V:04-1] Track bundlesizes for modules loaded on page load [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116806 (https://phabricator.wikimedia.org/T373017) (owner: 10Reedy) [14:11:16] (03PS1) 10Reedy: Fix Phan error related to message parameters [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116807 [14:11:26] (03PS2) 10Reedy: Track bundlesizes for modules loaded on page load [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116806 (https://phabricator.wikimedia.org/T373017) [14:11:33] (03PS3) 10Reedy: CNCampaignPager: Protect against some more null columns [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116231 (https://phabricator.wikimedia.org/T385061) [14:11:37] (03PS2) 10Reedy: SpecialCentralNoticeBanner: Null-coalescence $par [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116232 (https://phabricator.wikimedia.org/T385335) [14:14:00] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 13Patch-For-Review, 07payments-orchestration: Gravy dLocal Cash Payments (one-time) - https://phabricator.wikimedia.org/T385014#10517227 (10jgleeson) I couldn't find any dLocal cash, wallet, or QR payment mappings in Gravy's API docs. Based on their doc... [14:15:37] (03CR) 10CI reject: [V:04-1] Fix Phan error related to message parameters [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116807 (owner: 10Reedy) [14:26:39] jgleeson: dlocal only allows 1 webhook per instance. If we set the webhook to gr4vy's, we might lose out on notifications for our direct connections. [14:26:56] something to consider before we go live [14:31:53] (03CR) 10Reedy: "https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/1116807/ does the other" [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116231 (https://phabricator.wikimedia.org/T385061) (owner: 10Reedy) [14:32:18] 06Fundraising-Backlog, 07Epic, 07payments-orchestration: Gravy dLocal Webhooks - https://phabricator.wikimedia.org/T385016#10517346 (10Damilare) Note: Dlocal provides allows only one webhook URL for each class of notifications. Setting Gravy's url would mean we miss out on notifications for our direct connec... [14:33:18] ohhh good call out damilare [14:33:28] that is a big commitment [14:33:57] yep, I wonder if we can setup a proxy type hook. [14:34:01] damilare: wanna paste that into the epic? https://phabricator.wikimedia.org/T384967 that feels important [14:34:18] oh I added it to the webhook phab [14:34:31] ah cool ok that makes sense [14:35:10] I finally finished the book Domain Driven Design just now [14:35:27] the last few chapters were hard to get through [14:35:58] I'd strongly recommend that book, at least the first 2/3rds of it [14:37:54] (03PS1) 10Jforrester: Merge remote-tracking branch 'origin/master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116813 (https://phabricator.wikimedia.org/T353420) [14:38:33] (03CR) 10Jforrester: "As an alternative approach, the usual `git merge` way of updating this branch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Centr" [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1116807 (owner: 10Reedy) [14:39:22] Nice! it's always a refreshing feeling when you complete a book. [14:43:05] yeah definitely. that one felt like finishing a marathon. One that I started and dropped out of a few times in the past too, ha! [14:43:46] this is next on the list https://www.amazon.co.uk/Implementation-Patterns-Addison-Wesley-Signature-Kent/dp/0321413091 [14:45:28] and that's only 174 pages which feels less intimidating. thank you kent beck for your short books [14:46:10] lol bring back short reads [14:46:17] indeed! [14:46:58] I'm still on this: https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/ [14:47:50] oh nice [14:48:06] I hope to publish a summary when I'm done [14:48:55] still in chapter 6 of 10 chapters, so it'd be a while. [14:50:40] yeah sharing the wisdom is definitely a good thing. I was thinking of maybe pushing up some explainer patches showing how we could adopt some of the DDD "tactical patterns" on our code such as aggregates and repositories. Not really to propose we use but just a different way to look at what we already have [14:54:25] back soon! [14:56:03] that'd be awesome! [15:20:39] (03Abandoned) 10Raimond Spekking: Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115719 (owner: 10L10n-bot) [15:57:17] (03PS1) 10Damilare Adedoyin: WIP: Integrate dLocal netbanking bank transfer in India. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1116829 (https://phabricator.wikimedia.org/T385012) [16:29:14] 06Fundraising-Backlog, 10FR-Tech-Analytics: Update currency exchange rate data source to include rates for each day of the year - https://phabricator.wikimedia.org/T382359#10517821 (10IAckerman-WMF) Hi @Ejegg -- checking in on this [16:29:17] (03PS1) 10Ejegg: Avoid php notice when handling exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116832 [16:34:04] 06Fundraising-Backlog: Troubleshoot ilse Gerrit access - https://phabricator.wikimedia.org/T385497 (10IAckerman-WMF) 03NEW [16:35:42] (03CR) 10Wfan: [C:03+2] Avoid php notice when handling exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116832 (owner: 10Ejegg) [16:43:45] (03CR) 10Thiemo Kreuz (WMDE): [C:03+1] "Should be safe and compatible with all PHP versions we care about, as far as I'm aware of." [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1115949 (owner: 10Umherirrender) [16:44:19] (03CR) 10Thiemo Kreuz (WMDE): [C:03+2] Add caller to http request in XmlTransactionProcessor [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1115948 (owner: 10Umherirrender) [16:46:34] (03CR) 10Thiemo Kreuz (WMDE): [C:03+1] Replace call_user_func with dynamic function call [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115946 (owner: 10Umherirrender) [16:54:52] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002 - https://phabricator.wikimedia.org/T369565#10517991 (10Papaul) @VRiley-WMF it looks like all those servers are connected to 1G. Can you please move them to 10G ports and update the task i can h... [17:02:22] (03Merged) 10jenkins-bot: Avoid php notice when handling exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116832 (owner: 10Ejegg) [17:08:59] (03CR) 10CI reject: [V:04-1] Add caller to http request in XmlTransactionProcessor [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1115948 (owner: 10Umherirrender) [17:29:26] (03CR) 10Ejegg: "check experimental" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115157 (https://phabricator.wikimedia.org/T384338) (owner: 10Ejegg) [17:33:46] (03CR) 10Ejegg: "check experimental" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115152 (owner: 10Ejegg) [17:38:12] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10518268 (10Dwisehaupt) [17:50:52] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10518366 (10Dwisehaupt) [18:17:39] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10518493 (10Dwisehaupt) [18:30:09] PROBLEM - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [18:35:06] (03PS1) 10Ejegg: Update smash-pig and gravy [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116849 [18:35:13] (03CR) 10Ejegg: [C:03+2] Update smash-pig and gravy [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116849 (owner: 10Ejegg) [18:36:13] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116850 [18:41:10] (03CR) 10Ejegg: [V:03+2 C:03+2] Update smash-pig and gravy [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116849 (owner: 10Ejegg) [18:41:42] (03PS2) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116850 [18:44:34] (03PS1) 10Ejegg: Update composer dependencies for PHP 8.2 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116852 [18:44:54] hmm, those dependencies should probably go out with the deployment, actually [18:51:07] ACKNOWLEDGEMENT - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker Jeff_Green maint week fun https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [18:56:31] (03CR) 10Ejegg: [C:03+2] Update composer dependencies for PHP 8.2 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116852 (owner: 10Ejegg) [18:57:01] (03PS1) 10Damilare Adedoyin: Ensure recurring payments created on failed trustly contributions are cancelled on chargeback [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116854 (https://phabricator.wikimedia.org/T384593) [18:59:28] (03PS1) 10Ejegg: Update libraries for PHP 8.2 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116855 [18:59:44] (03CR) 10Ejegg: [C:03+2] Update libraries for PHP 8.2 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116855 (owner: 10Ejegg) [19:00:08] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116856 [19:02:00] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116850 (owner: 10Ejegg) [19:13:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [19:15:28] (03CR) 10Umherirrender: [V:03+2] Add caller to http request in XmlTransactionProcessor [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1115948 (owner: 10Umherirrender) [19:15:40] git log -p [19:15:46] derp, wrong window [19:18:15] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 1258493 Threads: 7 Questions: 103442173 Slow queries: 640 Opens: 1419 Open tables: 1255 Queries per second avg: 82.195 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [19:20:09] RECOVERY - check_coworker on civi1002 is OK: PROCS OK: 0 processes with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [19:22:17] (03PS5) 10Jgleeson: Add to test coverage for VerifyEmployerFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) [19:22:17] (03PS5) 10Jgleeson: Reduce if/else nesting and in verifyEmployerFile API action. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114472 (https://phabricator.wikimedia.org/T270677) [19:23:30] ok that should get my flaky unit tests working [19:23:43] back later [19:26:13] (03Merged) 10jenkins-bot: Update composer dependencies for PHP 8.2 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116852 (owner: 10Ejegg) [19:26:13] (03Merged) 10jenkins-bot: Update libraries for PHP 8.2 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1116855 (owner: 10Ejegg) [19:26:14] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116850 (owner: 10Ejegg) [19:26:57] (03PS3) 10Ejegg: Bump up minimum PHP and SmashPig versions [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115152 [19:28:13] (03CR) 10CI reject: [V:04-1] Bump up minimum PHP and SmashPig versions [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115152 (owner: 10Ejegg) [19:28:25] (03PS6) 10Jgleeson: Add to test coverage for VerifyEmployerFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) [19:28:26] (03PS6) 10Jgleeson: Reduce if/else nesting and in verifyEmployerFile API action. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114472 (https://phabricator.wikimedia.org/T270677) [19:28:53] always good to spot a typo after setting yourself to afk [19:29:06] * jgleeson|afk is actually afk [19:33:04] (03PS3) 10Jforrester: i18n: Remove 4 messages, now moved to DonationInterface [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1097498 (https://phabricator.wikimedia.org/T376514) (owner: 10Ejegg) [19:33:16] (03CR) 10Jforrester: "Done." [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1097498 (https://phabricator.wikimedia.org/T376514) (owner: 10Ejegg) [19:33:36] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116856 (owner: 10Ejegg) [19:34:55] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1116856 (owner: 10Ejegg) [19:36:01] (03CR) 10CI reject: [V:04-1] i18n: Remove 4 messages, now moved to DonationInterface [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/1097498 (https://phabricator.wikimedia.org/T376514) (owner: 10Ejegg) [19:36:48] !log fundraising civicrm upgraded from 3e566467 to abe0fc61 [19:36:49] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:01:49] (03PS1) 10Ejegg: Fix negative timings in audit logs [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116861 [20:02:15] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738612926 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 4 minutes 16 seconds - replication_delay is 1738612926, memory use is 873.41K (peak 892.36K, 0.07% of max, fragmentation 7.40%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:02:15] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay data is missing - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 81 days 20 hours - memory use is 1.87M (peak 2.07M, 0.08% of max, fragmentation 3.81%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:03:19] PROBLEM - check_redis_donor_prefs on frqueue1003 is CRITICAL: CRITICAL: connected_slaves is 0 1 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 5 minutes 58 seconds - memory use is 991.93K (peak 991.93K, 0.08% of max, fragmentation 7.01%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis_donor_prefs [20:07:11] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613226 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 9 minutes 16 seconds - replication_delay is 1738613226, memory use is 852.91K (peak 892.36K, 0.08% of max, fragmentation 8.09%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:07:11] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613226 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 1 minutes - replication_delay is 1738613226, memory use is 873.43K (peak 932.49K, 0.07% of max, fragmentation 7.51%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:08:19] RECOVERY - check_redis_donor_prefs on frqueue1003 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 10 minutes 58 seconds - memory use is 1.08M (peak 1.10M, 0.09% of max, fragmentation 7.06%), connected_slaves is 3 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis_donor_prefs [20:09:00] (03CR) 10Cstone: [C:03+2] Avoid duplicate createPayment after fetch data [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1102381 (https://phabricator.wikimedia.org/T381935) (owner: 10Wfan) [20:09:08] (03CR) 10Eileen: Add to test coverage for VerifyEmployerFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [20:09:27] (03CR) 10CI reject: [V:04-1] Avoid duplicate createPayment after fetch data [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1102381 (https://phabricator.wikimedia.org/T381935) (owner: 10Wfan) [20:10:56] (03CR) 10Eileen: Add to test coverage for VerifyEmployerFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [20:12:11] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613526 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 14 minutes 16 seconds - replication_delay is 1738613526, memory use is 852.85K (peak 892.36K, 0.08% of max, fragmentation 8.03%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:12:11] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613526 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 6 minutes - replication_delay is 1738613526, memory use is 852.93K (peak 932.49K, 0.07% of max, fragmentation 7.69%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:17:13] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613826 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 19 minutes 16 seconds - replication_delay is 1738613826, memory use is 852.91K (peak 892.36K, 0.07% of max, fragmentation 7.95%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:17:13] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738613826 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 11 minutes - replication_delay is 1738613826, memory use is 852.93K (peak 932.49K, 0.07% of max, fragmentation 7.67%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:19:01] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10518916 (10Dwisehaupt) [20:19:13] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10518917 (10Dwisehaupt) [20:22:13] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614125 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 24 minutes 15 seconds - replication_delay is 1738614125, memory use is 852.91K (peak 912.01K, 0.07% of max, fragmentation 7.89%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:22:13] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614125 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 15 minutes 59 seconds - replication_delay is 1738614125, memory use is 852.93K (peak 932.49K, 0.07% of max, fragmentation 7.66%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:24:39] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for jsimmons - https://phabricator.wikimedia.org/T385351#10518944 (10Dwisehaupt) certificate renewed and sent via email. password sent via sms. [20:27:13] PROBLEM - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614426 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 29 minutes 16 seconds - replication_delay is 1738614426, memory use is 852.91K (peak 912.01K, 0.07% of max, fragmentation 7.82%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:27:13] PROBLEM - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614426 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 21 minutes - replication_delay is 1738614426, memory use is 852.93K (peak 932.49K, 0.07% of max, fragmentation 7.65%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:28:16] looking at frqueue1004 [20:29:13] ACKNOWLEDGEMENT - check_redis on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614426 300 - REDIS 6.0.16 on 127.0.0.1:6379 has 0 databases (), up 29 minutes 16 seconds - replication_delay is 1738614426, memory use is 852.91K (peak 912.01K, 0.07% of max, fragmentation 7.82%) Dwisehaupt checking redis after origin swap https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:29:14] ACKNOWLEDGEMENT - check_redis_donor_prefs on frqueue1004 is CRITICAL: CRITICAL: replication_delay is 1738614426 300 - REDIS 6.0.16 on 127.0.0.1:6380 has 0 databases (), up 21 minutes - replication_delay is 1738614426, memory use is 852.93K (peak 932.49K, 0.07% of max, fragmentation 7.65%) Dwisehaupt checking redis after origin swap https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_p [20:30:04] 06Fundraising-Backlog, 10fundraising-tech-ops: DjangoBannerStats migration from fran1001/bullseye to fran2001/bookworm failure. - https://phabricator.wikimedia.org/T385523 (10Jgreen) 03NEW [20:35:16] (03CR) 10Cstone: Update thank you unsubscribe url to epc (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1115952 (https://phabricator.wikimedia.org/T359445) (owner: 10Wfan) [20:39:28] (03CR) 10Eileen: [C:03+2] Fix negative timings in audit logs [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116861 (owner: 10Ejegg) [20:41:20] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10519002 (10Dwisehaupt) [20:44:45] PROBLEM - Host frqueue1004 is DOWN: PING CRITICAL - Packet loss = 100% [20:45:14] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#10519008 (10AKanji-WMF) Moving to Sprint +1 for discussion [20:45:30] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#10519009 (10Cstone) a:05Cstone→03None [20:47:11] RECOVERY - Host frqueue1004 is UP: PING OK - Packet loss = 0%, RTA = 0.57 ms [20:47:13] RECOVERY - check_redis on frqueue1004 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 17 keys, up 2 minutes 7 seconds - replication_delay is 0, memory use is 258.66M (peak 258.69M, 3.29% of max, fragmentation 1.04%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis [20:47:13] RECOVERY - check_redis_donor_prefs on frqueue1004 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 2 minutes 7 seconds - replication_delay is 1, memory use is 990.16K (peak 990.48K, 0.09% of max, fragmentation 7.61%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_redis_donor_prefs [20:47:39] (03CR) 10Jgleeson: Add to test coverage for VerifyEmployerFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [20:52:29] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog: 2025 Sprint B Priorities - https://phabricator.wikimedia.org/T384974#10519016 (10Damilare) [20:54:01] (03CR) 10Cstone: [C:03+2] Fix negative timings in audit logs [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116861 (owner: 10Ejegg) [21:00:13] PROBLEM - check_mysql on frdb1005 is CRITICAL: Access denied for user nagios@localhost https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_mysql [21:02:58] (03CR) 10Eileen: Add to test coverage for VerifyEmployerFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [21:02:58] (03Merged) 10jenkins-bot: Fix negative timings in audit logs [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116861 (owner: 10Ejegg) [21:05:17] RECOVERY - check_mysql on frdb1005 is OK: Uptime: 238 Threads: 7 Questions: 1701 Slow queries: 0 Opens: 851 Open tables: 686 Queries per second avg: 7.147 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_mysql [21:13:47] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Backfill recurring payments missing Gravy backend processor info. - https://phabricator.wikimedia.org/T385115#10519070 (10XenoRyet) p:05Triage→03High [21:14:09] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Queue messages are logged twice - https://phabricator.wikimedia.org/T384561#10519074 (10XenoRyet) p:05Triage→03Low [21:14:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10MW-1.44-notes (1.44.0-wmf.15; 2025-02-04): CI needs to update some test for php 8.2 and REL1_43 upgrade - https://phabricator.wikimedia.org/T384656#10519075 (10XenoRyet) p:05Triage→03High [21:15:05] 06Fundraising-Backlog: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#10519076 (10AKanji-WMF) [21:16:42] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Automatic merging of organization records with same name - is a problem - https://phabricator.wikimedia.org/T384748#10519081 (10XenoRyet) p:05Triage→03Medium [21:17:15] PROBLEM - check_mysql on frdata2001 is CRITICAL: Access denied for user nagios@localhost https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_mysql [21:19:06] 06Fundraising-Backlog: Troubleshoot ilse Gerrit access - https://phabricator.wikimedia.org/T385497#10519087 (10AKanji-WMF) Hi @IAckerman-WMF - this is not FR-Tech, but we'll figure out who you need to talk to and follow up! [21:22:15] PROBLEM - check_mysql on frdata2001 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_mysql [21:22:15] 06Fundraising-Backlog: Should we switch YAML spacing to 2 spaces? - https://phabricator.wikimedia.org/T385026#10519110 (10AKanji-WMF) Team FR-Tech (present on the call) voted in favour of adopting YAML w 2-space indents! [21:27:42] (03CR) 10Jgleeson: Add to test coverage for VerifyEmployerFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [21:32:15] PROBLEM - check_mysql on frdata2001 is CRITICAL: Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_mysql [21:36:29] 06Fundraising-Backlog: convert unsupported operand types - https://phabricator.wikimedia.org/T385526 (10AnnWF) 03NEW [21:37:15] PROBLEM - check_mysql on frdata2001 is CRITICAL: Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_mysql [21:41:25] (03PS7) 10Jgleeson: Add to test coverage for VerifyEmployerFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) [21:41:26] (03PS7) 10Jgleeson: Reduce if/else nesting and in verifyEmployerFile API action. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114472 (https://phabricator.wikimedia.org/T270677) [21:41:28] 06Fundraising-Backlog, 10FR-Tech-Analytics: Update currency exchange rate data source to include rates for each day of the year - https://phabricator.wikimedia.org/T382359#10519156 (10AKanji-WMF) Hi @IAckerman-WMF - we discussed in our midsprint call today. We could do this (adding duplicate data rows as @Ejeg... [21:42:15] RECOVERY - check_mysql on frdata2001 is OK: Uptime: 1101 Threads: 4 Questions: 1369 Slow queries: 0 Opens: 288 Open tables: 127 Queries per second avg: 1.243 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_mysql [21:44:06] eileen: that test turned out to be a lot of work just to remove a few if statements :) [21:48:10] (03PS1) 10Ejegg: DoNotMerge - diagnostic for LoggingTest [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116875 [21:49:08] (03CR) 10Ejegg: "check experimental" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116875 (owner: 10Ejegg) [21:49:32] (03CR) 10CI reject: [V:04-1] DoNotMerge - diagnostic for LoggingTest [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116875 (owner: 10Ejegg) [21:52:45] (03CR) 10Jgleeson: [C:03+2] Ensure recurring payments created on failed trustly contributions are cancelled on chargeback [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116854 (https://phabricator.wikimedia.org/T384593) (owner: 10Damilare Adedoyin) [21:56:31] wfan: looks like this one could be merged with 2 small phpcs code style tweaks https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/1102381 [21:57:04] (03CR) 10Jgleeson: [C:03+2] "Thanks Peter" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115901 (owner: 10Pcoombe) [22:06:18] (03Merged) 10jenkins-bot: Increase lowest JPY monthly convert amount to be above minimum [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1115901 (owner: 10Pcoombe) [22:13:07] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install fransw1001.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T367801#10519204 (10VRiley-WMF) [22:15:07] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install fransw1001.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T367801#10519206 (10VRiley-WMF) a:03Jgreen Hey Jeff I was able to connect this unit up with 10G. It should now be ready to go for you. Let me know if you need anyt... [22:15:26] (03Merged) 10jenkins-bot: Ensure recurring payments created on failed trustly contributions are cancelled on chargeback [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1116854 (https://phabricator.wikimedia.org/T384593) (owner: 10Damilare Adedoyin) [22:25:50] (03CR) 10Eileen: [C:03+2] Add to test coverage for VerifyEmployerFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [22:32:17] (03CR) 10Eileen: [C:03+2] "Yep - definite readability improvement!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114472 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [22:34:09] (03PS1) 10Ejegg: Move test for bad encoding [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116882 (https://phabricator.wikimedia.org/T384656) [22:35:01] (03CR) 10CI reject: [V:04-1] Move test for bad encoding [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116882 (https://phabricator.wikimedia.org/T384656) (owner: 10Ejegg) [22:36:29] (03CR) 10Ejegg: "Check experimental" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1116882 (https://phabricator.wikimedia.org/T384656) (owner: 10Ejegg) [22:39:29] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for eileen - https://phabricator.wikimedia.org/T385350#10519244 (10Dwisehaupt) certificate renewed and sent via email. password and certificate in home directory on frpm1002 [22:50:49] (03Merged) 10jenkins-bot: Add to test coverage for VerifyEmployerFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114737 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [22:54:41] (03Merged) 10jenkins-bot: Reduce if/else nesting and in verifyEmployerFile API action. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1114472 (https://phabricator.wikimedia.org/T270677) (owner: 10Jgleeson) [23:04:42] (03Abandoned) 10Jdlrobson: Introduce the Donate skin [extensions/FundraiserLandingPage] - 10https://gerrit.wikimedia.org/r/1020408 (https://phabricator.wikimedia.org/T257862) (owner: 10Jdlrobson) [23:04:52] 06Fundraising-Backlog, 10fundraising-tech-ops: Standardize on uft8mb4 charset for fundraising-database role 'mysql' database - https://phabricator.wikimedia.org/T378979#10519314 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt This was taken care of with the upgrade to bookworm and msyql 10.11 on frdb... [23:05:15] 03Fundraising Sprint Bibimbap, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q3 maintenance window - https://phabricator.wikimedia.org/T337584#10519319 (10Dwisehaupt) [23:06:33] 06Fundraising-Backlog, 10fundraising-tech-ops: DjangoBannerStats migration from fran1001/bullseye to fran2001/bookworm failure. - https://phabricator.wikimedia.org/T385523#10519320 (10Jgreen) 05Open→03Resolved a:03Jgreen This fixed it, the assumption being that shift from python 3.9 -> 3.11 ultimatel... [23:09:53] (03PS5) 10Wfan: Avoid duplicate createPayment after fetch data [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1102381 (https://phabricator.wikimedia.org/T381935) [23:10:26] thanks jack! Just updated~ [23:14:43] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for emartin - https://phabricator.wikimedia.org/T385244#10519329 (10EMartin) I am unable to import the new certificate. May I get some assistance? {F58350252} [23:15:30] (03CR) 10Wfan: [C:03+2] "re-apply cstone's +2" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1102381 (https://phabricator.wikimedia.org/T381935) (owner: 10Wfan) [23:15:59] (03Merged) 10jenkins-bot: Avoid duplicate createPayment after fetch data [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1102381 (https://phabricator.wikimedia.org/T381935) (owner: 10Wfan) [23:32:54] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install franio100[1-3] - https://phabricator.wikimedia.org/T367820#10519367 (10VRiley-WMF) a:03Jgreen [23:33:57] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install franio100[1-3] - https://phabricator.wikimedia.org/T367820#10519369 (10VRiley-WMF) All these should be set and ready to go! @Jgreen Let me know if you need anything else! [23:59:26] (03CR) 10Ejegg: [V:03+2 C:03+2] Remove extensions not used on payments-wiki [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1113169 (https://phabricator.wikimedia.org/T384338) (owner: 10Ejegg) [23:59:38] (03CR) 10Ejegg: [V:03+2 C:03+2] Add payments-wiki extensions as submodules [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1113171 (https://phabricator.wikimedia.org/T384338) (owner: 10Ejegg) [23:59:46] (03CR) 10CI reject: [V:04-1] Remove extensions not used on payments-wiki [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1113169 (https://phabricator.wikimedia.org/T384338) (owner: 10Ejegg) [23:59:47] (03CR) 10CI reject: [V:04-1] Add payments-wiki extensions as submodules [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1113171 (https://phabricator.wikimedia.org/T384338) (owner: 10Ejegg)