[11:14:23] Fundraising Sprint Pseudopretzels, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Q2-FY2020-21-cleanup-list: Filtering Fund-to-Fund Transfer gift source out of reporting/search - https://phabricator.wikimedia.org/T258984 (RLewis) @LeanneS yep this works, thanks. [13:00:36] Fundraising-Backlog, fundraising Sprint Q 2020 for real, Patch-For-Review: Latest currency fields in silverpop export are incorrect - https://phabricator.wikimedia.org/T260708 (KHaggard) @Eileenmcnaughton @mepps - Hey all updating this task again - looks like the nightly import is showing tons of bad... [13:08:13] Fundraising-Backlog, fundraising Sprint Q 2020 for real, Patch-For-Review: Latest currency fields in silverpop export are incorrect - https://phabricator.wikimedia.org/T260708 (jgleeson) Ah shoot @KHaggard I'll take a look at the file generated this morning and see if I can track down the problematic... [13:31:05] (PS1) Jgleeson: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/622144 [13:33:55] Fundraising-Backlog, fundraising Sprint Q 2020 for real, Patch-For-Review: Latest currency fields in silverpop export are incorrect - https://phabricator.wikimedia.org/T260708 (jgleeson) Looks like the [[ https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/622144/ | patch ]] to fix the pro... [13:38:34] (CR) Jgleeson: [C: +2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/622144 (owner: Jgleeson) [13:39:09] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/622144 (owner: Jgleeson) [14:52:36] (PS1) Mepps: WIP Started translations for other failed recurring messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622162 [14:52:52] (CR) jerkins-bot: [V: -1] WIP Started translations for other failed recurring messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622162 (owner: Mepps) [14:57:01] hi ejegg! [14:57:26] hi! [14:57:48] jgleeson: thanks for deploying that fix! [15:02:40] Fundraising-Backlog, fundraising Sprint Q 2020 for real, Patch-For-Review: Latest currency fields in silverpop export are incorrect - https://phabricator.wikimedia.org/T260708 (KHaggard) Thanks @jgleeson - I ran the export job you sent over and it the bad data issue seems to be fixed. Pasting in rows... [15:04:00] Fundraising-Backlog, fundraising Sprint Q 2020 for real, Patch-For-Review: Latest currency fields in silverpop export are incorrect - https://phabricator.wikimedia.org/T260708 (jgleeson) Awesome! [15:08:27] does anyone else struggle with distraction between the time you run "vagrant up" and it asks for your password? [15:09:09] ha! yes mepps [15:09:14] and yes mepps [15:09:17] the vagrant struggle [15:09:22] glad it's not just me--phew! [15:16:57] I'm gonna head home my battery is on 12% and I forgot my charger (°ロ°) back later! [16:14:10] Fundraising-Backlog, fundraising-tech-ops: CiviCRM license for Tomas WhiteAntelope - https://phabricator.wikimedia.org/T260745 (Dwisehaupt) a:Jeff_G→Dwisehaupt Hi Evelyn, I'll get started on this. Could you please add Tomas's contact information to the Fundraising Contact list. https://collab.wi... [16:15:48] Fundraising-Backlog: Japanese last name, first name ordering issues - https://phabricator.wikimedia.org/T261144 (DStrine) [16:16:34] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, I18n: Donation form should use the appropriate name order - https://phabricator.wikimedia.org/T259945 (DStrine) I split this off to a different task: T261144 >>! In T259945#6402520, @CDenes_WMF wrote: > Hi All, > > Adding onto this that... [16:26:16] Fundraising-Backlog, fundraising-tech-ops: CiviCRM license for Tomas WhiteAntelope - https://phabricator.wikimedia.org/T260745 (Dwisehaupt) [16:44:39] Fundraising-Backlog, fundraising-tech-ops: CiviCRM license for Tomas WhiteAntelope - https://phabricator.wikimedia.org/T260745 (Dwisehaupt) Approval request sent to Lisa via email. [16:58:45] fundraising-tech-ops, fr-tech-ops-okr: Update staging to match production (php, buster) - https://phabricator.wikimedia.org/T260629 (Dwisehaupt) This will be done in maintenance window today. https://collab.wikimedia.org/wiki/2020-08-24-downtime-maintenance [17:11:39] Wikimedia-Fundraising-Banners: 2020-08-19 en4C pre-test: mobile small batch - https://phabricator.wikimedia.org/T260548 (spatton) Open→Declined [17:11:41] Wikimedia-Fundraising-Banners: QA for en4C pre-test on August 19 - https://phabricator.wikimedia.org/T260541 (spatton) [17:11:44] Wikimedia-Fundraising-Banners: 2020-08-19 en4C pre-test: mobile large batch - https://phabricator.wikimedia.org/T260547 (spatton) Open→Declined [17:11:46] Wikimedia-Fundraising-Banners: QA for en4C pre-test on August 19 - https://phabricator.wikimedia.org/T260541 (spatton) [17:11:48] Wikimedia-Fundraising-Banners: QA for en4C pre-test on August 19 - https://phabricator.wikimedia.org/T260541 (spatton) [17:11:50] Wikimedia-Fundraising-Banners: 2020-08-19 en4C pre-test: desktop small batch - https://phabricator.wikimedia.org/T260546 (spatton) Open→Declined [17:11:52] Wikimedia-Fundraising-Banners: 2020-08-19 en4C pre-test: desktop large batch - https://phabricator.wikimedia.org/T260543 (spatton) Open→Declined [17:11:54] Wikimedia-Fundraising-Banners: QA for en4C pre-test on August 19 - https://phabricator.wikimedia.org/T260541 (spatton) [17:12:08] Wikimedia-Fundraising-Banners: QA for en4C pre-test on August 19 - https://phabricator.wikimedia.org/T260541 (spatton) Open→Declined [17:14:30] Wikimedia-Fundraising-Banners: QA for enUS pre-test on August 26 - https://phabricator.wikimedia.org/T261147 (spatton) [17:15:58] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop large batch - https://phabricator.wikimedia.org/T261148 (spatton) [17:16:29] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop small batch - https://phabricator.wikimedia.org/T261149 (spatton) [17:18:26] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile large batch - https://phabricator.wikimedia.org/T261150 (spatton) [17:19:27] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile small batch - https://phabricator.wikimedia.org/T261151 (spatton) [17:19:43] Wikimedia-Fundraising-Banners: QA for enUS pre-test on August 26 - https://phabricator.wikimedia.org/T261147 (spatton) p:Triage→High [17:19:54] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop large batch - https://phabricator.wikimedia.org/T261148 (spatton) p:Triage→High [17:19:59] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop small batch - https://phabricator.wikimedia.org/T261149 (spatton) p:Triage→High [17:20:01] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile large batch - https://phabricator.wikimedia.org/T261150 (spatton) p:Triage→High [17:20:03] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile small batch - https://phabricator.wikimedia.org/T261151 (spatton) p:Triage→High [17:30:23] so phpstorm has decided today is the day it goes haywire. My battery drained pretty quickly earlier on today and I've been noticing file editor windows closing on their own throughout the day which I just put down to weirdness. turns out my phpstorm is stuck in an indexing loop which may have been started in me disabling read permissions on the project directory (magically) but after trying fix [17:30:25] that and watching it keep restarting the index I've decided to give up and install the latest version to see if that fixes it. It's so frustrating as it's making stuff run super slow :( [17:30:52] oh man and now the newest version is reindexing [17:32:00] I've been getting weird permission errors on the civicrm project for the last few weeks. The first instance was git telling me I couldn't check other branches out as I didn't have permissions to unlink the civicrm phpunit bootstrap file [17:32:10] I've broken something I guess [17:33:00] I guess this would have been a good time to have a recent backup to restore to.. if I had one [17:33:39] fr-tech this might be a good reminder to take a backup of your current machines if they're working how you like em [17:34:09] nooo bad phpstorm [17:34:26] but good reminder mines still in some weird limbo of a state [17:36:12] 2020-08-24 18:28:06,334 [ 17311] INFO - il.indexing.FileBasedIndexImpl - scheduleIndexRebuild, reason: File type change [17:36:14] 2020-08-24 18:28:06,356 [ 17333] INFO - indexing.UnindexedFilesUpdater - Unindexed files update canceled [17:36:16] hmmm [17:36:19] I see that in the log [17:39:45] it finished! [17:40:26] thank you PhpStorm 2020.2 [17:42:16] so let's create that backup [17:53:26] that reminds me that i should get a backup drive. [17:54:29] Wikimedia-Fundraising-Banners: QA for enUS pre-test on August 26 - https://phabricator.wikimedia.org/T261147 (jbolorinos-ctr) Thanks Sam! I'll get started on these now and let you know if I have any questions. [17:56:00] fundraising-tech-ops, fr-tech-ops-okr: Update staging to match production (php, buster) - https://phabricator.wikimedia.org/T260629 (Dwisehaupt) a:Dwisehaupt [18:23:58] (CR) Mepps: "A couple comments as I review" (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [18:25:34] ejegg looking at the lightncandy situation again and trying to remember what the issue was in the first place that we were going to try the core one, because it was checking that vendors version against DIs? [18:26:10] cstone yeah, we don't have a separate vendor dir for DonationInterface now, just the core one [18:26:19] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile large batch - https://phabricator.wikimedia.org/T261150 (jbolorinos-ctr) Screenshot Test Results - Mobile: - Control: https://app.crossbrowsertesting.com/public/i15b74a2dc92badb/screenshots/z020324360733a1bd60b - Variant: https://app.crossbrowse... [18:26:20] so we can't have two versions of the library [18:26:29] also just good to be on a newer version [18:27:50] (CR) Mepps: "I've been stepping through the code a bit today. It feels pretty dense but I believe that's just due to data complexity. There's this larg" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [18:29:11] Fundraising-Backlog, fundraising-tech-ops: CiviCRM license for Tomas WhiteAntelope - https://phabricator.wikimedia.org/T260745 (EMartin) I added him to the list. He is not a FT employee of Fundraising but helping out from another department. Not sure if that's relevant or not. Evelyn [18:37:28] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Matching Gifts database data cleanup - https://phabricator.wikimedia.org/T260935 (DStrine) [18:38:12] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Fix latest currency in wmf_donor table - https://phabricator.wikimedia.org/T260755 (DStrine) [18:44:25] (CR) Mepps: "Mostly the change looks good. See comment." (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/620123 (https://phabricator.wikimedia.org/T243005) (owner: Jgleeson) [18:44:38] Fundraising-Backlog: Queue investigation: model - https://phabricator.wikimedia.org/T261157 (DStrine) [18:45:08] Fundraising Sprint Octopus hugs, Fundraising Sprint Pseudopretzels, Fundraising-Backlog, fundraising Sprint Q 2020 for real: Fr-tech chores list - https://phabricator.wikimedia.org/T258527 (mepps) a:mepps [18:45:14] Fundraising-Backlog: Queue investigation: model donor experience - https://phabricator.wikimedia.org/T261157 (DStrine) [18:46:54] Fundraising-Backlog: Queue investigation: model donor experience - https://phabricator.wikimedia.org/T261157 (DStrine) [18:49:26] Fundraising-Backlog, fundraising-tech-ops: CiviCRM license for Tomas WhiteAntelope - https://phabricator.wikimedia.org/T260745 (Dwisehaupt) >>! In T260745#6407261, @EMartin wrote: > I added him to the list. He is not a FT employee of Fundraising but > helping out from another department. Not sure if th... [18:57:32] (CR) Jgleeson: Paypal Audit: Move the check for missing subscription ids to a later stage in the audit process to avoid overthrowing exceptions. (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/620123 (https://phabricator.wikimedia.org/T243005) (owner: Jgleeson) [19:08:38] Fundraising Sprint Octopus hugs, Fundraising Sprint Pseudopretzels, Fundraising-Backlog, fundraising Sprint Q 2020 for real, and 3 others: Banner history logger records incorrect status code following campaign fallback - https://phabricator.wikimedia.org/T255810 (mepps) from @AndyRussG: - so the... [19:28:25] Fundraising Sprint Octopus hugs, Fundraising Sprint Pseudopretzels, Fundraising-Backlog, fundraising Sprint Q 2020 for real, MW-1.36-notes (1.36.0-wmf.5; 2020-08-18): Move to 'Tokenize Parameter Field' for one-time tokens on initial recurring transactio... - https://phabricator.wikimedia.org/T248626 [19:35:08] Fundraising Sprint Octopus hugs, Fundraising Sprint Pseudopretzels, Fundraising-Backlog, fundraising Sprint Q 2020 for real, and 3 others: Banner history logger records incorrect status code following campaign fallback - https://phabricator.wikimedia.org/T255810 (mepps) Based on our call just now... [19:47:50] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop large batch - https://phabricator.wikimedia.org/T261148 (jbolorinos-ctr) Screenshot Test Results - Desktop: - Control: https://app.crossbrowsertesting.com/public/i15b74a2dc92badb/screenshots/z4712359af3597cc4906 - Variant: https://app.crossbrow... [19:49:39] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: desktop small batch - https://phabricator.wikimedia.org/T261149 (jbolorinos-ctr) Screenshot Test Results - Desktop: - Control: https://app.crossbrowsertesting.com/public/i15b74a2dc92badb/screenshots/z70009dcff00cde72015 - Variant: https://app.crossbrow... [19:51:37] Wikimedia-Fundraising-Banners: 2020-08-26 enUS pre-test: mobile small batch - https://phabricator.wikimedia.org/T261151 (jbolorinos-ctr) Screenshot Test Results - Mobile: - Control: https://app.crossbrowsertesting.com/public/i15b74a2dc92badb/screenshots/z06e7b762e8a73238c27 - Variant: https://app.crossbrowse... [19:53:17] Fundraising-Backlog: Queue investigation: model donor experience - https://phabricator.wikimedia.org/T261157 (DStrine) [20:02:55] Fundraising-Backlog, fundraising Sprint Q 2020 for real, FR-Ingenico, MediaWiki-extensions-DonationInterface: Ingenico donation fails on Hungarian descriptor too long - https://phabricator.wikimedia.org/T261058 (DStrine) [20:07:09] Fundraising-Backlog, FR-Q2-FY2020-21-cleanup-list: Investigation: Slow queue consumers - https://phabricator.wikimedia.org/T260818 (mepps) https://phabricator.wikimedia.org/T261157 [20:12:11] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Analyse queries in queue import - https://phabricator.wikimedia.org/T261163 (Eileenmcnaughton) [20:20:20] (CR) Mepps: [C: +2] Updated OANDA endpoint to www.oanda.com to fix weak DH key problem. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/621774 (https://phabricator.wikimedia.org/T260092) (owner: Jgleeson) [20:27:40] (Merged) jenkins-bot: Updated OANDA endpoint to www.oanda.com to fix weak DH key problem. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/621774 (https://phabricator.wikimedia.org/T260092) (owner: Jgleeson) [20:36:21] fr-tech https://www.youtube.com/watch?v=eijioOIBzpg [20:36:48] the person pronouncing the two different versions sounds like she's arguing with herself [20:41:20] ejegg: I saw your message in the chat about posting the queue stats to prometheus. I like that idea but I'm not sure how from those averages we can visualise the entire process of a message being processed with the proportions or each block we measure on one useful chart. I was thinking like a horizontal bar made up of each block in order that they run so we could maybe see when one block [20:41:22] becomes "fatter" due to certain conditions, pushing out the overall process [20:43:17] a little bit like kcachegrind's display [20:43:59] (PS17) Eileen: Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) [20:47:42] mepps: thanks for the review on the api endpoint update! this is the other patch that reverts the workaround function we added to use the old api endpoint https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/621494 [20:50:32] (CR) Eileen: Move email handling to deduper, fix email loss bug (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [20:53:59] (PS18) Eileen: Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) [20:55:33] (PS19) Eileen: Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) [20:55:42] (PS1) Jgleeson: WIP: Add stats counter to contribution tracking qc so that we can record duplicate ct exceptions. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622196 (https://phabricator.wikimedia.org/T256037) [20:58:16] (PS20) Eileen: Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) [21:00:20] (CR) Mepps: [C: +2] Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [21:03:37] (CR) jerkins-bot: [V: -1] WIP: Add stats counter to contribution tracking qc so that we can record duplicate ct exceptions. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622196 (https://phabricator.wikimedia.org/T256037) (owner: Jgleeson) [21:06:59] (Merged) jenkins-bot: Move email handling to deduper, fix email loss bug [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619075 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [21:22:15] PROBLEM - check_puppetrun on frdb1001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [21:27:17] RECOVERY - check_puppetrun on frdb1001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [21:43:05] PROBLEM - Host frdb1002 is DOWN: PING CRITICAL - Packet loss = 100% [21:45:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [21:46:09] PROBLEM - check_mysql on frdb2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [21:47:09] PROBLEM - check_mysql on frdb2001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [21:47:15] PROBLEM - check_mysql on frdb1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [21:50:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [21:51:09] ACKNOWLEDGEMENT - check_mysql on frdb1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) Dwisehaupt frdb1002 work. [21:51:09] ACKNOWLEDGEMENT - check_mysql on frdb2001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) Dwisehaupt frdb1002 work. [21:51:09] ACKNOWLEDGEMENT - check_mysql on frdb2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) Dwisehaupt frdb1002 work. [21:55:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [22:00:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [22:03:34] ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) Dwisehaupt frdb1002 work [22:08:01] ejegg: happy birthday it seems [22:08:45] thanks eileen ! [22:11:20] ejegg: do you want to hang out to discuss the outage? [22:12:13] sure eileen - in 15 minutes OK? [22:12:17] sure [22:12:22] and again using that last meeting's link? [22:12:53] (CR) Eileen: "I just removed turning logging on - doing it here doesn't enable the contribution triggers - but I think in wmf_civicrm.install would" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:13:05] (PS4) Eileen: Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) [22:15:06] yep [22:15:39] I'm just hanging out in there so join when you are ready [22:20:01] (CR) jerkins-bot: [V: -1] Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:27:29] (PS5) Eileen: Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) [22:31:31] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:37:09] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 1207526 Threads: 1 Questions: 88975876 Slow queries: 569 Opens: 139651 Flush tables: 1 Open tables: 712 Queries per second avg: 73.684 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [22:37:21] RECOVERY - Host frdb1002 is UP: PING OK - Packet loss = 0%, RTA = 0.39 ms [22:40:15] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 7886258 Threads: 3 Questions: 1551524089 Slow queries: 424772 Opens: 10755588 Flush tables: 8 Open tables: 199 Queries per second avg: 196.737 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [22:41:09] RECOVERY - check_mysql on frdb2002 is OK: Uptime: 1208186 Threads: 1 Questions: 88982646 Slow queries: 587 Opens: 139744 Flush tables: 1 Open tables: 728 Queries per second avg: 73.649 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [22:42:15] RECOVERY - check_mysql on frdb1001 is OK: Uptime: 5455 Threads: 1 Questions: 6484 Slow queries: 0 Opens: 701 Flush tables: 1 Open tables: 694 Queries per second avg: 1.188 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [22:46:29] (PS6) Ejegg: Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:47:04] (CR) Ejegg: [C: +2] Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:53:48] (Merged) jenkins-bot: Add config to support rpow in our dev environments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/620146 (https://phabricator.wikimedia.org/T237350) (owner: Eileen) [22:55:14] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [22:56:10] PROBLEM - check_mysql on frdb2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) [23:05:14] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 7887758 Threads: 4 Questions: 1551524979 Slow queries: 424831 Opens: 10755963 Flush tables: 8 Open tables: 200 Queries per second avg: 196.700 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [23:06:10] RECOVERY - check_mysql on frdb2002 is OK: Uptime: 1209685 Threads: 1 Questions: 88983461 Slow queries: 587 Opens: 139744 Flush tables: 1 Open tables: 728 Queries per second avg: 73.559 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [23:10:18] Fundraising-Backlog: Japanese last name, first name ordering issues - https://phabricator.wikimedia.org/T261144 (Eileenmcnaughton) Just as aside - from a dedupe point of view - if 2 contacts have exactly the same name but it's flipped the script should merge them - using the most recent donor's order as the... [23:26:05] (PS1) Eileen: Break circular dependency logic [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622233 [23:26:22] ejegg: well that was fun - trying to uninstall wmf_civicrm turned out to be impossible [23:26:22] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/622233 [23:26:50] circular dependencies eileen? [23:27:04] oh hah yeah, those [23:27:06] yep - I put up a patch to break them && [23:27:46] hmm, what on earth does it want wmf_communication for? [23:28:13] (PS1) Ejegg: Update Drupal to 7.72 [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622234 [23:28:15] (PS1) Ejegg: Re-apply WMF patches [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622235 [23:28:46] Twig handling? [23:29:09] oh hah, right [23:29:21] to generate php to copy into DonationInterface :P [23:29:40] yeah, that shouldn't be a problem to snip out [23:30:14] (CR) Eileen: [C: +2] "Yep that all looks painfully familiar - given we don't even use bartik theme I suspect that hack does nothing now...." [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622235 (owner: Ejegg) [23:30:23] (CR) Eileen: [C: +2] Update Drupal to 7.72 [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622234 (owner: Ejegg) [23:30:54] (CR) Ejegg: [C: +2] "The removed dependency is only needed for a drush command that generates code for DonationInterface - fine to leave out for tests etc." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622233 (owner: Eileen) [23:31:00] thanks eileen! [23:34:43] (CR) jerkins-bot: [V: -1] Update Drupal to 7.72 [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622234 (owner: Ejegg) [23:34:55] think I can force that [23:37:45] (CR) jerkins-bot: [V: -1] Re-apply WMF patches [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622235 (owner: Ejegg) [23:43:58] (Merged) jenkins-bot: Break circular dependency logic [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/622233 (owner: Eileen) [23:51:41] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, and 2 others: My testing suggests that REMOVING the contribution_status_id (& low-cardinality indexes like is_deleted) will spe... - https://phabricator.wikimedia.org/T247489 [23:59:33] (CR) Ejegg: [V: +2 C: +2] Update Drupal to 7.72 [wikimedia/fundraising/crm/drupal] - https://gerrit.wikimedia.org/r/622234 (owner: Ejegg)