[00:57:29] PROBLEM - check_mysql on frdb2001 is CRITICAL: Slave IO: Preparing Slave SQL: Yes Seconds Behind Master: (null) [01:02:29] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 198797 Threads: 11 Questions: 2398551 Slow queries: 81 Opens: 1668 Flush tables: 1 Open tables: 1196 Queries per second avg: 12.065 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [03:38:45] (CR) AndyRussG: "Cool!! Thanks so much for working on this!! :)" (2 comments) [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/664939 (https://phabricator.wikimedia.org/T268683) (owner: Jgleeson) [03:42:45] (PS2) AndyRussG: Payments: enable hook endowment for footer links [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/647904 (https://phabricator.wikimedia.org/T269717) (owner: Ejegg) [03:43:09] (CR) AndyRussG: "(just rebased on current master)" [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/647904 (https://phabricator.wikimedia.org/T269717) (owner: Ejegg) [04:05:53] (PS3) AndyRussG: Payments: enable hook endowment for footer links [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/647904 (https://phabricator.wikimedia.org/T269717) (owner: Ejegg) [04:08:01] (CR) AndyRussG: "Ahh thanks for this! Apologies I didn't see it earlier!" [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/647904 (https://phabricator.wikimedia.org/T269717) (owner: Ejegg) [05:19:26] (PS1) AndyRussG: Settings for MediaWiki logs on Payments [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/667425 [07:56:27] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/667457 (owner: L10n-bot) [08:09:14] (CR) Raimond Spekking: [V: +2 C: +2] "false positive" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/667457 (owner: L10n-bot) [12:53:39] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Updating multiple contribution - increase limit - https://phabricator.wikimedia.org/T276083 (RLewis) [14:24:04] fundraising-tech-ops: reload frdev testing/staging databases - https://phabricator.wikimedia.org/T275443 (Jgreen) Open→Resolved a:Jgreen This is done, for future reference the restore time for civicrm+drupal+fredge was ~58H. [14:37:17] fundraising-tech-ops: improve civicrm dump/restore strategy - https://phabricator.wikimedia.org/T276100 (Jgreen) [14:38:57] fundraising-tech-ops: improve civicrm dump/restore strategy - https://phabricator.wikimedia.org/T276100 (Jgreen) [15:30:05] Fundraising-Backlog: Mexico payment form error - https://phabricator.wikimedia.org/T275156 (DStrine) Updating this task prior to sprint planning. I'm moving this to blocked/not fr-tech as the payment form issues and translations copy editing are not in our control. If the psp pushed back or give us instructi... [15:34:04] Fundraising-Backlog, Fr-drupal-upgrade-2021: Investigate project create checklist of all steps needed for drupal upgrade - https://phabricator.wikimedia.org/T276107 (DStrine) [15:34:25] Fundraising-Backlog, Fr-drupal-upgrade-2021: Investigate project create checklist of all steps needed for drupal upgrade - https://phabricator.wikimedia.org/T276107 (DStrine) [15:34:27] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Epic: drupal upgrade - https://phabricator.wikimedia.org/T261779 (DStrine) [15:50:07] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issue with New Field in Individual Engage Import File - https://phabricator.wikimedia.org/T275442 (DStrine) [15:50:25] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Creating New Fidelity File Civi Import - https://phabricator.wikimedia.org/T275445 (DStrine) [15:52:24] Fundraising-Backlog: Brazil TY page in English and not Portuguese - https://phabricator.wikimedia.org/T275576 (DStrine) bumping this. This seems to be an issue with testing. If testers don't see any issues through the sidebar or test banners this issues isn't relevant. I am not prioritizing this before the L... [15:52:54] Fundraising-Backlog: TY page for failed BT transaction in Brazil - https://phabricator.wikimedia.org/T275577 (DStrine) bumping this. The translation section of this seems to be an issue with testing. If testers don't see any issues through the sidebar or test banners this issues isn't relevant. I am not prio... [15:53:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Email address isn't saving when updating in dedupe feature - https://phabricator.wikimedia.org/T275663 (DStrine) [16:09:23] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Search kit tool notes - contribution search - https://phabricator.wikimedia.org/T275707 (DStrine) [16:10:22] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-CiviCRM-Searchkit: Search kit tool notes - contribution search - https://phabricator.wikimedia.org/T275707 (DStrine) [16:10:59] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Query breaks when results include multiple records of same donor - https://phabricator.wikimedia.org/T275668 (DStrine) [16:11:22] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Query breaks when results include multiple records of same donor - https://phabricator.wikimedia.org/T275668 (DStrine) [16:12:29] Fundraising-Backlog, FR-Civi-Dedupe: Alternate name field for donors with different alphabet characters - https://phabricator.wikimedia.org/T275678 (DStrine) [18:09:26] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wikimedia-CentralNotice-Administration, Patch-For-Review, Wikimedia-production-error: Display a sane error if the CN banner name is invalid, instead of a cryptic «BannerDataException» - https://phabricator.wikimedia.org/T173782 (Ciell)... [18:12:24] (PS1) Cstone: Update es-419(es-mx) and pt-br thank you emails. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667681 [18:33:49] Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for spatton - https://phabricator.wikimedia.org/T273498 (Dwisehaupt) Certificate renewed and sent via email. Password sent via SMS. [18:37:03] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Patch-For-Review: es419 Thank You Email Edits for the March 9th Pretest - https://phabricator.wikimedia.org/T274319 (Cstone) @CDenes_WMF the updated thank you emails are on staging [18:56:48] (CR) jerkins-bot: [V: -1] Update es-419(es-mx) and pt-br thank you emails. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667681 (owner: Cstone) [19:01:38] (CR) Cstone: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667681 (owner: Cstone) [19:05:42] (CR) Krinkle: "Thanks, appreciate it." [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/663946 (owner: Krinkle) [19:22:32] Wikimedia-Fundraising-Banners: Mobile small: Long amounts lead to stacked nag buttons on phone screens - https://phabricator.wikimedia.org/T275208 (Pcoombe) Open→Resolved a:Pcoombe Fixed in current best: https://meta.wikimedia.org/w/index.php?diff=prev&oldid=21168345 {F34130513} [19:33:40] (PS1) Eileen: Submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667697 [19:34:14] (CR) Eileen: [C: +2] Submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667697 (owner: Eileen) [19:39:29] (PS1) Eileen: Add test to report template saying no results found if that is the case [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/667698 (https://phabricator.wikimedia.org/T275071) [19:53:02] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Jgreen) [19:54:56] (Merged) jenkins-bot: Submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667697 (owner: Eileen) [20:02:36] (PS1) Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667702 [20:05:22] Fundraising-Backlog, Analytics, FR-Tech-Analytics: Whitelist Portal and WikipediaApp event data for (sanitized) long-term storage - https://phabricator.wikimedia.org/T273246 (EYener) @mforns thanks for linking to T262433! I've added myself as a subscriber. Heads up for @mpopov, as I believe you autho... [20:06:20] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Jgreen) [20:08:10] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Jgreen) [20:15:26] (CR) Eileen: [C: +2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667702 (owner: Eileen) [20:21:59] eileen: thanks for the twig merge into deployment. I can confirm it's no longer coming up in the security scan for me locally [20:23:32] jgleeson: cool - I haven't done vendor update yet [20:24:10] yeah I was scanning master [20:25:27] (PS1) Eileen: Vendor update, twig, symfony polyfill [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/667705 [20:26:03] (CR) Eileen: [C: +2] Vendor update, twig, symfony polyfill [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/667705 (owner: Eileen) [20:27:23] this is what I see symfony/http-foundation v2.8.52 requires symfony/polyfill-php55 (~1.0) [20:29:12] wikimedia/smash-pig v0.6.7 requires symfony/http-foundation (^2.8.2) [20:29:24] Wikimedia-Fundraising-Banners: Do number formatting where we use ISO currency code for amounts - https://phabricator.wikimedia.org/T275206 (Pcoombe) Open→Resolved Fixed with [this change](https://meta.wikimedia.org/w/index.php?title=MediaWiki:FundraisingBanners/LocalizeJS-2017.js&diff=prev&oldid=2116... [20:36:04] (PS2) Eileen: Add text to report template saying no results found if that is the case [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/667698 (https://phabricator.wikimedia.org/T275071) [20:44:52] (Merged) jenkins-bot: Vendor update, twig, symfony polyfill [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/667705 (owner: Eileen) [20:54:23] (PS1) Eileen: Vendor update, twig, polyfill [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667710 [20:56:40] (CR) Eileen: [C: +2] Vendor update, twig, polyfill [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667710 (owner: Eileen) [21:00:05] Fundraising-Backlog, fundraising-tech-ops: examine the possibility of purging data from civicrm.civicrm_activity - https://phabricator.wikimedia.org/T261353 (DStrine) [21:07:09] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Jgreen) [21:13:11] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Email address isn't saving when updating in dedupe feature - https://phabricator.wikimedia.org/T275663 (DStrine) [21:13:24] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Query breaks when results include multiple records of same donor - https://phabricator.wikimedia.org/T275668 (DStrine) [21:16:26] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Query breaks when results include multiple records of same donor - https://phabricator.wikimedia.org/T275668 (Eileenmcnaughton) a:Eileenmcnaughton [21:18:15] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Query breaks when results include multiple records of same donor - https://phabricator.wikimedia.org/T275668 (Eileenmcnaughton) I'm pretty sure this is https://github.com/eileenmcn... [21:19:36] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Email address isn't saving when updating in dedupe feature - https://phabricator.wikimedia.org/T275663 (Eileenmcnaughton) A fix is incorporated in https://gerrit.wikimedia.org/r/c/w... [21:32:53] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Civi Upgrade UI issue: Some dashlets don't populate, others are hard to read now - https://phabricator.wikimedia.org/T275071 (CaitVirtue) @Eileenmcnaughton Yes, that looks right t... [21:55:43] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Jgreen) [22:04:24] (CR) Cstone: [C: +2] "Looks good!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667053 (https://phabricator.wikimedia.org/T275668) (owner: Eileen) [22:05:51] (CR) Cstone: [C: +2] Add text to report template saying no results found if that is the case [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/667698 (https://phabricator.wikimedia.org/T275071) (owner: Eileen) [22:07:09] (CR) Eileen: [C: +2] "recheck" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667702 (owner: Eileen) [22:10:44] (CR) Eileen: [C: +2] "recheck" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667710 (owner: Eileen) [22:13:02] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Patch-For-Review: es419 Thank You Email Edits for the March 9th Pretest - https://phabricator.wikimedia.org/T274319 (CDenes_WMF) Perfect, thank you Christine! Green light for production. [22:14:35] (CR) Eileen: "recheck" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/667698 (https://phabricator.wikimedia.org/T275071) (owner: Eileen) [22:14:49] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667053 (https://phabricator.wikimedia.org/T275668) (owner: Eileen) [22:16:23] !log civicrm revision changed from f07390ff87 to 04a029958c, config revision is 643477b35d [22:16:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:16:41] cstone: jgleeson I just deployed twig & the es_mx update [22:16:44] I ran updb [22:17:13] thanks eileen [22:17:45] nice eileen ! [22:17:47] cstone: I didn't double check but that update covers everything? I guess the db didn't store es-419 anyway so it wasn't in the db... [22:17:54] bye for now! [22:17:58] night jgleeson [22:21:44] !log stopping mysql replication on frdb2001 and starting utf8mb4 table alters under a root screen session [22:21:48] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:22:22] !log ran the following on frdb2001 to allow replication to continue after conversion to utf8mb4 charset: set global slave_type_conversions = ALL_NON_LOSSY; [22:22:26] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:22:34] eileen: its ok for now we havent deployed the DI side of it yet so it'll just end up as es in the civi db as it was before if any donations somehow happen [22:25:30] (Merged) jenkins-bot: Update Afform & dependencies [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667053 (https://phabricator.wikimedia.org/T275668) (owner: Eileen) [22:27:08] Fundraising-Backlog, fundraising-tech-ops: Fix our database to support emojis - convert tables to utf8mb4 - https://phabricator.wikimedia.org/T270218 (Dwisehaupt) These alters completed in 26 hours on frdb2002. After the alter completed, I set the following global variable to allow replication to continu... [22:29:20] Fundraising-Backlog, fundraising-tech-ops: Fix our database to support emojis - convert tables to utf8mb4 - https://phabricator.wikimedia.org/T270218 (Dwisehaupt) [22:34:21] (Merged) jenkins-bot: Add text to report template saying no results found if that is the case [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/667698 (https://phabricator.wikimedia.org/T275071) (owner: Eileen) [22:36:24] (PS1) Eileen: CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667722 [22:36:34] (CR) Eileen: [C: +2] CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667722 (owner: Eileen) [22:38:56] (CR) Eileen: [C: +2] "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667722 (owner: Eileen) [22:39:54] fundraising-tech-ops: Plan for FY 2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Dwisehaupt) [22:54:47] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising, Wikimedia-Fundraising-CiviCRM: Add es-419 as a supported 'preferred language' in Civi - https://phabricator.wikimedia.org/T199733 (Eileenmcnaughton) PR to do the same re-labelling upstream https://github.c... [22:55:19] (Merged) jenkins-bot: CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667722 (owner: Eileen) [22:56:39] (PS1) Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667725 [22:56:53] (CR) Eileen: [C: +2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/667725 (owner: Eileen) [22:59:21] (PS1) Eileen: De-symlink sites folder [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667727 (https://phabricator.wikimedia.org/T273822) [23:01:04] (PS1) Eileen: Update gitignore for sites move [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667728 (https://phabricator.wikimedia.org/T273822) [23:01:48] cstone: I just put up the patch to desymlink the sites folder [23:03:26] (CR) jerkins-bot: [V: -1] De-symlink sites folder [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667727 (https://phabricator.wikimedia.org/T273822) (owner: Eileen) [23:04:18] (CR) jerkins-bot: [V: -1] Update gitignore for sites move [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667728 (https://phabricator.wikimedia.org/T273822) (owner: Eileen) [23:05:16] !log civicrm revision changed from 04a029958c to e1dacbe348, config revision is 643477b35d [23:05:21] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:17:11] Fundraising Sprint Downed power line jump rope, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Civi Upgrade UI issue: Some dashlets don't populate, others are hard to read now - https://phabricator.wikimedia.org/T275071 (Eileenmcnaughton) @CaitVirtue I just deployed it - so... [23:20:42] Fundraising-Backlog: Automatically send status 800 on prior donors stopped at 600 - https://phabricator.wikimedia.org/T276183 (EMartin) [23:37:01] eileen: im seeing the same error thats on CI locally for the symlinks Error: main(): Failed opening required [23:37:01] '/src/wikimedia/fundraising/crm/drupal/sites/default/../../vendor/autoload.php' [23:38:47] (PS1) Eileen: Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 [23:38:52] cstone: is ^^ it? [23:40:21] cstone: hey btw, without wishing to interrupt other stuff ur working on, if you have a chance to paste any details of the port error you got, and where/when/how it appeared, that'd be cool :) thx in advance! [23:40:38] sure AndyRussG and I haven't tried it again yet let me do that [23:40:45] ah okok cool [23:42:33] (CR) jerkins-bot: [V: -1] Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 (owner: Eileen) [23:47:03] eileen: from the install script I think the one in drupal/sites/default/settings.php, line 556 but that path works out for me unless the text editor here is not updating [23:47:33] RECOVERY - check_puppetrun on frav1002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:48:44] or wait it should just be sites/default now? [23:49:27] no it should be drupal/sites/default [23:51:07] (PS2) Eileen: Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 [23:54:58] (CR) jerkins-bot: [V: -1] Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 (owner: Eileen) [23:55:29] (PS3) Eileen: Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 [23:59:24] (CR) jerkins-bot: [V: -1] Update path for autoload [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/667732 (owner: Eileen) [23:59:33] three was right I couldn't count haha