[00:46:08] Fundraising Dash, Fundraising-Backlog: Big English total raised figure and amount per second chart broken after new year UTC - https://phabricator.wikimedia.org/T241665 (Ejegg) [01:00:48] Fundraising Dash, Fundraising-Backlog: Big English total raised figure and amount per second chart broken after new year UTC - https://phabricator.wikimedia.org/T241665 (Ejegg) Hmm, looks like it may actually be broken even after new year local time, until we add a new campaign for 2020. [01:07:11] (PS1) Ejegg: Fix missing data error after new year UTC [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/561323 (https://phabricator.wikimedia.org/T241665) [01:08:32] cstone: I think ^^^ will restore the 'Total To Date' number in the dash for whoever's refreshing that right now [01:08:50] if you're not busy! [01:09:06] hah nope [01:10:03] (CR) Cstone: [C: +2] Fix missing data error after new year UTC [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/561323 (https://phabricator.wikimedia.org/T241665) (owner: Ejegg) [01:10:52] (Merged) jenkins-bot: Fix missing data error after new year UTC [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/561323 (https://phabricator.wikimedia.org/T241665) (owner: Ejegg) [01:41:28] thanks cstone! [01:45:42] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/561324 [01:46:06] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/561324 (owner: Ejegg) [01:46:47] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/561324 (owner: Ejegg) [02:01:56] Fundraising Dash, Fundraising Sprint YAMLton, the Musical, Fundraising-Backlog, Patch-For-Review: Big English total raised figure and amount per second chart broken after new year UTC - https://phabricator.wikimedia.org/T241665 (Ejegg) [10:22:15] (CR) Daimona Eaytoy: [C: +1] Italy Thank You email Body and Subject Line File [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/561313 (owner: Cdenes) [21:13:36] !log stopped fundraising cron jobs to calculate EOY summaries [21:13:37] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:41:15] PROBLEM - check_procs on civi1001 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [21:42:39] PROBLEM - check_zombie on civi1001 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [22:02:25] PROBLEM - check_load on civi1001 is CRITICAL: CRITICAL - load average: 42.20, 33.37, 19.37 [22:03:49] PROBLEM - check_procs on civi1001 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [22:07:17] PROBLEM - check_zombie on civi1001 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [22:11:19] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [22:16:19] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [22:20:47] argh, permissions error [22:21:19] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [22:26:19] RECOVERY - check_ipsec on americium is OK: Strongswan OK - 1 ESP OK [22:44:49] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Work to reduce smarty cruft on the server - https://phabricator.wikimedia.org/T241688 (Eileenmcnaughton) [22:46:31] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Work to reduce smarty cruft on the server - https://phabricator.wikimedia.org/T241688 (Ejegg) Is part of the problem that Civi is rendering templates when they will likely never be shown to a user, i.e. during background deduplication jobs? [22:51:04] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Work to reduce smarty cruft on the server - https://phabricator.wikimedia.org/T241688 (Eileenmcnaughton) So this is one of the patches I was thinking we could dig into https://github.com/civicrm/civicrm-packages/pull/234 [22:54:25] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Work to reduce smarty cruft on the server - https://phabricator.wikimedia.org/T241688 (Eileenmcnaughton) This is the other https://github.com/civicrm/civicrm-core/pull/14706 It's also possible we are processing greeting templates during dedupe - I thin... [22:54:54] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Work to reduce smarty cruft on the server - https://phabricator.wikimedia.org/T241688 (Eileenmcnaughton) (there are 2 other open PRs against smarty - might be good for this to dig into those too) [22:57:13] (PS1) Eileen: Fix a couple of typos [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/561384 [23:30:51] PROBLEM - Host civi1001 is DOWN: PING CRITICAL - Packet loss = 100% [23:31:19] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [23:35:17] RECOVERY - check_load on civi1001 is OK: OK - load average: 0.56, 0.81, 0.39 [23:35:19] RECOVERY - check_procs on civi1001 is OK: PROCS OK: 295 processes [23:35:21] RECOVERY - Host civi1001 is UP: PING OK - Packet loss = 0%, RTA = 0.38 ms [23:35:27] RECOVERY - check_zombie on civi1001 is OK: PROCS OK: 0 processes with STATE = Z [23:36:19] RECOVERY - check_ipsec on americium is OK: Strongswan OK - 1 ESP OK