[01:06:36] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:10:17] RECOVERY - check_minfraud_secondary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 385 bytes in 0.246 second response time [10:15:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 88% free (6673 MB out of 7629 MB) [10:20:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6608 MB out of 7629 MB) [10:25:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:30:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:35:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:40:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:44:01] (PS1) Hashar: Define npm peer dependency explicitly [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/368650 (https://phabricator.wikimedia.org/T172094) [10:45:12] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:46:44] (PS1) Hashar: Define npm peer dependency explicitly [extensions/ContributionTracking] - https://gerrit.wikimedia.org/r/368661 (https://phabricator.wikimedia.org/T172094) [10:48:52] (PS1) Hashar: Define npm peer dependency explicitly [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368663 (https://phabricator.wikimedia.org/T172094) [10:50:17] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [10:55:08] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:00:07] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:05:07] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:10:07] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:15:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:18:03] (PS1) Hashar: build: mark package.json as private [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368683 [11:20:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:21:08] (CR) Hashar: [C: 1] Define npm peer dependency explicitly [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368663 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [11:25:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:30:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:35:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:40:10] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:45:21] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:50:11] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:55:11] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [11:57:45] Jeff_Green: frdb1002 seems to have trouble. But I guess you are already on it :-} [11:59:51] hashar: yeah, it looks like mariadb had some kind of spike that pushed it into swap briefly, but now it's just nagios perpetually reporting that high water mark rather than actual ongoing swap utilization [12:00:11] PROBLEM - check_swap on frdb1002 is CRITICAL: SWAP CRITICAL - 87% free (6609 MB out of 7629 MB) [12:00:15] :( [12:00:19] Jeff_Green: good luck! [12:00:30] thanks! [12:10:21] RECOVERY - check_swap on frdb1002 is OK: SWAP OK - 100% free (7629 MB out of 7629 MB) [13:40:51] (PS1) Hashar: Define npm peer dependency explicitly [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368768 (https://phabricator.wikimedia.org/T172094) [13:52:18] (CR) Hashar: [C: 2] Define npm peer dependency explicitly [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368768 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [13:52:35] (CR) Hashar: [C: 2] Define npm peer dependency explicitly [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368663 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [13:53:12] (Merged) jenkins-bot: Define npm peer dependency explicitly [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368768 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [13:54:32] (Merged) jenkins-bot: Define npm peer dependency explicitly [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368663 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [14:43:08] (PS2) Hashar: build: mark package.json as private [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368683 [15:18:43] (CR) Hashar: [C: 2] build: mark package.json as private [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368683 (owner: Hashar) [15:21:17] (Merged) jenkins-bot: build: mark package.json as private [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368683 (owner: Hashar) [16:14:00] (PS1) Ejegg: Add filename to paypal audit failmail [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/368809 [16:29:42] hi ejegg! [16:29:58] hi mepps! [16:30:19] how's things? [16:30:48] oh not too bad, want to quickly check in? [16:30:59] sure! [16:42:42] (PS1) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 [16:44:09] (CR) jerkins-bot: [V: -1] WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 (owner: Mepps) [16:57:40] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/368813 [16:57:45] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/368813 (owner: Ejegg) [16:57:51] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/368813 (owner: Ejegg) [16:58:58] !log updated Misc fundraising tools from 457bddb843f69867f24cd072ede1f6733d59c853 to 58bcbf3ddecb945bf522c76127c5aecf8a36a96e [16:59:07] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:17:08] (PS1) Ejegg: Merge commit '9d7b9a522' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368815 [17:25:25] ejegg i solved my problem i think [17:25:36] oh cool! [17:25:48] i deleted code i'd thought i added but it had actually already been there [17:38:23] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Make sure silverpop export does not add default date for last_donation date - https://phabricator.wikimedia.org/T171920#3487017 (Ejegg) @CCogdill_WMF we just deployed a fix for th... [17:41:54] aha [17:42:11] (CR) Ejegg: [C: 2] Merge commit '9d7b9a522' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368815 (owner: Ejegg) [17:44:07] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368818 [17:44:11] (CR) Ejegg: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368818 (owner: Ejegg) [17:50:20] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Make sure silverpop export does not add default date for last_donation date - https://phabricator.wikimedia.org/T171920#3487068 (CCogdill_WMF) @Ejegg is it possible to the leave f... [17:52:39] (Merged) jenkins-bot: Merge commit '9d7b9a522' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368815 (owner: Ejegg) [17:52:42] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368818 (owner: Ejegg) [17:54:02] (PS2) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 [18:22:13] ok fr-tech, going to deploy some payments stuff, piece by piece [18:22:21] (CR) jerkins-bot: [V: -1] WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 (owner: Mepps) [18:30:35] !log updated payments-wiki from 084d0f99e454046ec4cc428d346dcd3d04fe4746 to c531e11f7eedf2738a6b9fb67bf996b9059d1a92 [18:30:46] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:40:22] (PS3) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 [18:40:35] ejegg got tests passing! [18:40:49] now have to see about adding in paypal rectifier [18:40:50] awesome! [19:09:22] (PS4) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 [19:10:36] (PS1) Ejegg: Merge commit '14ed96cff' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368832 [19:10:43] (CR) Ejegg: [C: 2] Merge commit '14ed96cff' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368832 (owner: Ejegg) [19:11:41] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368833 [19:11:57] (CR) Ejegg: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368833 (owner: Ejegg) [19:18:22] Fundraising Sprint Lou Reed, Fundraising-Backlog, § Fundraising Sprint Abba, § Fundraising Sprint Beastie Boys, and 5 others: Update GC audit to read WX file - https://phabricator.wikimedia.org/T86090#3487404 (Ejegg) p:Normal>High Update from Ingenico - some donations that donor services... [19:18:42] (Merged) jenkins-bot: Merge commit '14ed96cff' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368832 (owner: Ejegg) [19:18:45] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368833 (owner: Ejegg) [19:29:46] !log updated payments-wiki from c531e11f7eedf2738a6b9fb67bf996b9059d1a92 to 2d1080778c7806dae49ce2cac784930fb7715a14 [19:29:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:33:47] ok, now for the queue message handling.... [19:34:39] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368838 [19:34:41] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368838 (owner: Ejegg) [19:35:51] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368839 [19:35:55] (CR) Ejegg: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368839 (owner: Ejegg) [19:41:10] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/368838 (owner: Ejegg) [19:42:24] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/368839 (owner: Ejegg) [19:43:30] deployment confirm code: ehh [19:43:56] !log updated payments-wiki from 2d1080778c7806dae49ce2cac784930fb7715a14 to bd9f7305d7c46f2afab79b0f4d72ca528747878b [19:44:07] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:54:43] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Make sure silverpop export does not add default date for last_donation date - https://phabricator.wikimedia.org/T171920#3487522 (Ejegg) I think database-NULL (rather than the stri... [19:54:54] hehe [19:57:16] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Make sure silverpop export does not add default date for last_donation date - https://phabricator.wikimedia.org/T171920#3487533 (CCogdill_WMF) Will do! [20:04:57] (PS1) Ejegg: Update DonationInterface lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368848 (https://phabricator.wikimedia.org/T95647) [20:11:38] (CR) jerkins-bot: [V: -1] Update DonationInterface lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368848 (https://phabricator.wikimedia.org/T95647) (owner: Ejegg) [20:21:34] (PS2) Ejegg: Add ext.centralNotice.adminUi module back to campaign list view [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/368531 (https://phabricator.wikimedia.org/T172023) (owner: AndyRussG) [20:21:44] (CR) Ejegg: [C: 2] Add ext.centralNotice.adminUi module back to campaign list view [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/368531 (https://phabricator.wikimedia.org/T172023) (owner: AndyRussG) [20:24:04] (Merged) jenkins-bot: Add ext.centralNotice.adminUi module back to campaign list view [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/368531 (https://phabricator.wikimedia.org/T172023) (owner: AndyRussG) [21:20:34] (PS9) Ejegg: [WIP] GlobalCollect WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:22:14] (CR) jerkins-bot: [V: -1] [WIP] GlobalCollect WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:23:18] (PS10) Ejegg: [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:24:51] (CR) jerkins-bot: [V: -1] [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:25:08] (PS11) Ejegg: [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:27:03] (CR) Ejegg: "Thanks, Ingenico, for solving our Drupal module name collision!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [21:40:14] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Do we need to add handling for the 2 silverpop mails that would not retrieve properly - https://phabricator.wikimedia.org/T171433#3487845 (Eileenmcnaughton) [21:42:43] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3487846 [21:43:14] (PS1) Eileen: Make mailing get process skip on exception. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368940 (https://phabricator.wikimedia.org/T171433) [23:08:56] (PS1) Ejegg: WIP Ingenico WX audit parsing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/368948 (https://phabricator.wikimedia.org/T86090)