[00:55:06] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1229 [01:00:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1287 [01:05:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1399 [01:05:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3778 [01:05:43] uh oh [01:05:55] yeah - hmm [01:06:12] eileen: you got a thing going? [01:06:14] that job has been running throttled - but has been running for a while [01:06:32] so I guess if it runs throttled for a long time it eventually starts to add up [01:06:44] what's happening now? Is it going up or down? [01:07:09] looks like it is going up right now [01:07:23] ok - better kill it I guess [01:07:27] just based on the alerts [01:07:32] ah [01:07:38] but they are only every 5 minutes? [01:07:50] yeah [01:08:03] do you think it is coinciding with another job? [01:08:05] that one above is the first I saw - were there ones you got earlier? [01:08:26] first i have seen in awhile [01:08:33] hmm [01:09:00] so in 2 minutes time we should get some more alerts [01:10:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1458 [01:10:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3840 [01:10:35] so about the same - I think the job might have finished though? [01:11:44] yeah doesn't look like any new data is happening there [01:11:47] ccccccgdtfcjiuerekctrhhcnlcliltghnudjiiuuktt [01:12:21] ok so it should start going down now [01:12:30] I think so [01:12:40] assuming that job was the cause [01:13:42] is it still working on the first big bulk insert? [01:13:56] they are actually single inserts [01:13:58] many [01:14:12] although - we use mixed replication don't we? [01:14:21] so, when does it do row & when does it do statement? [01:14:21] yeah [01:14:31] heh good question [01:15:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3953 [01:15:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1513 [01:17:09] i have not been able to find a satisfactory answer [01:19:04] oh ok http://mysql.wingtiplabs.com/documentation/row639ae/configure-row-based-or-mixed-mode-replication.html [01:19:24] looks like it uses statement based by default [01:20:06] except in a few cases, i think this is the most notable: [01:20:10] Updates two tables with auto_increment columns (the binlog format only carries one auto_increment value per statement) [01:20:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4012 [01:20:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1512 [01:20:47] so sounds like mysql will figure it out? [01:21:36] I see, the job hasn't stopped. It is intermittently pausing because it is throttling [01:23:06] it's set to do 5000 in 60 seconds [01:23:55] i could see replication type having an effect on slave servers if the queries are processor intensive [01:24:26] the queries are all INSERT IGNORE, based on primary key [01:25:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1564 [01:25:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4069 [01:25:27] that's still going up [01:25:38] if i'm reading that right it means that unless they do something that would depend on machine state, it defaults to statement based replication [01:26:12] which is going to be slower in the case that these are complicated queries [01:26:20] is that the case? [01:26:32] or is it just a lot of them? [01:26:34] they are all INSERT IGNORE statements [01:26:37] & there are a lot [01:27:18] ok that makes me think replication type is probably not the issue [01:28:04] I thought that since the data being collected was no longer in Dec there would be a lot less emails & we could speed it up a bit [01:28:26] it could be colliding with another job [01:28:59] we need a process-control dashboard of some sort [01:29:03] yeah [01:30:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1679 [01:30:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4124 [01:30:51] well, it's still going up, i know jeff would say to kill it [01:31:05] yeah we might have to [01:31:26] was it off until recently? why is it blowing up now? [01:31:40] earlier today I increased the grab size [01:31:52] I thought we had 'proven' the throttling [01:32:05] but, it looks like it needs a smaller grab size and throttling [01:32:24] sorry i'm not even familiar with the job, it is pulling data from somewhere (?) and putting in mysql? [01:32:28] it mustn't be throttled to a speed it can keep up with long-term [01:32:34] yeah, pulling from silverpop [01:32:50] we can backfill 450 days of data, which is what it is doing [01:32:56] a flat file? [01:33:18] it requests a csv from silverpop & then downloads & imports it [01:33:32] ah yes that sounds familiar [01:33:35] thanks [01:33:52] uses INSERT IGNORE so it is ok if there is any silliness about re-processing the same data [01:34:27] it's like email addresses and stuff? [01:34:41] right - check the table civicrm_mailing_provider_data [01:34:52] email, timestamp, action, some identifiers [01:35:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1741 [01:35:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4180 [01:35:58] that's up again - TBH we were a bit slow - I think a new job managed to start up 15 mins ago [01:36:35] any more that start now will be grabbing much less data since I changed the setting but in 4 mins we should get a fail mail saying process lock caused job not to start [01:37:51] sounds like a balancing act, what do you envision long term? [01:37:59] just get the throttle settings right? [01:38:16] well once it has caught up it won't be trying to grab more than a short period of data [01:38:29] the minimum length it can grab is 12 hours [01:38:46] ah ok, how long till it catches up? [01:38:52] & it has been running a few days on 12 hours [01:39:10] it's up to March at the moment - [01:40:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4176 [01:40:07] starting when? [01:40:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1699 [01:40:36] we started back in May last year [01:40:45] but Dec was the most data [01:41:11] I thought I could set the grab really high for Feb/ March as not so much mail going out then [01:41:22] but, obviously got it wrong [01:41:36] (that last one was finally going down) [01:45:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1815 [01:45:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4228 [01:50:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1874 [01:50:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4165 [01:51:04] cwd - you'd better kill it - not worth the stress on people to let it run - it'll get there eventually [01:51:35] ok one sec [01:51:39] sorry got a phone call [01:51:50] wife is out of town on business [01:52:01] ok [01:54:03] ok i killed it [01:54:11] cool [01:54:52] eileen: grab == how much data it pulls at once? [01:55:00] from silverpop? [01:55:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4140 [01:55:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1979 [01:56:53] cwd yep - it has an time interval it requests [01:57:29] it would be really interesting now to know how re-running that same data would affect replication, since it will re-run a huge chunk of data already imported [01:57:46] would it replicate the rows or not given they didn't change the data [01:59:24] considering that article i posted i am guessing it would re run the queries [01:59:32] which would do nothing since the row is there [01:59:45] but it would still have to calculate the difference [02:00:02] yeah - that seems likely [02:00:07] it seems like row based replication would be better in that case [02:00:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2049 [02:00:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4089 [02:00:32] that one was supposed to be DOWN! [02:00:36] but "mixed" won't choose "row" unless some fairly unlikely thing happen [02:00:51] hrm [02:01:54] perhaps mysql has a queue of queries [02:02:04] these are the things i would know if i was an actual sys admin [02:05:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4028 [02:05:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2163 [02:10:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2223 [02:10:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4020 [02:15:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2280 [02:15:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4019 [02:20:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2393 [02:20:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3918 [02:25:06] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2451 [02:25:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3856 [02:30:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2566 [02:30:18] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3848 [02:35:06] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2626 [02:35:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3785 [02:40:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2741 [02:40:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3775 [02:45:06] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2542 [02:45:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3713 [02:50:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2300 [02:50:09] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3765 [02:55:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3760 [02:55:16] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2362 [03:00:16] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3438 [03:05:06] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3019 [03:05:16] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 1772789 Threads: 2 Questions: 112227673 Slow queries: 22036 Opens: 29819 Flush tables: 1 Open tables: 1007 Queries per second avg: 63.305 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [03:10:16] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 1772270 Threads: 2 Questions: 105704750 Slow queries: 9683 Opens: 29061 Flush tables: 1 Open tables: 602 Queries per second avg: 59.643 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [03:43:42] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3488364 (Eileenmcnaughton) I ran this from the... [03:47:28] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: Drush not handling spaces in quotes / schedule Si... - https://phabricator.wikimedia.org/T171435#3488366 [03:54:51] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3488372 (Eileenmcnaughton) I've added this job... [04:37:42] (PS1) Eileen: Omnimail MailingProviderData.get: extend specificity on id. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368977 (https://phabricator.wikimedia.org/T161760) [05:15:21] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: Drush not handling spaces in quotes / schedule Si... - https://phabricator.wikimedia.org/T171435#3488459 [05:15:36] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: Drush not handling spaces in quotes / schedule Si... - https://phabricator.wikimedia.org/T171435#3488460 [06:15:17] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2276 [06:20:17] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2044 [06:25:17] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2100 [06:30:08] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2158 [06:35:07] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1793 [06:45:17] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 1785170 Threads: 1 Questions: 106233370 Slow queries: 9683 Opens: 29548 Flush tables: 1 Open tables: 602 Queries per second avg: 59.508 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [06:53:50] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Restore live civicrm database to staging to get time to run 4.7.21 upgrade - https://phabricator.wikimedia.org/T168574#3488506 (Eileenmcnaughton) Open>Resolved [09:35:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1312 [09:40:07] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1427 [09:45:17] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 1796789 Threads: 1 Questions: 113242956 Slow queries: 22066 Opens: 30419 Flush tables: 1 Open tables: 1184 Queries per second avg: 63.025 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 286 [09:47:25] whyyyyyy [09:50:15] that job must be coinciding with another and causing the lag [10:15:05] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/ContributionReporting] - https://gerrit.wikimedia.org/r/369079 [10:15:55] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 [10:17:09] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/FundraiserLandingPage] - https://gerrit.wikimedia.org/r/369141 [10:17:11] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/FundraisingChart] - https://gerrit.wikimedia.org/r/369142 [10:17:13] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/369143 [10:17:15] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/FundraisingTranslateWorkflow] - https://gerrit.wikimedia.org/r/369144 [10:19:10] (PS1) Hashar: build: bump grunt and drop grunt-cli [extensions/LandingCheck] - https://gerrit.wikimedia.org/r/369201 [10:31:37] (CR) Hashar: "recheck" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 (owner: Hashar) [10:33:33] (CR) jerkins-bot: [V: -1] build: bump grunt and drop grunt-cli [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 (owner: Hashar) [10:45:21] (PS2) Hashar: build: bump grunt and drop grunt-cli [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 [10:45:23] (PS1) Hashar: build: bump grunt-contrib-watch [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369322 (https://phabricator.wikimedia.org/T172094) [13:13:20] (CR) Umherirrender: [C: 2] build: bump grunt and drop grunt-cli [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 (owner: Hashar) [13:13:43] (CR) Umherirrender: [C: 2] build: bump grunt-contrib-watch [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369322 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [13:18:10] (Merged) jenkins-bot: build: bump grunt-contrib-watch [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369322 (https://phabricator.wikimedia.org/T172094) (owner: Hashar) [13:19:55] (Merged) jenkins-bot: build: bump grunt and drop grunt-cli [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/369104 (owner: Hashar) [15:09:11] (CR) Ejegg: [C: 2] Make mailing get process skip on exception. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368940 (https://phabricator.wikimedia.org/T171433) (owner: Eileen) [15:11:21] (PS2) Ejegg: WIP Ingenico WX audit parsing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/368948 (https://phabricator.wikimedia.org/T86090) [15:15:29] (Merged) jenkins-bot: Make mailing get process skip on exception. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368940 (https://phabricator.wikimedia.org/T171433) (owner: Eileen) [15:46:31] (PS1) Ejegg: Update config documentation in README [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/369406 [15:49:59] hi K4-713 ! [15:50:42] I hear it's a bit warm up in Portland [15:58:57] hey ejegg. [15:59:02] Good morning. [15:59:22] Mornin! [15:59:32] Does your place have A/C ? [16:09:24] (PS12) Ejegg: [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [16:10:26] ^^lol [16:11:06] ejegg: My place does have AC. We had one installed as soon as we moved into the place. [16:11:08] (CR) jerkins-bot: [V: -1] [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [16:15:45] nice. is it as apocalyptic outside as the news makes out? [16:16:29] (CR) Ejegg: [C: 2] Omnimail MailingProviderData.get: extend specificity on id. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368977 (https://phabricator.wikimedia.org/T161760) (owner: Eileen) [16:20:35] ejegg: Well, it is hot. [16:20:57] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3489687 (CCogdill_WMF) @Eileenmcnaughton am I... [16:21:06] I'd be worried about the food I'm growing if I didn't have automatic watering. [16:22:40] (Merged) jenkins-bot: Omnimail MailingProviderData.get: extend specificity on id. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368977 (https://phabricator.wikimedia.org/T161760) (owner: Eileen) [16:22:47] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3489694 [16:25:07] go go gadget herb spiral! [16:28:43] ejegg: hey thanks for the prompt reply to CPS folks [16:29:03] fr-tech so the CN deploy plan I suggested by e-mail sounds fine? [16:29:04] Do you think the problem is the new machines? [16:29:21] yep, that plan sounds good! [16:29:39] sorry for no prompt (or even tardy) reply to that mail [16:30:25] ejegg: heh np, probably you replied in Standup and it didn't register in my brain 8p [16:31:44] (PS2) Ejegg: Limit Silverpop group import to Opt In by default [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368125 (owner: Eileen) [16:36:28] (CR) Ejegg: [C: -1] "Good stuff, just missing a function arg" (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368125 (owner: Eileen) [16:37:55] fr-tech I'm going to deploy CiviCRM. Little change to the imports that should fill in contribution_tracking.country for offline imports, plus a couple of Eileen's latest silverpop fixes [16:39:29] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/369415 [16:39:35] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/369415 (owner: Ejegg) [16:40:26] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/369415 (owner: Ejegg) [16:41:48] !log updated CiviCRM from 23f2bbf73557a7a88e783f68459112cf4bba1c79 to 5c741b1f42da80a30a93d26338eb89912f72f1eb [16:41:52] (PS1) AndyRussG: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/369416 [16:41:58] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:45:03] (CR) AndyRussG: [C: 2] Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/369416 (owner: AndyRussG) [16:47:01] (Merged) jenkins-bot: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/369416 (owner: AndyRussG) [16:52:44] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface: Create orphan rectifier for PayPal Express Checkout - https://phabricator.wikimedia.org/T172202#3489787 (Ejegg) [16:53:05] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface: Create orphan rectifier for PayPal Express Checkout - https://phabricator.wikimedia.org/T172202#3489800 (Ejegg) p:Triage>High [16:53:42] mepps does 4 points sound good for the PayPal EC orphan adapter? Feel free to change the new ticket ^^^ [16:57:12] gmail is SLOOOOOW today [16:59:25] Fundraising Sprint Lou Reed, Fundraising Sprint Navel Warfare, Fundraising-Backlog, § Fundraising Sprint Abba, and 7 others: Update GC audit to read WX file - https://phabricator.wikimedia.org/T86090#3489816 (Ejegg) a:Ejegg [17:00:52] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Fr-Ingenico-integration_17_18: Customize Ingenico hosted checkouts iframe - https://phabricator.wikimedia.org/T171346#3461789 (Ejegg) p:Triage>High a:Ejegg [17:03:05] (PS2) Ejegg: Update DonationInterface lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368848 (https://phabricator.wikimedia.org/T95647) [17:07:17] Fundraising-Backlog: Leave blank values in Silverpop file blank, not NULL - https://phabricator.wikimedia.org/T91692#3489865 (Ejegg) Oops, I'd totally forgotten about this ticket! I guess blank and NULL aren't actually the same from Silverpop's point of view. I'll fix the fields you listed above. [17:12:52] (CR) jerkins-bot: [V: -1] Update DonationInterface lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/368848 (https://phabricator.wikimedia.org/T95647) (owner: Ejegg) [17:30:16] (PS1) Ejegg: Final export should have blanks instead of NULL [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) [17:31:11] ejegg yeah that sounds right [17:31:18] Fundraising Sprint Navel Warfare, Fundraising-Backlog, FR-Email, Patch-For-Review: Leave blank values in Silverpop file blank, not NULL - https://phabricator.wikimedia.org/T91692#3489910 (Ejegg) a:Ejegg [17:31:31] cool mepps [17:31:45] I'm going to get some food soon - want to check in after that? [17:31:57] like an hour from now? [17:35:36] sounds good ejegg! [17:38:13] (CR) jerkins-bot: [V: -1] Final export should have blanks instead of NULL [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) (owner: Ejegg) [18:40:26] (PS5) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 [18:51:55] hi mepps, sorry to be so long [18:52:03] taking a look at the patch [18:52:03] hi ejegg that's okay [18:53:31] mepps I'm in the video chat - want to talk me through what you've done so far? [19:21:40] (Abandoned) Mepps: WIP Orphan rectifier for paypal express [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/368812 (owner: Mepps) [19:25:19] ejegg oops just realized we have tech backlog in 5 today :) [19:28:17] oh right, super soon! [19:31:06] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic, FR-Email: EPIC: Add key silverpop data into Civi - https://phabricator.wikimedia.org/T161757#3490588 (Eileenmcnaughton) [19:31:08] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3490587 [19:35:15] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3490591 (Eileenmcnaughton) @CCogdill_WMF I hav... [19:37:45] Hey Fundraising team. I need your input on https://phabricator.wikimedia.org/T171287 [19:41:26] fr-tech: ^ [19:41:29] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: On saving a banner, form shows values from save request without checking DB - https://phabricator.wikimedia.org/T171774#3490604 (DStrine) p:Triage>High [19:42:25] raynor you're in luck - we're in a meeting prioritising tasks at this very moment! [19:42:30] I'll toss that one into the mix [19:42:41] cool. thx [19:43:34] reading-web will apply the fix, we just need information how to pass popups.isEnabled information [19:47:26] Fundraising-Backlog, Patch-For-Review: process-control should make slow-starting jobs easier - https://phabricator.wikimedia.org/T171873#3490619 (DStrine) p:Triage>Normal [19:49:32] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: EPIC: CentralNotice: Tracking: Issues related to old JS widgets in Admin UI - https://phabricator.wikimedia.org/T172024#3490639 (DStrine) [19:49:42] Fundraising Sprint Lou Reed, Fundraising Sprint Navel Warfare, Fundraising-Backlog, § Fundraising Sprint Abba, and 7 others: Update GC audit to read WX file - https://phabricator.wikimedia.org/T86090#3490642 (Ejegg) [19:51:11] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3490648 (CCogdill_WMF) Oh, I think ignoring is... [19:55:09] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Ingenico: Deal with recurring donations stuck in 'In Progress' status - https://phabricator.wikimedia.org/T171868#3490664 (Ejegg) [19:55:19] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Direct Mail Appeal not reflecting in contribution records for event - https://phabricator.wikimedia.org/T171794#3490665 (DStrine) p:Triage>Normal [19:55:47] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Ingenico: Deal with recurring donations stuck in 'In Progress' status - https://phabricator.wikimedia.org/T171868#3490670 (Ejegg) From the task closed as duplicate, specifically about the 4 most recent ones: We need to get the donations themselv... [19:57:12] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM phpunit tests failing on second run - https://phabricator.wikimedia.org/T171680#3490673 (DStrine) p:Triage>Normal [20:00:06] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Direct Mail Appeal not reflecting in contribution records for event - https://phabricator.wikimedia.org/T171794#3476345 (mepps) @LeanneS When I went to Find Contributions > Direct Mail Appeal and looked for "DCEvent_17", I found the contribution by CID... [20:00:24] pcoombe: looks like the ticket raynor just mentioned ( T171287 ) has a question for you [20:00:24] T171287: Expose whether previews are enabled for third-party JavaScript - https://phabricator.wikimedia.org/T171287 [20:00:39] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Direct Mail Appeal not reflecting in contribution records for event - https://phabricator.wikimedia.org/T171794#3490685 (mepps) @LeanneS Let me know if that makes sense to you! [20:02:57] ejegg: https://github.com/gabrielbull/omnimail/pull/27#issuecomment-310162377 [20:05:04] oh boy, more responsibilities! [20:05:07] hehe [20:05:23] i mean, "awesome, we can upstream it all!" [20:18:30] Fundraising Sprint Navel Warfare, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), Patch-For-Review: Compound order IDs should never end in ".0" - https://phabricator.wikimedia.org/T171891#3490744 (Ejegg) Good news: th... [20:21:09] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Patch-For-Review: Strict error message in logs (quick tidy up) - https://phabricator.wikimedia.org/T171560#3490745 (Ejegg) a:Ejegg [20:22:16] ejegg: can you check the changes I made in process_control - will try deploying them [20:22:26] sure thing eileen [20:23:07] ejegg: thanks [20:24:17] eileen: not sure how that quoted command is going to work, lemme try it locally [20:30:21] eileen: try prepending that quoted thing with 'sh -c' [20:31:30] eileen: oh wait, also, the echo should be at the start, before all the drush stuff [20:33:32] sure thing raynor ! [20:34:29] also, hi! I don't think we've met. Is most of the reading team going to wikimania? [20:34:49] ejegg: ug so I got it wrong? [20:35:23] eileen: i think python's subprocess thing can't deal with a command line that includes pipes [20:35:30] but it can start a shell to do that [20:36:13] and I don't think there's a drush echo command... [20:36:25] no big deal [20:36:30] no it should be an echo piped into drush [20:36:34] k [20:37:01] want me to fix it while I'm on there? [20:37:08] yes please [20:37:12] sure thing [20:37:13] (I wish drush would just cope with those spaces ) [20:37:17] yarp [20:37:28] pretty clever workaround, though! [20:38:53] ok I'm gonna do morning things & then have a go at fixing the groupMember so it can cope if the download takes longer than the time out (the stuff in the recipient one & not in there [20:39:47] eileen: cool. I pushed a change to the file, but I'll leave the deployment to you [20:40:54] fr-tech there are some easy reviews ready if you want to help boost the sprint stats [20:41:38] https://gerrit.wikimedia.org/r/369430, https://gerrit.wikimedia.org/r/364616 are pretty darn trivial [20:41:52] https://gerrit.wikimedia.org/r/367748 is also pretty small [20:43:30] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, Performance-Team, WMDE-Fundraising-CN: WMDE banners failing to save - Timing out on save - https://phabricator.wikimedia.org/T170591#3490807 (ggellerman) + @Amire80 @K4-713 for visibility given importanc... [20:57:21] fundraising-tech-ops, Operations, netops: bonded/redundant network connections for fundraising hosts - https://phabricator.wikimedia.org/T171962#3481495 (faidon) (T119654 is a restricted task, I have no access to it) While I don't think bonded ports are particularly problematic, I think we should be... [21:52:39] ejegg: did you see the test fail on the silverpop patch [21:53:10] oh right, I need to swap that password back to blank! [21:53:11] thanks [21:53:53] (PS2) Ejegg: Final export should have blanks instead of NULL [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) [21:54:04] ok eileen, that one should pass [22:00:18] (CR) Eileen: [C: 1] "Looks good I just had one question - not so familiar with python so it could be a python thing" (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) (owner: Ejegg) [22:01:58] (CR) Ejegg: "I swapped the test to look at the view we actually export the CSV from. The view does some null filtering and formatting, including 1->YES" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) (owner: Ejegg) [22:08:29] (CR) Eileen: [C: 2] "OK - makes sense" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) (owner: Ejegg) [22:10:42] (CR) Mepps: [C: 2] Restore unsubscribe tag replacement [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364616 (https://phabricator.wikimedia.org/T170302) (owner: Ejegg) [22:11:32] (CR) Mepps: [C: 2] Get rid of return-a-reference option [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/367748 (https://phabricator.wikimedia.org/T171560) (owner: Ejegg) [22:12:42] I'm going to deploy those cron changes now, so keep an eye out for any concerns off them [22:14:57] (Merged) jenkins-bot: Final export should have blanks instead of NULL [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/369430 (https://phabricator.wikimedia.org/T91692) (owner: Ejegg) [22:16:35] (PS3) Ejegg: Ingenico WX audit parsing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/368948 (https://phabricator.wikimedia.org/T86090) [22:16:47] (Merged) jenkins-bot: Restore unsubscribe tag replacement [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364616 (https://phabricator.wikimedia.org/T170302) (owner: Ejegg) [22:17:05] (Merged) jenkins-bot: Get rid of return-a-reference option [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/367748 (https://phabricator.wikimedia.org/T171560) (owner: Ejegg) [22:17:05] thanks eileen and mepps! [22:17:24] mepps: was too quick for me - I started looking at the others & she had done them [22:17:34] and I'll deploy the silverpop export fix [22:17:37] !log update process_control to c198b7f9c96c2eee3aadfa766d679c27354ede90 [22:17:47] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:18:01] ejegg: I'm on deploy box if you want me to push it [22:18:27] I still have to merge to deploy [22:18:32] ok [22:18:59] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/369566 [22:19:05] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/369566 (owner: Ejegg) [22:19:11] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/369566 (owner: Ejegg) [22:19:42] eileen if you're still up there feel free to push the 'tools' repo! [22:20:20] !log tools updated from 58bcbf3ddecb945bf522c76127c5aecf8a36a96e to 95542299ac1d06c6292cdfce22431ab615a77bf5 [22:20:26] done [22:20:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:20:31] thanks! [22:20:46] is it the same rsync blaster [22:21:08] looks like it [22:21:51] Fundraising Sprint Navel Warfare, Fundraising-Backlog, FR-Email, Patch-For-Review: Leave blank values in Silverpop file blank, not NULL - https://phabricator.wikimedia.org/T91692#1093457 (Ejegg) OK @CCogdill_WMF, we just deployed something to fix empty dates. I think after tonight's export you'll... [22:22:41] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3491255 [22:22:43] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Do we need to add handling for the 2 silverpop mails that would not retrieve properly - https://phabricator.wikimedia.org/T171433#3491253 (Eileenmcnaughton) Open>Resolved... [22:29:15] Fundraising Sprint Judgement Suspenders, Fundraising Sprint Kickstopper, Fundraising Sprint Navel Warfare, Fundraising-Backlog, and 4 others: retrieve lists of contacts who received a particular mailing - https://phabricator.wikimedia.org/T161762#3491282 (Eileenmcnaughton) @CCogdill_WMF I think w... [22:32:57] (PS4) Ejegg: Ingenico WX audit parsing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/368948 (https://phabricator.wikimedia.org/T86090) [22:33:21] woot, I think the SmashPig bit of WX file parsing is ready to go! [22:33:51] (PS13) Ejegg: [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [22:41:49] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3491295 (Eileenmcnaughton) @CCogdill_WMF I thi... [22:47:48] Fundraising Sprint Navel Warfare, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Make sure silverpop export does not add default date for last_donation date - https://phabricator.wikimedia.org/T171920#3491303 (Eileenmcnaughton) a:Eileenmcnaughton>Ejegg [22:49:43] cwd I managed to work around the old-drush issue - but it would be good to get it updated - is it worth adding an issue for that - I guess without it blocking us it would be pretty low priority [22:55:31] (CR) jerkins-bot: [V: -1] [WIP] Ingenico WX audit module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/199653 (https://phabricator.wikimedia.org/T86090) (owner: Katie Horn) [22:57:37] eileen: building our own drush package you mean? [22:58:52] would it be feasible to use a local drush from vendor? [22:59:54] well - I guess so - normally I install drush using composer [23:00:04] i can see the argument for it being a system tool but since it is a bunch of php closely related to civi it does seem like the thing you should be able to administer [23:00:21] *a thing [23:00:46] yeah - so you think we would just add to composer.json & run from there - I guess there is no security issue there [23:01:04] we don't have vendor under web root [23:01:15] & I guess we can have executatbles in there? [23:01:24] would the challenge being making sure it's in the system path for CI? [23:01:34] i can't imagine it's the only +x file in vendor [23:01:56] but even if it wasn't you could call it with php [23:02:04] I don't know it matters for CI [23:02:20] it's in order to be able to run scheduled jobs on production [23:02:50] ok gotcha [23:02:59] (which we already can, but a later version would probably be a little easier to work with, assuming the spaces issue is from drush version) [23:03:18] maybe jeff's opinion would differ but it seems like part of civi to me [23:03:39] would it be annoying to find the right path? [23:04:06] like the vendor path to drush? [23:04:35] well it would all be from files in process_control - the current version works for most things [23:05:04] Fundraising-Backlog: Can't access CiviCRM - https://phabricator.wikimedia.org/T172233#3491414 (DKaufman) [23:05:31] that vendor dir is in a static location right? [23:05:46] yeah [23:05:58] we could put it in puppet so it gets the system path [23:05:59] (also https://phabricator.wikimedia.org/T172233 - is probably you cwd) [23:06:07] and uninstall drush .deb [23:06:37] hmm ok - well I guess we should try installing & using via composer first & see how that goes [23:06:40] yes indeedie [23:06:57] eileen: would you change the jobs to point to the absolute path? [23:07:27] cwd - they currently do - ie "/usr/local/bin/drush --user=1 -v -r.... [23:07:48] yeah i mean the vendor absolute path [23:07:56] probably easier for me to change that wrapper [23:08:02] unless i'm missing something? [23:08:46] right, rather than update the jobs [23:09:07] (btw ejegg that attempt to deploy the job to get around the spaces doesn't seem to be running) [23:09:21] eileen: oh shoot, really? [23:09:43] cwd - Ok I'll work on a patch to vendor now & once that is deployed we can see if we can get it to run on it [23:09:53] ejegg: yeah - just radio silence on that job [23:10:09] Fundraising-Backlog, fundraising-tech-ops: Can't access CiviCRM - https://phabricator.wikimedia.org/T172233#3491442 (DStrine) a:Eileenmcnaughton>None [23:10:17] ok, sounds good [23:11:10] Fundraising-Backlog, fundraising-tech-ops: Can't access CiviCRM - https://phabricator.wikimedia.org/T172233#3491414 (DStrine) @cwdent and @Jgreen is his cert expired? Or could he need help managing it? [23:11:19] eileen: the schedule is 47 6 * * * [23:11:27] did you mean to only run it once a day? [23:11:32] oh - once a day [23:11:36] yeah I guess I did [23:11:45] you can run it manually on the civi box [23:11:49] but - I don't like being patient [23:11:56] with run-job omnimail_mailing_load_older1 [23:12:00] I really just wanted to check it's running [23:12:06] I usually do that under 'screen' [23:12:09] or with nohup [23:12:24] ah ok - running that now - cool [23:12:44] walp, it's pretty late here. I'mma head home. good night, and good luck! [23:12:46] if that works I'll schedule one or 2 more [23:12:56] since that is getting 4-8 weeks & we have 0 -1 [23:14:00] Fundraising-Backlog, fundraising-tech-ops: Can't access CiviCRM - https://phabricator.wikimedia.org/T172233#3491455 (cwdent) Hi @DKaufman, sorry you're having trouble Did this just start happening? What browser (version) are you using, and what OS? I have had a similar experience, nothing I have found... [23:14:36] it finished [23:14:50] so 4-8 weeks took only 2 minutes [23:15:16] so I'll schedule 1 to 4 weeks twice a day & that should cover it [23:17:43] great [23:17:46] thanks for tuning that [23:23:54] !log update process_control to ecb6669da763a5f1f79bc7cb0817770e5a1ad3bc [23:24:03] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:25:15] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Direct Mail Appeal not reflecting in contribution records for event - https://phabricator.wikimedia.org/T171794#3491502 (LeanneS) @mepps Thanks - the "DCEvent_17" is the value of the same direct mail appeal, but "2017 DC Event" is the label that should... [23:28:30] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3491515 [23:31:38] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3491525 (CCogdill_WMF) Sounds good! How long i... [23:32:45] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, and 8 others: retrieve the text/ html and statistics data for m... - https://phabricator.wikimedia.org/T161758#3491529 [23:34:00] Fundraising Sprint Navel Warfare, Fundraising-Backlog, FR-Email, Patch-For-Review: Leave blank values in Silverpop file blank, not NULL - https://phabricator.wikimedia.org/T91692#3491530 (CCogdill_WMF) Great, thanks! I'll purge the data from that field. Le mar. 1 août 2017 à 15:21, Ejegg !log update process control to33a0262d87aa1512888c82ac6b4ed91de3d3cac5 [23:46:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:50:27] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3491578 (Eileenmcnaughton) Actually I've incre... [23:50:43] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: retrieve the contacts who have signed up for email without going through our donor system - https://phabricator.wikimedia.org/T161759#3491580 (Eileenmcnaughton) [23:50:45] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising Sprint Navel Warfare, and 4 others: Reflect all unsubscribes via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161760#3491579 (Eileenmcnaughton) Open>Resolved