[00:00:02] (Merged) jenkins-bot: Handle banner loader errors on client [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/320317 (https://phabricator.wikimedia.org/T149107) (owner: AndyRussG) [00:09:45] (CR) Eileen: "I just pushed a change to do the file_exists check. I'm not sure about more forgiving handling of the case of csv and the file path since " (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319758 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [00:10:02] (PS2) Eileen: Add option to download file of skipped, ignored, and error from upload. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319758 (https://phabricator.wikimedia.org/T88460) [00:10:45] (CR) Eileen: "NB - am working on a follow up patch right now to address the desire for an extra output file & a test - perhaps it's better just amended " [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319758 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [00:17:36] fr-tech cn deploy peachy [00:18:21] \o/ [00:18:54] eileen1: if you saw "is not a valid option for field custom_26" would you immediately know what table that's talking about? [00:19:05] custom group? option group? [00:19:19] or ejegg ^ [00:19:59] what was the option value again? [00:20:58] in this file it is \'Unresctricted - General\' [00:21:29] is 26 an ID? [00:22:03] ok, that's the 'Restrictions' field in 'Gift Data' [00:22:27] ah ok yeah so 26 is the id [00:22:50] where are the options for a custom field stored? [00:23:08] yeah, I guess its ID in civicrm_custom_field [00:23:24] let's see - I think they're in civicrm_custom_group [00:23:25] yep looks to be, that must be a foreign key somewhere? [00:24:20] err, no - that's civicrm_option_group [00:24:25] and civicrm_option_value [00:24:43] cwd - so the table civicrm_custom_field tells you 2 things [00:24:50] 1 the column_anme [00:24:52] name [00:24:58] 2) the custom_group_id [00:25:15] the civicrm_custom_group table has a table_name column [00:25:29] aaaah [00:25:42] & the civicrm_custom_field table has an option_group_id - which links to an option set [00:25:53] btw you can get the options for most fields using the api [00:26:04] the old kv store in the relational db trick [00:26:10] how do i use the api? [00:26:20] $options = civicrm_api3('Contribution', 'getoption', array('name' =>'custom_26') [00:26:31] do you drush that? [00:26:32] (I think that's it - it might be field rather than name - let me check) [00:26:47] drush cvapi Contribution.getoptions name=custom_26 [00:27:10] neat! [00:27:31] you can do any api call using drush cvapi [00:28:04] ah no -it's field not name [00:28:14] drush cvapi Contribution.getoptions field=custom_26 [00:28:40] awesome [00:29:13] also we have a function that takes the value in civicrm_custom_field.name & returns (for example) custom_26 [00:29:15] wmf_civicrm_get_custom_field_name('parent_contribution_id') [00:29:29] if you need to convert that way [00:29:52] haha damn it...it says Unresctricted [00:30:31] this was a roundabout but educational way to notice a typo [00:30:43] oh man [00:32:32] (PS1) Cdentinger: s/Unresctricted/Unrestricted/ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320327 [00:32:39] oh right - I didn't realise you hadn't spotted that! [00:32:55] * cwd feels shame [00:33:21] hmm - I was hoping some errors would be left - still probably clearer to fix them all & put some back [00:33:22] (CR) Ejegg: [C: 2] s/Unresctricted/Unrestricted/ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320327 (owner: Cdentinger) [00:33:41] (CR) Eileen: [C: 2] s/Unresctricted/Unrestricted/ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320327 (owner: Cdentinger) [00:34:53] file is missing mega required fields. i should probably just make a new one. [00:35:09] but i wonder if this file ever imported properly, given the typo [00:38:28] (Merged) jenkins-bot: s/Unresctricted/Unrestricted/ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320327 (owner: Cdentinger) [00:39:10] cwd - I don't know - if was perfect for me because I was working on the output of errors! [00:39:45] hehe well that's an upside [00:40:18] :-) [00:41:22] eileen1: so as not to duplicate effort do you already have a method of importing some test contributions? [00:42:24] cwd not really - I was working with failed ones [00:42:32] which ticket are you on [00:43:12] eileen1: well i was just hoping to test the silverpop export [00:43:38] for this one: https://phabricator.wikimedia.org/T148578 [00:43:40] ah OK - so you just wanted to import in order to export? [00:44:10] but, with silverpop aren't you better to bite the bullet & test on staging - incase the query turns out to take 8 hours longer than expected..... [00:44:44] hmm yeah that's a good point [00:45:28] yes import just to export [00:48:12] eileen1: I'm doing that test there now [01:01:22] (PS1) Eileen: CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320329 [01:04:54] !log disable jobs GlobalCollect Recurring Donations [01:04:54] Globalcollect WR1 Log Audit [01:04:58] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:05:58] !log disable jobs http://localhost:9000/job/Dedupe%20CiviCRM%20contacts/ http://localhost:9000/job/Dedupe%20CiviCRM%20contacts%20(name-match)/ Thank you mail send [01:05:58] Donations queue consume [01:06:04] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:06:53] !log disable GlobalCollect audit file [01:06:58] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:14:08] (CR) Eileen: [C: 2] CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320329 (owner: Eileen) [01:17:05] !log disabled Public Data Export [01:17:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:17:28] (Merged) jenkins-bot: CiviCRM submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/320329 (owner: Eileen) [01:23:19] !log enabled Thank you mail send [01:23:20] Donations queue consume [01:23:20] Globalcollect WR1 Log Audit [01:23:24] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:27:30] !log enabled GlobalCollect Recurring Donations Dedupe CiviCRM Contacts GlobalCollect audit file download Fundraiser Public Data Export [01:27:35] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:40:55] (PS1) Eileen: Merge Submodule update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320336 [01:43:55] (CR) Eileen: [C: 2] Merge Submodule update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320336 (owner: Eileen) [01:44:05] (Merged) jenkins-bot: Merge Submodule update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320336 (owner: Eileen) [01:48:33] !log update CiviCRM from 45a1b9a0e1665b1fc71165e1bc4bcfdca2a9adf7 to 63d35fc5f5c065168efd2250c2741b3603b2eb92 [01:48:38] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:59:05] Fundraising Sprint Unbreaking Now, Fundraising-Analysis, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: Contribution report: add column for preferred language - https://phabricator.wikimedia.org/T146942#2778573 (Eileenmcnaughton) @LeanneS I added preferred_language to some of the repor... [11:10:05] hey the-wub Seddon do you happen to know the impact of the black/white wp logo in the wmf mails? [11:10:54] I guess caitlin is located in SF and therefore not available before late afternoon [13:05:58] Till_WMDE: caitlin thought it was potentially a small improvement. we'll probably re-test it soon [13:06:05] (and you're right, she is in SF) [13:43:57] thanks the-wub [13:44:38] I will check with her later. will she be in this chat the-wub ? [13:47:13] Till_WMDE: yes usually. her username is ccogdill [13:47:57] thanks [14:08:08] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Operations, Traffic: CentralNotice: Review and update Varnish caching for Special:BannerLoader - https://phabricator.wikimedia.org/T149873#2767284 (Gilles) >>! In T149873#2767335, @aaron wrote: > The first approach might work using Varnish... [16:28:14] (PS5) Ejegg: Fix missing timezone / postal code bug [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/320233 (https://phabricator.wikimedia.org/T148578) [16:37:41] hi cwd. I timed that new query and it's not too bad [16:37:58] how much worse than before? [16:38:00] still only getting zip and tz on like 3/4 of US donors though [16:38:09] ah hrmph [16:38:33] I think a few minutes - let me look at the timing on last night's run [16:39:29] wondering if there are many US donors (as determined by contribution_tracking table) where we don't have an address [16:39:40] but those queries are taking forever [16:41:15] ah could those be the missing ones? [16:41:23] it's one possibility [16:41:26] or if the address was invalid? [16:41:46] well, for the 1/4, there's no zip code at all in the export [16:42:02] and tz looking is based on zip? [16:42:06] *lookup [16:42:09] yeah [16:42:29] but that deduplication in the export is somewhat sketchy [16:43:42] I dunno about the 'address country has to match latest contribution_tracking country) where clause [16:45:31] heh, sorry about the syntax error ^^ [16:46:46] hehe [16:48:04] i am not seeing that clause... [16:48:22] oh wait i'm on an old ps [16:48:42] just trying to match the previous logic [16:49:20] so latest ct country is paramount, and you don't get a nonsense country/state combination [16:54:26] i'm confused about how this is the ct country, all the tables say civicrm [16:54:53] hmm? lemme see where that came from [16:55:19] you are talking about this query? https://gerrit.wikimedia.org/r/#/c/320233/5/silverpop_export/update_table.sql [16:56:06] ah, it's the query on line 108, not shown in that diff [16:57:48] man gerrit diff is terrible [16:58:06] I'll see if we get more by relaxing that is_primary = 1 like the old one had [16:58:57] oop, got to hit a meeting [17:00:16] fr-tech: The cable TV sex channels don't expand our horizons, don't make us better [17:00:16] people, and don't come in clearly enough. [17:00:16] -- Bill Maher [17:00:16] -- discuss. [17:03:48] fundraising-tech-ops, Operations, ops-codfw: payments2002 disk failure - https://phabricator.wikimedia.org/T149646#2780082 (Papaul) Talked with HP again today . I will be receiving a 1TB 7.2k 3.5" disk by tomorrow. According to them they no longer carry 500GB 7.2K 3.5" disks [17:37:01] ejegg: this log_civicrm_db doesn't seem to be around on vagrant, or in the silverpop export config in /etc/fundraising. i added it there but wondering where the schema is? [17:37:25] erm i should say i just had provision bomb and haven't dug into it [17:37:29] probably do that first [17:38:48] cwd ohh yeah, log tables aren't turned on by default for vagrant - we should do that [17:39:02] lemme see what that'll take. hopefully there's a drush command [17:41:44] huh, not seeing it. [17:41:57] cwd you can get those tables by flipping the setting on in the UI: [17:42:12] https://civicrm.org/blog/shot/new-civicrm-33-feature-logging [17:42:38] oooh, thanks [17:43:36] When we push a same-day fix in response to a customer’s tweet, agonize over the best way to slip some humor into release notes [17:43:42] NECKBEARD [17:43:45] god [17:44:04] ?? [17:44:07] ...sorry wrong channel... [17:44:15] but for context https://slackhq.com/dear-microsoft-8d20965d2849#.cg8iek6yw [17:44:41] * cwd ruffles feathers [17:45:32] oh jeez, that was a really cringeworthy open letter [17:45:42] seriously [17:47:41] but also how microsoft is try to elbow its way in with the relevant crowd again [17:53:45] there are few worse feelings than typing into the wrong channel [17:57:38] Fundraising-Backlog, fundraising-tech-ops: fundraising-drupal log repeats itself - https://phabricator.wikimedia.org/T150221#2780208 (Jgreen) Looks to me as though the application is logging twice, slightly differently each time. I don't think it's a syslog issue. [18:04:56] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, and 8 others: [Epic] Do not show donation form error message: "No processors available". Fix... - https://phabricator.wikimedia.org/T117872#2780246 [18:09:14] huh, if I use another intermediate table I get way more US address info for the mailing list. gotta be a join problem someplace [19:14:33] (PS6) Ejegg: Fix missing timezone / postal code bug [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/320233 (https://phabricator.wikimedia.org/T148578) [19:23:17] Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, Fundraising-Backlog, FR-2016-17-Q2-Campaign-Support, Patch-For-Review: Add time zone data back into Silverpop export file - https://phabricator.wikimedia.org/T148578#2780525 (Ejegg) That last patch should make sure a... [19:27:58] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, and 8 others: [Epic] Do not show donation form error message: "No processors available". Fix... - https://phabricator.wikimedia.org/T117872#2780547 [19:28:24] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, and 8 others: [Epic] Do not show donation form error message: "No processors available". Fix... - https://phabricator.wikimedia.org/T117872#2780548 [19:32:24] Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, Fundraising-Backlog, FR-2016-17-Q2-Campaign-Support, Patch-For-Review: Add time zone data back into Silverpop export file - https://phabricator.wikimedia.org/T148578#2780560 (CCogdill_WMF) Yeah, 85% about matches wha... [19:34:10] (CR) Cdentinger: [C: 2] Fix missing timezone / postal code bug [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/320233 (https://phabricator.wikimedia.org/T148578) (owner: Ejegg) [19:35:01] (Merged) jenkins-bot: Fix missing timezone / postal code bug [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/320233 (https://phabricator.wikimedia.org/T148578) (owner: Ejegg) [19:37:21] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/320442 [19:37:39] (CR) Ejegg: [C: 2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/320442 (owner: Ejegg) [19:37:46] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/320442 (owner: Ejegg) [19:39:21] ugh my touchpad just went nutso [19:39:22] !log updated fundraising tools from 7ff719a466bb9ecbdb5f444f67d67903456f6fdb to d14d47a83bd822d28da0f2d03afbd74008e215a1 [19:39:27] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:39:27] gonna restart [19:39:29] cwd oh noes! [19:41:45] that seems to have fixed it [19:41:47] whew [19:42:13] i got apple juice on it and it started bouncing around clicking on everything [19:56:53] eileen1: are you online? [19:57:28] ejegg: and I are discussing this: https://phabricator.wikimedia.org/T150281 [19:57:55] MG is having a lot of problems with imports as of yesterday [19:59:55] dstrine: yes [20:00:24] dstrine: this is the existing issue & you want an update re the fix? or a new issue? [20:01:32] nm I think ejegg found the issue [20:01:38] He is commenting [20:01:52] ok [20:02:48] I have the patch to improve the error experience - I'm just writing unit tests & have decided there is another minor tidy up to do - but expect to put that to QA very soon [20:36:23] Fundraising Sprint Testing on Production, Fundraising Sprint Unbreaking Now, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Spike: Spike: Central Notice: Consider options for monitoring versions of banners shown - https://phabricator.wikimedia.org/T148593#2727179 (AndyRussG) Here's... [21:37:09] (CR) Ejegg: [C: 2] "Looks good!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319757 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [21:38:02] ejegg: I'm reworking the import part with a test & some changes - I should push it up soon [21:38:17] cool, thanks [21:38:19] not sure if you think I should amend it into the commit in the review q [21:38:24] or add a second commit [21:41:13] either way is fine [21:44:38] ejegg: OK - I'm just trying to think about getting it out to MG since current stuff is about 20 mins away - but it directly impacts on their day & I am aware I will need you to find time to review [21:45:08] ahh, so if it doesn't change the functionality I can just review the existing patch [21:45:23] well it does a little - but just a few more enhancements [21:45:30] the existing one could go out [21:45:50] actually - the new version adds the 'download all skipped' [21:47:25] looking at the existing one in any case [21:47:30] (Merged) jenkins-bot: Alter WMFException to permit us to present a simpler error for users. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319757 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [21:48:01] I'm super close actually (famous last words) [21:54:00] well, I can't find anything functional to complain about! [21:55:59] maybe merge that & we'll deploy & then treat the next one as a second patch [21:59:10] (CR) Ejegg: [C: 2] "This oughtta work! Comments about comments inline" (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319758 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [22:04:07] (Merged) jenkins-bot: Add option to download file of skipped, ignored, and error from upload. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/319758 (https://phabricator.wikimedia.org/T88460) (owner: Eileen) [22:18:41] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320525 [22:20:29] Fundraising-Backlog: Engage isn't able to edit addresses due to latitude and longitude - https://phabricator.wikimedia.org/T150305#2781348 (LeanneS) [22:20:34] (CR) Eileen: [C: 2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320525 (owner: Eileen) [22:28:26] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Unable to delete a banner with bad HTML code - https://phabricator.wikimedia.org/T150265#2781380 (AndyRussG) [22:28:36] Fundraising-Backlog: Engage isn't able to edit addresses due to latitude and longitude - https://phabricator.wikimedia.org/T150305#2781381 (ggellerman) p:Triage>High [22:28:51] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/320525 (owner: Eileen) [22:29:19] Fundraising Sprint Unbreaking Now, Fundraising-Backlog: Engage isn't able to edit addresses due to latitude and longitude - https://phabricator.wikimedia.org/T150305#2781348 (ggellerman) [22:31:09] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Unable to delete a banner with bad HTML code - https://phabricator.wikimedia.org/T150265#2781402 (ggellerman) p:High>Normal [22:32:49] Fundraising-Backlog, fundraising-tech-ops: fundraising-drupal log repeats itself - https://phabricator.wikimedia.org/T150221#2781406 (ggellerman) p:Triage>Normal [22:36:46] Fundraising-Backlog, FR-2016-17-Q2-Bugs: Ingenico audit: donation from 11/1 not in Civi - https://phabricator.wikimedia.org/T150212#2781412 (ggellerman) p:Triage>Normal [22:37:48] Fundraising-Backlog: Are we using 3DS APi call for Ingenico? - https://phabricator.wikimedia.org/T150192#2781421 (ggellerman) p:Triage>Normal [22:38:53] Fundraising-Backlog: Are we using 3DS APi call for Ingenico? - https://phabricator.wikimedia.org/T150192#2777138 (K4-713) Check for enabled 3DS currencies in LocalSettings. I believe there's an array we can alter fairly trivially if enabling doesn't hurt conversion too much. [22:39:06] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Reconsider minfraud failure mode - https://phabricator.wikimedia.org/T150072#2781427 (ggellerman) p:Triage>Normal [22:47:26] Fundraising-Backlog, Patch-For-Review, WMF-deploy-2016-11-08_(1.29.0-wmf.2): Investigate and bring back data & functionality previously held in contribution tracking table - https://phabricator.wikimedia.org/T150055#2781444 (ggellerman) p:Triage>Normal [22:57:17] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Operations, Traffic: CentralNotice: Review and update Varnish caching for Special:BannerLoader - https://phabricator.wikimedia.org/T149873#2781465 (ggellerman) p:Triage>Normal [22:58:35] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Autofocus on first field (First Name) on payments.wiki - https://phabricator.wikimedia.org/T149803#2781467 (ggellerman) p:Triage>Normal [22:59:56] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: Deal with diacritic conflicts on name checks - https://phabricator.wikimedia.org/T149763#2781469 (ggellerman) p:Triage>Normal [23:25:58] !log update CiviCRM from 63d35fc5f5c065168efd2250c2741b3603b2eb92 to 7ee2ce47973c691e32d97ca6fed55df037cf585b [23:26:03] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:30:41] (PS4) Ejegg: Add highest_native_amount to Silverpop export [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/318360 (https://phabricator.wikimedia.org/T135407) [23:31:53] (PS5) Ejegg: Add highest_native_amount to Silverpop export [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/318360 (https://phabricator.wikimedia.org/T135407) [23:32:04] checking the timing on ^^^ now [23:53:53] Fundraising Sprint Unbreaking Now, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Engage isn't able to edit addresses due to latitude and longitude - https://phabricator.wikimedia.org/T150305#2781624 (DStrine)