[00:00:02] cwd no - I haven’t been on there today [00:00:38] OK ty, I'll log in in a minute [00:00:46] (PS23) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [00:01:50] eileen: I've got the menu item appearing, but it never runs the code in SmashPigSettings.php [00:02:03] (PS24) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [00:02:35] So, the connection between the menu item and the actual page class is made via the xml/menu/*.xml file? [00:02:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1542 [00:03:53] ejegg: yep - but you are using civix? [00:04:08] you can add a page using civix generate:page & it creates that [00:05:09] oh hey, good call [00:07:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1842 [00:12:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2054 [00:12:52] NoraN: are you around? this alert ^ is from a query started by your civicrm user, guessing it's a search [00:13:07] sometimes those run away [00:13:25] eileen and/or ejegg might have tips on how to make it work... [00:13:34] anyway i'm going to kill it so it stops alerting [00:13:48] cwd yeah kill [00:14:01] if it igets to the point where it notifies you the user has long lost interest [00:14:19] (you can log a phab to get it fixed at some point with the query detail in it) [00:14:20] eileen: actually it's an insert, into civireport_activity_temp_target [00:14:25] heh yeah that's what i figured :) [00:14:26] :-( [00:14:42] is that not a search? [00:14:57] hmm - no it’s a report [00:15:03] should i not kill it? [00:15:07] yes kill [00:15:22] i can silence the alert if killing it will hurt anything [00:15:28] no kill it [00:15:30] kk [00:15:39] it’s the same as a search in terms of user engagement [00:15:54] they don’t have the ability to set them running & then ‘collect later' [00:16:19] yeah [00:17:37] (PS25) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [00:17:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [00:22:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [00:24:50] grr, still can't seem to make them connect even with the civix code [00:26:04] huh well i guess it is the replication thread that is going slow [00:26:08] so i can't kill it [00:27:31] oh that might not even be the problem query [00:27:36] there is one by user jsamra [00:27:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2954 [00:27:50] SELECT * FROM civicrm.civicrm_mailing_provider_data... [00:27:54] i'm going to kill that one [00:32:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3254 [00:33:30] hrm [00:37:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) [00:38:06] https://grafana-admin.wikimedia.org/dashboard/db/fundraising-database?orgId=1&from=1526344673115&to=1527727073131&panelId=28&fullscreen [00:38:22] it does seem that something started opening more temp tables but nothing alarming like before [00:39:48] eileen: is this report query a new thing? [00:40:19] cwd it is not a new report but they might have been trying it out [00:40:41] & params matter - e.g if there were no filters... [00:40:50] i don't understand why it's lagging only on the one server [00:40:58] it's the replication of this one query [00:41:14] “SELECT * FROM civicrm.civicrm_mailing_provider_data.” looks like a mysqldump query [00:41:39] eileen: this is: [00:41:42] INSERT INTO civireport_activity_temp_target [00:41:47] the other was a red herring [00:41:50] ah ok [00:41:58] yeah can def be kiled [00:42:13] well i can't seem to kill it cause it's the replication thread [00:42:20] ah :-( [00:42:35] FWIW that is the sort of thing that we might get some wins on through Tim’s work [00:42:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3854 [00:42:43] but the other boxes are not complaining so i don't get what the difference is [00:42:59] yeah i am very interested to see what he comes up with [00:43:45] so if i kill this thread replication stops [00:43:51] but the query just goes again when i restart [00:44:09] presumably it is starting from scratch so i am just delaying it finishing :P [00:45:51] :-( [00:47:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4154 [00:50:36] ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4154 Casey Dentinger replication is lagging, not sure why - The acknowledgement expires at: 2018-05-31 13:49:11. [00:52:08] well that should be good till the morning [00:52:23] hopefully the query finishes and it's an isolated incident :) [00:52:30] * cwd continues cleaning garage [00:57:41] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 2332952 Threads: 1 Questions: 45461990 Slow queries: 169953 Opens: 20705 Flush tables: 1 Open tables: 1198 Queries per second avg: 19.486 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [00:58:15] yay [03:35:48] (PS26) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [03:40:05] (CR) jerkins-bot: [V: -1] WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) (owner: Ejegg) [06:49:05] (PS1) Eileen: WIP privacy extension. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/436465 [06:55:11] (PS2) Eileen: WIP privacy extension. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/436465 [15:14:40] hi jgleeson! [15:14:44] how was the vacation? [15:28:43] hey ejegg, great thanks! [15:28:49] glad to be back now [15:32:16] right on [15:32:52] jgleeson: so, I spent a while working on updating the audit processing to work with txns from both APIs [15:32:59] that stuff is waiting for some code review [15:33:38] there is on patch in SmashPig: https://gerrit.wikimedia.org/r/435294 [15:33:43] *one [15:34:05] (the other SmashPig patches are speculative right now, I should -1 them till we decide they're a good idea [15:34:38] welll, one is probably good, but just a distraction now [15:34:49] the other would change how we store the gateway account [15:35:36] Anyway, there are two patchs in the wmf_audit module that could be reviewed too: https://gerrit.wikimedia.org/r/435220 and https://gerrit.wikimedia.org/r/435286 [15:35:57] then there's a third in that same module in WIP, waiting for the SmashPig bit [16:32:33] fundraising-tech-ops, Operations: Long term storage for frack prometheus data - https://phabricator.wikimedia.org/T175738#4246366 (RobH) [16:33:30] Socialist cybernetics in Chile: https://en.wikipedia.org/wiki/Project_Cybersyn [16:33:52] sadly, destroyed by Pinochet [16:35:20] ooh, there's an alternate history novel about it, based in a 1979 where Allende wasn't overthrown! [16:37:15] (PS27) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [16:37:47] wow, I never knew about that [16:39:58] also covered on the 99% invisible podcast: https://99percentinvisible.org/episode/project-cybersyn/ [16:45:59] I can't help but wonder what if, after reading that [16:47:35] the biggest criticism of theoretical centrally planned economies is that no one person or council can know all things and therefore not organise efficiently. [16:50:53] "Each chair had an ashtray, a place for your whiskey glass and a set of buttons that controlled the display screens on the walls." [16:51:08] all the essentials covered... [17:22:21] Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Fr-Ingenico-integration_2017-18: Ingenico recurring messages missing token - https://phabricator.wikimedia.org/T195488#4246615 (jgleeson) a:jgleeson [20:26:30] (PS1) Ejegg: WIP: add translations for monthly donation message [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/436632 [20:27:09] fr-tech: ugh, I'm having a hard time figuring out how to get the monthly donation message translated [20:27:33] so one problem is that the built-in i18n functions are all based on the idea that there is a single ambient locale for any thread [20:27:33] hmm ejegg i don't know a lot aobut how, how do normally go about it? [20:27:56] mepps we're currently using the DonationInterface i18n json files within CRM [20:28:41] there's this whole shim: https://github.com/wikimedia/wikimedia-fundraising-crm/blob/16e5944cbcdbbbf817f6300d9aba50bfcc5f142a/sites/all/modules/wmf_communication/MediaWikiMessages.php [20:30:03] I could copy all that over to the extension, but it would still need some editing, since the DonationInterface messages we need (monthly-donation-description) specifically mention 'the Wikimedia Foundation' [20:30:48] there's a setting in config for 'customTranslateFunction' [20:31:29] and I supposed we could write a wrapper that used MediaWikiMessages when a special parameter was present, like 'message-key' [20:32:23] and just fell back to CRM_Core_I18n::crm_translate when the parameter is missing [20:32:32] but that would intercept every single ts() call [20:32:41] so... maybe really inefficient? [20:36:26] is this for the ingenico recurring extension ejegg? [20:36:33] mepps exactly [20:36:51] hmm i don't love making a civi extension dependent on wmf custom code... [20:37:00] there's one line that we need to send for the description that goes on the CC statement [20:37:05] could we do use crm_translate natively? [20:37:08] right, that's the big trick [20:37:27] so crm_translate assumes there's one language being used throughout the whole process [20:37:47] and we need to use a different language for each donation [20:37:55] ahh [20:39:22] We can copy all the mw messages, plus the MediaWikiMessages shim, over into the new extension [20:39:27] ejegg, is it possible to open up or overwrite the config being used by CRM_Core_I18n::crm_translate using something dirty like reflection ? [20:39:40] on a case by case basis [20:40:17] jgleeson: there is a way for an extension to use a custom translate function [20:40:41] So we could copy the relevant messages into the extension as in https://gerrit.wikimedia.org/r/436632 [20:40:53] then add the MediaWikiMessages shim to access those [20:41:24] time for a fundraising specific translation program? :) [20:41:46] mw-- [20:42:16] We'd get something fairly good, with some weirdness like dropping 'the' and having 'Wikimedia Foundation' always being in English [20:42:48] So people's statements in English would go from 'Monthly donation to the Wikimedia Foundation' to 'Monthly donation to Wikimedia Foundation' [20:43:28] while in Spanish it would go from 'Donación mensual a la Fundación Wikipedia' to 'Donación mensual a Wikimedia Foundation' [20:43:57] jgleeson there's the other option, to set the customTranslateFunction in config [20:44:36] and just add a couple of parameters to our translate call in the extension, which would only matter to our (local) customTranslateFunction [20:45:13] the local-only customTranslate function could live in wmf_communication, and just look for those params (like message-key) [20:45:35] if those params exist, it uses the MediaWikiMessage, if not, it passes through to the existing logic [20:45:53] that way, we get to use exactly the same messages as ever, with no editing [20:46:33] but... we are adding a couple of stack frames and an array key lookup to every single ts() call in CRM [20:47:26] hmmm [20:47:38] mepps any preference between those two ^^^ [20:48:33] hmm if i'm understanding the second one correctly it sounds like it might be better [20:49:47] mepps using the config->customTranslateFunction ? [20:49:55] It would certainly make the extension simpler! [20:49:55] yes ejegg [20:51:07] ok, I'll proceed with the extension without messing with translation for now [20:51:27] maybe just add 'language' and 'message-key' parameters to that one ts call [20:51:51] is there a requirement elsewhere in the extension to use the vanilla ts() behaviour? [20:52:36] I'm wondering if you could just write something independent instead of merging the two [20:53:18] hard to propose anything solid with my limited knowledge :) [20:54:03] jgleeson: we could use https://github.com/wikimedia/simplei18n or the like [20:54:24] I'm just a little wary of bloating the thing [20:56:01] hey that looks nice [20:57:20] so the config to override the default translation message would come with the extension? [20:57:48] jgleeson: I'd leave it out of the extension itself [20:57:49] and I'm guessing you'd just add a conditional in that space to take effect if your parameter exists, and if not proxy on to original behaviour? [20:57:55] oh I see [20:58:14] but i'd set it in the wmf_civicrm.install script that activates the extension [20:58:35] got it [20:58:46] right, only change the behavior on param existing [21:00:19] oooh, wait, there might be a way to switch languages after all! [21:00:23] let's see... [21:00:33] (within the existing Civi i18n) [21:01:02] honestly it would surprise me if there weren't ejegg, but my initial exploration didn't find it [21:01:42] catch up with you both tomorrow to see how it went [21:01:47] hopping off for tonight [21:01:51] g'night fr-tech [21:03:27] mepps, looking at this: https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/I18n.php#L634 [21:04:22] slightly wary of that setNativeGettextLocale call [21:05:53] http://php.net/manual/en/function.setlocale.php [21:06:11] The locale information is maintained per process, not per thread. If you are running PHP on a multithreaded server API like IIS, HHVM or Apache on Windows, you may experience sudden changes in locale settings while a script is running, though the script itself never called setlocale(). This happens due to other scripts running in different threads of the same process at the same time, [21:06:17] changing the process-wide locale using setlocale(). [21:06:34] So.... as long as nobody runs that script from the web UI we should be fine :P [21:06:56] otherwise everybody's UIs would be liable to bounce around to different languages [21:06:59] boo [21:07:43] getting antsy and snacky here at the apt, gonna head to the library. [21:34:11] ok, back! [21:39:19] so... punting on the translation, just need to add more tests [22:53:17] (PS28) Ejegg: WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) [22:57:29] (CR) jerkins-bot: [V: -1] WIP: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T1888678) (owner: Ejegg)