[12:38:40] Wikimedia-Fundraising: Following link to donate fails to return user to previous page upon completion - https://phabricator.wikimedia.org/T60506#2206585 (Pcoombe) [13:03:03] Fundraising-Backlog: Ingenico: iDEAL form error with Abn-Amro bank - https://phabricator.wikimedia.org/T132680#2206643 (MBeat33) [14:53:12] Jeff_Green: hello! I was hoping to learn your current thinking about the logging triggers, if this is a good time? [14:53:19] sure [14:53:52] Mostly, we wanted to know if the replication failover trigger thing seems like a blocker to you? [14:54:22] as of this moment I would say replication is a blocker altogether, because I don't know/understand how triggers behave in mixed replication [14:54:57] yah. That was about to be a subquestion--whether mixed-mode replication is trustworthy when we build all our data using triggers. [14:55:09] right [14:55:16] I guess we're using mixed rather than row-based replication for performance reasons? [14:55:49] we made that switch a while ago, I don't entirely remember why we did it at the time [14:56:02] might be notes on a ticket somewhere, looking.... [14:56:04] Keeping up with the Jonses :) [14:56:23] nahh, there was actually a problem that we were working around [14:56:44] The plot thickens... [14:58:39] I don't think this is a good idea, but... one fringe suggestion might be use row-based replication for just the logging database. [14:59:35] I don't think you get to choose by database, but maybe I'm wrong [15:00:08] there's like 0.02% chance I will find the history on this... [15:00:14] hehe. k [15:00:26] why do ticketing systems never have usable search? [15:00:28] I'll poke at my email a bit [15:00:58] replication specifics aside, I have a very dim view of triggers overall [15:02:13] sorry i'm new to this conversation but wouldn't you not want triggers on a replicate? assuming the triggers are writing rows. [15:02:35] cwd you want different things depending on your replication method [15:02:45] Me too, but they do have some properties which make them decent for simulating a multidimensional database, like running regardless of application code screwups [15:03:23] i was able to offload a _ton_ of work to some triggers at the last job [15:03:24] cwd https://dev.mysql.com/doc/refman/5.7/en/replication-features-triggers.html [15:03:41] Jeff_Green: "MariaDB Logging: Going from STATEMENT to MIXED" [15:03:55] Email from mwalker on 2013-10-25 [15:03:58] awight ooh looking [15:04:45] That seems like it was a good call. But this certainly doesn't preclude row-based replication. [15:05:15] ah right [15:05:48] when we started working with mariadb it spewed all kinds of errors about our queries [15:06:12] cos it was built from slightly firmer cheese than mysql... [15:06:27] so yeah this is the kind of thing that gives me serious concern with triggers, we don't ~totally~ understand replication, and we don't ~totally~ understand triggers [15:07:11] i have only used them in postgres so i definitely can't vouch for them in mysql :P [15:07:17] and now we're talking about deploying ~350 triggers, which are essentially code, and few people will have full visibility or comprehension [15:07:25] Strange, this doesn't even mention mixed replication: https://dev.mysql.com/doc/refman/5.7/en/replication-features-triggers.html [15:07:45] i don't think it's so much mysql vs other db's, I think it's a basic design philosophy thing [15:08:19] awight: right, so what happens with triggers in that case? mariadb is making a decision about which replication mode is appropriate for the context [15:08:33] so which does it choose? how does that affect transactions/rollback [15:08:43] oh, I just meant the fact that mariadb actually told us about the warning conditions [15:08:58] Jeff_Green: yeah, I'm reading through that now. https://dev.mysql.com/doc/refman/5.7/en/replication-formats.html and https://dev.mysql.com/doc/refman/5.7/en/binary-log-mixed.html [15:09:09] It's occuring to me that I should read the mariadb manual instead, however [15:09:10] i was responding to the postgres comment [15:09:29] http://www.oracle.com/technetwork/testcontent/o58asktom-101055.html [15:09:49] Jeff_Green: like keeping business logic out of the database you mean? [15:09:53] that ^^ says the things that rattle around in my brain much better than I can [15:10:15] cwd yeah, not putting code in a database seems like a good call from an ops perspective [15:10:28] Jeff_Green: about transactions--eileen and I talked about that, and she's mostly convinced me that it's actually fine to have un-rolled-back information in the logging table, about things which were actually rolled back. [15:10:31] like right at this moment you can't tell what's happening [15:10:39] * awight collides actuallies [15:11:22] yeah it's a cost benefit thing. it definitely complicates everything, but in the last case anyway we simply couldn't do this at another layer cause it was too slow [15:11:59] i documented/version controlled all the triggers and the functions they called [15:12:25] Jeff_Green: skimming the Ask Tom critique, I don't think that applies to our case. The triggers within the civicrm database which do business logic things are horrendously evil, of course. But this logging thing is very orthogonal. [15:12:52] Pretty much what cwd said. [15:12:55] awight: I have no idea, I haven't reviewed the >350 triggers to determine what they do [15:13:09] I can imagine :-) but I'm not going to have time to do that ever [15:13:21] yeah that's a shit load of them [15:13:29] In theory, they are only copying our civicrm tables into a Type 2 slowly changing dimension form in the logging db [15:13:49] do you know how many different functions they are calling? [15:14:00] err https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_Add_New_Row [15:14:47] cwd: Seems that they aren't calling any custom functions, but a ton of IFNULL() [15:14:54] is there somewhere i could check out these triggers? [15:15:32] cwd: crm/civicrm/sql/trigger_create.mysql [15:15:32] awight: this almost seems like something we could do by storing the replication logs :-P [15:15:54] Jeff_Green: ino. But having them in this known and accessible form lets us actually use them from application code [15:16:41] awight: ya [15:17:02] awight: is this on master yet? [15:17:12] and admittedly I don't totally understand how this all works from an application side [15:17:15] i see trigger.mysql but not _create [15:17:18] cwd: only on lutetium [15:17:18] Jeff_Green: What would you suggest for how we can make an educated judgement about this? Should we pull in jcrespo, or just reason through the docs? [15:17:24] thanks [15:17:30] cwd: also you need root privs to see triggers [15:17:36] which is another part of the problem [15:17:50] cwd: I believe it is on our civicrm master [15:17:50] the civi design is based on civi having root db privs [15:17:59] it is not on the civi master [15:18:01] * awight greens [15:18:36] cwd: I have it on civicrm Ic8a998d19583ad78da5821d1f50d2fa7f86a97de at least [15:18:48] so in that other file...looks like there are some existing triggers in civi? [15:18:58] there are a couple yeah [15:18:59] cwd: yeah, those are just horribly unfortunate. [15:19:01] that look like they should be handled by foreign keys [15:19:05] We can actually live without them [15:19:12] which we've periodically lost over the years, causing snafus [15:19:39] The other triggers actually do call stored custom procedures. [15:20:02] and I vaguely remember we're *not* be able to live without the procedures, cos they're called from app queries now. [15:20:02] 10734 lines [15:20:07] o_o [15:20:20] cwd: it's all machine-generated, though. You could just look at the code to build the triggers... [15:20:44] ./CRM/Logging/Schema.php [15:21:36] The triggers are built using schema introspection. Wheee! [15:21:43] What could go wrong. [15:22:03] imagine a world where 2 triggers fail to get installed :-) [15:22:10] ino [15:22:42] so these are all just edit-time replication to a log? [15:22:58] "just" [15:23:42] yeah [15:24:23] what goes into the log? essentially each value that changed? [15:24:47] an full copy of the new record's values [15:25:18] oic [15:25:32] * awight abandons grammar to the winds [15:25:56] lessee. It's the new row, plus a couple of logging-specific columns. [15:25:58] isn't this what replication already does? [15:26:32] The logging columns are log_conn_id, log_user_id, log_action [15:26:36] cwd: yeah, although what presents itself the replication log depends on your mode of replication [15:26:42] filled like, CONNECTION_ID(), @civicrm_user_id, 'insert' [15:27:05] cwd: The difference is that this log is accessible from the application. [15:27:14] Check it out on the staging Civi. [15:27:24] it just seems like there must be a more systemic way to do this than a ton of individual triggers [15:27:49] Well. There are databases that support this sort of thing natively, but they're big $$ and we'll never port Civi to them [15:28:13] fwiw, https://en.wikipedia.org/wiki/Log_trigger [15:28:44] i mean, I can see the logic of doing it this way from a purely civi developer's perspective [15:28:53] most civi installs probably do not involve replication [15:29:23] possibly, we're one of two installations that might have replication yeah [15:29:36] yeah i've totally made log triggers that called functions, but it seems like a straight up row copy should be simpler [15:30:18] cwd: a straight up row could be done from the application it seems to me [15:30:49] the mysql API should tell you if the row changed, so have a followup action to copy it into the log table [15:30:53] But you can't enforce good habits on CiviCRM extensions, for example. [15:31:05] but if you block every query on another round trip from php...it might crash and burn [15:31:52] cwd: maybe the query can be fired without waiting for it to complete? [15:32:02] either way mysql is probably doing roughly the same amount of work [15:32:14] The thing that relieves my own anxiety about this whole arrangement is that we're only writing to the logging database. We could drop the triggers and the new database a day later if something seems to be wrong, and most likely there'll be no harm done. [15:32:15] hmm, so like do this at the db connector level in php? [15:33:15] it would be easy if everything went through a query builder, easy to tell if something was a write or not [15:33:16] cwd: I haven't seen what civi's innards look like, but one would hope there's a central library for database interaction, maybe that could be extended to support something like this [15:33:34] but i'm guessing there are some string literal updates going down [15:34:10] awight agreed re. triggers only affecting the logging database [15:35:46] Jeff_Green: cwd: There's no query abstraction whatsoever. That'll hopefully come with Civi 5.x, by basing everything on Doctrine. [15:36:03] It's all string concatenation. [15:36:08] * Jeff_Green dies [15:36:26] * Jeff_Green comes back from the dead. [15:36:31] ok that explains things! [15:36:38] yeah so it'd be like...strstr('update') strstr('delete') [15:37:24] so is this a feature we're doing uniquely, or is this something happening for core civi for the world? [15:37:32] i bet a stored procedure that copied on writes would be pretty easy [15:37:41] then all the triggers could call the same thing [15:37:53] Jeff_Green: It's been around in core Civi for a while now, but there's a feature toggle. [15:37:58] since they are generated anyway [15:38:07] eileen has enabled it for all of her clients, who are numerous [15:38:08] oh this is how core did it? [15:38:16] yep [15:38:45] ah well i guess i'd defer to their decision [15:38:52] i'm sure they considered all the options [15:39:22] although the code base is a fiery train wreck [15:39:29] I think the stored procedure sounds nice, but there would have to be a different function for each table. Then, the trigger actually received a copy of the row, so you'd be performing an extra lookup for no reason. [15:39:32] but that's pretty much code bases [15:40:01] awight: hmm i bet you can pass the changes into the function [15:40:24] ah, good point. I wonder... [15:40:35] worked in postgres [15:40:44] the programming language is called pl/pgsql [15:40:48] really rolls off the tongue [15:41:13] postgres is a thing of much greater beauty [15:41:40] MySQL is a perfect match for PHP... the slower, less likely to ever improve, corporate way [15:42:49] hehe, yep [15:43:08] you know i'm sure these generated triggers are going to be the least overhead of any way of doing this [15:43:53] putting all the work in the compile step [15:44:11] I think so. [15:44:36] We still need to work out whether the trigger+replication thing is gonna screw us, though. [15:44:46] Especially cos we might be the only ones using this combination. [15:45:16] i could also see turning this on and the disk filling up [15:45:19] my suggestion would be to run the whole plan by jaime [15:45:53] is the replication row based or statement based? [15:45:58] I'm not too worried about disk, now that we dropped the faulkner db we have some serious space, although we should look at eventually expiring log entries somehow [15:46:15] replication is currently 'mixed' [15:46:33] at least I think so, lemme confirm that [15:47:09] yeah mixed [15:47:25] I don't know why we didn't go all in on row-based, we should run that by jaime too [15:48:21] Cool. I'll see if I can capture some of this stuff in the "console selves" ticket [15:48:53] is that better? it seems like statement based would be better with triggers... [15:48:57] is there a weeping admin emoticon in phabricator? [15:49:19] wait no [15:49:21] i'm backwards [15:49:51] i think row is probably the way to go to avoid corruption [15:49:51] cwd: right, I think statement based is where you throw the query over a fence and roll the dice [15:50:14] yes that ^^^ [15:50:38] It can be fun ;) [15:52:28] The second point on https://dev.mysql.com/doc/refman/5.7/en/binary-log-mixed.html sure sounds like, MySQL will always use row-based replication on updates which invoke a trigger. [15:52:56] oh that's interesting [15:53:23] But I'd like the snug feeling of just always using row logging, unless we burn through the localnet [15:54:07] the other thing about tech like mysql and php is by the time you realize they're broken it's too late to move away [15:55:28] most of this stuff would probably be better written as index cards. [15:56:11] hypercard! [15:57:01] at sparkfun we got bit by a mysql bug where select would literally disobey "where" in a specific situation [15:57:18] wat. I'd curious [15:57:19] and return the wrong rows [15:57:25] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-Ingenico, and 2 others: Failure to regenerate Order ID on iDEAL form - https://phabricator.wikimedia.org/T131983#2207287 (DStrine) [15:57:32] Fundraising Sprint Bloodletting 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice impressions beacon corrupts 'mixins' property - https://phabricator.wikimedia.org/T120082#2207288 (DStrine) [15:57:41] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice: DB timeouts when enabling more than one campaign at once from Spec... - https://phabricator.wikimedia.org/T128869#2207289 [15:57:47] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: GlobalCollect audit: lots of "impossible" errors for refunds - https://phabricator.wikimedia.org/T120430#2207290 (DStrine) [15:57:54] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Improve the change log report - https://phabricator.wikimedia.org/T130164#2207291 (DStrine) [15:57:59] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, and 2 others: TY mails failing silently for recurring donations - https://phabricator.wikimedia.org/T131200#2207292 (DStrine) [15:58:05] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Make a plan for moving stuff out of cookies - https://phabricator.wikimedia.org/T131319#2207293 (DStrine) [15:58:14] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, and 2 others: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2207294 (DStrine) [15:58:17] i spent the better part of a year regexing daily 10gb db dumps into something that would import into psql [15:58:19] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, and 2 others: Using back button from AstroPay lands donor on unusable form - https://phabricator.wikimedia.org/T130673#2207295 (DStrine) [15:58:23] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Document mass email - https://phabricator.wikimedia.org/T132369#2207296 (DStrine) [15:58:28] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, and 2 others: iDEAL cancel returns donor to Thank You page - https://phabricator.wikimedia.org/T131904#2207297 (DStrine) [15:58:35] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-Astropay, and 2 others: Astropay: 13 digit number will hide the fiscal number in mexico - https://phabricator.wikimedia.org/T131793#2207298 (DStrine) [15:59:50] cwd: The original data warehouse [15:59:57] data, meet warehouse [16:00:10] hehe [16:04:58] awight, Jeff_Green any way you could check out my last comment here? https://phabricator.wikimedia.org/T129265 sound safe? [16:06:18] looking [16:06:40] ty...and if you think it's cool maybe moving those files? [16:06:46] then i'll deploy and kick that job [16:07:07] Jeff_Green: The other thing eileen wanted me to ask was to schedule an outage that's convenient for you. It sounds like we're not very close to that point yet--does it still make sense to schedule a provisional window, or shall we just wait to do that? [16:07:15] cwd i have no problem moving the files but I also am not very familiar with the code processing them [16:07:49] awight: we should probably do some testing around replication before we schedule [16:07:50] cwd: That's how I would do it, too. Audit processing is idempotent. [16:08:04] Jeff_Green: cool, that works for me. [16:08:35] shall I move the files? [16:08:45] yes please and thank you! [16:08:48] k [16:10:40] done [16:12:42] question re. payments GET strings [16:12:54] what populates &referrer= ? [16:14:05] hrm, i have only seen that passed in post data [16:14:31] ah they're merged together in the context I'm looking at [16:15:27] Good question, that needs documentation... It's set by donatewiki if the donor lands there. Otherwise, I think paymentswiki pulls that from the HTTP headers. [16:15:28] ah ok, i'm pretty sure it gets set on donate wiki in a hidden field [16:15:58] cause payments wants the original referrer from WP or whatever [16:16:22] +1. And if the banner goes straight to payments, I think we use headers [16:16:27] awight: when it comes to deploying smash pig...it's been awhile [16:16:43] composer install and deploy civi? [16:16:51] ./gateway_common/GatewayPage.php: $referrer = $request->getHeader( 'referer' ); [16:17:03] * cwd twitch [16:17:28] cwd: Do you want to deploy SP itself, for a listener or audit thing? Or want to bump the library as it's used as a dependency of DI or CRM? [16:17:31] :( [16:18:03] cwd: thx [16:19:14] awight: hehe...audit thing [16:19:30] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Reassure ourselves about triggers & replication - https://phabricator.wikimedia.org/T132527#2207382 (awight) More discussion at http:... [16:19:52] Jeff_Green: cwd: Please lmk if I captured your concerns in that comment ^ [16:20:19] cwd: If it's standalone, then just merge to the deployment branch and fundraising_code_update -p SmashPig [16:20:36] deploying changes to SP used as a library obviously takes the extra steps. [16:20:39] Oh. [16:20:52] Which is exactly what you want to do, cos the WP audit is running under CRM, huh. [16:21:08] awight: ya that's good [16:21:35] yeah [16:21:55] cwd: I'm not totally sure, I'd have to mess around with it :/ [16:22:07] I think you can just do "composer update" from crm [16:22:14] and it will pull the new master [16:22:15] lol and the job is going to run in 10 minutes [16:22:18] * cwd disables it [16:22:21] +1 [16:23:17] However, you might have to update the SHA in composer.lock to the version you want, then check that in on master, merge to deployment, and composer install on the deployment branch so that it updates the vendor/ submodule.. and check in those changes, plus the submodule pointer bump. [16:23:30] * awight mourns lost sanity [16:23:38] hehe [16:23:44] some day i'll frig all this away [16:24:52] <3 lmk how I can help. [16:26:22] Your requirements could not be resolved to an installable set of packages. [16:26:25] good times [16:28:30] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Reassure ourselves about triggers & replication - https://phabricator.wikimedia.org/T132527#2207390 (awight) @jcrespo We're hoping t... [16:28:46] i guess this is where composer.lock came from. officially saying fuck it [16:28:52] composer update --no-dev ? [16:29:10] yeah that "amzn" thing has been killing us. [16:30:03] yeah same thing, i don't understand what it's saying though [16:30:29] Removal request for amzn/login-and-pay-with-amazon-sdk-php == 9999999-dev [16:32:36] What I did was horrible. I removed that dependency from whatever the crap was asking for it :( and made sure that the change didn't get accidentally checked in. [16:32:42] ejegg|afk understands better [16:33:02] something about using dev-master? [16:33:28] * cwd just edits the lock file [16:34:05] (PS1) Cdentinger: bump smash-pig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/283477 [16:35:47] oh hai [16:36:04] hehe [16:36:09] sorry. It's about that amzn lib [16:36:21] Looks like cwd worked around already, though [16:36:50] I don't get the issue--we already forked, so this is just a bug in composer when using custom repos? [16:37:56] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Reassure ourselves about triggers & replication - https://phabricator.wikimedia.org/T132527#2207404 (awight) Here's an example trigge... [16:38:38] weird... [16:38:50] i gotta do some taxing things, will be semi-semi online [16:39:16] oh, is it because we forked but I didn't make a new package name? [16:39:25] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice banner filter is case sensitive - https://phabricator.wikimedia.org/T55751#2207406 (DStrine) [16:39:38] ejegg: do you normally just update the lockfile on crm? [16:39:38] So you always have to specify the repo [16:39:59] since composer update bombs [16:40:04] cwd lately yes, just 'cause DI is currently incompatible [16:40:12] wait, composer update bombs too? [16:40:39] I got it to run, but the tests then fail b/c of the mediawiki-specific code on the gc recurring critical path [16:40:53] https://gerrit.wikimedia.org/r/281485 [16:41:08] oh right, need to list that rep [16:41:11] *repo [16:42:03] aaah, in the higher-level composer.json eh [16:45:22] ejegg: which lib update makes the tests break? [16:47:02] DonationInterface [16:47:35] I need to wrap the updates I made to session and request var access [16:49:05] oh man DI is in crm vendor [16:49:09] i didn't realize [16:49:54] "frig". [16:49:59] ;) [16:51:23] wouldn't it be better to fix this in crm than DI? [16:51:34] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Yo La Tengo, Fundraising-Backlog, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#1864197 (Pcoombe) @AndyRussG Should thi... [16:53:16] derp i did this from a topic branch...does it matter? https://gerrit.wikimedia.org/r/#/c/283477/ [16:54:01] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Yo La Tengo, Fundraising-Backlog, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#2207459 (AndyRussG) Hi @Pcoombe! I'd li... [16:55:29] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Easy: Make CentralNotice banner completely believe it to be in a certain country - https://phabricator.wikimedia.org/T111071#2207467 (Pcoombe) p:High>Normal [16:55:43] pretty sure it doesn't...i'm gonna move ahead with this deploy [16:56:01] (CR) Cdentinger: [C: 2] bump smash-pig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/283477 (owner: Cdentinger) [16:56:22] cwd: topic branches are always kosher, they're ignored by gerrit anyway. Only the .gitreview setting determines which branch your change will land on. [16:56:39] (or, you can use "git push HEAD:refs/for/BRANCH" to override the config file) [16:57:14] ah yes, gerrit without git review [16:57:32] http://garfieldminusgarfield.net/ [16:57:52] (Merged) jenkins-bot: bump smash-pig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/283477 (owner: Cdentinger) [16:58:47] yes! so good [17:02:30] casey@okay:/srv/crm$ frig prep vendor [17:02:32] updated deployment to 78b5e964ade24cffaff5bb8b800208d64ed44f53 [17:02:37] works here too! [17:03:32] (PS1) Cdentinger: Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283482 [17:04:27] (CR) Cdentinger: [C: 2] Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283482 (owner: Cdentinger) [17:08:05] wait i think i screwed up [17:08:26] hm? [17:08:37] maybe not...does the submodule pointer update have to be a follow on commit to the deployment merge? [17:09:03] my brain... [17:09:51] cwd either way works, sometimes I amend the merge and tack it on [17:10:08] i merged the commit from master that was just editing the lock file... update the submodule... now i must run composer install on deployment [17:10:12] (Merged) jenkins-bot: Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283482 (owner: Cdentinger) [17:10:17] and commit that? [17:10:42] it only occurs to me now how divergent that makes the history between master and deployment [17:11:12] hey ppena! any news on whether we want to test Mexico today? [17:11:22] wait, there's no vendor submodule on master [17:11:38] so yeah, anything that touches vendor is going to diverge [17:12:42] the-wub: still looks like we're seeing the fiscal number field [17:13:17] The .git directory is missing from /srv/crm/vendor/wikimedia/smash-pig [17:13:34] cwd yeah, more annoyance [17:14:03] it wouldn't let us check the folder in as a folder till i deleted that [17:14:13] git kept adding it as a submodule [17:14:39] the latest version of composer asks if you want to reinstall when that's missing though [17:14:44] instead of just dying [17:16:19] our composer use is in that annoying half-mature state between running composer in production to deploy, and hosting our own private package repos [17:22:36] (PS1) Cdentinger: update smash-pig for worldpay audit [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/283483 [17:27:48] ejegg: does this look right to you? https://gerrit.wikimedia.org/r/#/c/283483/ [17:29:28] it's too bad that vendor submodule also loses the commit history [17:29:43] yeah, that is annoying [17:29:53] but that looks like the right set of changes [17:30:02] added .gitmodules, but meh... [17:30:54] should that not be there? [17:31:19] i zapped it initially 'cause I was zapping .git, but nbd either way [17:31:38] hi all :) Banners just went up for Argentina, Chile, Colombia, Peru and Uruguay. we should start seeing Astropay donations [17:31:57] ejegg: ah yeah, to save space or what? [17:32:05] the-wub: rockin1 [17:32:10] ^ MBeat jessicarobell ppena ejegg [17:32:17] cwd nah, no reason at all I guess [17:32:29] might as well leave it, one less thing to delete next time [17:33:04] it does feel weird to be installing .git dirs with composer [17:33:26] but when it comes to submodules i just close my eyes and pray anyway [17:34:00] (CR) Cdentinger: [C: 2] update smash-pig for worldpay audit [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/283483 (owner: Cdentinger) [17:34:06] heh, at least git can remove submodules these days [17:34:28] is this the repo that needs a manual submit? [17:34:37] vendor ones do, yah [17:35:43] (CR) Cdentinger: [V: 2] update smash-pig for worldpay audit [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/283483 (owner: Cdentinger) [17:36:36] painless smashpig deploy would be the gold standard [17:37:58] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Yo La Tengo, Fundraising-Backlog, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#2207681 (Pcoombe) @AndyRussG We don't h... [17:38:44] Hermit Crab Husbandry?? these sprint names get better and better [17:38:56] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Yo La Tengo, Fundraising-Backlog, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#2207683 (AndyRussG) @Pcoombe OK thanks!... [17:39:12] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Yo La Tengo, Fundraising-Backlog, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#2207684 (AndyRussG) Open>Resolved [17:39:42] the-wub: hi! in theory this should be fixed, too: https://phabricator.wikimedia.org/T128869 [17:40:51] (PS1) Cdentinger: update vendor submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283490 [17:40:56] AndyRussG: that's great, thanks. will see what happens next time I make a bulk change [17:41:27] Fundraising Tech Backlog, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Error popup is not translated - https://phabricator.wikimedia.org/T89376#1034930 (Ejegg) Maybe related to https://phabricator.wikimedia.org/T129277 - is it fixed now? Can you still reproduce the error @awight? [17:42:20] Astropay looking good so far from my POV :) [17:44:04] the-wub: yeah... OK, do you have any bulk changes coming up? BTW the code handling changes from the list of campaigns was completely overhauled, so we should be on the lookout for any other weirdness, too... [17:45:20] (CR) Cdentinger: [C: 2] update vendor submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283490 (owner: Cdentinger) [17:45:38] (Merged) jenkins-bot: update vendor submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/283490 (owner: Cdentinger) [17:52:02] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice: DB timeouts when enabling more than one campaign at once from Spec... - https://phabricator.wikimedia.org/T128869#2207712 [17:54:01] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice: DB timeouts when enabling more than one campaign at once from Spec... - https://phabricator.wikimedia.org/T128869#2207716 [17:59:40] Fundraising Sprint Bloodletting 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice impressions beacon corrupts 'mixins' property - https://phabricator.wikimedia.org/T120082#2207728 (AndyRussG) Fix now on pr... [18:00:21] Fundraising Sprint Bloodletting 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, and 3 others: CentralNotice impressions beacon corrupts 'mixins' property - https://phabricator.wikimedia.org/T120082#2207729 (AndyRussG) Open>R... [18:01:12] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Patch-For-Review: Pull payment methods out of adapters - https://phabricator.wikimedia.org/T130056#2207742 (DStrine) [18:02:03] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, MediaWiki-extensions-DonationInterface, and 2 others: Use consistent parameter and variable names in DI - https://phabricator.wikimedia.org/T130939#2207744 (DStrine) [18:03:15] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, and 3 others: Move staging functions into helper classes - https://phabricator.wikimedia.org/T130075#2207747 (DStrine) [18:03:51] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Opt out field not working with Engage import - https://phabricator.wikimedia.org/T130768#2207764 (DStrine) [18:04:30] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Fix up core logging spec - https://phabricator.wikimedia.org/T130161#2207766 (DStrine) [18:04:51] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Sync relevant files with 4.7 to make porting easier - https://phabricator.wikimedia.org/T131223#2207767 (DStrine) [18:05:28] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Use a unique ID to store logging information per : https://issues.civicrm.org/jira/browse/CRM-18193 - https://phabricator.wikimedia.org/T131222#2207768 (DStri... [18:05:54] !log updated crm from 5877ee71abc21140e60934aef627003cfb2d11cc to 2f40e829195dec1cec71d08dc9c656eb247631ae [18:05:59] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:06:10] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi should record merge activities on both old and new contacts - https://phabricator.wikimedia.org/T119426#2207771 (DStrine) [18:06:25] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Backport CRM-18178 exclude _bak & _backup tables from trigger creation - https://phabricator.wikimedia.org/T131221#2207772 (DStrine) [19:02:02] ok, let's see if the GC audit script still works [19:03:27] might even run a little faster now that it's not looking for parent txn on non-refunds [19:05:17] (PS2) Ejegg: CRM-18366 Fix contact logging detail to show all related changes & revert all related changes [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282091 (owner: Eileen) [19:06:29] (CR) Ejegg: [C: 2] "Works, looks good. At some point it will be nice to add detail about the type of entity to each row, not just property and changed values." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282091 (owner: Eileen) [19:12:12] (Merged) jenkins-bot: CRM-18366 Fix contact logging detail to show all related changes & revert all related changes [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282091 (owner: Eileen) [19:19:26] Fundraising-Backlog, Hovercards, MediaWiki-extensions-CentralNotice: Measure impact of HoverCards on Central Notice interaction - https://phabricator.wikimedia.org/T131366#2208030 (JKatzWMF) [19:21:02] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: GlobalCollect audit: lots of "impossible" errors for refunds - https://phabricator.wikimedia.org/T120430#2208035 (Ejegg) Open>Resolved Tested, seem... [19:26:08] (PS2) Ejegg: CRM-18193 do not require log_date [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282092 (owner: Eileen) [19:29:56] (CR) Ejegg: [C: 2] "Looks good!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282092 (owner: Eileen) [19:31:51] (Merged) jenkins-bot: CRM-18193 do not require log_date [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/282092 (owner: Eileen) [19:36:21] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Show loading overlay on iDEAL redirect - https://phabricator.wikimedia.org/T132715#2208048 (Ejegg) [19:44:08] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Show loading overlay on iDEAL redirect - https://phabricator.wikimedia.org/T132715#2208090 (Ejegg) Open>Invalid Never mind, we're setting a 'disabled' class on the button and checking that [19:46:44] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-Ingenico, and 2 others: Failure to regenerate Order ID on iDEAL form - https://phabricator.wikimedia.org/T131983#2208103 (Ejegg) Haven't seen any since 4/11, but that might have something to do wit... [20:05:19] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-Astropay, and 2 others: Astropay: 13 digit number will hide the fiscal number in mexico - https://phabricator.wikimedia.org/T131793#2208187 (Ejegg) Open>Resolved a:Ejegg Fixed on AstroP... [20:05:20] AndyRussG: meeting? [20:18:10] (PS1) Ejegg: More efficient old custom value lookup [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/283513 [20:30:58] cwd|afk: I think this is the user that office IT would use to grant u rights? https://meta.wikimedia.org/wiki/User:WMFOffice [20:31:04] oh sorry dstrine [20:42:03] ah yes, it was philippe who i had been in contact with re: cn admin [20:42:07] and he no longer works here [20:43:10] dstrine: u want Jamesofur for that [20:43:15] just send him an email [20:44:52] Actually, that may have changed--I think you need someone from this list: https://meta.wikimedia.org/w/index.php?title=Special:ListUsers&group=bureaucrat [20:47:29] Fundraising Sprint Freshmaking, Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, and 2 others: Reassure ourselves about triggers & replication - https://phabricator.wikimedia.org/T132527#2208334 (awight) [20:48:56] huh, philippe is at wikia now [20:49:31] I don't see any COI with JWales presiding over both companies... :-/ [20:50:05] awight: thanks [20:50:58] Jamesofur: Are you still the person to bother about getting a WMF employee (dstrine) CentralNotice admin privs? [20:51:12] yup, email is best (at a training now) but will get by tonight [20:51:18] k thanks! [20:51:30] * awight puzzles through rights... [20:52:11] wikia is to wikipedia as _ is to _ [20:54:51] Jamesofur: I just sent an email. [20:54:54] Thanks all! [20:54:56] Oracle is to MariaDB? [20:56:35] i was going to say, wings is to the beatles [22:12:14] cwd: That's cold [22:15:42] how about an advertising deconstructionalist reality show called Brand on the Run [22:16:33] don't forget to tip your waitress [22:16:40] whoa [22:16:53] You are taxing my nonexistent Wings knowledge [22:19:04] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, and 2 others: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2208597 (cwdent) @Jgreen doh, I missed another pile of files, c... [22:19:47] happily those refunds are making it in now [22:20:20] sadly i missed a bunch of the files but whenever jeff moves them back the remaining should process [22:20:27] woot! [22:20:51] anything wrong with me leaving --run_all on the worldpay job? it runs in like 2 seconds even when there's a ton of new files [22:23:19] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Easy: Add CentralNotice keyboard shortcuts - https://phabricator.wikimedia.org/T132731#2208602 (Pcoombe) [22:23:29] That sounds great to me [22:23:46] thanks! [22:23:51] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Easy: Add CentralNotice keyboard shortcuts - https://phabricator.wikimedia.org/T132731#2208614 (Pcoombe) p:Triage>Low [22:30:44] awight: have you ever used ssh escape sequences? [22:30:49] TIL, and sort of blew my mind [22:30:49] dstrine: Oh. I've been meaning to run this by you: /me facepalms [22:30:57] dstrine: https://phabricator.wikimedia.org/T131904 [22:31:02] I don't think we care about this bug. [22:31:03] hey! awight [22:31:14] cwd: eh? like "~." ? [22:31:31] yeah [22:31:38] or ~ctrl+z [22:31:42] yah [22:31:48] extremely handy [22:32:04] i can't believe i never knew about this [22:32:13] hehe there's always a first time? [22:32:23] awight: why would we not care about T131904 ? [22:32:24] T131904: iDEAL cancel returns donor to Thank You page - https://phabricator.wikimedia.org/T131904 [22:32:29] but tbh ~ is a ridiculous escape sequence [22:32:49] dstrine: Have any donors actually complained? [22:32:57] awight: no [22:33:08] :D [22:33:13] Let's drop it like a brick. [22:33:16] we can punt but keep it open for next year? [22:33:19] sure! [22:33:24] kk one second... [22:33:36] Fundraising Sprint Ghostbusting , Fundraising-Backlog, MediaWiki-extensions-DonationInterface, FR-Ingenico, Patch-For-Review: iDEAL cancel returns donor to Thank You page - https://phabricator.wikimedia.org/T131904#2208632 (DStrine) [22:33:43] There's a very useful task which is an enormous rabbithole of this one, which I started working on. [22:34:04] It's to refactor the Ingenico result switcher to use our generalized controller. [22:34:21] awight: ok well I moved it way far away [22:34:26] That would be a nice chunk of tech debt to cut off of the spaceship [22:34:30] thanks! [22:35:29] awight: I was meaning to talk to you too... we pull some things into the hermit crab sprint. But we did not pull any queue stuff as you were not there to consult. [22:35:35] Fundraising Sprint Ghostbusting , Fundraising-Backlog, MediaWiki-extensions-DonationInterface, FR-Ingenico, Patch-For-Review: iDEAL cancel returns donor to Thank You page - https://phabricator.wikimedia.org/T131904#2208654 (awight) [22:35:39] ooh [22:35:51] your phase 1 stuff is in sprint +1 if you want to pull any in [22:35:54] just FYI [22:36:39] brb [22:46:31] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, I18n: Allow CentralNotice banner variables to be marked as non translated - https://phabricator.wikimedia.org/T53470#2208668 (DStrine) [22:46:47] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Easy: Add history link for banners - https://phabricator.wikimedia.org/T132732#2208670 (DStrine) [23:04:44] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-ActiveMQ: Pick a PHP Kafka client library - https://phabricator.wikimedia.org/T131268#2161707 (awight) [23:05:39] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-ActiveMQ: Write PHP-Queue module for Kafka - https://phabricator.wikimedia.org/T131269#2161726 (awight) [23:07:24] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, fundraising-tech-ops, FR-ActiveMQ: [Epic] Provision high-availability Kafka cluster for Fundraising - https://phabricator.wikimedia.org/T130283#2208700 (awight) @Jgreen I'm putting this task in the current sprint, not expecting to fi... [23:21:28] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-ActiveMQ: Pick a PHP Kafka client library - https://phabricator.wikimedia.org/T131268#2161707 (awight) a:awight [23:35:31] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-ActiveMQ: Pick a PHP Kafka client library - https://phabricator.wikimedia.org/T131268#2208796 (awight) [23:39:16] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, fundraising-tech-ops, FR-ActiveMQ: [Epic] Provision high-availability Kafka cluster for Fundraising - https://phabricator.wikimedia.org/T130283#2208800 (awight) [23:46:43] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, fundraising-tech-ops, FR-ActiveMQ: [Epic] Provision high-availability Kafka cluster for Fundraising - https://phabricator.wikimedia.org/T130283#2208808 (awight) [23:58:36] Fundraising Sprint Hermit Crab Husbandry, Fundraising-Backlog, FR-ActiveMQ: Pick a PHP Kafka client library - https://phabricator.wikimedia.org/T131268#2208817 (awight) [23:59:10] dstrine: You might want to look over what I pulled in... [23:59:14] See you tomorrow!