[00:01:35] eileen: cool, maybe that would be a good time to do the triggers too? [00:03:05] cwd I need the triggers done first (shouldn't need an outage I don't think) & then I can update some fields -well I suppose we could do then - it will make the outage longer as the db update I can do without an outage if the triggers are in place [00:03:26] I need to add some fields & indexes - should take about 30 mins for query I think [00:04:11] i am hesitant to run it live, anything at all goes wrong and we have a bad situation [00:04:19] potentially trying to roll the db back [00:04:37] which i have never done but sounds terrible [00:04:48] So we have a new field already - once triggers are reloaded I can populate it - I was thinking to do it in advance of any outage [00:04:48] hmm ok - basically it just adds a field to the existing trigger [00:05:23] yeah if everything goes as expected [00:05:30] definitely low risk but high consequence [00:05:52] like if i fumble a paste for instance :P [00:06:25] ok we can just make outage longer [00:06:25] well it's not a paste - just source triggers.mysql [00:06:34] historically we drop and recreate all the triggers every time, do you think that's not necessary? [00:07:25] everytime we upgrade? Well it's necessary if we are adding fields we want to track or dropping fields [00:07:25] the latter is the critical one [00:07:28] well I guess if we want to track the former is too... [00:07:49] it seems like the safest thing to do [00:07:55] but only the latter causes failure [00:08:23] would it make the outage longer if step 1 is running this update? [00:08:33] or you have to do stuff after running the update? [00:09:57] so 2 slow things 1) need to transfer data from field we are getting rid of to our new field - the new field exists but if we want the update to be logged triggers should be on 2) need to add some fields & indexes (later will drop some others but not this outage) [00:10:17] after the second one is done some more triggers need to be loaded - [00:10:37] step 1 is what we could do beforehand? [00:10:42] if we are doing both at once I guess we will add the fields & indexes first & then move the data [00:10:46] right step 1 could be first [00:11:40] ok [00:11:42] are you in tomorrow? [00:11:58] i would like to run it by jeff if it's not a major inconvenience for you [00:12:12] yes am in then [00:12:16] if he says it's ok i can run it before you get on [00:12:21] cool [00:12:42] otherwise we can do in an outage - just makes it longer & a bit more complex [00:13:01] iirc it runs in like 1s [00:13:17] but if i hit enter and there is a pause i will have a coronary incident [00:13:58] :-) [00:14:18] cwd for a longer outage when suits you? [00:14:58] next monday works for me [00:18:30] as in https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018&month=4&day=23&hour=23&min=0&sec=0&p1=264&p2=179&p3=224&p4=136 cwd [00:18:51] hmm still 4pm in sf there - do we normally do an hour later? [00:19:44] i think usually [00:19:51] we could email and see if anyone minds [00:20:05] it is a high value hour to save :) [00:20:25] karate comp sounds great btw [00:20:50] i was in karate as a kid for a while but my parents made me quit because the dojo was a basement full of asbestos [00:21:50] not so good for your health then! [00:22:02] our kids have been doing karate for 5 years now - they are getting quite good [00:22:27] wow [00:22:36] this is the first time we've traveled this far for a karate comp but I guess we'll be under pressure to keep doing it now [00:22:53] heh [00:23:05] and probably get asked to bring other people's kids [00:23:13] the kids club is quite competitive - other kids have represented NZ [00:23:55] (actually Jack will technically be representing NZ in May but in the particular tournament it's open entry for under 14s) [00:24:12] still he gets a karate NZ tracksuit (which we will have to pay for) [00:24:24] haha [00:24:26] sounds baller though [00:25:36] my 23yo cousin recently quit her job at google in manhattan and moved to thailand to fight muay thai competitively [00:25:49] it is a betting sport there as i understand [00:26:02] i hope everything is going well for her [00:29:21] wow - I don't think anyone makes money off karate! A guy from our club represented NZ at the world champs & he even had to pay for his on NZ team badge [00:29:36] just sent an outage email [00:42:24] thanks! [01:36:30] (PS8) Eileen: Add donor totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424179 (https://phabricator.wikimedia.org/T186168) [01:36:55] (Abandoned) Eileen: Add location hunt for civicrm_settings_location.php so we can put it in sites dir [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427079 (owner: Eileen) [02:12:31] (PS9) Eileen: Transfer data to new custom field (deploy AFTER triggers re-established) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424186 (https://phabricator.wikimedia.org/T186168) [05:29:23] (PS9) Eileen: Add donor totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424179 (https://phabricator.wikimedia.org/T186168) [05:29:25] (PS10) Eileen: Transfer data to new custom field (deploy AFTER triggers re-established) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424186 (https://phabricator.wikimedia.org/T186168) [05:29:27] (PS16) Eileen: Add triggers to manage yearly totals & associated custom fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/425219 (https://phabricator.wikimedia.org/T186168) [05:29:29] (PS1) Eileen: Apply drupal formatting to refundTest [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427317 [06:32:06] (PS1) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [06:39:21] (PS2) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [06:40:52] (PS3) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [06:58:51] (PS1) Eileen: Fix trigger generation for modified_date on custom data [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427320 (https://phabricator.wikimedia.org/T192051) [07:17:17] (PS4) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [07:20:51] (PS5) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [07:22:04] (PS6) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [07:23:02] (PS7) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [07:24:56] (PS8) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [14:30:27] (PS4) Jgleeson: 1) Mapper component to for transforming input to array and json output. [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/427198 [14:30:49] (CR) jerkins-bot: [V: -1] 1) Mapper component to for transforming input to array and json output. [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/427198 (owner: Jgleeson) [14:32:58] (PS5) Jgleeson: 1) Mapper component to for transforming input to array and json output. [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/427198 [14:34:14] (PS6) Jgleeson: Mapper component / Mapper Implementation within PaymentProviders/Ingenico/PaymentProvider.php::createPayment() [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/427198 [14:38:19] ejegg|afk fr-tech hi! just booked a deploy slot for the CN CSP header on banner previews, 10 am Pacific [14:39:09] (PS7) Jgleeson: WIP - Mapper component / Mapper Implementation within PaymentProviders/Ingenico/PaymentProvider.php::createPayment() [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/427198 [14:40:00] It's actually the same deploy slot we all love to hate [14:40:15] (Wednesday during Scrum of Scrums, hope someone's around to deploy...) [14:43:10] ejegg|afk, how are you manually testing the ingenico createPayment stuff atm? [14:43:22] I realised I haven't tried it since adding in my patch [14:44:14] Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog, Patch-For-Review: Refund the unintended recurring donations from the tests pre big english - https://phabricator.wikimedia.org/T190854#4139501 (mep... [14:58:28] thanks AndyRussG! [14:58:53] jgleeson: I had a patch in DonationInterface that Í've since abandoned [14:59:00] but still have a copy locally [15:00:09] (PS2) Ejegg: Delay TY letters for imported contributions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/423799 (https://phabricator.wikimedia.org/T191338) [15:01:21] ejegg: thank u also for all the help with this stuff!! :) [15:02:06] ejegg: I just dropped a nickname-pinging-less question in -operations asking about ideas for how to get some alerts from this... no answers yet [15:02:19] actually realized that is totally not scoped out yet [15:05:11] where did the old banner screenshot script run? [15:09:09] AndyRussG: I think aluminum? [15:09:19] Hmm K [15:09:24] That was the fr cluster, right? [15:09:28] yes [15:09:55] We don't need a sandboxed mw install, just a job that runs a headless browser, triggered by banner saves [15:10:12] right [15:10:19] Should we try to set one up first on labs maybe, see how it goes, and then figure out where it might eventually live? [15:10:37] yeah, I'd go with labs first [15:10:39] but.... [15:10:59] It might be fun to start out with a setup that we can tweak easily without too many permissions issues getting in the way... [15:11:03] I think the eventlogging -> pgehres is definitely a higher priority [15:11:12] ah yeah fer sure [15:11:18] and probably something we can wrap up pretty soon [15:11:24] okok [15:12:09] cwd / Jeff_Green So, just to recap, the blocker to running a job pulling a second kafka topic to files is upgrading aluminum to the new OS ? [15:12:40] ejegg: quick parenthesis: if this looks good to you, and you feel like +2ing, it could maybe go out on the same SWAT later today: https://gerrit.wikimedia.org/r/426096 [15:12:53] ejegg: I think the main blocker is there not being a second kafka topic yet [15:12:56] ah yeah, lemme look [15:13:01] Jeff_Green: ah, but there is! [15:13:03] T190089 [15:13:03] T190089: Add Banner Guidelines link to banner edit screen - https://phabricator.wikimedia.org/T190089 [15:13:12] ejegg: oh ho, well that is news! [15:13:21] alnitak is already stretch [15:13:43] we have banner logging going to a new topic, but so far just for a test campaign. [15:13:46] also the OS and librdkafka version shouldn't matter, it's just a matter of config [15:13:51] once we figure out how to consume it [15:13:56] i have been building out a situation in virtualbox where i can write the new collector with the python library [15:14:12] we can turn it on for all campaigns [15:14:23] the new puppet stuff from prod for kafkatee did not work with our puppet [15:14:32] although that might change as we update towards puppet 4 [15:14:34] AndyRussG: what's the new topic? CentralNoticeImpressions? [15:15:06] puppet aside, we mainly need to decide what we want the pipeline to look like short term [15:15:30] Right, I know there are new options now that we're consuming an EventLogging stream [15:15:34] if we're just replicating what we have, using a different topic, in parallel, we can basically use our existing puppet code and scripts and drop stuff in files on disk like we do now [15:16:04] ejegg: I think that'd be it... That's the schema name... https://meta.wikimedia.org/wiki/Schema:CentralNoticeImpression [15:16:09] i'm not sure what new options there are? i thought we're just talking about a pre-filtered heap of data so what we consume is slimmer [15:16:26] i don't think there's a good way to run 2 in parallel unless we update kafkatee puppet [15:16:37] we could have a hard cutover [15:16:45] cwd: Jeff_Green: it's also so far slightly newer data format, so we have to adapt the ingress script or mechanism [15:16:54] (PS2) Ejegg: Add privacy warning and allow wikitext in banner content field summary [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/426096 (https://phabricator.wikimedia.org/T190089) (owner: AndyRussG) [15:16:58] yeah we don't want that anyway, if we're trying to show that the new pipeline doesn't lose data we want to put it on a separate box that isn't already overloaded [15:17:10] AndyRussG: still JSON? [15:17:18] ejegg: what's the standard tool for Kafka->mysql that u mentioned was recommended? [15:17:25] Jeff_Green: is it currently JSON? [15:17:34] otto suggested python-kafka [15:17:42] (CR) Ejegg: [C: 2] "Maybe light on details, but it's a good reminder!" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/426096 (https://phabricator.wikimedia.org/T190089) (owner: AndyRussG) [15:18:09] AndyRussG: i think that's the way it's stored in kafka, and librdkafka/kafkatee is where we reformat it to standard log format [15:18:13] All I know is currently there's a bunch of files dumped somewhere that a Python script reads and puts the munged contents in a database [15:18:23] Ah hmmm [15:18:29] I'm not familiar with that step [15:18:35] sec [15:18:38] Jeff_Green: repo for that code/setup somewhere? [15:18:47] cwd: ah okok thx! [15:18:49] AndyRussG: it's all in frack puppet [15:18:53] AndyRussG: Jeff_Green The new format is still JSON, and comes off the wire parsed nicely [15:19:11] So if we're dumping to files, we shouldn't need to transform it at all [15:19:31] this is what's in kafkatee.conf [15:19:43] # Output format for JSON -> string transformation. [15:19:43] output.format = %{hostname} %{sequence} %{dt} %{time_firstbyte} %{ip} %{cache_status}/%{http_status} %{response_size} %{http_method} http://%{uri_host}%{uri_path}%{uri_query} - %{content_type} %{referer} %{x_forwarded_for} %{user_agent} %{accept_language} %{x_analytics} [15:20:16] yeah, we shouldn't need a lot of that now [15:20:18] and kafkatee pipes to a simple script that parses for the lines we want [15:20:40] and we shouldn't need to filter it either [15:20:47] that's great [15:20:52] well, not in the script to dump it to files [15:20:53] Jeff_Green: gerrit repo? (sorry I haven't worked with that much) [15:21:12] there was a little filtering in the djangobannerloader script [15:21:13] AndyRussG: frpm1001:/var/lib/git/puppet.git [15:21:18] but even that may be unnecessary [15:21:37] I think it was mainly filtering out the banner screenshot tool [15:21:38] AndyRussG: it's a local repo b/c we didn't want it exposed to the public [15:21:38] Oh so it's not in a public git repo? [15:21:46] ah hmmm [15:21:55] k thx :) [15:22:01] AndyRussG: you should still be able to get at that one on frpm1001 itself! [15:22:01] no, maybe someday we can strip out everything interesting and do that, but for now it is Too Hard [15:22:19] yeah got it [15:22:45] indeed even the public wikimedia config repo for prod has some seekret bits (passwords and such) [15:22:55] ya [15:24:33] (CR) Mepps: [C: 2] Fix trigger generation for modified_date on custom data [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427320 (https://phabricator.wikimedia.org/T192051) (owner: Eileen) [15:25:19] (CR) Mepps: [C: 2] Re-add settings_location.php [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427279 (owner: Eileen) [15:27:51] (Merged) jenkins-bot: Add privacy warning and allow wikitext in banner content field summary [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/426096 (https://phabricator.wikimedia.org/T190089) (owner: AndyRussG) [15:29:09] (Merged) jenkins-bot: Fix trigger generation for modified_date on custom data [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427320 (https://phabricator.wikimedia.org/T192051) (owner: Eileen) [15:31:52] (Merged) jenkins-bot: Re-add settings_location.php [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/427279 (owner: Eileen) [15:36:48] ejegg: eileen was saying she thinks this could be done live: https://phabricator.wikimedia.org/T192411 [15:37:12] i was thinking that might be optimistic if we care about the integrity of what the triggers are doing [15:37:37] cwd how about just running the updated lines in that file? [15:37:53] dropping the geocoder triggers should be fine live [15:38:25] and the new trigger creation is on what's currently a read-only field [15:38:44] introduces possibility for more human error [15:39:01] in the case of live writes to the master db it seems not worth risking anything [15:39:15] I'll drop something in a pastebin for review [15:39:17] one mistake and we have downtime plus unknown quantity of trying to roll back the db [15:42:06] oh hey, those have an error where they update mailing timestamps :( [15:42:21] need an edit anyway [15:43:03] what's the problem? [15:43:15] UPDATE civicrm_mailing SET modified_date = CURRENT_TIMESTAMP WHERE id = NEW.entity_id; [15:43:28] but the custom field doesn't extend mailing, it extends contact [15:43:37] eileen's got a patch in upstream to fix the issue [15:43:44] but that file was generated by the old code [15:44:04] would it have bombed if i tried to run it? [15:44:11] not a huge deal, but it is some unnecessary writes and potential inaccurate last modified dates on mailings [15:44:23] ah gotcha [15:44:33] since we've got so many more contacts than mailing, it's unlikely to match the id on many [15:44:46] just when updating the oldest few thousand contacts [15:45:00] but it'll waste some time on the db [15:45:54] https://phabricator.wikimedia.org/P7008 [15:46:33] I'd be comfortable running just that much live [15:47:40] well, after I try it local, anyway [15:48:35] wah wah... Can't create a TRIGGER from within another stored routine [15:51:00] aha, silly me, copy-pasted both old and new lines from the diff [15:54:32] oh, and my 'communications' custom group has the wrong table name cause I applied an early version of that patch. fixing [16:33:12] (CR) AndyRussG: [C: 2] Emit CSP headers on banner previews [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/427235 (https://phabricator.wikimedia.org/T190100) (owner: Ejegg) [16:34:20] fr-tech patches going out a bit early because so production schedule-shifting [16:35:03] ok, cool [16:47:15] (Merged) jenkins-bot: Emit CSP headers on banner previews [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/427235 (https://phabricator.wikimedia.org/T190100) (owner: Ejegg) [16:52:13] (PS1) AndyRussG: Add privacy warning and allow wikitext in banner content field summary [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/427439 (https://phabricator.wikimedia.org/T190089) [16:52:54] (CR) AndyRussG: [C: 2] Add privacy warning and allow wikitext in banner content field summary [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/427439 (https://phabricator.wikimedia.org/T190089) (owner: AndyRussG) [17:03:53] (Merged) jenkins-bot: Add privacy warning and allow wikitext in banner content field summary [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/427439 (https://phabricator.wikimedia.org/T190089) (owner: AndyRussG) [17:08:54] Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Bug in trigger creation from recent upgrade is causing civicrm_mailing.modified_date to be inappropria... - https://phabricator.wikimedia.org/T192051#4125928 [17:12:17] (PS1) Ejegg: Update CiviCRM submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427446 [17:13:06] fr-tech: I'm going to update the civicrm submodule on prod so we can regenerate the triggers without those mailing table updates [17:13:11] (CR) Ejegg: [C: 2] Update CiviCRM submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427446 (owner: Ejegg) [17:13:53] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427448 [17:13:58] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427448 (owner: Ejegg) [17:14:05] ejegg: cool [17:18:12] (Merged) jenkins-bot: Update CiviCRM submodule [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427446 (owner: Ejegg) [17:18:14] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427448 (owner: Ejegg) [17:19:17] !log updated CiviCRM from 64b26ad377 to cddfe9416c [17:19:22] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:02:31] (PS1) Ejegg: Fix triggers to remove spurious mailing updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427459 (https://phabricator.wikimedia.org/T192051) [18:03:07] fr-tech ok, just regenerated those triggers with eileen's latest fix ^^^ [18:43:42] Wikimedia-Fundraising-Banners: Fix Lint errors for current fundraising banners - https://phabricator.wikimedia.org/T192480#4140361 (Pcoombe) p:Triage>High [18:44:44] Wikimedia-Fundraising-Banners: Fix Lint errors for current fundraising banners - https://phabricator.wikimedia.org/T192480#4140378 (Pcoombe) [19:52:04] (CR) Eileen: [C: 2] Fix triggers to remove spurious mailing updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427459 (https://phabricator.wikimedia.org/T192051) (owner: Ejegg) [19:56:31] (Merged) jenkins-bot: Fix triggers to remove spurious mailing updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427459 (https://phabricator.wikimedia.org/T192051) (owner: Ejegg) [20:00:08] PROBLEM - check_puppetrun on rigel is CRITICAL: CRITICAL: Puppet has 21 failures. Last run 5 minutes ago with 21 failures. Failed resources (up to 3 shown): File[/etc/motd.tail],File[/usr/bin/check-raid.py],File[/etc/rssh.conf],File[/usr/local/bin/yubikey_otp_filter] [20:00:08] PROBLEM - check_puppetrun on alnitak is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 5 minutes ago with 6 failures. Failed resources (up to 3 shown) [20:01:51] Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi: fix Batch Merge Selected Duplicates - https://phabricator.wikimedia.org/T192491#4140595 (MBeat33) [20:05:08] RECOVERY - check_puppetrun on rigel is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [20:05:08] RECOVERY - check_puppetrun on alnitak is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [20:11:29] Fundraising-Backlog: Pivot and Analytics access for all FR-Tech - https://phabricator.wikimedia.org/T181629#3796522 (mepps) Opened this for myself: https://phabricator.wikimedia.org/T192472 [20:40:45] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4140714 [20:45:48] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4140719 [20:56:30] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4140759 [21:05:19] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427542 [21:05:48] (CR) Eileen: [C: 2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427542 (owner: Eileen) [21:07:15] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4140787 [21:07:17] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427542 (owner: Eileen) [21:17:04] !log civicrm revision changed from cddfe9416c to 00870af548, config revision is 853fcc9111 [21:17:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:17:19] cwd - ejegg version of trigger script now deployed [21:17:35] thanks, i'll run it in a minute [21:17:56] cool [21:18:41] ejegg: did you see my comment on here https://gerrit.wikimedia.org/r/#/c/423799/ (it got swallowed when rebased) [21:21:38] looking at this file i have a memory where there is a syntax error with the delimiters [21:21:50] eileen: oh right, I did see that. Taking one more look at the example you put in [21:26:38] cwd - can run on staging as a test first [21:26:47] yeah [21:28:32] break in the rain & not-so-heavy traffic = I'mma relocate [21:28:42] be back on from the apt in half an hr-ish [21:29:04] cwd hmm didn'trun on staging as a table is not on staging now [21:29:35] yep just saw that [21:30:09] what did we miss? [21:32:00] (PS1) Eileen: Remove triggers from removed language table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427548 [21:32:25] cwd - presumably table still exists on live & was generated there - just putting up a gerrit to remove those [21:32:36] cool [21:32:39] https://gerrit.wikimedia.org/r/#/c/427548/ [21:34:04] cwd are you ok to review that one? I didn't mean to remove the DROP TRIGGER IF EXISTS civicrm_value_1_language_after_delete // but it's actually duplicated in the file so it's fine [21:35:38] where else is it? [21:37:49] there are a whole lot of drops near the top of the file [21:38:04] they basically ALL appear twice [21:39:11] heh cool [21:39:30] away for 15 [21:41:56] (CR) Cdentinger: [C: 2] Remove triggers from removed language table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427548 (owner: Eileen) [21:46:22] (Merged) jenkins-bot: Remove triggers from removed language table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427548 (owner: Eileen) [21:52:55] cwd ran on staging & all ok [21:53:58] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427552 [21:54:08] (CR) Eileen: [C: 2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427552 (owner: Eileen) [21:55:00] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/427552 (owner: Eileen) [21:55:06] eileen: great [21:56:24] ejegg|afk: not sure why update 7640 [21:56:25] didn't delete language cust group on live but might do manually when triggers are updated [21:57:28] !log civicrm revision changed from 00870af548 to 0ac27e7c0d, config revision is 853fcc9111 [21:57:33] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:57:36] cwd ^^ deployed now [21:59:52] MBeat: I just tried exporting to csv from fraud report on staging & it worked? [22:04:17] eileen: ok here goes [22:04:26] cwd cool [22:04:43] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4141047 [22:04:57] eileen: it's taking a long time [22:05:02] * cwd sweats [22:05:07] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4141048 [22:05:09] ok appears to have finished [22:05:14] but took about 10x the time on staging [22:05:52] i think it's ok [22:05:54] 446 triggers [22:06:21] ty eileen I am retesting export on staging [22:14:57] cwd hmm 467 on staging but that might be OK [22:15:15] 446 tables on staging [22:15:44] is the 446 def triggers not tables? [22:16:50] eileen: unless i'm confused [22:17:32] eileen: show triggers returns 446 [22:18:09] and yeah 467 on staging [22:18:15] that's unnerving [22:18:25] cwd what does show tables give [22:19:18] I have 464 locally :-) [22:21:00] eileen: 330 on prod [22:21:17] wow prod lost some tables! [22:21:35] (334 tables locally) [22:21:45] including 'my_table [22:22:15] i'm confused [22:22:21] some tables on staging that I don't have wmf_eoy_receipt_donor - possibly temp ones [22:23:10] & cwd probably staging has some extra tables with triggers that existed before the script. I'll try dropping all & re-running [22:23:20] cool [22:37:27] fr-tech as per -operations, the new CN patches didn't go out to meta just yet. Train and other odd deploy issues unrelated to CentralNotice [22:38:03] thanks for the update AndyRussG [22:38:42] ejegg: :) I guess no news would've been good news [22:39:33] I'm just gonna pull the current core release branch for 30 (the version that was supposed to go out) to check that the CN submodule pointer where it should be [22:39:52] I can't imagine there's anything off about the patches, the CSP checks out fine on mediawiki.org, where it is live [22:44:53] cwd 446 on staging now (after some clean up) [22:45:37] ejegg: so we can proceed with this one https://gerrit.wikimedia.org/r/#/c/424186/ [22:46:03] eileen: looking it over [22:51:42] eileen: I get a DB Constraint Violation - custom_group_id should possibly be marked [error] [22:51:45] as mandatory for CustomField,create API [22:51:51] when I run that locally [22:52:02] guess it's just lines 3779-80 [22:53:28] ejegg: yeah I don't think I correctly figured local DB states :-( [22:53:29] Well, .git/modules/extensions/CentralNotice/HEAD points to the right sha on the core release branch that was supposed to go out [22:53:41] (afcf52a2) [22:54:22] AndyRussG: sounds like everything's good from our end then [22:54:39] and we just have to wait for them to clear the cow guts off the train tracks [22:57:32] eileen: awesome, thanks [23:12:41] (PS2) Eileen: Apply drupal formatting to refundTest [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427317 [23:12:43] (PS10) Eileen: Add donor totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424179 (https://phabricator.wikimedia.org/T186168) [23:12:45] (PS11) Eileen: Transfer data to new custom field (deploy AFTER triggers re-established) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424186 (https://phabricator.wikimedia.org/T186168) [23:12:47] (PS17) Eileen: Add triggers to manage yearly totals & associated custom fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/425219 (https://phabricator.wikimedia.org/T186168) [23:12:49] (PS9) Eileen: Wip drush fill script [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [23:15:27] (CR) Ejegg: [C: 2] Apply drupal formatting to refundTest [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427317 (owner: Eileen) [23:15:49] that was the easy one [23:17:45] :) [23:21:30] (Merged) jenkins-bot: Apply drupal formatting to refundTest [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427317 (owner: Eileen) [23:26:49] (CR) Ejegg: [C: 2] Add donor totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424179 (https://phabricator.wikimedia.org/T186168) (owner: Eileen) [23:28:54] (PS10) Eileen: Drush script to fill totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/427319 [23:31:29] eileen: so.. I re-ran the 7670 one and didn't get the constraint violation [23:31:31] (Merged) jenkins-bot: Add donor totals fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424179 (https://phabricator.wikimedia.org/T186168) (owner: Eileen) [23:32:01] was that likely because I didn't have the same field? [23:32:07] ejegg: I think that's good? [23:32:15] did it bork early? [23:32:45] no, I think it got through the whole thing fine, I just don't see anything changed in the patch [23:32:53] I didn't change it [23:32:53] hmm, lemme swap the is_view and run one more time [23:32:58] I think it's local db state [23:34:17] I ran on staging but before I added batching - will try to reset staging state [23:34:49] eileen if I swap the is_view back to pre-update, then run it again, I get the constraint error again [23:35:28] oh, it's slightly different this time [23:35:31] Mandatory key(s) missing from params array: custom_group_id, label [23:36:01] so is_view causing get call to fail? [23:40:46] trying to run on staging now [23:41:09] oh, that's a possibility [23:41:37] i just kikcked it - waiting now [23:42:37] it's running [23:49:07] kinda slowly but.... [23:49:34] not in a ''locking' sort of way [23:50:38] ok - I think I can speed it up by only copying across '1' values [23:52:38] (PS12) Eileen: Transfer data to new custom field (deploy AFTER triggers re-established) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/424186 (https://phabricator.wikimedia.org/T186168) [23:53:40] MUCH quicker