[01:04:21] So looking at the csv we have been using to test performance I note that [01:05:27] 1) when we restart the process it restarts from the beginning of the csv,meaning that the next time we test we get a unspecified delay before it hits the ones it needs to insert [01:06:23] 2) there seem to be a number of remind-me-later contacts that are already in our DB, but they have no contact_id in silverpop because they are opted out, on_hold or otherwise blocked from being send to silverpop [01:06:45] over time these seem to be hogging more of the start of the file - leading to #1 above [01:07:33] I feel like I need to split the csv up somehow / remove read lines but have not spotted how to in http://csv.thephpleague.com/8.0/instantiation/ yet (the library I'm using) [01:24:55] Fundraising Sprint Navel Warfare, Fundraising Sprint Outie Inverter, Fundraising Sprint Prank Seatbelt, Fundraising Sprint Quill Pencil, and 4 others: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3599241 (Eileenmcnaughton) @CCogd... [01:45:51] Fundraising Sprint Quill Pencil, Fundraising-Backlog: Establish methodology for creating load to replicate replag - https://phabricator.wikimedia.org/T175665#3599281 (Eileenmcnaughton) [02:42:40] (PS1) Eileen: CRM-20722: 'Lock wait timeout exceeded' error triggered on smart group cache rebuild [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377385 [02:51:52] (PS1) Eileen: CRM-21109 only clear caches once on cli script, consolidate code [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377386 [03:36:30] Fundraising Sprint Navel Warfare, Fundraising Sprint Outie Inverter, Fundraising Sprint Prank Seatbelt, Fundraising Sprint Quill Pencil, and 4 others: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3599365 (Ejegg) @Eileenmcnaughton... [04:41:45] Fundraising Sprint Navel Warfare, Fundraising Sprint Outie Inverter, Fundraising Sprint Prank Seatbelt, Fundraising Sprint Quill Pencil, and 4 others: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3599377 (Eileenmcnaughton) ohh th... [09:05:03] Fundraising-Backlog, Wikimedia-Fundraising-Banners, fundraising-tech-ops, MediaWiki-extensions-CentralNotice, and 4 others: 2017 USA banners may freeze your browser for 1-2 seconds - https://phabricator.wikimedia.org/T174267#3599648 (Krinkle) [09:12:42] Fundraising-Backlog, Wikimedia-Fundraising-Banners, fundraising-tech-ops, MediaWiki-extensions-CentralNotice, and 4 others: 2017 USA banners may freeze your browser for 1-2 seconds - https://phabricator.wikimedia.org/T174267#3599665 (Gilles) [13:54:41] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Discussion: Identify rogue banner impressions to logged in user - https://phabricator.wikimedia.org/T175690#3600341 (Jseddon) [14:03:28] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Geoip lookup - Misidentifying country due to travelling - https://phabricator.wikimedia.org/T175691#3600361 (Jseddon) [15:07:19] (PS7) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [15:12:23] (CR) jerkins-bot: [V: -1] WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [15:13:52] hey ejegg! [15:14:10] hi mepps! when i just want to recalculate the composer hash, I do something like update psr/log [15:14:25] (if I don't want to actually change any packages) [15:14:34] since psr/log is pretty darn stable [15:16:38] (PS8) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [15:17:09] oh i see, you're updating all the things anyway... [15:20:50] ejegg i'm curious why LintYaml.php isn't a calss [15:20:52] class [15:21:12] I think it's just a maintenance script type thing [15:21:19] I ask because it's throwing an error because those functions are global without a wf prefix [15:21:29] do we want a wf prefix? [15:21:37] or should i exclude it? [15:21:39] hmm, I think we can ignore that rule [15:21:50] universally or just for that file? [15:22:19] Oh, I'm not sure how to do it file-specific [15:22:32] haha me neither [15:22:45] either way seems fine [15:23:05] found a few phpcbf errors, will add comments [15:23:20] okay all the errors we have left are about accessing $_POST and $_GET [15:23:25] (CR) Ejegg: "phpcbf errors noted inline" (7 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [15:24:36] should we suppress those or fix ejegg? [15:25:13] let's suppress those for now [15:25:28] in the interest of getting this merged! [15:38:06] okay! [15:43:27] (PS1) Ejegg: Update PHP-Queue to 1.0 [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/377490 [15:43:43] (CR) Ejegg: [C: 2] Update PHP-Queue to 1.0 [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/377490 (owner: Ejegg) [15:44:02] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/377491 [15:44:05] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/377491 (owner: Ejegg) [15:45:02] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/377491 (owner: Ejegg) [15:52:04] (CR) Ejegg: [V: 2 C: 2] Update PHP-Queue to 1.0 [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/377490 (owner: Ejegg) [15:55:40] Jeff_Green: I seem to have gotten SmashPig in a bad state on the deploy box [15:55:49] ok [15:55:57] The submodule update hadn't merged when I tried updating the first time [15:56:21] and now it's asking if it should delete every file in vendor, whatever I try to do [15:56:28] ok [15:56:30] looking [15:56:44] I tried doing an update -p SmashPig= [15:56:50] then -p SmashPig=HEAD [15:56:55] but no dice [15:57:09] thanks! [15:57:56] try now? [15:58:19] still trying to delete vendor file by file [15:58:28] maybe I should say y ? [15:59:01] i saved a copy of the whole tree aside, so it's ok to fubar this copy if it goes awry [16:06:20] Fundraising Sprint Quill Pencil, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wikimedia-CentralNotice-Administration, Unplanned-Sprint-Work: CN Campaign Suppression prior to scheduled start time - https://phabricator.wikimedia.org/T175358#3600913 (Jseddon) https://pivot.wikimedia.... [16:10:01] thanks Jeff_Green, deleting seemed to work [16:10:07] ok cool [16:18:46] !log updated SmashPig from dce4f0a to dc62203910 [16:18:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:35:07] Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, and 11 others: Move already-encapsulated useful classes into SmashPig - https://phabricator.wikimedia.org/T163868#3212798 (Ejegg) Open... [16:36:11] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/377503 [16:36:19] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/377503 (owner: Ejegg) [16:37:12] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/377503 (owner: Ejegg) [16:40:06] !log updated CiviCRM from 63288ae8b3bac2f15804d4076c4ede55ba4a3759 to ee7dda38ed7071d3100907d11c2a77e82a2a852c [16:40:19] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:48:04] XenoRyet: have you looked at any CentralNotice stuff lately? [16:48:31] There are a couple of WIPs from AndyRussG that might want comments [16:48:48] and it might be a good idea to refresh our memory of that codebase [16:48:59] * AndyRussG sweeps trash under the carpet 8p [16:55:06] Fundraising Sprint Baudelaire Bowdlerizer, Fundraising Sprint Deferential Equations, Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, Fundraising-Backlog: fill out PCI SAQ-A form for 2017 - https://phabricator.wikimedia.org/T155779#3601101 (Ejegg) [16:56:03] ejegg: No, I've not been in there for a long time. You're right that it might be a good idea to take a look. [16:56:16] I'll take a peek at those WIPs [16:56:22] Fundraising Sprint Kickstopper, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, MW-1.30-release-notes (WMF-deploy-2017-06-06_(1.30.0-wmf.4)): Reconsider minfraud failure mode - https://phabricator.wikimedia.org/T150072#3601106 (Ejegg) [16:56:56] Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Thank you letter from civi mangles non-$ currencies - https://phabricator.wikimedia.org/T151957#3601107 (Ejegg) p:Triage>Normal [16:57:31] ejegg: Also, did you and mepps get anywhere on the 'variant' terminology for the A/B test patch? [16:57:34] great, thanks XenoRyet [16:57:44] I was lurking that conversation [16:57:54] ah, no, I should take a look at a thesaurus [16:58:17] Yea, it's weird. I get where she's coming from, but I can't think of a better word for what's going on there. [16:58:52] I can make 'variant' make sense in my head, but it doesn't flow as naturally as it might. [16:58:57] variation? [16:59:12] version? But that seems a bit too nebulous [16:59:30] yea, version gets used for different ideas as well. [17:00:59] man, I almost want to use branch [17:01:04] not quite right either though. [17:02:17] yeah, I can't see anything I like better [17:02:56] if I just search a/b testing, the blurbs all use 'version' and 'variant' [17:03:23] If that is the existing terminology, maybe we better just stick with that. [17:03:39] mepps: What do you think? ^ [17:03:50] first sentence of the wikipedia a/b testing article: In marketing and web analytics, A/B testing (bucket tests or split-run testing) is a controlled experiment with two variants, A and B.[1] [17:04:20] That's a pretty strong case in favor of variant. [17:04:45] Fundraising Sprint Ivory Tower Defense Games, Fundraising-Backlog, FR-PayPal-ExpressCheckout: Refine PayPal EC donation itemization - https://phabricator.wikimedia.org/T137730#3601125 (Ejegg) [17:07:41] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising-Backlog, fundraising-tech-ops: Migrate all jobs to process-control - https://phabricator.wikimedia.org/T162163#3601134 (Ejegg) p:Triage>High [17:08:09] Fundraising Sprint Gondwanaland Reunification Engine, Fundraising-Backlog: process-control failmail should include hostname - https://phabricator.wikimedia.org/T162214#3601138 (Ejegg) p:Triage>Normal [17:08:30] Fundraising Sprint Homebrew Hadron Collider, Fundraising-Backlog, FR-PayPal-ExpressCheckout, MW-1.29-release (WMF-deploy-2017-04-25_(1.29.0-wmf.21)), and 2 others: payments frontend sending duplicate malformed queue messages for EC recurring - https://phabricator.wikimedia.org/T163281#3601140 (Eje... [17:30:17] ejegg XenoRyet, if that's what's suggested i guess i can see it [17:34:39] Let me at least look through the code one more time and see if something more clear jumps out at me. [17:35:06] 'variation' maybe? but it's on the bubble too so i'm not sure i feel like that's a change that's needed [17:36:55] ejegg: i am gonna fire up dedupe to generate some heat, any objections? [17:36:57] Between variation and variant, I think we do stay variant due to it being the accepted term for A/B testing, but I sort of want to see if there's maybe a different way to describe what that variable actually is. [17:37:08] cwd go for it [17:37:21] I just deployed a thing to retry deadlocked contribution imports [17:37:29] so this might be a nice test of that too [17:37:31] woah [17:37:37] i just learned what 'druthers' means [17:37:47] i've heard the phrase for so long and hadn't thought aobut it [17:38:32] Heh, I don't think I've ever thought about that word either. [17:38:39] a couple ideas i'm throwing out XenoRyet: 'option', 'sub' [17:39:25] option feels like a single setting, not a group of changes [17:40:00] looking through the code, I think I'm talking myself into variant. [17:40:02] !log turned on dedupe [17:40:15] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:41:27] We have a form that has two or more variants, and this is how we specify which one we want. [17:43:06] dedupe: it's super effective https://grafana.wikimedia.org/dashboard/db/frack-db?orgId=1&from=now-30m&to=now [17:43:30] okay cool, i'm fine with moving forward with variant [17:44:15] Ok, let's do that then. Definitely a good conversation to have had though, I'm glad you brought it up. [17:44:45] You want to push the ol' +2 or shall I? [17:46:20] all you XenoRyet! [17:46:29] 10-4, will do. [17:47:48] (CR) XenoRyet: [C: 2] "Talked it out and we're sticking with 'variant' due to common A/B testing terminology." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/376850 (https://phabricator.wikimedia.org/T151769) (owner: Ejegg) [17:49:28] (PS6) Ejegg: Short array syntax [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/376307 [17:49:44] (Merged) jenkins-bot: Allow alternate configurations with 'variant' option [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/376850 (https://phabricator.wikimedia.org/T151769) (owner: Ejegg) [17:56:41] Thanks, mepps and XenoRyet [17:57:04] No worries [18:12:17] Fundraising Sprint Quill Pencil, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wikimedia-CentralNotice-Administration, Unplanned-Sprint-Work: CN Campaign Suppression prior to scheduled start time - https://phabricator.wikimedia.org/T175358#3601439 (AndyRussG) OK, thanks. So the cam... [18:16:29] (CR) Ejegg: [C: 2] "Looks good." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377385 (owner: Eileen) [18:21:01] hey ejegg small thing but i'm a little undecided how to handle those comments errors [18:21:19] mepps let's leave them as they were [18:21:21] should i just exclude the rule that caused those to be fixed, or should i move the comments to multiple lines and delete the duplicates [18:21:48] oh huh, do you know what rule it was? [18:22:06] and does phpcs complain about them as they were? [18:22:10] ejegg might be a line length but i thought i'd already excluded it, maybe not before fix [18:22:28] Hmm, I wonder if this has been reported as a phpcbf bug [18:22:55] weirdly, the 'fix' increases line lenght [18:25:16] (Merged) jenkins-bot: CRM-20722: 'Lock wait timeout exceeded' error triggered on smart group cache rebuild [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377385 (owner: Eileen) [18:29:49] (PS9) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [18:29:54] so ejegg we're down to just warnings [18:29:57] (CR) jerkins-bot: [V: -1] WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [18:30:16] nice! [18:30:30] ack, and a rebase [18:30:35] hope that's not too hairy [18:31:34] (PS10) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [18:32:13] nope ejegg it was pretty straightforward [18:35:09] cool! Looking at the lastet [18:35:12] *latest [18:35:25] also the lastest [18:37:13] eww, I really don't like the comments all on one line [18:38:02] the findCodeAction comment in gateway.adapter.php is still duplicating lines [18:38:18] (CR) XenoRyet: UI modules are part of the configuration (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) (owner: Ejegg) [18:42:12] (CR) Ejegg: [C: 1] "Nice! Can we just fix the two comments noted in gatway.adapter.php? Feels wrong to give in to a phpcbf bug and make things less readable." (2 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [18:45:08] hmph, failmail on a deadlock [18:45:15] doesn't look like we retried it [18:46:23] ejegg i think line 645 is how it was beore [18:46:39] oh nevermind i see now [18:46:41] weird! [18:46:44] i don't know why it did that [18:47:00] yeah, definitely not helpful behavior [18:47:54] (PS2) Ejegg: UI modules are part of the configuration [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) [18:47:56] ejegg we are kind of inconsistent with comments and line breaks [18:48:00] in the code base generally [18:48:05] (CR) jerkins-bot: [V: -1] UI modules are part of the configuration [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) (owner: Ejegg) [18:48:48] (CR) Ejegg: "fixed" (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) (owner: Ejegg) [18:49:33] mepps yeah, we can fix the rest of that down the road [18:49:53] (PS3) Ejegg: UI modules are part of the configuration [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) [18:52:05] (PS11) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [18:55:14] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2479 [18:57:17] (CR) XenoRyet: [C: 2] UI modules are part of the configuration [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) (owner: Ejegg) [18:57:49] Fundraising Sprint Quill Pencil, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wikimedia-CentralNotice-Administration, Unplanned-Sprint-Work: CN Campaign Suppression prior to scheduled start time - https://phabricator.wikimedia.org/T175358#3601679 (Jseddon) The suppression was only... [18:58:53] (CR) Ejegg: [C: 2] "Stylin!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [18:59:10] Fundraising-Backlog, FR-Adyen: adyen Stand-alone secured fields - https://phabricator.wikimedia.org/T175732#3601680 (DStrine) [18:59:25] oh shoot, those last two +2ed patches are probably going to conflict [19:00:08] Also, lets un-WIP that! [19:00:14] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1976 [19:01:59] (Merged) jenkins-bot: UI modules are part of the configuration [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/377312 (https://phabricator.wikimedia.org/T173869) (owner: Ejegg) [19:02:01] (CR) jerkins-bot: [V: -1] WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [19:02:54] (CR) Ejegg: "Sorry, looks like one more rebase is needed! Also, let's take off that 'WIP'" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [19:05:51] (PS12) Mepps: WIP Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [19:06:04] (PS13) Mepps: Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) [19:06:52] (CR) Ejegg: [C: 2] "Stylin!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [19:09:40] (CR) Ejegg: [C: 2] "Looks like a solid improvement. Looking forward to testing this with our import and audit scripts!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377386 (owner: Eileen) [19:10:13] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 969901 Threads: 1 Questions: 28785201 Slow queries: 20211 Opens: 11604 Flush tables: 1 Open tables: 603 Queries per second avg: 29.678 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [19:12:48] mepps I'm going to deploy the revision right before that, just to isolate the library updates from my ui / config changes [19:13:44] (PS1) Ejegg: Update php-queue and smash-pig [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/377532 [19:14:11] (Abandoned) Ejegg: Update SmashPig, PHPQueue [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/376722 (owner: Ejegg) [19:14:25] (CR) Ejegg: [C: 2] Update php-queue and smash-pig [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/377532 (owner: Ejegg) [19:14:44] (Merged) jenkins-bot: Add phpcs script [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/371770 (https://phabricator.wikimedia.org/T170314) (owner: Mepps) [19:15:03] (CR) Ejegg: [V: 2 C: 2] Update php-queue and smash-pig [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/377532 (owner: Ejegg) [19:16:07] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/377533 [19:16:10] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/377533 (owner: Ejegg) [19:18:06] (Merged) jenkins-bot: CRM-21109 only clear caches once on cli script, consolidate code [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/377386 (owner: Eileen) [19:18:21] Fundraising Sprint Quill Pencil, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wikimedia-CentralNotice-Administration, Unplanned-Sprint-Work: CN Campaign Suppression prior to scheduled start time - https://phabricator.wikimedia.org/T175358#3601773 (AndyRussG) Though if you [[ https... [19:18:28] fr-tech I'd like to deploy this diagnostic thing: https://gerrit.wikimedia.org/r/376760 [19:18:30] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/377533 (owner: Ejegg) [19:18:53] ejegg: looking [19:18:59] (PS3) Mepps: Log a stack trace for all dedupe queries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/376760 (https://phabricator.wikimedia.org/T175382) (owner: Ejegg) [19:19:00] thanks [19:19:04] (CR) Mepps: [C: 2] Log a stack trace for all dedupe queries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/376760 (https://phabricator.wikimedia.org/T175382) (owner: Ejegg) [19:19:15] ty mepps [19:19:42] (CR) Mepps: [C: 2] Short array syntax [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/376307 (owner: Ejegg) [19:19:48] woohoo! [19:20:16] hehe, now to rebase all my old patches around those style cleanups [19:20:51] (Merged) jenkins-bot: Short array syntax [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/376307 (owner: Ejegg) [19:24:04] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/377537 [19:24:10] (PS1) AndyRussG: BannerHistoryLogger: Fix ajax options for when sendBeacon is not available [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/377538 (https://phabricator.wikimedia.org/T174719) [19:24:32] (CR) Ejegg: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/377537 (owner: Ejegg) [19:27:41] (Merged) jenkins-bot: Log a stack trace for all dedupe queries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/376760 (https://phabricator.wikimedia.org/T175382) (owner: Ejegg) [19:32:02] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/377537 (owner: Ejegg) [19:32:20] ejegg: meeting? [19:34:01] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Discussion: Identify rogue banner impressions to logged in user - https://phabricator.wikimedia.org/T175690#3601867 (AndyRussG) Hmmm... Pivot doesn't show any [[ https://goo.gl/vgqyrR | FR campaigns going to logged-in users ]]... Do we have any mor... [19:37:24] (PS7) Ejegg: Log client-side errors on payments forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/259795 (https://phabricator.wikimedia.org/T121800) [19:52:26] Fundraising-Backlog, FR-Adyen: adyen Stand-alone secured fields - https://phabricator.wikimedia.org/T175732#3601914 (DStrine) [19:52:58] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic, FR-Email: Reflect all undeliverables via Silverpop in CiviCRM - https://phabricator.wikimedia.org/T161761#3601917 (CCogdill_WMF) @Eileenmcnaughton I think we discussed this over chat? But recording here: These can be marked on-hold * Hard... [19:54:31] fundraising-tech-ops, Operations: Long term storage for frack prometheus data - https://phabricator.wikimedia.org/T175738#3601924 (cwdent) [19:56:26] (PS8) Ejegg: WIP fraud points for email address at nonexistant tld [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/290615 [19:58:20] (CR) jerkins-bot: [V: -1] WIP fraud points for email address at nonexistant tld [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/290615 (owner: Ejegg) [19:59:00] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Geoip lookup - Misidentifying country due to travelling - https://phabricator.wikimedia.org/T175691#3600361 (AndyRussG) It's not a negligible number of users whose GeoIP location does not correspond to the location identified using their IP on the... [20:08:08] cwd are you about later on to do a couple of load tests? [20:10:52] Fundraising Sprint Quill Pencil, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Deal with diacritic conflicts on name checks - https://phabricator.wikimedia.org/T149763#3602026 (Eileenmcnaughton) [20:14:22] eileen: yes i will be around for awhile [20:14:40] fwiw i turned dedupe back on, it thrashed for awhile but did not create any real lag [20:14:45] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18, Epic: Update wmf_civicrm import normalisation to replace htmlampersand with & - https://phabricator.wikimedia.org/T175744#3602050 (Eileenmcnaughton) [20:14:50] and i have been unable to make the omnimailer thing lag either [20:15:23] cwd hmm - I wonder if something takes a while to 'build up' again after the restart [20:16:05] I found a few articles related to dirty pages yesterday [20:16:24] added them to the etherpad - the https://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_flushing_avg_loops seems of interest [20:18:11] interesting [20:18:25] the dbas did not seem to think our dirty pages looked abnormal [20:18:46] though it is interesting that frdb2001 seems to operate with a lot fewer dirty pages generally [20:19:23] which would seem to insinuate that it's flushing to disk more often, which might have to do with it having less ram than the others (~1/4) [20:19:50] all the settings should be the same so that's the only reason i think of for it behaving differently [20:20:49] with that it mind, it does not seem to lag any worse than the others [20:22:52] (PS4) Ejegg: WIP: getAllValidForms uses isValidForm [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/338458 (https://phabricator.wikimedia.org/T136254) [20:23:17] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Update wmf_civicrm import normalisation to replace htmlampersand with & - https://phabricator.wikimedia.org/T175744#3602094 (Eileenmcnaughton) [20:24:00] (PS4) Ejegg: Batch look up order status for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/328858 [20:24:27] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Implement merge rule on whitespace to prefer ones with more whitespace - https://phabricator.wikimedia.org/T175746#3602096 (Eileenmcnaughton) [20:24:36] (CR) jerkins-bot: [V: -1] WIP: getAllValidForms uses isValidForm [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/338458 (https://phabricator.wikimedia.org/T136254) (owner: Ejegg) [20:25:36] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Implement merge rule to allow overwriting any name fields that are wholly numeric in the case of a conflict - https://phabricator.wikimedia.org/T175747#3602111 (Eileenmcnaughton) [20:26:40] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Implement merge rule to allow resolution on conflicts on punctuation to prefer more punctuation - https://phabricator.wikimedia.org/T175748#3602125 (Eileenmcnaughton) [20:41:47] ejegg with phpcs closed out i'm not sure what the next priority is--there's still some outstanding ingenicoconnect work but i'm not sure if it's paused, just feel free to pint or email me before tomorrow if there's anything i should prioritize, otherwise i might look at CentralNotice a bit [20:50:10] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1444 [20:50:22] mepps let's try to get the DonationInterface part of getStatus rebased and merged [20:55:10] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 976200 Threads: 1 Questions: 30756580 Slow queries: 20416 Opens: 11716 Flush tables: 1 Open tables: 603 Queries per second avg: 31.506 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 132 [21:23:05] (PS6) Ejegg: WIP getHostedCheckoutStatus [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/366167 (https://phabricator.wikimedia.org/T163948) [21:33:50] !log changed contact de-duplication duty cycle to 25 minutes [21:34:03] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:35:56] ejegg: that dedupe stuff I'm not sure the right answer [21:36:13] the thing is - we used to use the fact that jenkins would not run concurrent jobs as a feature [21:36:21] yah [21:37:09] ie. if there is a lot of dupes in the batch it takes longer but often they are pretty quick [21:37:15] well, we could add an option to process-control to suppress that error [21:37:53] ie. the yaml would specify? [21:37:57] yeah [21:38:12] dunno what you'd call it [21:38:24] does it make sense to go as far as having a threshhold? ie. failmail after x skips? Might be overengineering [21:38:28] ignore-overtime or something [21:38:35] yeah that makes sens [21:38:36] e [21:53:51] eileen: did you have any new ideas about creating lag? [21:55:18] cwd just working on adjusting that omnimail script to allow us to set an offset in the file - I can see you must have been running it though as number of contacts in the group has shot up [21:55:29] we have indeed [21:55:40] and i was wondering if the writes are getting delayed by the completed rows [21:55:59] cwd - yeah so adding in an offset should help [23:02:30] (PS1) Eileen: Update Omnimail plugin to allow setting offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377663 [23:02:33] (PS1) Eileen: Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377664 (https://phabricator.wikimedia.org/T175665) [23:04:36] (CR) jerkins-bot: [V: -1] Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377664 (https://phabricator.wikimedia.org/T175665) (owner: Eileen) [23:19:00] (PS2) Eileen: Update Omnimail plugin to allow setting offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377663 [23:19:45] (PS1) Eileen: Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) [23:20:14] (Abandoned) Eileen: Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377664 (https://phabricator.wikimedia.org/T175665) (owner: Eileen) [23:28:54] (CR) jerkins-bot: [V: -1] Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) (owner: Eileen) [23:34:46] eileen: having friends over and i need to help cook, but i will check in later, and be watching for replag :) [23:34:49] (PS2) Eileen: Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) [23:39:09] ejegg: any chance you can check https://gerrit.wikimedia.org/r/#/c/377666/ - to allow better replag testing? (just waiting for zul to do it's thing) [23:40:31] for sure! [23:41:13] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Update wmf_civicrm import normalisation to replace htmlampersand with & - https://phabricator.wikimedia.org/T175744#3602708 (Eileenmcnaughton) [23:41:34] go zuul go [23:41:49] Fundraising Sprint Navel Warfare, Fundraising Sprint Outie Inverter, Fundraising Sprint Prank Seatbelt, Fundraising Sprint Quill Pencil, and 4 others: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3602710 (CCogdill_WMF) @Eileenmcn... [23:51:42] Fundraising Sprint Navel Warfare, Fundraising Sprint Outie Inverter, Fundraising Sprint Prank Seatbelt, Fundraising Sprint Quill Pencil, and 4 others: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3602752 (Eileenmcnaughton) I don'... [23:53:26] (PS1) Eileen: Submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377673 [23:54:05] (CR) Eileen: [C: 2] Submodule update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377673 (owner: Eileen) [23:54:41] (PS3) Ejegg: Update Omnimail plugin to allow setting offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377663 (owner: Eileen) [23:54:56] (CR) Ejegg: [C: 2] Update Omnimail plugin to allow setting offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377663 (owner: Eileen) [23:55:14] (PS3) Ejegg: Update Omnimail to use offset [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) (owner: Eileen) [23:56:31] (CR) Ejegg: [C: 2] "Looks good, assuming we're always resuming from the spot we left off at, and not changing params after an incomplete run." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) (owner: Eileen) [23:58:16] (CR) Eileen: "We can alter the DB setting if we want to jiggy with it after an incomplete run - or pass in offset in the drush command" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/377666 (https://phabricator.wikimedia.org/T175665) (owner: Eileen)