[00:06:43] thanks [00:07:51] (CR) Eileen: [C: -1] "Needs to be after https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/610412 is deployed" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609238 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:27:46] (PS11) Ejegg: Silverpop refactor - remove last UPDATE from the silverpop script [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608991 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:28:08] (CR) Ejegg: [C: +2] Silverpop refactor - remove last UPDATE from the silverpop script [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608991 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:28:44] thanks ejegg [00:28:59] (Merged) jenkins-bot: Silverpop refactor - remove last UPDATE from the silverpop script [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608991 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:29:07] ejegg: this is also pretty small ... https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/610196 [00:31:39] (CR) Ejegg: [C: +2] Extract update function [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610196 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:31:47] thanks ejegg [00:32:30] (Merged) jenkins-bot: Extract update function [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610196 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:42:00] (PS2) Eileen: Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) [00:42:16] OK - this one (just rebased) is also pretty trivial https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/609896 [00:42:38] (no pressure - that one isn't causing me the headaches the other was) [00:43:12] oh right, i was looking at that last night [00:43:18] and just forgot to press the button [00:43:21] oh cool [00:44:07] I think this is OK now too - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/610412 [00:44:34] eileen: oh right, the removal of the join to silverpop_export_staging gave me pause there [00:44:46] i which one [00:45:15] the exid one? [00:45:29] yep, just wanted to make sure we weren't getting any extra rows [00:46:00] hmm just 'join' is that left or inner [00:46:21] ah yes it's an inner join [00:47:27] so I guess we could make the donor join INNER [00:47:51] to ensure no change [00:48:56] (PS3) Eileen: Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) [00:48:58] hmm, actually that would really drop the number of rows in that table [00:49:26] since export_staging has the RML contacts with no donations [00:49:32] Well if we COALESCE in the view it's OK - let me check that [00:49:37] (CR) jerkins-bot: [V: -1] Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:49:59] oh hang on - I see [00:50:12] yeah, PS2 should be fine [00:50:18] since we're grouping on email [00:51:01] I think INNER is wrong - for the view [00:52:19] (PS4) Eileen: Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) [00:53:18] ah, yeah, keep the stat table small [00:53:36] yeah - but we aren't coalescing in the view.... [00:53:57] oh right [00:54:20] later on we filter bringing into INSERT INTO silverpop_export ( [00:55:11] does that FORCE INDEX ui_email still make sense now that we're not joining on email? [00:56:04] so I did a looooootttt of queries last night [00:56:17] I can put my list into a gerrit for you to see [00:56:32] but basically if you are doing GROUP BY on a field then indexing it is good [00:56:44] ok, cool [00:56:53] better is a combined index including that field AND any you aggregate [00:57:45] (CR) Ejegg: [C: +2] Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:57:51] (PS1) Eileen: Adventures in query-land [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610447 [00:58:17] ejegg: - that last PR is just for interest on queries I tried [00:58:38] (Merged) jenkins-bot: Remove exid from export_stat table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609896 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [00:59:57] ejegg: you can see the combined index in this one - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/610412 [01:02:04] now I can try to unjumble all my WIP [01:03:25] I need to deploy all the ones like ^^ that change the fields before deploying any that switch to incremental [01:03:41] but I won't deploy today to let the process run [01:22:08] (PS5) Eileen: Add all funds last donation date to export_staging table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610412 (https://phabricator.wikimedia.org/T253152) [01:34:05] (PS6) Eileen: Add all funds last donation date to export_staging table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610412 (https://phabricator.wikimedia.org/T253152) [01:34:07] (PS1) Eileen: Ensure num_rows is defined [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610479 (https://phabricator.wikimedia.org/T253152) [01:34:09] (PS1) Eileen: Remove a bunch of fields from silverpop_export_staging that are no longer used [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610480 (https://phabricator.wikimedia.org/T253152) [01:41:20] (CR) Ejegg: [C: +2] "Impressive index tweak!" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610412 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [01:42:13] (Merged) jenkins-bot: Add all funds last donation date to export_staging table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610412 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [01:42:36] ejegg: also 2 super-trivial ones in this chain - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/610359 [01:45:29] cool, I can def review the rename one [01:47:28] yep feel free to stop - I'll keep pointing them out - you just have to walk away :-) [01:48:06] (PS1) Eileen: Alter master email table to use latest donor rather than highest email id [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610495 (https://phabricator.wikimedia.org/T253152) [01:48:29] I switched it to used the email id of the latest donor (not the highest email id) - it was bugging me [01:53:09] (PS2) Eileen: Remove a bunch of fields from silverpop_export_staging that are no longer used [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610480 (https://phabricator.wikimedia.org/T253152) [01:53:11] (PS2) Eileen: Alter master email table to use latest donor rather than highest email id [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610495 (https://phabricator.wikimedia.org/T253152) [01:53:21] (PS2) Ejegg: Fix column names & table name on prospect fields for silverpop [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/609557 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [01:53:44] (CR) Ejegg: [C: +2] "Thanks for the consistency fix!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/609557 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [01:55:47] (CR) jerkins-bot: [V: -1] Fix column names & table name on prospect fields for silverpop [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/609557 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [01:56:27] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/609557 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [02:02:24] (PS2) Ejegg: Exclude discussed tables from logs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/610359 (https://phabricator.wikimedia.org/T257232) (owner: Eileen) [02:02:56] (CR) Ejegg: [C: +2] "Looks good!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/610359 (https://phabricator.wikimedia.org/T257232) (owner: Eileen) [02:03:10] ok, heading out! [02:03:15] cool thanks [02:04:29] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610507 [02:07:42] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610507 (owner: Eileen) [02:08:17] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610507 (owner: Eileen) [02:14:28] (Merged) jenkins-bot: Fix column names & table name on prospect fields for silverpop [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/609557 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [02:14:30] (CR) jerkins-bot: [V: -1] Exclude discussed tables from logs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/610359 (https://phabricator.wikimedia.org/T257232) (owner: Eileen) [02:14:32] (PS3) Eileen: Alter master email table to use latest donor rather than highest email id [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610495 (https://phabricator.wikimedia.org/T253152) [02:14:34] (PS3) Eileen: Remove a bunch of fields from silverpop_export_staging that are no longer used [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610480 (https://phabricator.wikimedia.org/T253152) [06:41:16] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/610359 (https://phabricator.wikimedia.org/T257232) (owner: Eileen) [06:42:09] (PS1) Eileen: Extract running of queries to a function [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610663 (https://phabricator.wikimedia.org/T253152) [06:42:11] (PS1) Eileen: Iterate array of updates to run [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610664 (https://phabricator.wikimedia.org/T253152) [07:21:19] (CR) AndyRussG: "Hi! Thanks much for the changes!! Yeah almost there... So, it's not a blocker, but I would really like to suggest that $currentEmployerFil" (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608089 (https://phabricator.wikimedia.org/T251201) (owner: Jgleeson) [12:15:16] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (Pcoombe) Checked the color contrast because I wasn't sure, and the black on orange is fine. Minor thing, but should we also change the "info i" to be the same orange? I think it would look better. [12:49:24] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, and 3 others: Add and delete fields from the _all_Wikimedia database (civi export to ESP) - https://phabricator.wikimedia.org/T252245 (KHaggard) Rec... [12:53:54] (CR) Jgreen: [C: +1] Allow specifying a channel key for access to a channel [wikimedia/fundraising/slander] - https://gerrit.wikimedia.org/r/610398 (https://phabricator.wikimedia.org/T122104) (owner: Dwisehaupt) [13:05:47] (CR) Jgreen: [C: +1] Enable irc connections over ssl if defined [wikimedia/fundraising/slander] - https://gerrit.wikimedia.org/r/610386 (https://phabricator.wikimedia.org/T122104) (owner: Dwisehaupt) [13:14:02] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising Sprint Grep works IRL, and 4 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (Pcoombe) @MBeat33 You can just point them di... [13:17:14] (CR) Jgreen: [C: +1] Changes for slander to work with python2 or python3 [wikimedia/fundraising/slander] - https://gerrit.wikimedia.org/r/595217 (https://phabricator.wikimedia.org/T122104) (owner: Dwisehaupt) [13:38:41] Fundraising-Backlog, FR-Adyen, donate.wikimedia.org: Adyen recurring option for France on landing page - https://phabricator.wikimedia.org/T257450 (Pcoombe) Open→Resolved I updated the Adyen Credit Card button to allow monthly. https://donate.wikimedia.org/w/index.php?title=Template:Paymentme... [14:00:47] fundraising-tech-ops: Configure payments2xxx for MediaWiki 1.35 upgrade testing - https://phabricator.wikimedia.org/T256153 (Jgreen) [14:18:45] Fundraising-Backlog, fundraising-tech-ops: Setup shell access and civi access for civi staging for Coleman Watts - https://phabricator.wikimedia.org/T252693 (Dwisehaupt) Stalled→Resolved [14:19:42] fundraising-tech-ops: Upgrade payments to MediaWiki 1.35 - https://phabricator.wikimedia.org/T254298 (Jgreen) Removed Fundraising-Backlog tag. The upgrade (this task) is a fr-tech-ops project. Broke out the Fundraising Tech parts to separate tasks for clearer task dependency tracking. [14:20:07] Fundraising-Backlog, fundraising-tech-ops: Setup shell access and civi access for civi staging for Coleman Watts - https://phabricator.wikimedia.org/T252693 (Dwisehaupt) Resolved→Stalled [14:23:05] Fundraising-Backlog, FR-Adyen, donate.wikimedia.org: Adyen recurring option for France on landing page - https://phabricator.wikimedia.org/T257450 (MBeat33) Thank you! [14:29:05] Wikimedia-Fundraising-Banners: [Enhancement] Update the i color on banners to orange also - https://phabricator.wikimedia.org/T257580 (jbolorinos-ctr) [14:30:21] Wikimedia-Fundraising-Banners: [Enhancement] Update the i color on banners to orange also - https://phabricator.wikimedia.org/T257580 (jbolorinos-ctr) [14:31:43] Wikimedia-Fundraising-Banners: [Enhancement] Update the i color on banners to orange also - https://phabricator.wikimedia.org/T257580 (jbolorinos-ctr) [14:32:51] fundraising-tech-ops, Patch-For-Review: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Dwisehaupt) Stalled→Open [14:32:53] fundraising-tech-ops: 2019-2020 Q3 fundraising hardware refresh and capex - https://phabricator.wikimedia.org/T242257 (Dwisehaupt) [14:33:47] fundraising-tech-ops, Gerrit: move main fundraising puppet repo to gerrit - https://phabricator.wikimedia.org/T222840 (Dwisehaupt) p:Low→Medium [14:40:47] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (jbolorinos-ctr) Haha, yeah I just made this same task for that @Pcoombe T257580 [14:54:53] Wikimedia-Fundraising-Banners: [Enhancement][Dsk Sm] Update color of nag on Desktop Small to orange - https://phabricator.wikimedia.org/T257584 (jbolorinos-ctr) [14:58:04] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising Sprint Grep works IRL, and 4 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (MBeat33) Thank you @Pcoombe this is great. D... [15:03:46] fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227 (Jgreen) Stalled→Resolved [15:05:24] fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227 (Jgreen) Closing task because we did about as much ruleset tuning as was feasible given the application. Spun off T122322 to look at the possibility of modifying payments-wiki behavior to make it easier... [15:07:15] Fundraising-Backlog, fundraising-tech-ops: Investigate the rebrand url changes - https://phabricator.wikimedia.org/T248700 (Jgreen) [15:19:24] Fundraising-Backlog, fundraising-tech-ops, Operations, Traffic: SSL cert for links.email.wikimedia.org - https://phabricator.wikimedia.org/T188561 (Jgreen) Checking back on this it looks like https://links.e.uso.org has slipped back to a B rating because they haven't ceased TLS 1.0/1.1 support. I... [15:19:51] fundraising-tech-ops, Operations, Traffic: SSL cert for links.email.wikimedia.org - https://phabricator.wikimedia.org/T188561 (Jgreen) [15:32:45] Fundraising-Backlog: Investigate the rebrand url changes - https://phabricator.wikimedia.org/T248700 (Jgreen) Removing fundraising-tech-ops tag because the fr-tech-ops actionable part was broken out to a separate task T247846 and completed. [15:37:00] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Create virtual server for user testing of new features - https://phabricator.wikimedia.org/T223636 (Jgreen) Removed tag because there isn't anything actionable from a fundraising-tech-ops perspective. [15:39:41] Fundraising-Backlog: Enable Content Security Policy on donate-wiki - https://phabricator.wikimedia.org/T223658 (Jgreen) Removed tag because there's nothing actionable for fundraising-tech-ops, donate-wiki is not administered by our team. [15:42:26] Wikimedia-Fundraising-Banners: [Enhancement][Dsk Sm] Update color of nag on Desktop Small to orange - https://phabricator.wikimedia.org/T257584 (TSkaff) @jbolorinos-ctr This was on purpose & should be tested separately, could you resolve? thanks! [15:44:22] fundraising-tech-ops: icingawm is missing from #wikimedia-fundraising channel - https://phabricator.wikimedia.org/T202314 (Jgreen) Open→Resolved p:Triage→Medium a:Jgreen [15:44:29] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (TSkaff) Awesome, thanks both! @jbolorinos-ctr Regarding the black font; that's the same black/orange mix we used in 2018--it's a little odd looking for sure--the white/orange isn't accessible unfortunately. [15:45:55] Fundraising-Backlog, observability: Sort out fr-tech work phone situation - https://phabricator.wikimedia.org/T207511 (Jgreen) Open→Declined [15:45:57] Fundraising-Backlog, fundraising-tech-ops: Page fr-tech about spikes in frtechmail - https://phabricator.wikimedia.org/T202419 (Jgreen) [15:45:59] fundraising-tech-ops, observability, Epic: [Epic] overhaul fundraising cluster monitoring - https://phabricator.wikimedia.org/T91508 (Jgreen) [15:46:26] Fundraising-Backlog: CPS contractor contact information - https://phabricator.wikimedia.org/T204832 (Jgreen) Open→Resolved a:Jgreen [15:47:06] Fundraising-Backlog: Figure out where json_encode will screw us in php 7.1+ - https://phabricator.wikimedia.org/T199471 (Jgreen) [15:48:32] fundraising-tech-ops: Testing infrastructure for EventLogging ingress of banner impression and landing page data - https://phabricator.wikimedia.org/T195259 (Jgreen) Open→Resolved p:Triage→Medium a:Jgreen [15:48:47] fundraising-tech-ops: Testing infrastructure for EventLogging ingress of banner impression and landing page data - https://phabricator.wikimedia.org/T195259 (Jgreen) [15:49:38] fundraising-tech-ops: Testing infrastructure for EventLogging ingress of banner impression and landing page data - https://phabricator.wikimedia.org/T195259 (Jgreen) We ended up testing on the otherwise unused codfw banner logger server. [16:41:20] Wikimedia-Fundraising-Banners: [Enhancement][Dsk Sm] Update color of nag on Desktop Small to orange - https://phabricator.wikimedia.org/T257584 (Pcoombe) Open→Declined [16:41:22] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (Pcoombe) [16:49:14] Wikimedia-Fundraising-Banners: [Enhancement] Update the i color on banners to orange also - https://phabricator.wikimedia.org/T257580 (Pcoombe) Open→Resolved Thea did desktop small: https://meta.wikimedia.org/w/index.php?title=MediaWiki:Centralnotice-template-B2021_0708_enNL_dsk_p2_sm_dsn_ong&diff=20... [16:49:16] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (Pcoombe) [16:51:21] Wikimedia-Fundraising-Banners: [Enhancement] Update the i color on banners to orange also - https://phabricator.wikimedia.org/T257580 (TSkaff) Thanks @Pcoombe! [17:04:14] holy moley cstone: "The standard recipe calls for one pound of salt for every four pounds of potatoes" [17:04:41] Fundraising-Backlog: Add one additional field to silverpop export - https://phabricator.wikimedia.org/T257597 (CCogdill_WMF) p:Triage→High [17:04:48] haha [17:06:41] The grocery stores here have like prepackaged potatoes in a bag with all the salt included [17:08:04] Fundraising-Backlog: Add two additional fields to silverpop export - https://phabricator.wikimedia.org/T257597 (CCogdill_WMF) [17:09:10] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Conflicting fields in some Acoustic contact records - https://phabricator.wikimedia.org/T254304 (KHaggard) Just adding in one more cid here, pertaining to the annual fund latest_donation_date field conversation we had yesterday in the Donor Services cal... [17:11:48] dstrine: my grandma would make something similar to that chicken pot pie but i dont remember what she called it [17:12:35] cstone: it sounds like a very "old timey" sort of thing. the use of the word pie was always confusing [17:17:11] Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog: Process to transfer MG employers data file between civi1001 and frpm1001 - https://phabricator.wikimedia.org/T256825 (mepps) My understanding from recent talks is that we'd looked into git for this, but can't use it with localsettings the wa... [17:25:53] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising Sprint Grep works IRL, and 4 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (Pcoombe) @MBeat33 That's normal for Firefox.... [17:25:56] Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising-tech-ops, Patch-For-Review: map out a civicrm.log_% table data expiration/delete strategy - https://phabricator.wikimedia.org/T257232 (mepps) @Ejegg @Eileenmcnaughton should this be pending deployment? I see the patch abo... [17:28:48] (CR) Mepps: [C: +2] Ensure num_rows is defined [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610479 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [17:29:45] (Merged) jenkins-bot: Ensure num_rows is defined [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610479 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [17:32:15] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising Sprint Grep works IRL, and 4 others: Implement redirect for hide banner cookie issue - https://phabricator.wikimedia.org/T251780 (MBeat33) Great, thank you for confirming @Pe... [17:34:00] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (Pcoombe) @TSkaff Regarding the black on orange not looking good: a tip I learned from a design course was to use a darker/lighter variation of the background color instead of pure black/white. [This tool](ht... [17:43:09] (CR) Mepps: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [17:45:27] fr-tech I need to run an errand so I'll be back later on today. [17:45:30] (PS10) Mepps: Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) [17:47:05] AndyRussG did you say you were going to ping someone on traffic about the patch above? My understanding is that it's super high priority at this point [17:48:42] mepps: ah yes thanks will do! [17:49:03] Wikimedia-Fundraising-Banners: QA: Orange banners for NL - https://phabricator.wikimedia.org/T257388 (TSkaff) Great idea @Pcoombe, we'll definitely have test space for the smalls and can try this out--even if it's a wash, it would certainly look better. [17:49:14] (CR) jerkins-bot: [V: -1] Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [17:50:08] (PS11) Mepps: Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) [17:50:42] AndyRussG XenoRyet i wonder if we want to set a time limit for the traffic review, just given the urgency on the hide banners task (apparently it's affecting active campaigns) [17:51:28] mepps: the point is that is has to work [17:51:53] Fundraising-Backlog, Analytics: Dashboard for CentralNotice impression rates using Druid, centralnotice_analytics and CN events - https://phabricator.wikimedia.org/T254792 (Milimetric) Looked at this a little bit closer. I don't see any reason it can't be implemented whenever you prioritize it. One sma... [17:52:31] could we test on beta AndyRussG? would it have the same caching? i just don't know what the traffic team's bandwidth is, or maybe XenoRyet could have erika or Grant elevate it with them [17:52:42] mepps: AndyRussG: There is a fair bit of urgency on this one, it's causing donor services pain with the NL, and it is donor facing. [17:52:58] *the NL campaign [17:53:38] mepps: we need input from Traffic folks. I'm looking up who to ping now. Testing on the beta cluster I don't think gives us much. [17:53:57] (CR) jerkins-bot: [V: -1] Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [17:54:23] XenoRyet do you think you could reach out to their manager? [17:54:23] mepps: Erika and Grant won't have the detailed knowledge of the front-end cache, they'd just ask the same people we'll ask [17:54:41] AndyRussG if it goes to their manager, it might get prioritized higher [17:55:02] XenoRyet: mepps:I don't think it's at the point of scaling it up like that. Just give me a few minutes to ping on IRC please [17:55:03] but i have no idea what's going on with their team, i just don't want it to linger [17:55:13] sure AndyRussG [17:55:59] Yea, we have time to ping folks. It's not quite UBN, but it is getting close and we should keep that in mind. [17:56:16] shoot, sorry I didn't note this in scrum-o-scrums [18:00:01] (CR) Mepps: "Did something change in Phan? The failing check is coming from outside this code and is because of a PhanTypeMismatchReturn in code that h" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [18:00:13] sorry i forgot to mention it ejegg! [18:01:38] fundraising-tech-ops: Add logging and lockfiles to fr analytics scripts where needed - https://phabricator.wikimedia.org/T256924 (EYener) Added new cube (impressions_hourly) that runs once per week on Thursdays [18:01:40] ejegg AndyRussG also the patch seems to be failing CI due to a possible new Phan check for another file [18:02:05] mepps it looks like phan is being run on changed files, even if the part of the file you touched is not the part that's failing [18:02:11] mepps: ufff well that we can just override for an urgent patch [18:02:29] ejegg but it's not a file i changed.. [18:02:37] thanks AndyRussG--good to know! [18:02:50] oh weird [18:03:05] ah yeah, bannerpager [18:03:16] see my comment [18:06:19] ejegg mepps we might want to turn off Phan voting until that stabilized, no? [18:06:42] yep AndyRussG [18:07:41] mepps ejegg XenoRyet OK I just pinged everyone I could find to ping on #wikimedia-operations [18:07:53] great AndyRussG [18:07:54] https://phabricator.wikimedia.org/project/members/1201/ [18:08:23] mepps ejegg XenoRyet potentially we could still do a backport deploy (formerly SWAT deploys) this evening. Otherwise it'll go to Monday [18:10:09] though if it's just that one issue we might be able to put a quick skip comment [18:11:14] AndyRussG looks like it should be in bblack and dzahn's working hours right now [18:11:46] yeah I pinged bblack. dzahn wasn't on IRC [18:12:07] I'm sure everyone's hours are messed up because COVID [18:12:08] yeah i saw your pings, just looks like others are after hours now [18:13:18] ejegg mepps we could look for documentation on Wikitech or Mediawiki about how our varnish setup deals with special pages and cookies [18:13:25] however the actual answers about this are likely to be in the code [18:13:43] (PS1) Mepps: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 [18:13:58] which at least for me is very difficult to understand. It's a weird special language that is compiled to C and then to native executable, or something like that [18:14:11] I can try to do that a bit later, though [18:14:47] so if we deploy this and it doesn't work as expected, we could just not do the redirect...i don't think it would break exising functionallity (though tell me if it would) [18:15:06] i can see that AndyRussG--i find varnish tough to understand [18:17:04] mepps: existing functionality in CN should be unaffected by any of this [18:17:16] however I don't know how many browsers it's no longer working for [18:17:20] (CR) jerkins-bot: [V: -1] Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [18:17:36] AndyRussG so if we deploy it and it doesn't work as expected, then we can just roll back the thank you page setting [18:17:55] mepps: The latest deploy slot today is at 4 pm Pacific time, in 5 hours. https://wikitech.wikimedia.org/wiki/Deployments [18:18:05] thanks AndyRussG [18:18:26] i can't be online at that time unfortunately :(--i'm on my own with kid bedtime tonight [18:18:45] mepps: about deploying and it not working, yeah, that's an option [18:18:55] mepps: I can be online then, it's ok [18:19:02] thanks AndyRussG [18:20:00] mepps XenoRyet ejegg just got a reply from cdanis saying he's OOO but can look at it on Monday if no one else is able to before then [18:23:29] XenoRyet what are your thoughts in terms of priority? we could put this out tonight with the possibility it won't work (in which case, we just won't redirect donors there at all), or wait for review on monday [18:24:20] cccccchrbkbkrbthjtrrhjitceukgcbfhgrluibtnfbj [18:24:23] cccccchrbkbkbetnfngguhfugctrhivchciilnrebbtg [18:24:29] whoops [18:24:31] (PS2) Mepps: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 [18:24:44] haha cstone [18:27:17] cccccchrbkbkccdjckcbicrtcbubnbbuckehefrnntgu [18:27:23] cursed haha [18:27:58] (CR) jerkins-bot: [V: -1] Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [18:29:10] (PS3) Mepps: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 [18:33:15] (PS4) Mepps: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 [18:33:20] (CR) jerkins-bot: [V: -1] Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [18:33:30] oh weird, yeah, that looks like it should actually be a string [18:33:42] wonder how we're getting away with the array return [18:34:50] hmm, $out->addHTML( $pager->getNavigationBar() ) [18:35:01] and addHTML expects text [18:35:15] let's actually fix that soon [18:35:48] (PS5) Mepps: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 [18:35:50] (PS12) Mepps: Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) [18:36:54] (CR) Mepps: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [18:39:09] mepps: I got a reply from ema but it's late for him [18:39:14] ejegg should we make a task? [18:39:43] mepps sure, i'll do that [18:39:51] AndyRussG how do we feel going forward knowing we can not use the feature if it doesn't work as expected? XenoRyet said it's not UBN but it's been pretty tough on DS [18:40:10] mepps: I'd like to try to read the varnish code/doc a little later [18:40:16] by how do we feel, i meant how do you feel ;)--too much time with toddlers [18:40:23] Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising-tech-ops, Patch-For-Review: map out a civicrm.log_% table data expiration/delete strategy - https://phabricator.wikimedia.org/T257232 (Jgreen) [18:40:32] I have to eat something now then meeting at the top of the hour, then I can look at it [18:40:35] actually, the right amount of time with toddlers [18:40:45] sorry, I didn't know hor urgent this is in any case [18:41:04] i think it's been escalated recently AndyRussG because of NL [18:42:01] i can take a look too AndyRussG--i'm curious how it all works [18:43:31] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Fix return type of CNBannerPager::getNavigationBar - https://phabricator.wikimedia.org/T257610 (Ejegg) [18:43:42] ok, ^^^ [18:45:41] mepps: ok cool! thanks!! [18:45:55] Wikimedia-Fundraising-Banners: QA for en6C pre-test on July 15 - https://phabricator.wikimedia.org/T257509 (spatton) [18:46:22] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: desktop small batch - https://phabricator.wikimedia.org/T257510 (spatton) [18:51:29] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: desktop large batch - https://phabricator.wikimedia.org/T257611 (spatton) [18:51:42] mepps: Sorry just getting back to the priority question [18:51:54] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: desktop large batch - https://phabricator.wikimedia.org/T257611 (spatton) [18:52:03] So if it doesn't work, donor experience is the same as what's happening now? [18:52:25] Or does something more broken happen? [18:52:45] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: desktop large batch - https://phabricator.wikimedia.org/T257611 (spatton) [18:53:42] Actually reading backscroll myself too, looks like some of this already got talked about. [18:56:27] If we get some more information about our options, I can ping MBeat about it and see how DS is feeling either way. [18:57:02] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: mobile large batch - https://phabricator.wikimedia.org/T257612 (spatton) [18:59:05] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: mobile small batch - https://phabricator.wikimedia.org/T257613 (spatton) [18:59:15] Wikimedia-Fundraising-Banners: 2020-07-15 en6C pre-test: mobile small batch - https://phabricator.wikimedia.org/T257613 (spatton) [19:00:28] Wikimedia-Fundraising-Banners: QA for en6C pre-test on July 15 - https://phabricator.wikimedia.org/T257509 (spatton) p:Triage→High a:jbolorinos-ctr [19:02:06] XenoRyet, our options are to deploy this evening with AndyRussG and i reading through docs a little, or to wait until Monday [19:02:44] the risk of deploying this evening is that even with review, it could turn out Varnish blocks this from working as we expect [19:03:46] What's the donor experience if Varnish blocks it? [19:05:26] Basically the thing we're weighing against here is that there are current NL donors who are frustrated because they donated and are still seeing banners, and we don't really have anything to help them. [19:05:46] XenoRyet donor experience will be exactly as it is now [19:06:12] we will turn on the redirect setting, try it, and if it doesn't work, just go back to sending donors to the ty page [19:06:48] In that case, I'd lean towards pushing it out. I'll defer to you or AndyRussG if you're uncomfortable about it though. [19:09:21] thanks XenoRyet i'm leaning that way too, but AndyRussG would be the one to handle the deploy [19:13:06] ^ so far we’re not getting crushed by banner compliants, we’ve sent the NL workaround macro 12 times. I expect that complaint total will rise as the campaing continues, but we’re going to deploy Peter’s new cookie IRL into our macris (NL and everywhere) so at least the donors who reach out to us will know how to suppress, mepps XenoRyet [19:13:27] *campaign, macros typoking! [19:14:17] thanks MBeat that's helpful context! in that case, it seems okay to go with either today or monday based on current information [19:14:18] if that info is helpful in terms of deploying a patch [19:14:24] :) ty mepps [19:14:24] Ah, Peter was able to come up with a workaround? That's good to know, takes a bit of the pressure off. [19:14:32] but of course, probably preferred sooner [19:14:48] XenoRyet, Peter just has DS sending donors to the Special:HideBanner page directly [19:14:56] so they still have to reach out to DS [19:15:04] and of course there may be folks frustrated who haven't contacted DS [19:15:20] +1 no telling how many we’re irritating who don’t contact us [19:15:37] Yea, that does work for the ones who do reach out, which is some pressure off. But you're right that we do still have to think about the rest of the folks who will run into it between now and Monday. [19:16:27] I still lean towards today, but it is good to have the additional context. [19:31:00] Fundraising-Backlog, fundraising-tech-ops, FR-Tech-Analytics: investigate moving non-essential databases (faulkner, pgehres, fredge) off of fundraising database cluster - https://phabricator.wikimedia.org/T255456 (Jgreen) From today's analytics coordination meeting: The pghehres database is queried d... [19:34:20] Fundraising-Backlog, FR-Tech-Analytics, MediaWiki-extensions-CentralNotice: Banner history logger records incorrect status code following campaign fallback - https://phabricator.wikimedia.org/T255810 (EYener) [19:45:33] XenoRyet mepps sorry meeting... I'll try to get this out today [19:45:49] thanks for talking it out all out AndyRussG! [19:46:07] and in a case like this, if we can't for some reason, really a Friday deploy might be obtainable in such a case [19:46:13] mepps: thank u for highlighting the urgency! [19:46:21] and for writing the actual patch! [19:46:30] :) [19:46:38] :) AndyRussG [19:47:25] K I'm gonna get a head start on some cooking now, and that should give me a clear 2 hours this afternoon to figure it out [20:44:07] (PS6) Ejegg: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [20:44:41] (CR) Ejegg: [C: +2] "Looks like a good band-aid for now! Phab task to fix underlying issue is https://phabricator.wikimedia.org/T257610" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [20:48:31] (Merged) jenkins-bot: Suppress PhanTypeMismatchReturn for CNBannerPage::getNavigationBar() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/610892 (owner: Mepps) [20:53:30] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610923 [20:54:28] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610923 (owner: Eileen) [20:55:12] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/610923 (owner: Eileen) [20:55:20] I'm going to deploy latest changes to silverpop, ping katie, kick off silverpop daily & head to the dentist & check in a bit [20:55:59] actually can ping katie after it runs - goal is just to let her know to keep an eye on tonights & it will run again by then [20:58:00] (PS10) Jgleeson: Command to check matching gifts employer data file [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608089 (https://phabricator.wikimedia.org/T251201) [20:58:17] gl eileen [20:58:17] !log https://phabricator.wikimedia.org/T253152 [20:58:22] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:58:32] with the dentist.. and silverpop! :) [20:58:45] :-) [20:58:56] OK the job is running now [20:59:27] eileen: I'm gonna take a peek at this tomorrow https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/610664 [20:59:32] from your eod [20:59:46] Cool [21:00:08] is that the thing you want to add to? [21:00:09] yeah I want to have routines to do incremental updates & to fully rebuild if necessarry [21:00:14] cool [21:00:14] yeah [21:00:40] so this needs updating - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/609238 [21:00:50] to not review-ready [21:01:01] but, you can see it adds a file just to update the main table [21:01:12] & also one to rebuild if we really need to [21:01:33] ideally we could run the update one every hour or so [21:02:58] got it [21:03:45] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, and 3 others: Add and delete fields from the _all_Wikimedia database (civi export to ESP) - https://phabricator.wikimedia.org/T252245 (Eileenmcnaught... [22:11:01] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, and 2 others: Add silverpop privacy delete routine - https://phabricator.wikimedia.org/T257001 (Eileenmcnaughton) If someone wants to pick this up I... [22:14:29] (CR) AndyRussG: Allow redirect on Special:HideBanners on limited redirectUrl (3 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [22:14:39] I just added notes in https://phabricator.wikimedia.org/T257001 in case anyone wants to grab that task [22:23:28] ejegg: we seem to get omnimail fails at a fairly consistent time of day - maybe we should just edit the schedule? [22:23:44] or am I imagining the consistency? [22:27:06] dstrine: I'm just looking at Pats email [22:27:37] & I guess the time they want might be when our backlog grooming is on [22:27:43] & whether I should [22:27:49] 'offer' thaat [22:32:18] eileen sure, let's try changing the schedule! [22:33:25] I'm just trying to see if there really is a pattern - I feel like they happen 'overnight' for me [22:35:03] OK - I can see fails at '6.50 pm' on more than one day [22:35:40] seems to be 6.50am utc [22:38:58] dang remind me where past process-control logs are - trying to see if that co-incides with the upload job [22:46:56] (PS1) Eileen: Fix test to have modified date for all contacts [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610946 (https://phabricator.wikimedia.org/T253152) [22:49:54] eileen: if you are on civi1001: /srv/archive/civi1001/process-control [22:50:10] thanks dwisehaupt [22:50:13] I always forget [22:50:14] np [22:51:13] if you are on the central logger, the path is slightly different and the logs are named with the host they ran on. [22:51:29] useful if you want to check runs that were done on civi2001 [22:52:20] nice [22:53:11] Just thinking I might monitor fail mails on omnirecipient uploads for a few days against silverpop_daily as timings keep changing [22:53:28] so probably new data is more useful [22:54:05] (PS1) Eileen: Add rebuild file for full table rebuild [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610949 (https://phabricator.wikimedia.org/T253152) [22:55:30] i'd love for process-control to log job status and timing stuff into prometheus, but that's a long term goal. [22:57:53] (PS11) Eileen: Silverpop - build main table incrementally [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609238 (https://phabricator.wikimedia.org/T253152) [22:58:38] (CR) jerkins-bot: [V: -1] Silverpop - build main table incrementally [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609238 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [23:02:41] (PS12) Eileen: Silverpop - build main table incrementally [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/609238 (https://phabricator.wikimedia.org/T253152) [23:19:12] (PS13) Ejegg: Allow redirect on Special:HideBanners on limited redirectUrl [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/609810 (https://phabricator.wikimedia.org/T251780) (owner: Mepps) [23:24:43] AndyRussG: ok, ^^^ is the text-revised (and rebased) version [23:29:28] ejegg: ok, see operations chat with Krinkle [23:29:58] drat, i wasn't in the channel so I don't see the backscroll [23:30:00] in there now though