[02:16:10] (CR) MaxSem: [C: +2] build: Upgrade mediawiki-codesniffer to v28.0.0 [extensions/ContributionTracking] - https://gerrit.wikimedia.org/r/541955 (owner: Jforrester) [02:20:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [02:25:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [02:25:51] (Merged) jenkins-bot: build: Upgrade mediawiki-codesniffer to v28.0.0 [extensions/ContributionTracking] - https://gerrit.wikimedia.org/r/541955 (owner: Jforrester) [02:30:13] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1783 [02:35:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2083 [02:40:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [02:45:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2044 [02:50:15] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 633029 Threads: 2 Questions: 57809829 Slow queries: 22527 Opens: 6629 Flush tables: 1 Open tables: 200 Queries per second avg: 91.322 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [04:41:41] Fundraising Sprint Trojan Horse Wisperer, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: More small fixes needed for Campaign fallback - https://phabricator.wikimedia.org/T234248 (AndyRussG) Here are some suggestions on what to smoke test for any changes to the fallback loop. You could chec... [05:37:17] Wikimedia-Fundraising-CiviCRM: Issues with the Benevity Import - https://phabricator.wikimedia.org/T234370 (RLewis) @Eileenmcnaughton i'm just bumping this as the import still isn't importing the file{F30613254}. Can you take a look please and advise on the next steps? [12:53:24] Fundraising-Backlog, Core Platform Team, Editing-team, Operations, and 11 others: RFC: Serve Main Page of Wikimedia wikis from a consistent URL - https://phabricator.wikimedia.org/T120085 (phuedx) [13:27:12] fundraising-tech-ops: test database for fundraising analytics queries - https://phabricator.wikimedia.org/T235171 (Jgreen) [13:27:35] fundraising-tech-ops: test database for fundraising analytics queries - https://phabricator.wikimedia.org/T235171 (Jgreen) p:Triage→Normal [13:43:22] fundraising-tech-ops: test database for fundraising analytics queries - https://phabricator.wikimedia.org/T235171 (Jgreen) I'm going to backup and repurpose dev_analytics for this, which hasn't been touched since 2017. [14:10:58] fundraising-tech-ops: test database for fundraising analytics queries - https://phabricator.wikimedia.org/T235171 (Jgreen) Open→Resolved [14:31:58] (PS1) Cdenes: removed extra parentheses character [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/542120 [15:10:26] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues with the Benevity Import - https://phabricator.wikimedia.org/T234370 (DStrine) [15:11:01] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues with the Benevity Import - https://phabricator.wikimedia.org/T234370 (DStrine) This hadn't been on our backlog but I just added it. [15:13:28] Fundraising-Backlog: Recurring notification email text showing subsequent donation in USD when donation was submitted in GBP - https://phabricator.wikimedia.org/T235105 (Cstone) The 2nd email is sending $msg['currency'] when it should be sending $msg['original_currency'] to the template. [15:29:25] cstone: is that a quick one to push up a patch for? [15:30:34] Fundraising-Backlog, fundraising-tech-ops: Reconfigure fundraising check_endpoints - https://phabricator.wikimedia.org/T212252 (Jgreen) [15:47:34] jgleeson: yeah it would be just that one change [15:54:04] awesome [15:56:09] fr-tech, I realised this morning that vagrant snapshotting is to used with care. I lost a local database and contents after snapshotting back to an earlier version of my vm without realising this would be lost @_@ [15:56:24] ouch. [16:01:54] fr-tech did we mean to move standup a half hour later today? [16:03:45] I hope so as I'm going to pick Oscar up shortly [16:05:17] i think Dylan moved it cause there was a fr retro from 10-11 but then that got cancelled [16:05:58] Ah okay, if it works for others, it works for me! [16:16:06] (CR) AndyRussG: [C: +2] "Yay! Thanks much!!!! :)" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538967 (https://phabricator.wikimedia.org/T232859) (owner: Mepps) [16:18:59] (CR) Jgleeson: [C: +1] "nice work! this is working for me. could we chat a little about the single data cell update from two files. I am struggling to see where t" [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/541155 (https://phabricator.wikimedia.org/T234737) (owner: AndyRussG) [16:19:02] (Merged) jenkins-bot: Change name, default value, and document CentralNoticeMaxCampaignFallback [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538967 (https://phabricator.wikimedia.org/T232859) (owner: Mepps) [16:26:48] jgleeson: feel free to find a space on my calendar if you want to talk over the FRUEC patch with the multiples log files for a single aggregate data row, as per your comment... Or if you prefer, text mode is also fine, of course! [16:27:19] I just adjusted my "kid pick-up busy time" on my calendar to reflect that Thursdays they both now go to after-school art class, bt [16:27:46] thanks AndyRussG! gotta run but will check in with you when I get back. bye for now ! [16:44:21] Fundraising-Backlog, fundraising-tech-ops: Reconfigure fundraising check_endpoints - https://phabricator.wikimedia.org/T212252 (Jgreen) @ejegg this is mostly done, with some notes: I'm not sure what's up with the paypal endpoints. Neither of them connect with a browser, curl, wget, etc. The endpoints ar... [16:56:20] Fundraising-Backlog, FR-Paypal, Patch-For-Review: Don't cancel PayPal recurring donations after 3 failed attempts - https://phabricator.wikimedia.org/T235022 (EMartin) PLEASE NOTE: when we draft this email to the donor to log into paypal to update their info there, Paypal has advised the following:... [17:12:10] fundraising-tech-ops: fundraising access request for Nora Nichols - https://phabricator.wikimedia.org/T232623 (Jgreen) >>! In T232623#5553098, @NNichols wrote: > yes thank you! @NNichols Great. There's additional setup to be done for ssh access. You'll need to generate an ssh key (see https://collab.wikimed... [17:12:46] fundraising-tech-ops: Access to the ssh frdev1001 server - https://phabricator.wikimedia.org/T232401 (Jgreen) Open→Resolved p:Triage→Normal [17:13:18] fundraising-tech-ops: fundraising access request for Nora Nichols - https://phabricator.wikimedia.org/T232623 (Jgreen) p:Triage→Normal [17:46:02] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Future version of Chrome might break Special:HideBanners cookies - https://phabricator.wikimedia.org/T235204 (Pcoombe) p:Triage→High [17:58:20] Fundraising-Backlog: Smashpig failmail referencing Amazon whitelisting - https://phabricator.wikimedia.org/T235205 (mepps) [18:05:55] jgleeson: did you want to go over FRUEC stuff now or later? [18:06:01] (or tomorrow?) [18:11:24] jgleeson: wondering in part 'cause I'd like to relocate pretty soon, though also no problem to wait a bit :) [18:32:05] sorry AndyRussG, I got d/c and forgot to ssh back into my irc client [18:32:21] I can chat now if you've not already moved on? [18:32:25] or tomorrow is fine [18:33:15] jgleeson hey no worries... However I'm currently on the way home [18:33:47] Maybe you can explain here where the confusion lies?jgleeson [18:34:13] sure thing [18:34:19] Apologies if the ticket or other dog isn't clear [18:35:01] It's a problem we ran into on production [18:35:15] *doc not dog [18:35:25] so in the commit message you mention that the test data updates a single cell from two test files and I can see the updated test file in the patch although the log entries are quite long and I couldn't work out which field would be the field that matches from the other log files [18:35:30] arrrg soft keyboards [18:36:59] jgleeson ah it's be all the fields contributing to the unique key for the banner impressions table [18:39:31] just checking the db now AndyRussG [18:39:33] if you look at the old Django Banners Stats code, the sql for ingressing into that table has a similar update on duplicate key clause, that is similar to what's in the Gerrit change [18:41:24] So the original fruec code assumed that wasn't needed because it assumed the time of events in the files faithfully respected the times block indicated by the filenames [18:41:34] oooo is it this? UNIQUE KEY (timestamp, banner, campaign, project_id, language_id, country_id), [18:41:41] Yep [18:41:54] ah ok [18:42:04] So we thought there wouldn't be overlaps of timestamps among fibres [18:42:11] *files [18:42:28] but there are on production [18:43:18] so fruec does need to aggregate data from multiple files into the same row sometimes [18:43:39] To keep backward compatibility with legacy [18:43:49] AndyRussG: in the test log file as I right in assuming that the TZ on that entry should be the same as one in another log file to evoke the unique key collision ? [18:43:58] yes [18:43:59] am I right* [18:44:10] you got it [18:44:26] ok I did search for that and don't seem to find another entry with that TZ [18:44:29] let me double checl [18:44:30] You can see which one it is looking at the diff for the test data [18:44:32] check [18:44:51] and also if you run the new code on the test data [18:45:48] you'll see the new files and impressions link table with data for two files pointing to a single row of impressions data [18:46:22] hence the need for the link table, to make the many-to-many association [18:48:10] ahhh I think I see it now within _INSERT_OR_UPDATE_DATA_CELL_SQL [18:52:47] I've figured out a way to test that behaviour [18:54:30] I can import the existing test data and then capture the sql generated for your new test log file (the duplicate). I can then chop off the 'ON DUPLICATE KEY' part to simulate the failure and then let the code run to prove it works! [18:54:58] simulate in a separate mysql query outside the code [19:02:35] Fundraising-Backlog: Monthly convert: set hide cookie when modal loads - https://phabricator.wikimedia.org/T235209 (spatton) [19:05:02] jgleeson ah sure clever approach! [19:06:01] What I tried was to apply the patch's change for the test data but try to ingress it with the previous code [19:06:29] So that showed the same error I got on production [19:09:40] jgleeson also there's the code for purging incompletely processed files [19:11:39] sorry my approach proved a little fiddlier than I expected, almost done! [19:14:32] For that I you could just set the status to processing for one of the files with a duplicate timestamp [19:15:01] then run the purge incomplete command [19:15:09] I managed to trigger that scenario bombing out of a debugging session while midprocessing a file [19:15:28] conveniently by accident! :) [19:20:56] Oooh yay [19:23:54] jgleeson btw I'm not extremely confident in the SQL I wrote for the purge feature [19:25:29] I was kinda following a suggestion I saw on stack exchange but am not so sure about it [19:25:54] DELETE FROM files .. ? [19:26:02] worked ok for me just then [19:26:18] I've also the update on duplicate behaviour too [19:26:24] also confirmed* [19:28:01] I feel like it would be good to pull in the unittests ticket/epic for FRUEC to make it easier to confirm the behaviour going forward [19:28:53] I feel like the code is nice and modular so writing some basic tests should hopefully be doable [19:29:11] maybe a good ticket for some of us less familiar with FRUEC andy to take on to learn on the job? [19:29:18] AndyRussG* [19:30:14] (CR) Jgleeson: [C: +2] "This is working and looks good to me :) Thanks for the guidance on the areas I didn't understand!" [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/541155 (https://phabricator.wikimedia.org/T234737) (owner: AndyRussG) [19:31:01] jgleeson thanks! [19:31:14] np, thanks for explaining it all! [19:31:23] Yeah agreed about the unit tests [19:32:45] ok I'm gonna call it a day! have a good evening all o/ [19:32:56] The query I was unsure about was the update one for the banner impressions table for the other function [19:33:12] cya! [20:10:59] Fundraising-Backlog: Civi is not updated with actual Dlocal volume/$$ - https://phabricator.wikimedia.org/T235212 (EMartin) [20:21:11] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: New credential for Engage: Naomi K - https://phabricator.wikimedia.org/T234992 (Jgreen) >>! In T234992#5558361, @Eileenmcnaughton wrote: > @LeanneS she should have a civi email now but I assume you need @Jgreen to do the certificate & he'll need an e... [20:21:39] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: New credential for Engage: Naomi K - https://phabricator.wikimedia.org/T234992 (Jgreen) p:Triage→Normal [20:21:55] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: New credential for Engage: Naomi K - https://phabricator.wikimedia.org/T234992 (Jgreen) [20:57:13] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues with the Benevity Import - https://phabricator.wikimedia.org/T234370 (RLewis) @DStrine thanks, let me know if you need anything else from me. I think Leanne is looking to reconcile September's figures within the next day or two so sorry it might... [21:06:59] Fundraising-Backlog, FR-PayPal-ExpressCheckout: Duplicate donations caused by PayPal's “Final Approval” messaging - https://phabricator.wikimedia.org/T235220 (MBeat33) [21:30:40] Fundraising Sprint Trojan Horse Wisperer, Fundraising-Backlog, FR-PayPal-ExpressCheckout: Duplicate donations caused by PayPal's “Final Approval” messaging - https://phabricator.wikimedia.org/T235220 (DStrine) [21:49:58] Fundraising Sprint Trojan Horse Wisperer, Fundraising-Backlog, FR-PayPal-ExpressCheckout: Duplicate donations caused by PayPal's “Final Approval” messaging - https://phabricator.wikimedia.org/T235220 (XenoRyet) a:XenoRyet