[02:07:20] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=2585 [critical =2000] [02:12:20] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=3177 [critical =2000] [02:17:20] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=3854 [critical =2000] [02:22:20] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=4369 [critical =2000] [02:27:20] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=3696 [critical =2000] [02:32:18] PROBLEM - check_missing_thank_yous on frdb1002 is CRITICAL: CRITICAL missing_thank_yous=2381 [critical =2000] [02:37:20] RECOVERY - check_missing_thank_yous on frdb1002 is OK: OK missing_thank_yous=1127 [14:26:06] (PS59) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [14:26:28] Fundraising Sprint A Wrinkle in Timezones, Fundraising Sprint Heaven is a page on Wikipedia, Fundraising Sprint Visual Basic Instinct, Fundraising Sprint X-rays, and 6 others: Address final code review comments on geotargeting - https://phabricator.wikimedia.org/T250751 (mepps) I'm finding the XS... [14:27:53] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [14:30:16] (PS60) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [14:36:26] Fundraising Sprint A Wrinkle in Timezones, Fundraising Sprint Heaven is a page on Wikipedia, Fundraising Sprint Visual Basic Instinct, Fundraising Sprint X-rays, and 6 others: Address final code review comments on geotargeting - https://phabricator.wikimedia.org/T250751 (mepps) Okay! I verified t... [14:38:35] (PS61) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [14:40:26] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [15:03:02] :-) I went a bit feral on process-control, no rush to look at soon if ever! [15:04:01] awight: nice! [15:05:55] Got interested because I just did a similar subprocess-spawning-via-python thing in "quibble" and wanted to compare pipe handling in both code bases. [15:07:38] Fundraising Sprint A Wrinkle in Timezones, Fundraising Sprint Heaven is a page on Wikipedia, Fundraising Sprint Visual Basic Instinct, Fundraising Sprint X-rays, and 6 others: Address final code review comments on geotargeting - https://phabricator.wikimedia.org/T250751 (mepps) @Ejegg @AndyRussG... [15:08:01] yeah that was a snaggly issue with python iirc, were you able to make it behave better? [15:08:32] (PS62) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [15:08:41] cool awight! ejegg mentioned you'd been doing some python at work :) [15:09:10] Not much nicer, it ended up being really complicated this other way, too: https://gerrit.wikimedia.org/r/#/c/integration/quibble/+/587885/26/quibble/util.py [15:09:49] What I finally realized is that "tail -f *" is pretty much the ideal. [15:10:10] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [15:10:41] In the quibble case, the trickiness is because it merges output from parallel subprocesses, so there's no nice compromise between buffering and interleaving. [15:11:18] IMO it's great to just buffer each process, and flush all buffers periodically, prefixing with loud === proc1 === separators. [15:13:21] ah yeah, that sounds tangly [15:14:58] I do like the context managers to capture std* and logging, but since there's no functional difference, it probably doesn't make sense to port to process-control. [15:16:33] If I can emulate `tail -f` however, I'll ring this channel again. Is there a use case for tailing multiple subprocesses under one process-control parent? [15:18:41] That's a good question. I haven't really had to look at it since we fired it up (that is kind of amazing when you think back to the jenkins days). I'm not aware of any case where we're using it to run anything more than a single subprocess, but Elliott would know better. [15:19:53] great news. Let's burn this chat log ;-) [15:22:47] * awight continues silently admiring the fun sprint names [15:23:18] awight: lol [15:28:52] (PS63) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [15:33:17] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [16:05:26] (PS64) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [16:09:25] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [17:12:37] Fundraising Sprint Heaven is a page on Wikipedia, Fundraising-Backlog: switch NL credit card processing to adyen - https://phabricator.wikimedia.org/T251172 (DStrine) [17:34:56] ugh, internet completely down at home [17:35:38] and I had no cell phone credit as I haven't been using it [17:35:56] oh no ejegg! [17:36:32] (PS65) Mepps: Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [17:37:16] i got a 2GB packet on it after mooching off my wife's data to add the credit. Now just waiting for a human on the support line [17:39:37] (CR) jerkins-bot: [V: -1] Finer geo targeting [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/518449 (https://phabricator.wikimedia.org/T102848) (owner: Vedmaka Wakalaka) [18:02:23] (PS1) Mepps: Change adyen-ideal blocks to adyen-nl and add cc [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592729 (https://phabricator.wikimedia.org/T251172) [18:02:29] ejegg ^^ [18:03:40] fundraising-tech-ops: fr-log-announcer reporting logins to payments2xxx servers multiple times - https://phabricator.wikimedia.org/T251175 (Dwisehaupt) [18:04:20] (PS2) Mepps: Change adyen-ideal blocks to adyen-nl and add cc [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592729 (https://phabricator.wikimedia.org/T251172) [18:07:07] sorry to hear about your connectivity issues ejegg ! [18:07:23] is dialup still a thing in columbia? maybe use that as a last resort [18:12:09] The thing about lockdown too, you can't run down to the local coffee shop and get wifi either. We had a power outage here on Thursday and had to try to figure some of that out. [18:15:44] hah, no, we have our 'landline' via the cable provider so that's out too [18:16:09] mepps that looks good to me, but I'll wait a sec before +2ing [18:17:27] actually, gonna go eat some food quick. [18:17:27] k ejegg [18:19:05] need some neighbors to get together and deploy mesh network devices across all of the backs of the houses/apartments. as long as some are on a different upstream it might work. [18:21:45] Not a bad idea. I think all my neighbors are on the same thing here. Pretty much just comcast available in this area. [18:24:20] i know there is diversity here, but i think it's getting less due to the speedy service and cheap price of the fiber rollouts. [18:58:14] fundraising-tech-ops: update risky_package_updater to use systemctl for restarting services instead of the system V service command - https://phabricator.wikimedia.org/T251183 (Dwisehaupt) [18:59:10] (PS1) Jgleeson: Remove some leftover debug code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592742 [19:02:49] Fundraising-Backlog, Epic: Epic: matching gifts HEP integration (was employer field V2) - https://phabricator.wikimedia.org/T228765 (DStrine) [19:03:13] ejegg mepps hi! Quick question, am I correct in understanding that for the time being, in the patches as they are, there is no localization of region names, only country names (when the CLDR extension is enabled)? Or am I missing something? Thanks in advance! [19:11:45] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice Geotargeting: More code cleanup - https://phabricator.wikimedia.org/T251185 (AndyRussG) [19:16:21] that's a good question AndyRussG [19:18:14] AndyRussG: right [19:18:29] CLDR just doesn't have the region names localized [19:20:47] fundraising-tech-ops: update risky_package_updater to use systemctl for restarting services instead of the system V service command - https://phabricator.wikimedia.org/T251183 (Dwisehaupt) Pushed this small change that should do it. Will check when rolling updates on eqiad hosts soon. `[frack::puppet] 5e975... [19:22:06] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice Geotargeting: Improve regions lists - https://phabricator.wikimedia.org/T251186 (AndyRussG) [19:27:37] (CR) DannyS712: [C: +1] "LGTM" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592742 (owner: Jgleeson) [19:30:30] (CR) Jgleeson: "thanks for the review DannyS712!! :)" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592742 (owner: Jgleeson) [19:32:10] sorry fr-tech be there in a minute [19:32:23] Oh, me too. Lost track of time. [19:38:02] fundraising-tech-ops, Operations, decommission, ops-eqiad: decommission frav1001.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T222109 (Papaul) Open→Resolved Complete [19:52:15] fundraising-tech-ops, Operations, decommission, ops-eqiad: decommission bismuth.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T248516 (Papaul) ` [edit interfaces interface-range vlan-administration] - member "ge-[0-1]/0/18"; [edit interfaces interface-range disabled] member ge-0/0... [19:53:50] fundraising-tech-ops, Operations, decommission, ops-eqiad: decommission bismuth.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T248516 (Papaul) [20:11:16] Fundraising-Backlog: Ingenico/Globalcollect SSL cert failures 04-23-2020 - https://phabricator.wikimedia.org/T250950 (DStrine) [20:16:35] (CR) Ejegg: [C: +2] Change adyen-ideal blocks to adyen-nl and add cc [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592729 (https://phabricator.wikimedia.org/T251172) (owner: Mepps) [20:18:52] (Merged) jenkins-bot: Change adyen-ideal blocks to adyen-nl and add cc [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592729 (https://phabricator.wikimedia.org/T251172) (owner: Mepps) [20:28:59] fundraising-tech-ops, Operations, decommission, ops-eqiad: decommission americium.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T245038 (Papaul) ` [edit interfaces interface-range vlan-fundraising] - member "ge-[0-1]/0/10"; [edit interfaces interface-range disabled] member "ge-[0-... [20:30:33] fundraising-tech-ops, Operations, decommission, ops-eqiad: decommission americium.frack.eqiad.wmnet - https://phabricator.wikimedia.org/T245038 (Papaul) [20:44:01] (PS1) Awight: [POC] Incrementally introduce namespacing [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592759 (https://phabricator.wikimedia.org/T133969) [20:46:35] (CR) jerkins-bot: [V: -1] [POC] Incrementally introduce namespacing [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592759 (https://phabricator.wikimedia.org/T133969) (owner: Awight) [20:47:36] fr-tech that's the series I mentioned https://www.netflix.com/gb/title/81273378 [20:47:54] jgleeson: https://en.wikipedia.org/wiki/Coronavirus,_Explained ? [20:48:09] maybeeeee [20:48:39] yep I think that's the one [20:49:45] jgleeson: cool, thanks much!!!! I've added it to my list of edifying content to suggest to my kids [20:49:56] Looks like the other "Explained" documentaries are pretty cool, too! [20:50:00] ah I didn't realise JK Simmons was a narrator. The best drum teacher ever! [20:50:11] yes I saw that too AndyRussG lol [20:51:57] :) [20:52:05] (CR) Awight: [C: -1] [POC] Incrementally introduce namespacing (2 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592759 (https://phabricator.wikimedia.org/T133969) (owner: Awight) [21:07:57] (PS2) Awight: [POC] Incrementally introduce namespacing [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592759 (https://phabricator.wikimedia.org/T133969) [21:08:30] nice work awight! [21:08:34] * awight bows before the marvel of setting up local testing for DonationInterface [21:09:15] Error from line 12 of /X/DonationInterface/tests/phpunit/Adapter/Adyen/AdyenApiTest.php: Class 'DonationInterfaceApiTestCase' not found [21:09:18] meow [21:10:04] awight: you can probably get it out the box with mw-vagrant if you have the time to sit and let it build [21:13:20] jgleeson: Thanks for the confirmation, I'll study the vagrant role for clues. Also peeked at the continuous integration job for DI... [21:15:24] awight: https://github.com/wikimedia/mediawiki-extensions-DonationInterface/blob/d07c5341d8c432092f0f955acad55e5d33225312/DonationInterface.class.php#L71 [21:15:57] looks like you wrote that code back in 2015 [21:16:40] hell of a 5 years inbetween right? [21:17:24] I found the poison dart, thanks for the clue! At like 62, it bails out if that TestConfiguration.php file is mising. [21:18:33] ah [21:19:57] oops, no that just detects whether the tests/ directory has been removed, I guess. I'd better keep this to myself :-) [21:23:04] Actually got it this time: calling phpunit.php with the "tests/" directory doesn't work, instead the tests have to be auto-detected and selected using the "--group DonationInterface" parameter. [21:26:41] Fundraising Sprint Heaven is a page on Wikipedia, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Backend - Create MW API to expose employer data - https://phabricator.wikimedia.org/T251199 (jgleeson) [21:27:23] PROBLEM - check_procs on frdb1002 is CRITICAL: PROCS CRITICAL: 1067 processes [21:27:37] Fundraising Sprint Heaven is a page on Wikipedia, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Frontend - Make employer field autocomplete which integrates with new backend MW employer data API - https://phabricator.wikimedia.org/T251200 (jgleeson) [21:31:05] Fundraising-Backlog, Epic: Create job to update/generate employer dataset for usage within paymentswiki - https://phabricator.wikimedia.org/T251201 (jgleeson) [21:33:47] Fundraising-Backlog, Epic: Create job to update/generate employer dataset for usage within paymentswiki - https://phabricator.wikimedia.org/T251201 (jgleeson) Due to us needing to search this data with substring queries we probably wanna put this data into secondary location capable of supporting that fe... [21:34:07] Fundraising-Backlog, fundraising-tech-ops: Upgrade fundraising grafana to v6.6 - https://phabricator.wikimedia.org/T246119 (Dwisehaupt) Open→Resolved Everything looks good and clear. Closing. [21:34:09] Fundraising-Backlog, fundraising-tech-ops, fr-tech-ops-okr: OKR 2019-2020 Q3: Increase visibility and awareness of Fundraising system health and wellness - https://phabricator.wikimedia.org/T243110 (Dwisehaupt) [21:47:21] PROBLEM - check_procs on frdb1002 is CRITICAL: PROCS CRITICAL: 1152 processes [21:50:12] (CR) Eileen: [C: +2] "This looks like a pretty clear case of temporary code" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592742 (owner: Jgleeson) [21:50:33] (Abandoned) Eileen: Update failmail default address [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/588497 (owner: Eileen) [21:53:07] Fundraising-Backlog, Epic: Create job to update/generate employer dataset for usage within paymentswiki - https://phabricator.wikimedia.org/T251201 (Ejegg) It looks like there's a builtin php function that could help here: https://www.php.net/manual/en/function.preg-grep.php With preg_grep, we could jus... [21:57:21] PROBLEM - check_procs on frdb1002 is CRITICAL: PROCS CRITICAL: 1086 processes [22:07:21] PROBLEM - check_procs on frdb1002 is CRITICAL: PROCS CRITICAL: 1003 processes [22:09:39] (Merged) jenkins-bot: Remove some leftover debug code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592742 (owner: Jgleeson) [22:22:21] RECOVERY - check_procs on frdb1002 is OK: PROCS OK: 500 processes [22:40:44] (PS1) Awight: [POC] Browser test for fun [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592782 [22:40:58] * awight runs away [22:48:04] (CR) Awight: [C: -1] "Looks like the Jenkins job would need a slight tweak, to also run npm-test and selenium-test" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/592782 (owner: Awight) [23:40:22] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Import new donor list back into Civi - https://phabricator.wikimedia.org/T250755 (Eileenmcnaughton) @leannes I imported just the first 2 contacts in the file using the UI - civicrm/group/search?force=1&context=smog&gid=802 - the only change I made was a... [23:50:23] hmph, deadlocks