[00:02:11] dstrine: the answer I'm getting in #wikimedia-operations about when the change will be made on production is that it's not known, has to go through various stages of tests, but at this point we may be blocking everyone [00:02:57] Basically everything has to pass on Jenkins (where it's just been enabled), then on the beta cluster, then on a little test corner of the main cluster, then live everywhere [00:03:04] So they're rolling it out quite carefully [00:03:17] But we shouldn't hold all that up too much [00:04:03] why is it suddenly a problem? [00:04:32] they're about to turn on stricter rules [00:06:04] dstrine: it's basically a cleanup that's happening throughout the whole codebase that runs on the main cluster. It turns on a stricter checking for something-or-other MariaDB-SQL-query-related [00:06:16] can anyone say how much of an interruption this will be? Is it a quick fix or will it distract anyone? [00:06:18] By changing this config we'll catch more better errors faster [00:06:42] dstrine: I think it'd take about 1 hr to get a better answer to your question [00:07:04] All we know now is that our tests are not passing with the new rule enabled [00:08:06] ok... make the task. [00:08:32] at some point I'd like to know if it's interrupting any of our planned work [00:08:44] wow, refund import is sloooowww [00:08:55] here you can see what our tests are saying: https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm/23195/consoleFull [00:09:39] but I don't know if all the tests fail due to just 1 or 2 bad queries, or if there are other queries that will die but that aren't covered by tests [00:10:48] fr-tech CN deploy is all goodness gooey goo [00:11:17] I'm going to sign off for the day. Let's check in on this new random task tomorrow [00:11:19] Sorry don't let me say that ever again [00:11:25] dstrine: K see ya! [00:12:36] AndyRussG: if you are cool with the ui I made for the banner sequence then that's all I want to go over [00:12:59] * dstrine runs away [00:21:40] srsly, do I have some O(n^n) loop in there? it's doing like one per minute [00:24:54] ok, four per minute. gonna need to restart with nohup [00:26:46] * ejegg greps refund code for 'sleep' [00:27:08] sleefund [00:30:24] k, it's nothing in the drush batch import code, queue consumer is similarly slow [00:35:22] I think I heard about a work slowdown protest the CPUs were planning today. I think they want to unionize [00:35:39] enwiki was also kinda sluggish a little while ago [00:35:55] jk [00:36:09] I'm all for the proletariat, but I want my data now! [00:42:17] ejegg: can i axe a question? [00:42:24] sure [00:42:52] if this paypal consumer just stores the unedited message as step 1, should it go to the jobs queue? [00:43:11] that's what I was thinking [00:43:13] sorry, paypal listener i mean [00:43:42] ok, and there's already a JobQueueConsumer in there [00:44:08] so the idea would be that i make a job with an execute function that stores the message in one of the 3 other queues? [00:44:24] yeah, we are currently running things off the Stomp job queue, but we're going to switch to that one [00:44:32] yep, exactly! [00:44:55] Feel free to create a jobs-paypal Redis queue [00:45:30] would the same jobs consumer eat that one? [00:45:49] We've only been using jobs for adyen so far, so the generic stomp jobs queue is being consumed by a maintenance script that specifies the adyen config node [00:46:54] I guess we'll need a different consumer running for each processor, unless we start having the job queue consumer start flipping between configs depending on the job's gateway attribute [00:47:08] which... is maybe the thing to do, actually! [00:47:28] would it be overload to have them all use the same queue? [00:47:56] cwd I kind of like the granularity of a different queue and consumer for each gateway [00:48:21] in case one of them is going terribly wrong, we don't have to interfere with the rest [00:49:10] but all the consumers will be the same program? [00:51:32] but running with different configs [00:51:45] yah, exactly [00:52:10] they'll all just pull jobs off the queue, call execute, and drop em in the damaged bin if they fail [00:56:40] ok sounds good, i think i know where to go from here, thanks! [00:56:51] rockin [01:19:54] k, think I'm gonna bounce. Have a good evening fr-tech! [01:20:19] ejegg|away: cya! [01:24:57] see ya! [01:54:59] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, and 4 others: CN: Stop using the geoiplookup HTTPS service (always use the Cookie) - https://phabricator.wikimedia.org/T143271#2621905 (BBlack) Open... [01:55:03] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: [EPIC] GeoIP lookup for IPv6 connections delays banner loading - https://phabricator.wikimedia.org/T121925#2621910 (BBlack) [01:55:05] Fundraising-Backlog, Epic, Tracking: [EPIC] Fix and improve geolocation, and ensure it's working properly for FR campaigns - https://phabricator.wikimedia.org/T121937#2621909 (BBlack) [01:56:21] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: [EPIC] GeoIP lookup for IPv6 connections delays banner loading - https://phabricator.wikimedia.org/T121925#1891807 (BBlack) [01:56:24] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Fix GeoIP cookie synthesis or loading to prevent repeated calls to geoiplookup for IPv6 users who can't be geolocated by their address on that protocol - https://phabricator.wikimedia.org/T122092#2621911 (BBlack) Open>Resolved... [01:57:04] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: [EPIC] GeoIP lookup for IPv6 connections delays banner loading - https://phabricator.wikimedia.org/T121925#1891807 (BBlack) Open>Resolved a:BBlack Fixed incidentally with the work in T100902 [03:41:51] (PS2) Awight: Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 [03:41:53] (PS2) Awight: Static interface to Configuration [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 [03:42:41] (CR) jenkins-bot: [V: -1] Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 (owner: Awight) [03:42:44] (CR) jenkins-bot: [V: -1] Static interface to Configuration [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 (owner: Awight) [03:43:13] (CR) Awight: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309190 (owner: Awight) [03:46:22] (PS3) Awight: Reuse transactionFinalized logic [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309190 [03:46:24] (PS1) Awight: Update composer libs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309506 [03:49:25] (CR) jenkins-bot: [V: -1] Reuse transactionFinalized logic [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309190 (owner: Awight) [04:05:48] (PS3) Awight: Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 [04:05:50] (PS3) Awight: Static interface to Configuration [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 [04:10:24] (CR) Awight: [C: -1] "I don't like the mush of responsibilities here. TODO: move all payments_initial manipulation into SmashPig before rewriting this patch." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309190 (owner: Awight) [04:17:47] (PS8) Awight: Don't be nice about missing contribution_tracking_id [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304116 [04:19:21] (CR) jenkins-bot: [V: -1] Don't be nice about missing contribution_tracking_id [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304116 (owner: Awight) [04:19:34] (CR) Awight: "@ejegg: Remind me whether there are blockers?" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306310 (https://phabricator.wikimedia.org/T141487) (owner: Awight) [05:26:05] (PS10) Awight: Don't use frontend classes from fraud filters [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306853 (https://phabricator.wikimedia.org/T141487) [05:26:07] (PS13) Awight: Be less magical about unstaging order status things [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306852 (https://phabricator.wikimedia.org/T141487) [05:26:09] (PS1) Awight: Update composer libs [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/309516 [05:27:59] (CR) jenkins-bot: [V: -1] Don't use frontend classes from fraud filters [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306853 (https://phabricator.wikimedia.org/T141487) (owner: Awight) [05:28:39] (CR) jenkins-bot: [V: -1] Be less magical about unstaging order status things [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306852 (https://phabricator.wikimedia.org/T141487) (owner: Awight) [05:54:08] (PS11) Awight: Don't use frontend classes from fraud filters [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306853 (https://phabricator.wikimedia.org/T141487) [05:54:10] (PS14) Awight: Be less magical about unstaging order status things [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/306852 (https://phabricator.wikimedia.org/T141487) [06:26:06] (PS2) Awight: Flesh out required modules for all projects in this repo [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/309080 [06:49:50] (PS3) Awight: Make tests real [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306599 (https://phabricator.wikimedia.org/T141654) [06:50:38] (CR) jenkins-bot: [V: -1] Make tests real [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306599 (https://phabricator.wikimedia.org/T141654) (owner: Awight) [06:51:24] (CR) Awight: "@cwd: ready for more squashing..." [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306599 (https://phabricator.wikimedia.org/T141654) (owner: Awight) [06:55:08] (PS2) Awight: [WIP] Payments API for PayPal [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306601 [06:55:57] (CR) jenkins-bot: [V: -1] [WIP] Payments API for PayPal [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306601 (owner: Awight) [06:56:31] (PS2) Awight: [WIP] Test unhappy validation path [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306603 (https://phabricator.wikimedia.org/T141654) [06:57:19] (CR) jenkins-bot: [V: -1] [WIP] Test unhappy validation path [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306603 (https://phabricator.wikimedia.org/T141654) (owner: Awight) [07:13:57] (PS1) Awight: Make constructor parameters optional [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/309523 [07:15:03] (Abandoned) Awight: Go carets on the required versions [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/301398 (owner: Awight) [07:19:32] (PS6) Awight: [Cleanup] Fix annoying punctuation and unnecessary strict equality [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/293688 [07:21:00] (CR) jenkins-bot: [V: -1] [Cleanup] Fix annoying punctuation and unnecessary strict equality [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/293688 (owner: Awight) [14:03:08] fundraising-tech-ops, Operations, ops-eqiad: rack/setup beryllium replacement frauth1001 - https://phabricator.wikimedia.org/T143902#2623039 (Cmjohnson) @Jgreen frauth1001 is cabled to pfw1 2/0/9 [15:10:25] fundraising-tech-ops, Operations, ops-eqiad: Rack/Setup pay-lvs1003 and pay-lvs1004 - https://phabricator.wikimedia.org/T143900#2623158 (Jgreen) [15:11:02] fundraising-tech-ops, Operations, ops-eqiad: Rack/Setup pay-lvs1003 and pay-lvs1004 - https://phabricator.wikimedia.org/T143900#2582571 (Jgreen) [15:57:27] Fundraising-Backlog, FR-Smashpig: SmashPig config should be more namespaced - https://phabricator.wikimedia.org/T145205#2623291 (cwdent) [16:35:26] (PS1) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 [18:07:56] fundraising-tech-ops, Operations, ops-eqiad: Rack/Setup pay-lvs1003 and pay-lvs1004 - https://phabricator.wikimedia.org/T143900#2623826 (Jgreen) [18:08:23] fundraising-tech-ops: replace beryllium with new hardware running jessie - https://phabricator.wikimedia.org/T145101#2623830 (Jgreen) p:High>Normal [18:09:39] (PS13) Cdentinger: WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) [18:58:17] (PS2) Ejegg: Defaults for more queues [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/309181 [18:58:25] (CR) jenkins-bot: [V: -1] Defaults for more queues [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/309181 (owner: Ejegg) [19:04:10] (PS3) Ejegg: Defaults for more queues [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/309181 [19:26:11] (CR) Ejegg: "This is really coming together! Sorry we have to temporarily disfigure it with a StoredOpaqueyadayada subclass to mirror to stomp till the" (9 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) (owner: Cdentinger) [19:45:23] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, FR-ActiveMQ: Move recurring queue consumer off ActiveMQ - https://phabricator.wikimedia.org/T145233#2624206 (Ejegg) p:Triage>High [19:47:07] (PS1) Ejegg: Move recurring queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309618 (https://phabricator.wikimedia.org/T145233) [19:48:32] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, FR-ActiveMQ: Migrate refunds to new queue - https://phabricator.wikimedia.org/T145234#2624214 (Ejegg) [19:48:42] ejegg: meeting? [19:48:49] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, FR-ActiveMQ: Migrate refunds to new queue - https://phabricator.wikimedia.org/T145234#2624227 (Ejegg) p:Triage>High [20:39:45] ejegg: the crm consumers expect those KeyedOpaqueEtcetc ? [20:42:00] oh i get it. what would the execute function on a completed message do? [20:47:53] AndyRussG: banner UI spec is updated [20:50:53] cwd the completed message doesn't have any execute function [20:51:08] it's just that the stomp data store expects a keyedopaquestorable.... [20:51:24] and my thought was to avoid touching that codepath at all [20:52:18] dstrine: cool thx!! [20:52:44] i wasn't thinking we'd need to write new listeners with the transitional multiQueueWriter [20:52:49] yeah that makes sense. so any old thing that extends it will do? can i just reuse my job class? [20:53:29] I'd just make an empty concrete Paypal\Message class [20:53:34] and assign fields dynamically [20:54:09] or... you could use the existing DonationInterfaceMessage for the completed queue [20:54:52] although at this point it has nothing to do with donation interface really [20:55:06] yah, needs rename [21:07:52] ejegg: once we shut off active mq we won't have to use keyedopaque stuff anymore? [21:13:24] cwd right [21:13:43] so we'll do a round of cleanup on the code [21:14:28] Again, I just wanted the first round of changes be as limited as possible, just to get us off the old infrastructure [21:15:14] would suck to run out of time before the cleanup round, but it would be better than having activemq still on for big english [21:26:14] yeah, for sure [21:26:18] i think we're on the right track [21:48:41] (PS14) Cdentinger: WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) [21:50:34] (PS15) Cdentinger: WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) [21:55:34] (PS16) Cdentinger: WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) [22:17:38] (CR) Ejegg: WIP smashpig paypal listener (4 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) (owner: Cdentinger) [22:19:25] ejegg: thank you for the excellent review! [22:19:54] feel like i've made a lot of progress understanding smash pig in a short time [22:20:31] nice! [22:21:40] I'm excited to get working on some of your suggestions for improvement once this project is done [22:26:05] me too! [22:27:04] man there have been these orb weaver spiders growing huge around here lately, but in the last couple days praying mantises showed up and are chowing down on them [22:27:14] some giant bug action [22:27:56] Fundraising-Backlog, FR-Amazon: Allow Amazon users who have selected to stay logged in to bypass login on a second donation - https://phabricator.wikimedia.org/T145251#2624688 (DStrine) [22:31:56] ooh, orb weavers. Sounds like they'd make cool body-snatchers style pods [22:32:19] back in a few... [22:44:26] goin to catch a movie, have a good weekend! [23:07:31] I'm heading out in a few minutes as well. [23:07:39] not work related: http://www.startrek.com/article/tng-bluetooth-communicator-badge-available-now [23:07:49] * dstrine has already placed his order [23:19:51] (PS2) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 [23:21:26] (CR) jenkins-bot: [V: -1] Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (owner: XenoRyet) [23:23:21] (PS3) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) [23:25:09] (CR) Ejegg: "Cool! Looks like this might only need a couple names fixed." (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [23:25:19] (CR) jenkins-bot: [V: -1] Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [23:25:40] looks like my copypasta killed me. [23:29:58] heh, i love the stuff, but it can be treacherous! [23:30:50] I was just jumping around all over the place, forgetting to make sure I actually changed the parts I was using. [23:31:34] I changed that namespace in PS2, but it looks like Jenkins still doesn't like it. You know what I'm doing wrong there? [23:35:21] (PS4) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) [23:37:22] (CR) jenkins-bot: [V: -1] Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [23:39:36] (PS5) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) [23:43:36] (CR) jenkins-bot: [V: -1] Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [23:45:14] ejegg: What command do you use for running the Civi tests? [23:45:35] (CR) Ejegg: "Got to work around my dumb constructor arg order. Also some of the test stuff looks fishy." (4 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [23:45:57] XenoRyet: you're on an OS version with php7, right? [23:46:15] I think there's some stuff crm still doesn't like with hhvm [23:46:35] oh wait, you'd be running these in vagrant, right? [23:46:43] Yea, in vagrant [23:47:17] then I think you can just run phpunit --group Queue2Civicrm [23:47:21] from the crm root dir [23:47:50] or if that defaults to hhvm, php5 /usr/bin/phpunit --group Queue2Civicrm [23:55:01] grr, i really don't want to install 350MB of KDE libraries to see my profiling results [23:55:20] That does seem excessive [23:57:21] (PS6) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) [23:58:00] Something's funky with running the tests for me. Complaining about missing a class in the Antifraud tests, but I'm outta time for the day. [23:58:14] Just throw that latest version at Jenkins and pick it up again on monday. [23:59:40] XenoRyet: maybe has to do with enabled modules? [23:59:55] Yea, maybe. [23:59:58] oh, or just go into the drupal dir and do drush cc all