[00:00:29] I think I did that in the merge [00:00:51] oops, didn't git review the last version [00:00:55] sure, go ahead [00:03:14] or i can [00:03:29] i still see my last bad commit in vendor [00:03:33] on deployment [00:03:54] did i abandon that revert? [00:04:00] yeah, my git review timed out [00:04:07] just trying to send it again [00:04:13] ah word [00:05:34] ah, cmon, what's the holdup gerrit? [00:05:50] (PS1) Ejegg: Update vendor [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/329528 [00:05:58] k, ^^^ should do it [00:07:23] cwd [00:07:53] (CR) Cdentinger: [V: 2 C: 2] Revert "update phpmailer for CVE" [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/329526 (owner: Ejegg) [00:08:45] (CR) Cdentinger: [V: 2 C: 2] Update (only) phpmailer [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/329527 (owner: Ejegg) [00:09:02] (CR) Cdentinger: [C: 2] Update vendor [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/329528 (owner: Ejegg) [00:09:18] ok that's looking good [00:09:45] (Merged) jenkins-bot: Update vendor [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/329528 (owner: Ejegg) [00:11:47] ejegg: want i should deploy sp? [00:12:35] (PS1) Ejegg: DO NOT MERGE: Update payments_initial when donation completes [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/329529 [00:14:26] cwd please do! [00:17:47] damn gerrit is slow indeed [00:18:53] cwd oho, I see the issue with php-queue [00:18:57] !log updated smash-pig from c3eaadb737a02db27f3e97a8e60981c7182e877c to 3da597fdfc9718d10fae9c8e94825729a561d890 [00:19:00] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:19:00] it [00:19:13] 'll throw an exception every time it empties a queue [00:19:21] ha [00:19:23] so, noisy, but not actually messing up the data [00:19:26] yeah [00:19:41] exception when nothing is available? [00:20:15] when the data popped off json_decodes to null, it thinks that meant a JSON failure [00:20:32] but with an empty queue, the data was null to begin with [00:20:57] isn't that the same function that does not error from json syntax errors? [00:21:28] right, so awight wrapped it to throw an exception [00:21:38] ah ha [00:21:47] we should just check for null before we feed it to that thing [00:21:55] yeah [00:23:25] freenode staff huh [00:23:53] (PS1) Ejegg: Decoding null to null is not an error [wikimedia/fundraising/php-queue] - https://gerrit.wikimedia.org/r/329531 [00:24:06] cwd ^^^ [00:24:17] for when we do want to update the rest of the libs [00:25:01] i wonder how come that exception didn't bubble up to jenkins? [00:26:26] yeah, that's odd [00:27:04] ejegg: feel like updating packagist again so we can fix DI? [00:27:22] then i think we should be able to turn payments back on [00:27:53] cwd not sure if we actually need to touch packagist for that [00:28:28] when i update it still gets https://github.com/wikimedia/wikimedia-fundraising-SmashPig/commit/63d30e8c6dfaee0b38db746a21a993c282424ec8 [00:29:43] huh, I didn't think packagist pulled in .lock files from libs [00:30:18] (PS1) Ejegg: Revert "update composer.lock" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329532 [00:30:20] (PS1) Ejegg: Update (only) phpmailer in composer.lock [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329533 [00:31:17] cwd we can just update the dependencies of a dependency if we want ^^^ [00:33:05] works for me [00:33:16] (CR) AndyRussG: [C: 1] "LGTM...!! A nitpick: I'm not sure that any calling code should really send null for json decoding... However, I imagine that check should " [wikimedia/fundraising/php-queue] - https://gerrit.wikimedia.org/r/329531 (owner: Ejegg) [00:33:43] * AndyRussG spins wheels distractingly in invisible mud [00:34:40] (CR) Cdentinger: [C: 2] Revert "update composer.lock" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329532 (owner: Ejegg) [00:35:21] ejegg: at this point that change would still get reverted by composer update right? [00:35:28] (CR) Cdentinger: [C: 2] Update (only) phpmailer in composer.lock [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329533 (owner: Ejegg) [00:36:32] (Merged) jenkins-bot: Revert "update composer.lock" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329532 (owner: Ejegg) [00:36:50] cwd nope, composer update just looks at composer.json for the second level dependencies [00:37:24] (Merged) jenkins-bot: Update (only) phpmailer in composer.lock [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/329533 (owner: Ejegg) [00:37:34] ugh, this is a really annoying mismatch in deployment though [00:37:47] but since that dep is inherited from SP and it's looking at old SP? [00:39:51] cwd it doesn't bring in the composer.lock files from libraries [00:39:57] just composer.json [00:40:35] man that is confusing [00:41:18] ejegg: should i merge master or cherry pick those last two? [00:41:28] nothing in the merge looks dangerous [00:42:42] I'm trying the merge [00:42:48] cool [00:44:51] (PS1) Ejegg: Update phpmailer (and restore smash-pig) [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/329534 [00:45:22] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/329535 [00:45:46] cwd there was a SmashPig version mismatch between vendor and composer.lock [00:46:14] damn [00:46:14] doesn't look like the change affected anything used in DI though [00:46:16] did i do that? [00:46:24] I dunno when it happened [00:46:38] but that merge and the vendor patch should get them in sync again [00:46:40] like someone went into vendor and git pull'd ? [00:46:45] and deployed it? [00:46:52] err, I dunno [00:47:14] like someone merged a composer.lock change and did a composer update x/y instead of composer install [00:47:45] wouldn't that update composer.lock again? [00:48:45] yeah, guess they threw that change out later or something [00:48:58] ah yeah [00:49:04] so complicated [00:49:09] oh, or just merged a composer.lock change and didn't update vendor [00:49:20] oh yeah or that [00:49:36] i can't help but think a bunch of submodules might be easier than using composer [00:49:49] hey, I should go out to dinner, but I'll have my phone on me [00:49:54] there is too much overlap between what composer handles and what git handles [00:50:00] ok i'm happy to deploy this [00:50:04] thanks! [00:50:13] no prob! thanks again, have a good night [00:50:20] you too [00:52:04] (CR) Cdentinger: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/329535 (owner: Ejegg) [00:53:04] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/329535 (owner: Ejegg) [00:55:29] (CR) Cdentinger: [V: 2 C: 2] Update phpmailer (and restore smash-pig) [extensions/DonationInterface/vendor] - https://gerrit.wikimedia.org/r/329534 (owner: Ejegg) [00:59:04] (PS1) Cdentinger: update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/329536 [01:01:20] (CR) Cdentinger: [C: 2] update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/329536 (owner: Cdentinger) [01:07:42] (Merged) jenkins-bot: update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/329536 (owner: Cdentinger) [01:10:00] !log updated payments-wiki from 5b4ced90352ec2c51186f93946840af24eaef290 to 21ea9bc4a46c96f84685b33ef06fd1b4c40e797e [01:10:02] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:30:12] cwd: woot? [01:30:41] AndyRussG: cautiously optimistic [01:31:40] cwd: woot woot! [01:31:44] Can I test anything? [01:31:50] Maybe peek at logs? [01:32:01] I could watch some logs and ping if something looks amiss? [01:34:45] AndyRussG: i have been watching them, donations coming in, no errors so far [01:34:58] there are still a lot of jobs turned off for reasons i'm not sure of [01:35:06] guessing it has to do with the jenkins meltdown yesterday [01:35:36] Hmmm right [01:35:41] But that's not user-facing, no? [01:35:49] Supposedly the queue can handle that, right? [01:36:17] AndyRussG: yeah not especially user facing, i turned the thank you mail back on [01:36:35] the others are mostly stuff like downloading and processing audit files [01:37:04] right cool [01:37:59] Yeah not a huge number of donations coming in, maybe 5-6/minute, via e-mail and donate wiki [01:38:31] yup [01:38:49] sure feel terrible about all those people that clicked on the email link and got service unavailable today [01:44:27] Eh well unexpected outages do happen... [01:44:43] I wish I could have been of more help! [01:45:09] I guess at least I got a sense of... stuff we could improve on, as well as better document? [01:46:24] oh yeah! [01:46:42] i mostly just thrashed around all day myself [01:46:52] reverted most of the commits i pushed [01:47:21] the thing i'd like to do most is re-evaluate our composer usage [01:47:39] hmmm right [01:48:11] That using master SP in vendor but a different branch in the standalone deploy at least needs a reason to be and documentation [01:48:43] i think we should just use submodules, at least for our own projects [01:49:11] i used to fear submodules but this job has forced me to critically reason about them [01:49:50] hmmm [01:50:13] The sense I got also when I was trying (mostly unsuccessfully) to document the queue stuff was that the different codebases could be better organized and partitioned [01:50:36] Could be wrong about that... [01:51:06] yeah, for instance the hack adam put in to make smashpig short circuit on startup only applied to when it is used as an http end point [01:51:23] but it has a whole grab bag of other uses, mostly as various job runners [01:51:45] mixed concerns for days [01:52:28] hmmm yeah [01:52:41] and the thing about it running different code when run as an application vs a dependency [01:52:52] that is due to the "dev-master" feature of composer [01:53:00] hmm [01:53:20] My sense is that if that were cleaned up and stuff were documented it'd be easier to think about onboarding new folks [01:53:24] if it was a submodule we'd just use the deployment branch everywhere [01:53:28] But it's easy for me to armchair criticize [01:53:44] Ah hmmm [01:53:46] heh sure, but you're right [01:54:16] What's the case for using composer? Just basically to treat all dependencies the same? [01:54:36] i think it's just convention [01:55:13] and i suppose to be able to use all the stuff on packagist [01:55:26] though i think it probably all happens on github too by now [01:59:52] anyway that's enough for today, have a good one! [02:00:59] cwd|afk: cya! congrats on fixing all this!!!! :) [02:36:59] https://github.com/PHPMailer/PHPMailer/pull/927#issuecomment-269447237 [02:37:02] sigh [02:37:12] https://github.com/PHPMailer/PHPMailer/issues/928 [02:37:14] more sigh [02:37:45] cba with upstreams like that [02:37:46] X| [02:38:47] Reedy: thanks for the link, this one has been a blast [02:43:09] nothing like not using branches properly [03:20:31] (Abandoned) Awight: Update libs [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/310920 (owner: Awight) [03:23:15] (PS1) Awight: Update PHPMailer [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329541 (https://phabricator.wikimedia.org/T154209) [03:23:30] (PS1) Awight: Update PHPMailer [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/329542 (https://phabricator.wikimedia.org/T154209) [03:24:26] o/ [03:25:04] holla \o [03:26:55] (CR) jerkins-bot: [V: -1] Update PHPMailer [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329541 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [03:27:51] (PS1) Awight: Short array syntax tweaks. [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329543 (https://phabricator.wikimedia.org/T154209) [03:29:13] (PS2) Awight: Update PHPMailer [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/329542 (https://phabricator.wikimedia.org/T154209) [03:29:33] cwd: if you wish to peruse ^ [03:31:03] (CR) Awight: [V: 2 C: 2] "Force merge cos sad." [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329541 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [03:31:20] (CR) jerkins-bot: [V: -1] Short array syntax tweaks. [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329543 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [03:31:20] awight: yep looks reasonable [03:31:38] but...is it on top of my broken patch that added dev deps? [03:32:13] oh? [03:32:49] seems not. CAn you link me the patch you're refererring to? [03:32:52] referring [03:33:37] awight: https://gerrit.wikimedia.org/r/#/c/329497/ [03:33:51] aha, interesting [03:35:12] I don't think you added dev deps, actually! [03:35:19] just updates that we don't want [03:35:41] weird though, vendor#master doesn't show that? [03:35:57] oh there's no vendor update [03:36:08] ok I think I need to surger [03:36:09] ya, was never massaged into deployment [03:36:32] ooh good ok than it's sort of correct and I can ignore. [03:37:57] (CR) Awight: [V: 2 C: 2] Short array syntax tweaks. [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/329543 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [03:38:17] (CR) Awight: [C: 2] Update PHPMailer [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/329542 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [03:41:12] (Merged) jenkins-bot: Update PHPMailer [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/329542 (https://phabricator.wikimedia.org/T154209) (owner: Awight) [05:22:53] !log updated civicrm from 038e166269667e7b1c0e9ef54b06ae79b546c76e to f78c894ba6686f0b512e8120eb75e928ef8c92fe [05:22:55] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [05:30:08] Fundraising-Backlog, MediaWiki-Vagrant: Replace upstart with systemd unit in crm::dash - https://phabricator.wikimedia.org/T154264#2905964 (bd808) Patches should be submitted to the `jessie-migration` gerrit branch. [05:31:55] cwd: o/ :) [05:35:29] AndyRussG: \o [05:35:35] that should be the last of it [05:35:41] and now...bed time [18:00:32] fr-tech: Meantime, in the slums below Ronnie's Ranch, Cynthia feels as if some one [18:00:33] has made voodoo boxen of her and her favorite backplanes. On this fine [18:00:33] moonlit night, some horrible persona has been jabbing away at, dragging [18:00:33] magnets over, and surging these voodoo boxen. Fortunately, they seem to [18:00:33] have gotten a bit bored and fallen asleep, for it looks like Cynthia may [18:00:33] get to go home. However, she has made note to quickly put together a totem [18:00:33] of sweaty, sordid static straps, random bits of wire, flecks of once meaningful [18:00:33] oxide, bus grant cards, gummy worms, and some bits of old pdp backplane to [18:00:34] hang above the machine room. This totem must be blessed by the old and wise [18:00:34] venerable god of unibus at once, before the idolatization of vme, q and pc [18:00:34] bus drive him to bitter revenge. Alas, if this fails, and the voodoo boxen [18:00:35] aren't destroyed, there may be more than worms in the apple. Next, the [18:00:35] arrival of voodoo optico transmitigational magneto killer paramecium, capable [18:00:36] of teleporting from cable to cable, screen to screen, ear to ear and hoof [21:19:31] (PS1) Eileen: Add support for importing campaign source, medium for CoinBase. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/329642 (https://phabricator.wikimedia.org/T153791) [21:22:18] (CR) jerkins-bot: [V: -1] Add support for importing campaign source, medium for CoinBase. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/329642 (https://phabricator.wikimedia.org/T153791) (owner: Eileen) [22:11:41] (PS2) Eileen: Add support for importing campaign source, medium for CoinBase. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/329642 (https://phabricator.wikimedia.org/T153791) [22:14:24] (CR) jerkins-bot: [V: -1] Add support for importing campaign source, medium for CoinBase. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/329642 (https://phabricator.wikimedia.org/T153791) (owner: Eileen)