[00:47:31] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi credential for new fundraising contractor, Peter - https://phabricator.wikimedia.org/T183207#3846858 (LeanneS) [00:48:04] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Civi credential for new fundraising contractor, Peter - https://phabricator.wikimedia.org/T183207#3846868 (DStrine) [00:50:38] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Civi credential for new fundraising contractor, Peter - https://phabricator.wikimedia.org/T183207#3846858 (DStrine) If he does not have a WMF email address just yet we might not be able to start this [13:31:19] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, MW-1.31-release-notes (WMF-deploy-2018-01-02 (1.31.0-wmf.15)), Patch-For-Review: Currency selector dropdown: arrow overlaps last letter - https://phabricator.wikimedia.org/T181435#3847852 (Pcoombe) Open>Resolved [14:26:45] (PS1) Ejegg: Revert "Reset deploy branch to match production" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399180 [14:35:12] (CR) Jgleeson: [C: 2] Revert "Reset deploy branch to match production" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399180 (owner: Ejegg) [14:36:03] (Merged) jenkins-bot: Revert "Reset deploy branch to match production" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399180 (owner: Ejegg) [14:49:09] (PS1) Jgleeson: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399185 [14:53:38] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399185 (owner: Jgleeson) [14:55:48] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/399185 (owner: Jgleeson) [15:04:07] !log switched off donations queue consumer and thank you mailer service in preparation for new release [15:04:16] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:09:14] !log updated civicrm from e0ee2d189c to 9d6ba74f57 [15:09:24] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:25:57] !log switched back on donations queue consumer and thank you mailer service [15:26:14] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:28:18] hi ejegg! [15:28:46] he's just left mepps [15:29:00] ah jgleeson [15:29:11] I ate into his holiday :( [15:29:24] he was helping me release the fix to my prometheus stuff [15:29:38] oh cool [15:29:46] all live now though :) :) :) [15:29:53] first release [15:29:56] woo woo!!! [15:30:01] (working) release :) :) [15:30:43] haha [15:31:10] i'm still recovering but i'm hoping i can repair my recent release [15:31:11] * jgleeson adopts sniper-mode while watching for failmail [15:31:16] in a bit of git tangle [15:32:18] ah hopefully you will pick up. Need to look after yourself though because you definitely don't wanna be ill next week [15:32:56] yeah i think it's likely food poisoning so hoping to be recovered pretty soon [16:26:14] mepps: food poisoning is the worst :( [16:50:56] AndyRussG: any new insight on https://phabricator.wikimedia.org/T182446 [16:58:19] (PS2) Jgleeson: Updated stats-collector composer package to version 1.0.0 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/398495 [17:19:07] dstrine: no, was waiting for Seddon... I can dig tho [17:20:50] Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3848842 (jgleeson) [17:22:22] jgleeson: if you are talking about the .prom files, if one has not been updated since the last scrape yeah it will get the same file again [17:22:31] is that what you mean? [17:22:43] not quite [17:22:47] I thought that at first [17:23:00] the best example I can use it this [17:23:10] first scape, a=1, b=2, c=3 [17:23:14] grafan shows them all [17:23:31] second scrape a=1,b=2 [17:23:44] grafana shows the same result as before, although c is not present [17:23:54] c=paypal in this example [17:24:37] hmmm, i've definitely seen gaps in graphs when data goes missing [17:25:16] yeah what makes it extra tricky is I don't think it's a consist bug [17:25:23] https://grafana.wikimedia.org/dashboard/db/fundraising-overview?orgId=1&from=now-15m&to=now&panelId=15&fullscreen&refresh=1m [17:25:28] that board you linked [17:25:29] consistent [17:25:34] i see gaps where paypal is absent [17:25:45] agreed [17:25:54] also 0 != absent [17:27:12] however if you monitor the stats output in /var/spool/prometheus, which I think is feeding this dashboard, there are cases where no paypal donations are in the batch and grafana reports the new data but then adds in a paypal entry with data of the previous run with actual paypal donations [17:28:03] hmm [17:28:22] I'll edit my bug report as it's not 100% accurate stating that paypal is present for every new data point [17:29:34] Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3848882 (jgleeson) [17:32:36] cwd you can reproduce it by jumping on to civi1001, and in your terminal run `watch -n1 cat /var/spool/prometheus/donations.prom` while watching in another window https://grafana.wikimedia.org/dashboard/db/fundraising-overview?orgId=1&from=now-15m&to=now&panelId=15&fullscreen&refresh=1m and comparing the two sources [17:34:33] Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3848902 (jgleeson) [17:35:00] Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3848842 (jgleeson) [17:36:53] Fundraising Sprint Winter Wanderland, Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3848909 (jgleeson) [18:12:17] Fundraising Sprint Winter Wanderland, Fundraising-Backlog: Bugfix: Prometheus serving old data in the absence of new data, to Grafana - https://phabricator.wikimedia.org/T183275#3849031 (cwdent) @jgleeson is right, wonky behavior, looks like Prometheus doesn't deal well with a highly variant list of metr... [19:19:31] fr-tech: Hey, our cat is having some trouble all of a sudden. Looks worrying enough that I'm going to take him to the vet right now, and I might not be back for Sprint Planning. I'll set up the retro doc so it's ready, and I'll send an email for folks who aren't on here right now. [19:45:51] Fundraising-Backlog: AUD minimum donation amount error on mobile form? - https://phabricator.wikimedia.org/T183191#3849226 (MBeat33) Thanks, @Pcoombe. We've never seen any negative feedback about non-whole minimum amounts from donors, but rounding up (if I understand the cause of the screenshot above) seems... [20:22:37] (PS1) Eileen: Extract function to get Importer class (to re-use from another place) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/399251 (https://phabricator.wikimedia.org/T182808) [20:22:37] (PS1) Eileen: Allow Benevity currencies [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/399252 (https://phabricator.wikimedia.org/T182808) [20:23:39] fr-tech do we have a place where we define a definitive list of gateways within the code base? [20:23:53] Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18, Epic: EPIC: Ingenico reintegration 1 hour test - https://phabricator.wikimedia.org/T183289#3849321 (DStrine) [20:24:05] Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18: reassess tasks leading up to 1 hour test in ingenico reintegration - https://phabricator.wikimedia.org/T183290#3849330 (DStrine) [20:24:27] Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18: reassess tasks leading up to 1 hour test in ingenico reintegration - https://phabricator.wikimedia.org/T183290#3849340 (DStrine) [20:27:56] (PS9) Jgleeson: DonationStats now tracking average processing times by second [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/392425 [20:56:09] Fundraising-Backlog: Document severity of the various forms of outage - https://phabricator.wikimedia.org/T183293#3849435 (Eileenmcnaughton) [21:00:35] Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18: reassess tasks leading up to internal test in ingenico reintegration - https://phabricator.wikimedia.org/T183290#3849457 (DStrine) [21:01:06] Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18, Epic: EPIC: Ingenico reintegration internal test - https://phabricator.wikimedia.org/T183289#3849458 (DStrine) [21:17:29] Fundraising Sprint Winter Wanderland, Fundraising-Backlog, FR-Ingenico, Fr-Ingenico-integration_2017-18: reassess tasks leading up to internal test in ingenico reintegration - https://phabricator.wikimedia.org/T183290#3849514 (DStrine) [21:35:46] (PS3) Jgleeson: Updated stats-collector composer package to version 1.0.0 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/398495 [21:37:43] Fundraising Sprint Uptight Piano, Fundraising Sprint Vaporwerewolf, Fundraising Sprint Winter Wanderland, Fundraising-Backlog: Track rate of new donations added to civicrm by the donations queue consumer, in prometheus - https://phabricator.wikimedia.org/T179214#3849539 (jgleeson) I'm now thinkin... [22:01:55] signing off for tonight fr-tech, catch you all tomorrow o/ [22:20:02] (PS2) Eileen: [wip]Allow Benevity currencies [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/399252 (https://phabricator.wikimedia.org/T182808)