[00:00:03] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Impressions slow to update on jaJP dsk campaign - https://phabricator.wikimedia.org/T145446#2631405 (AndyRussG) [00:00:28] this is pedantic, but what's the advantage of validating a job being an instance of runnable when that should never happen, instead of just blowing up? [00:01:16] Just for the error message, I guess [00:02:01] But I guess in the Configuration->object style, we don't care [00:02:45] couldn' [00:02:56] coudln't the underlying error provide a similarly detailed error? [00:03:06] message that is [00:03:55] ah, there's a reason to get the nice error: I think calling a nonexistent method is fatal, so we would lose the opportunity to clean up and dispatch the error somewhere [00:05:14] shouldn't the fatal handler be set up for that? [00:05:32] might be totally out of scope [00:05:35] but in a perfect world [00:06:54] Yeah I agree, my preference for error-checking is pretty mild [00:13:54] f10 [00:15:36] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Impressions slow to update on jaJP dsk campaign - https://phabricator.wikimedia.org/T145446#2631433 (AndyRussG) If donations and impressions were down at the same time, that's a pretty certain indication people weren't seeing banners. My first incl... [00:17:05] awight: sorry i don't know where to stop derailing the conversation [00:17:45] It's great--let's lower the drawbridge and invite all the iconoclasts within the castle walls :p [00:18:23] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Patch-For-Review, Unplanned-Sprint-Work: Banners not allocated correctly - https://phabricator.wikimedia.org/T144393#2631438 (AndyRussG) I'm moving this to the Done column but not closing. It's hopef... [00:18:37] do you mean irregular http requests? [00:19:30] (PS6) Awight: [WIP] Higher level serialization codec [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310163 [00:21:27] cwd: I think that might be enough for us to get to the next step here ^ [00:21:54] All it does is let you poke new style jobs into the queue [00:22:02] we can refine the struct as needed [00:22:12] ... could even do it live now that we have versioning [00:22:23] awight: is the version stuff in case we change the consumers? [00:22:43] it's so we can tweak the formats [00:23:03] like, we should immediately push the keyedopaque params under a 'object-properties' key [00:24:29] awight: why ReflectionClass instead of new $class ? [00:24:45] cos you can't call "new" with variable arguments [00:24:55] e.g. call_user_func_array [00:30:04] fr-tech hey... any words of wisdom on how to store banner sequence config in mixin parameters? https://github.com/wikimedia/mediawiki-extensions-CentralNotice/blob/ca450900a9b6598d1ef121dda98b2694b6072cfd/CentralNotice.sql#L85-L93 ? [00:30:15] It could just be a JSON blob [00:31:38] AndyRussG: ooh right... that sounds good for now! [00:32:03] unless you feel like hypernormalizing [00:32:08] ...Or we could create array and object types of parameters, and try to keep track of their members with unique enclosing-param-name-based param names [00:32:57] awight: yeah I'm tending towards such unqueryable simplicity... Banner assignments themselves will still be available in the usual places, so the blob will just be where they go in the sequence and any other details of the step they correspond to [00:34:18] cwd: hmmmmm yurfff... I guess not many reasonablenormalizing options come to mind... == [00:34:21] ?? [00:34:39] I need to cook some veggies... See you tomorrow! [00:34:43] awight: cya! [00:35:40] AndyRussG: if only it was postgres: https://www.postgresql.org/docs/9.5/static/datatype-json.html [00:36:00] you can index and query into json [00:39:04] (PS7) Awight: [WIP] Higher level serialization codec [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310163 [00:42:23] awww [00:42:54] aaarg even the postgres web documentation has nicer design! [01:19:59] (CR) Ejegg: "Deployment note: production settings should duplicate all inst-args nodes to constructor-parameters nodes before the first SP-using app is" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/309523 (owner: Awight) [01:40:18] (CR) Ejegg: "Nice! Just got to work around the odd 'verified' naming convention for now." (6 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305761 (https://phabricator.wikimedia.org/T141654) (owner: Cdentinger) [01:50:07] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: Epic: Dedupe V2: resolve top conflicts - https://phabricator.wikimedia.org/T143057#2631516 (Eileenmcnaughton) [01:51:14] (CR) Ejegg: [WIP] Payments API for PayPal (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306601 (owner: Awight) [01:53:07] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, and 2 others: Spike: Civi Failmail Error: IMPORT_CONTACT Couldn't store email for the contact. - https://phabricator.wikimedia.org/T142954#2552223 (E... [02:02:17] (PS6) Ejegg: Move donation queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) [02:04:25] (CR) Ejegg: "Is manifest version 2 OK to use under REL1_27 ?" [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/307044 (https://phabricator.wikimedia.org/T142392) (owner: Paladox) [02:04:26] ejegg: ty for the CR! makin dinner but will peruse in a bit [02:05:13] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/293688 (owner: Awight) [02:11:20] (CR) Ejegg: [C: -1] "This looks fine, but let's hold off merge till we get the queue stuff done, since it'll mean a bit of work in each of the consuming repos." (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 (owner: Awight) [02:16:08] ok, checking out for the night. See ya all tomorrow! [02:49:46] (CR) AndyRussG: [C: 2 V: 2] "PHPUnit tests pass locally! Failures on CI are due to MariaDB strict mode there..." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/293688 (owner: Awight) [02:51:17] (CR) jenkins-bot: [V: -1] [Cleanup] Fix annoying punctuation and unnecessary strict equality [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/293688 (owner: Awight) [02:52:19] aaarg strict mode [03:20:33] (PS1) Eileen: Add Index to contact.preferred_language [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310192 (https://phabricator.wikimedia.org/T96410) [06:39:32] (PS2) Eileen: Add Index to contact.preferred_language [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310192 (https://phabricator.wikimedia.org/T96410) [06:39:34] (PS1) Eileen: Clean up legacy junk data from preferred_language field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310218 (https://phabricator.wikimedia.org/T96410) [06:43:43] (PS1) Eileen: Update preferred language based on contribution_tracking. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310219 (https://phabricator.wikimedia.org/T96410) [06:54:54] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, and 2 others: Silverpop export not capturing language changes in Civi (not sure if this is language-o... - https://phabricator.wikimedia.org/T96410#2631714 [12:52:07] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CN Campaign Setting oddities - https://phabricator.wikimedia.org/T145447#2630364 (Pcoombe) @spatton I would guess some Javascript isn't loading correctly for you. Do you see any errors in the browser console when it happens? [14:56:13] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, and 2 others: Silverpop export not capturing language changes in Civi (not sure if this is language-o... - https://phabricator.wikimedia.org/T96410#2633075 [17:33:49] fr-tech: anyone have things to chat about? [17:34:06] awight: sure, brt [17:35:16] holler if you need the dialup [17:48:42] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CN Campaign Setting oddities - https://phabricator.wikimedia.org/T145447#2633790 (spatton) @Pcoombe et al, you're right; here's what I see in my console: load.php:176 Exception in module-execute in module ext.centralNotice.adminUi.campaignManager:... [18:33:49] (CR) Ejegg: [C: 2] Static interface to Configuration [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 (owner: Awight) [18:34:34] (Merged) jenkins-bot: Static interface to Configuration [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306885 (owner: Awight) [18:37:07] (PS4) Ejegg: Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 (owner: Awight) [18:38:15] (CR) Ejegg: [C: 2] Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 (owner: Awight) [18:38:58] (Merged) jenkins-bot: Clean up AdyenTestConfiguration interface [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/306886 (owner: Awight) [18:41:39] (PS1) Ejegg: Update SmashPig lib & config usage [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/310372 [18:52:04] (CR) Awight: [C: 2] "thanks!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/310372 (owner: Ejegg) [18:53:43] (Merged) jenkins-bot: Update SmashPig lib & config usage [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/310372 (owner: Ejegg) [18:56:05] (PS1) Awight: Configuration fixup for maintenance jobs [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310376 [18:57:01] ejegg: ^ [18:57:10] oh, lemme take a look [18:57:31] (CR) Ejegg: [C: 2] Configuration fixup for maintenance jobs [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310376 (owner: Awight) [18:57:38] Scrutinizer found it, actually! Sketchy that PHP lint passed it over [18:57:56] good stuff! [18:58:33] I'll have a patch for CRM updating both smashpig and di libs in a couple minutes [18:59:51] thx--are you also making the DI patch? I was about to otherwise [19:00:51] (PS1) Ejegg: Update SmashPig & DonationInterface libraries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310377 [19:01:26] Figured I'd roll it in since there'd be conflict otherwise [19:01:33] great! [19:01:55] k, lemme rebase those queue consumer thangs [19:04:28] (Merged) jenkins-bot: Configuration fixup for maintenance jobs [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310376 (owner: Awight) [19:08:01] (PS7) Ejegg: Move donation queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) [19:41:50] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CN Campaign Setting oddities - https://phabricator.wikimedia.org/T145447#2634445 (AndyRussG) @spatton Thanks for reporting this!!! Certainly worrisome... Can you tell me your browser, browser version and OS and OS version pls? Could you pls try t... [19:49:22] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, FR-ActiveMQ, and 3 others: Delete from the pending database when a transaction is failed or completed - https://phabricator.wikimedia.org/T143945#2584307 (awight) Production pending datab... [19:56:12] ejegg: hey, have a second to go over packagist "auto-update" with me? [19:56:36] sure! [19:56:47] I only vaguely remember some connection like we need to introduce proper version numbers if we're auto-updating, else we'll get unintended updates? [19:57:10] i think that was the only issue [19:57:14] cool. [19:58:08] I don't see any settings in Packagist such as getting new versions from #deployment :( [19:59:27] It's annoying that new patches merged on master will fail "open" in that they will become the new deployment commit *unless* we update composer .version [20:00:03] durn, out of time. I have a meeting trap [20:30:01] (CR) Ejegg: "Sorry, we snuck in the SmashPig configuration changes under the parent patch, and now this needs manual rebase!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) (owner: XenoRyet) [20:32:25] (CR) Awight: [C: 1] "Some cleanup suggestions for the moved code, but I'd be fine with merging as-is if you prefer..." (6 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) (owner: Ejegg) [20:35:34] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Wiki-Loves-Monuments (2016): Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565#2634622 (awight) [20:35:49] (PS8) Ejegg: Move donation queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) [20:35:49] AndyRussG: dstrine-away: ^ That could be pretty grim for fundraising as well... [20:35:57] err, T145565 [20:35:57] T145565: Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565 [20:36:40] (CR) Ejegg: "Thanks awight! I'll prep a follow-on cleanup patch" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) (owner: Ejegg) [20:37:23] (CR) Awight: [C: 2] "/me unwraps champagne foil" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/308800 (https://phabricator.wikimedia.org/T131277) (owner: Ejegg) [20:37:43] awight: thx for the headsup! Hmmm I wonder, maybe some geolocation thing? [20:37:59] but then how would we be correctly identifying location on the server but not on the client... [20:38:01] It seems to be hitting multiple countries though--I haven't checked whether it's global [20:38:08] but IT looked the same [20:38:58] Could be the same as the other recent CN bugs, which (I'm suspsecting) may be related to banner serving or translation-related [20:39:19] ejegg: I'm no longer in a rush to get the new rectifier out, found that the pending vs payments-initial patch isn't doing its job. [20:39:47] AndyRussG: Lmk if I can do anything! [20:40:49] awight: durn, what's failing there? [20:40:55] I do think we should dig into it right away [20:41:06] AndyRussG: +1 [20:41:33] ejegg: It's just not deleting/preventing insertion to the pending db [20:41:34] fr-tech dstrine-away K I'm going to drop sequence at least until our meating to look into stuffs [20:41:56] awight: any places you can think to dig, pls go for it and lmk!!! thx!!!!! [20:42:18] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CN Campaign Setting oddities - https://phabricator.wikimedia.org/T145447#2634650 (Pcoombe) It's not something I've seen at all. (Chrome 52, Mac OS X El Capitan) [20:42:44] AndyRussG: Sure, I'll keep you posted. I did browse to kowiki...?country=KR and got the WLM banner as expected. [20:43:10] AndyRussG: I guess I can dig in stat1002 and try to find readers who should have been served a banner [20:44:36] dstrine-away: We're provisionally pulling that CentralNotice bug into the sprint, please let us know your thoughts about still getting other work done... [20:44:53] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Wiki-Loves-Monuments (2016): Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565#2634672 (awight) [20:49:32] AndyRussG: fwiw, I'm dumping pageviews from a random hour, where uri_host="ko.wikipedia.org" and year=2016 and month=9 and day=10 and hour=8 [20:51:26] (PS1) Ejegg: Clean up DonationQueueConsumer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310415 (https://phabricator.wikimedia.org/T131277) [20:52:31] (CR) Awight: [C: 2] "Much more readable!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310415 (https://phabricator.wikimedia.org/T131277) (owner: Ejegg) [20:53:51] (CR) Ejegg: [C: 2] "Looks fine to me" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310168 (https://phabricator.wikimedia.org/T96410) (owner: Eileen) [20:53:52] awight: cool! [20:54:09] I think Hive has a geo tag on the views too [20:54:53] Could also be a data pulling issue? How was the data in the bug gotten? Maybe there are a lot of views on ko wiki that are not in the country? [20:55:37] Definitely possible--check yr email for "banner data from South Korea" [20:55:47] (query is in there) [20:56:50] Ah K [20:57:04] Also wondering about error responses on Special:BannerLoader [20:57:43] (Merged) jenkins-bot: Clean up legacy junk data from contribution_tracking.language field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310168 (https://phabricator.wikimedia.org/T96410) (owner: Eileen) [20:57:45] hmm, ok i'll try to pull that from hive [20:58:04] (PS2) Ejegg: Re-enable recurring GC tests, add more [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306806 (https://phabricator.wikimedia.org/T141487) [20:58:57] (Abandoned) Ejegg: Update donation-interface lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306704 (owner: Ejegg) [21:00:20] (Abandoned) Ejegg: Always look for and delete pending entries [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305679 (https://phabricator.wikimedia.org/T141477) (owner: Ejegg) [21:01:58] (CR) jenkins-bot: [V: -1] Re-enable recurring GC tests, add more [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306806 (https://phabricator.wikimedia.org/T141487) (owner: Ejegg) [21:05:05] (PS2) Ejegg: FIXME [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305580 [21:06:48] (CR) Cdentinger: "I'm confused about the DataStores directory, is it just anything related to data storage? Maybe serialization is a different kind of conce" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310163 (owner: Awight) [21:07:11] (PS1) Awight: Fix "complete" status label [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310437 (https://phabricator.wikimedia.org/T143945) [21:07:55] (CR) jenkins-bot: [V: -1] Fix "complete" status label [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310437 (https://phabricator.wikimedia.org/T143945) (owner: Awight) [21:08:41] (PS1) Awight: Fix status label [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310439 (https://phabricator.wikimedia.org/T143945) [21:09:31] ejegg: Found why the code was failing... "completed" vs "complete". Sigh [21:09:39] Patches are ready for CR whenever. [21:10:06] yay magic strings :P [21:10:27] (CR) jenkins-bot: [V: -1] Fix status label [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310439 (https://phabricator.wikimedia.org/T143945) (owner: Awight) [21:10:40] (PS2) Awight: Fix "complete" status label [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310437 (https://phabricator.wikimedia.org/T143945) [21:13:19] nice [21:13:55] AndyRussG: You have any sample hive queries I can copy from? It's soo slow to figure that out with trial and error [21:14:06] awight: i thought i saw the stub for the job producer in this patch but now i don't see it, did i miss something? [21:14:08] I'll paste them into https://collab.wikimedia.org/wiki/Fundraising/Engineering/Fun_SQL_Queries [21:14:43] cwd: I think that snuck into #master already [21:14:54] oh yeah! [21:14:55] I didn't manage to split it out of the parent commit in time [21:15:07] thanks [21:15:15] no prob! [21:15:16] it's... not much ;) [21:15:35] awight: not just yet, but not far off.. [21:19:00] AndyRussG: I found two tasks with useful pasta, T90635 and T132500 [21:19:00] T132500: Clicktracking data not matching up with donation totals - https://phabricator.wikimedia.org/T132500 [21:19:00] T90635: Strategy banner impressions - https://phabricator.wikimedia.org/T90635 [21:20:58] hmm, more or less... [21:21:20] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Login for Engage User - https://phabricator.wikimedia.org/T145577#2634811 (LeanneS) [21:23:43] (PS9) XenoRyet: Move refund queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309603 (https://phabricator.wikimedia.org/T145234) [21:25:22] awight|brb: recurring tests are failing with an exception in unstaging after the DonationInterface update: https://gerrit.wikimedia.org/r/306806 [21:26:29] ejegg: Cool, I'll take a look [21:31:56] (PS2) Ejegg: Move recurring queue consumer off ActiveMQ [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/309618 (https://phabricator.wikimedia.org/T145233) [21:32:34] awight: https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/Queries#Pageviews [21:33:16] AndyRussG: verra nice, thanks! [21:33:59] yw [21:34:33] 2+ hours of CPU time per query is causing me stinginess [21:34:57] hummm I recall having gotten good stuff much faster before [21:35:31] It's only a few minutes of wall time, but many cpus seem to be involved in the stampede [21:35:32] ejegg|afk: do you have anywhere still the link to the etherpad where we were making notes for Hive queries last year? (/me laments lost notes from sdd fail...) [21:35:54] fr-tech https://etherpad.wikimedia.org/p/CentralNotice_pageviews_and_impressions [21:36:10] That one is empty [21:36:13] yes [21:36:20] (just created it) [21:36:23] I found this in my history, fwiw [21:36:23] https://etherpad.wikimedia.org/p/FR_unsolved_mysteries [21:36:35] AndyRussG: ah great [21:37:27] yeah though there was another one [21:37:38] thx though that looks also maybe useful!! [21:38:01] I'm wondering why this comes up empty: [21:38:01] select * from wmf.webrequest where uri_host="ko.wikipedia.org" and uri_path like '%BannerLoader%' and year=2016 and month=9 and day=10 and hour=8 [21:39:51] oh. nvm [21:39:58] that's obvious, cos BannerLoader is on meta. [21:40:43] right [21:40:56] but we could get the country [21:41:46] k, I'll add that [21:42:00] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues manually merging records - https://phabricator.wikimedia.org/T144440#2634874 (LeanneS) @dstrine we are still finding that some contacts are merging incorrectly. I've had a few in which either the wrong email address or mailing address gets merged. [21:42:51] too bad--webrequest doesn't have .country [21:45:22] awight: it does! geocoded_data... (I think...) https://wikitech.wikimedia.org/wiki/Analytics/Data/Webrequest [21:46:00] oh, funky! [21:46:56] I don't recall exactly how to where it into a query tho... [21:49:13] nice--I might just do a grep on the dumped rows to get a ballpark idea of what's in there [21:50:06] ah nice idea [21:50:13] I found something interesting so far--28k requests for BannerLoader in a sample hour, which is already much higher than the impressions. I'm pulling Hyve impressions for the same period now [21:50:58] woohoo! Can we get HTTP response? [21:51:42] http_status [21:52:55] yep [21:53:18] maybe it's just a problem with the pghers db pipeline [21:53:24] huh. a breakpoint in recordImpression isn't firing [21:53:39] Maybe this is the FF debugger not handling promises though [21:53:42] chroming... [21:54:30] awight: try URL param recordImpressionSampleRate=1 [21:54:48] huh. That might be all it is. [21:55:07] We're sampling impressions but not multiplying in the pgehres pipeline. [21:55:20] Something about using the default? [21:55:53] Cos, the counts seem to be multiples of 10 which doesn't sound right [21:58:46] huh? [21:59:12] ri is sampled on the client and further sampled in the db pipeline [21:59:29] The pgehres.bannerimpressions.count field seems to be Nx10, but that wouldn't make sense would it [21:59:38] It should be N00 or N000 [22:01:24] yeah, beaconImpressions are sampled 1:10 and we're configured to sample /beacon/impression at 1:100 [22:01:52] so bannerimpressions.count numbers should be Nx1000. Looks like we're failing to multiply by the beacon sample rate. [22:03:01] oh [22:03:04] hmm! [22:03:29] ejegg: AndyRussG awight meeting? [22:03:35] thx [22:08:55] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 4 others: Investigate Civi Load Time issue - https://phabricator.wikimedia.org/T138334#2634959 (DStrine) Open>Resolved [22:10:53] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Support CiviCRM dash changes - https://phabricator.wikimedia.org/T142051#2634960 (DStrine) Open>Resolved [22:13:27] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, FR-Adyen, Patch-For-Review: Reenable Amex on France Adyen form - https://phabricator.wikimedia.org/T144236#2634980 (XenoRyet) Open>Resolved [22:14:42] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 4 others: CRM reads 'completion messages' from pending DB and ActiveMQ, scream if no match - https://phabricator.wikimedia.org/T140484#2634984 (Ejegg) [22:14:46] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, FR-Smashpig, and 4 others: [Epic] Consolidate "pending" queue usages - https://phabricator.wikimedia.org/T130897#2634985 (Ejegg) [22:14:48] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, and 7 others: Quit looking in pending queue for completion message information - https://phabricator.wikimedia.org/T122641#2634983 (Ejegg) ... [22:14:58] Fundraising Sprint Pretending This Isn't Happening, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, and 2 others: [Epic] Rewrite all queue clients to use a single shim library, improve library - https://phabricator.wikimedia.org/T133108#2634987 (Ejegg) [22:15:01] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, and 4 others: Migrate fredge to new queue - https://phabricator.wikimedia.org/T131273#2634986 (Ejegg) Open>Resolved [22:15:17] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, and 3 others: Store damaged and delayed messages in a database not a queue - https://phabricator.wikimedia.org/T142028#2634988 (Ejegg) ... [22:16:26] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, and 2 others: Migrate banner history queue off ActiveMQ - https://phabricator.wikimedia.org/T141555#2634990 (Ejegg) Open>Resolved [22:17:38] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Login for Engage User - https://phabricator.wikimedia.org/T145577#2634811 (Eileenmcnaughton) I have created an account for him & he should have an email - so he just needs his certificate now... [22:18:14] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Wiki-Loves-Monuments (2016): Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565#2634993 (awight... [22:28:09] (PS3) Ejegg: Re-enable recurring GC tests, add more [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306806 (https://phabricator.wikimedia.org/T141487) [22:32:21] Fundraising Sprint Jabberwock Slaying, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, and 4 others: Slow queries on staging related to groups - https://phabricator.wikimedia.org/T136158#2635084 (Eileenmcnaughton) Open>... [22:32:42] (CR) jenkins-bot: [V: -1] Re-enable recurring GC tests, add more [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/306806 (https://phabricator.wikimedia.org/T141487) (owner: Ejegg) [22:33:00] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Write regression test for T144489 - https://phabricator.wikimedia.org/T144557#2635086 (awight) [22:34:19] (CR) Ejegg: [C: 2] Fix "complete" status label [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310437 (https://phabricator.wikimedia.org/T143945) (owner: Awight) [22:34:23] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Longterm fix + regression test for T144489 - https://phabricator.wikimedia.org/T144557#2635088 (Eileenmcnaughton) [22:34:44] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Longterm fix + regression test for T144489 - https://phabricator.wikimedia.org/T144557#2603473 (Eileenmcnaughton) a:Eileenmcnaughton [22:35:20] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: Epic: CentralNotice Banner squence - https://phabricator.wikimedia.org/T144615#2635097 (DStrine) [22:35:23] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice banner sequence: more precise, new UI mock-up - https://phabricator.wikimedia.org/T144452#2600221 (DStrine) Open>Resolved [22:35:30] ejegg: thanks! There's a crm piece, too, since I haven't finished the reuse patch. https://gerrit.wikimedia.org/r/#/c/310439/ [22:35:40] (Merged) jenkins-bot: Fix "complete" status label [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/310437 (https://phabricator.wikimedia.org/T143945) (owner: Awight) [22:36:38] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review, and 2 others: Date bug on Aug 31st causing recurring Ingenico donations to process in error - https://phabricator.wikimedia.org/T144489#2635112 (awight) Open>Resolved [22:36:48] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review, and 2 others: Date bug on Aug 31st causing recurring Ingenico donations to process in error - https://phabricator.wikimedia.org/T144489#2635113 (Eileenmcnaughton) Closing this - but there is... [22:38:03] (CR) Awight: [C: 2] "/me waves it goodbye" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310137 (https://phabricator.wikimedia.org/T143142) (owner: Ejegg) [22:40:53] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues manually merging records - https://phabricator.wikimedia.org/T144440#2599873 (Eileenmcnaughton) @LeanneS per discussion yesterday - are you still hitting issues since that meeting. If so can you please send me the contact ids & detail of what hap... [22:42:17] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Update to use MariaDB Strict Mode - https://phabricator.wikimedia.org/T145591#2635164 (AndyRussG) [22:42:52] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Update to use MariaDB Strict Mode - https://phabricator.wikimedia.org/T145591#2635182 (AndyRussG) [22:42:58] (Merged) jenkins-bot: Remove wmf_zendesk_reports [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/310137 (https://phabricator.wikimedia.org/T143142) (owner: Ejegg) [22:43:23] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Support MariaDB Strict Mode - https://phabricator.wikimedia.org/T145591#2635164 (AndyRussG) [22:44:34] !log update paymentswiki config to da01ae9045af7b8b5f3d34a5f81419fde3bde1d5 [22:44:40] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:46:57] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Support MariaDB Strict Mode - https://phabricator.wikimedia.org/T145591#2635194 (AndyRussG) See https://mariadb.com/kb/en/mariadb/sql_mode/#strict-mode. [22:49:18] (PS1) Ejegg: WIP MariaDB strict mode [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/310457 (https://phabricator.wikimedia.org/T145591) [22:50:52] (CR) jenkins-bot: [V: -1] WIP MariaDB strict mode [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/310457 (https://phabricator.wikimedia.org/T145591) (owner: Ejegg) [22:53:02] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues manually merging records - https://phabricator.wikimedia.org/T144440#2599873 (DStrine) p:Low>High [22:53:08] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Wiki-Loves-Monuments (2016): Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565#2635220 (AndyRus... [22:54:47] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: Dedupe: resolve Conflicts on capitalisation of names - https://phabricator.wikimedia.org/T145032#2635226 (Eileenmcnaughton) [22:55:05] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: Epic: Dedupe V2: resolve top conflicts - https://phabricator.wikimedia.org/T143057#2635229 (Eileenmcnaughton) [22:55:07] Fundraising Sprint Qwerty Thwacking, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: Dedupe: resolve Conflicts on capitalisation of names - https://phabricator.wikimedia.org/T145032#2617605 (Eileenmcnaughton) Open>Resolved [22:58:19] (PS2) Ejegg: WIP MariaDB strict mode [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/310457 (https://phabricator.wikimedia.org/T145591) [22:59:46] (CR) jenkins-bot: [V: -1] WIP MariaDB strict mode [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/310457 (https://phabricator.wikimedia.org/T145591) (owner: Ejegg) [23:02:04] Fundraising Sprint R 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Patch-For-Review: CentralNotice: Support MariaDB Strict Mode - https://phabricator.wikimedia.org/T145591#2635251 (DStrine) [23:02:08] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, FR-ActiveMQ, Patch-For-Review: Migrate refunds to new queue - https://phabricator.wikimedia.org/T145234#2635255 (DStrine) [23:02:12] Fundraising Sprint R 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CN Campaign Setting oddities - https://phabricator.wikimedia.org/T145447#2635254 (DStrine) [23:02:15] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, and 2 others: Investigate the discrepancy between pageview counts and banner impressions for WLM 2016 - https://phabricator.wikimedia.org/T145565#2635253 (DStrine) [23:02:17] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, FR-ActiveMQ, Patch-For-Review: Move recurring queue consumer off ActiveMQ - https://phabricator.wikimedia.org/T145233#2635257 (DStrine) [23:02:20] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice banner sequence: implement feature for MVP - https://phabricator.wikimedia.org/T144453#2635259 (DStrine) [23:02:22] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues manually merging records - https://phabricator.wikimedia.org/T144440#2635261 (DStrine) [23:02:24] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog: SPRINT Q (8/31 - 9/14) Goal: finish Queues and fix bugs - https://phabricator.wikimedia.org/T144449#2635260 (DStrine) [23:02:27] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, and 2 others: Banners not allocated correctly - https://phabricator.wikimedia.org/T144393#2635262 (DStrine) [23:02:30] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, and 4 others: Delete from the pending database when a transaction is failed or completed - https://phabricator.wikimedia.org/T143945#2635265 (DStrine) [23:02:32] Fundraising Sprint Octopus Untangling, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, Epic: Epic: Create frack vm cluster - https://phabricator.wikimedia.org/T142533#2635267 (DStrine) [23:02:34] Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, Patch-For-Review, Spike: Spike: Zendesk migration from Web Portal to Helpdesk - https://phabricator.wikimedia.org/T143142#2635266 (DStrine) [23:02:36] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, and 6 others: Move legacy PayPal listener to SmashPig - https://phabricator.wikimedia.org/T141654#2635268 (DStrine) [23:02:39] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, and 10 others: Run the orphan rectifier job from CRM Jenkins - https://phabricator.wikimedia.org/T141487#2635269 (DStrine) [23:02:42] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, and 6 others: Audit processors should mirror outbound queue messages - https://phabricator.wikimedia.org/T141484#2635272 (DStrine) [23:02:45] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, and 8 others: Rewrite orphan rectifier to use the pending database and WmfFramework - https://phabricator.wikimedia.org/T141486#2635271 (DStr... [23:02:47] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, and 3 others: Donation import should merge and delete pending db info - https://phabricator.wikimedia.org/T141477#2635273 (DStrine) [23:02:49] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Receipt/Thank you letters from Civi - Fix template? - https://phabricator.wikimedia.org/T137496#2635278 (DStrine) [23:02:53] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, and 6 others: Continuous integration: DonationInterface needs composer variant - https://phabricator.wikimedia.org/T141309#2635276 (DStrine) [23:02:55] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, and 4 others: Decimal dates are breaking our queue consumer - https://phabricator.wikimedia.org/T140667#2635277 (DStrine) [23:02:58] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising Sprint Killing Time, and 15 others: [Epic] All pending producers should write to a single pending queue - https://phabricator.wikimedia.org/T133195#2635279 (DStrine) [23:03:01] Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, Fundraising-Backlog, and 5 others: Migrate donations to new queue - https://phabricator.wikimedia.org/T131277#2635280 (DStrine) [23:03:03] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising Sprint Qwerty Thwacking, Fundraising Sprint R 2016, and 3 others: Silverpop export not capturing language changes in Civi (not sure if this is lang... - https://phabricator.wikimedia.org/T96410#2635282 [23:07:52] Fundraising Sprint R 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issues manually merging records - https://phabricator.wikimedia.org/T144440#2635305 (LeanneS) @Eileenmcnaughton it has been happening today. I was just able to recreate with CID 1880803. The duplicate contact email address... [23:27:19] (PS3) Ejegg: WIP MariaDB strict mode [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/310457 (https://phabricator.wikimedia.org/T145591) [23:38:00] ok gonna jump on later tonight to sort out the T145584 fail.