[00:00:13] AndyRussG: that's the thing, I would prefer to not clear counts [00:00:28] awight: Hmmm K [00:00:29] AndyRussG: should that be the same email? [00:00:36] Sure [00:00:42] I was planning on sending one with several options [00:01:21] But yeah that's a good one, hadn't thought of that [00:02:25] awight: ppena - (atgomez) I can look at that reconciliation issue on (my) Monday - but I'm just wondering if there is more information I can gather first [00:03:07] in particular normally if I look at anything that relates to reconciliation its good to have a copy of at least a subset of what it is reconciled against [00:04:20] ie - what do we receive from GC? [00:04:21] I'll ask PPena when she's around again... [00:04:32] I don't know myslef [00:05:46] awight: cool - - are you both in the office today? [00:06:07] eileen: great! monday! [00:06:31] atgomez: yep - I just need to gather as much info as I can as no-one will be around to ask! [00:07:34] atgomez: I've tried a few ways to nudge getting staging data updated but no joy yet (& of course Jeff is gone now for the day) - if you have any tricks please try - it would help for diagnosing that reconciliation stuff [00:07:47] ok. do you have the ticket number? [00:09:56] atgomez: https://phabricator.wikimedia.org/T119748 [00:10:57] Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Can we get the staging (CiviCRM & Drupal) DB updated from live? - https://phabricator.wikimedia.org/T119748#1871715 (atgo) p:Normal>High [00:11:34] Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Can we get the staging (CiviCRM & Drupal) DB updated from live? - https://phabricator.wikimedia.org/T119748#1834623 (atgo) @jgreen we need this to debug multiple tasks now. Any chance it could happen Friday? [00:12:46] thx atgomez [00:13:19] AndyRussG: On second thought, you mind if I keep these issues separate? I'd like to avoid further confusion... [00:13:38] awight: yeah that sounds good [00:13:54] I'd also say emphasize that this is just the desktop data [00:14:17] BTW I realized I also omitted some needed BH filtering, rrrg... Still it should be close enough [00:16:08] (PS4) Ejegg: fiddle with iframe container/content height [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/258080 (owner: Cdentinger) [00:16:37] (CR) Ejegg: [C: 2] "Little extra page height is way better than a scrolling frame!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/258080 (owner: Cdentinger) [00:17:18] AndyRussG: +1, I'll put "desktop" in the subject [00:17:31] awight: cool thanks! [00:17:40] (Merged) jenkins-bot: fiddle with iframe container/content height [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/258080 (owner: Cdentinger) [00:22:39] (CR) Ejegg: "It might be nice for the adapter to just say whether address is required or not. Then the mustache class would look at country, decide wh" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/258099 (owner: Cdentinger) [00:24:16] awight: https://phabricator.wikimedia.org/P2406 [00:29:48] Fundraising-Backlog, Wikimedia-Fundraising, MediaWiki-extensions-CentralNotice: Invalidate "waitdate" status whenever the impression diet maximum impressions limit is increased - https://phabricator.wikimedia.org/T121178#1871777 (awight) NEW [00:29:52] (CR) Cdentinger: "When I try to switch the year to 2014 the modal sticks and I see this: TypeError: data[(dataCount - 1)] is undefined" [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/257677 (https://phabricator.wikimedia.org/T94110) (owner: Ejegg) [00:29:53] AndyRussG: if you agree ^^ [00:30:20] I think we should start work on that right away... I can try to do something this afternoon, with your blessing. [00:31:20] awight: sounds fine, but I dunno aboutt the UI side... Sounds a bit unclear sideeffect-y [00:31:53] like how? [00:32:18] I think this is the behavior the admins expected (apparently) [00:34:20] I'll wander towards this event for a minute... [00:39:48] awight|corrupt: cool! [00:40:20] awight|corrupt: arrrrg I forgot to mention the actual explanation, it was the problem with banners closed due to the close button contributing to impression counts of the impression diet [00:43:35] awight|corrupt: minde if I send a follow-up? [00:48:14] awight|corrupt: https://phabricator.wikimedia.org/P2407 [00:48:20] ^ doesn't work... [00:48:45] back soon, I'll get bassqueal though [00:53:04] (CR) Ejegg: "Bet it's the test data! I'll add some stuff for past years too." [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/257677 (https://phabricator.wikimedia.org/T94110) (owner: Ejegg) [01:08:05] AndyRussG: do you think we should take the mobile campaign down overnight? [01:08:08] megan wants to know [01:08:42] atgomez1: no, I think leave them up [01:08:57] I can take them down if I come across anything evil later [01:09:22] the concern is that we may be cookie-ing wihtout showing the large banner [01:09:41] so we would effectively be losing that more impactful impression [01:10:02] so we should address that concern [01:10:11] atgomez1: Ah I see [01:10:37] It sounds unlikely [01:10:50] ok, can you please reply on one of those threads and explain that? [01:10:55] atgomez1: K [01:10:57] it's 2am in paris and megan is sending emails :P [01:15:56] atgomez1: yeah I understand! [01:20:14] thank you :) [01:20:20] atgomez1: likewise! [01:40:22] awight: K I'll send e-mails on that... pls feel free to correct if u see incorrectness.. [01:40:55] awight: also any idea wassup w/ my query? I guess you can't filter on dynamic fields like this? https://phabricator.wikimedia.org/P2407 [01:41:02] looking [01:41:05] thx!! [01:41:25] There's no `country` field in webrequest [01:41:30] that's in pageview_hourly [01:41:55] show create table webrequest [01:42:59] awight: right but I guess I'm not creating it w/ this line ? parse_url(concat('http://bla.org/woo/', uri_query), 'QUERY', 'country') as country [01:43:33] ooh [01:44:12] I'm afraid you might need to repeat the function call in the where. [01:45:52] Ah hmmm interesting [01:45:55] Lemme try [01:48:18] awight: bwahahaha looks like that did it, thx! [01:49:50] sad story... [01:50:02] we'll just run the same function 6 times per row... [01:52:28] awight: hahah no can't be right.... https://phabricator.wikimedia.org/P2407 [01:53:35] nope. [01:53:53] I think I have the right time... Looks like campaigns went up at 19:43 UTC that day https://meta.wikimedia.org/w/index.php?title=Special:CentralNoticeLogs&offset=&limit=100 [01:54:08] webrequest_source = 'mobile' [01:54:14] I believe that's the issue. [01:55:49] from doc: webrequest_source string Source cluster [01:56:19] That's not very informative! [01:56:28] What is the field about? [01:56:51] All I know :( is that I had an OR clause using 'text' and 'mobile' in my .hivehistory [01:56:56] I think someone told me to do that. [01:57:00] K [01:57:07] * AndyRussG obeys someone :) [01:57:09] (reading ellery) [01:57:15] :( [01:57:23] The banality of evil!! [01:57:41] Here's something in ewulczyn's repo: fr/chinese_variants.sql:AND webrequest_source in ('mobile', 'text') [01:57:55] fr/dashboard/python/dashboard_instances/Google Wallet Android vs Iphone.ipynb: " AND ((webrequest_source = 'text') OR (webrequest_source = 'mobile'))\n", [01:58:02] 1,000 snippets *can't* be wrong [01:58:35] hahaha [01:59:36] Heh well that was it! https://phabricator.wikimedia.org/P2407 [01:59:57] beautiful! [02:01:34] Totals don't add up [02:01:46] that's... promising [02:02:18] I'm sort of just drooling onto resources/subscribing/ext.centralNotice.impressionDiet.js so any entertainment is appreciated ;-) [02:02:54] awight: 2387789 pageviews vs. total 1569016 calls to /beacon/impression [02:02:58] https://achanceofeclecticism.files.wordpress.com/2014/05/img_1441.jpg but without the cute [02:03:17] Can I see the pageviews query? [02:03:29] https://phabricator.wikimedia.org/P2406 [02:04:07] Did Phabricator just go down? [02:04:16] We need to break this down according to user agents [02:04:33] hah appears so [02:04:59] yeah, wm-ops is abuzz [02:06:08] * awight grimaces [02:06:26] I'm becoming less confident about my proposed fix to the waitdate issue [02:06:39] We really did intend the logic to work as it is now [02:06:55] gah.. okay my moment of reflection is over [02:06:58] If we have a major dip in mobile impressions, that'll be the most urgent thing, no? [02:07:08] donno. [02:07:36] awight: https://tools.wmflabs.org/paste/view/36cb3319 [02:07:43] but desktop donations dwarf mobile, so 60% hidden impressions was top o' my list [02:08:37] sorry, please repaste the impressions query, too [02:11:45] Ah right sorry [02:12:34] awight: https://tools.wmflabs.org/paste/view/83507315 [02:13:11] awight: yea that was urgent, glad that one's cleared up! [02:13:44] P.S. Phab's back phabbing about [02:16:36] Those queries look consistent. [02:17:12] awight: how so? [02:17:22] The total number of impressions is much less than the pageviews [02:18:06] sorry--your queries look right; numbers do not. [02:18:13] Ah heh OK [02:18:27] +1 to breaking down the impressions by UA [02:18:54] K... other steps that come to mind? [02:19:26] Check pgehres.bannerimpressions... [02:19:31] subtract desktop impressions... [02:29:31] awight: right! [02:32:27] awight: offhand know what hive function to parse and test just the "browser family" property out of this value (user_agent_map field)? [02:32:41] {"browser_major":"9","os_family":"iOS","device_family":"iPod","os_major":"9","browser_family":"Mobile Safari","wmf_app_version":"-","os_minor":"1"} [02:32:46] (for exampe) [02:33:46] see also ./record_impression/old_approaches/record_impression_helpers.py: device_family = ParseDevice(user_agent)['family'] [02:33:50] no, I don't know [02:34:13] but I would guess: group by user_agent_map['browser_family'] [02:34:43] Ah K that sounds lawgihcal [02:37:22] I really want to get rid of mixinParams.skipInitial... I wonder if it's ever been used. [02:39:37] Heh I used it... when I was smoke testing! [02:40:10] Fun thing is it gets accrued even before waitdate is up, if the user goes back and gets waitdate-based hides [02:40:14] awight: https://phabricator.wikimedia.org/P2408 [02:40:37] Gonna break out a google doc, anyway there's the raw fleish [02:44:48] too bad, they actually use it. select * from cn_notice_mixin_params p join cn_notice_mixins m on p.nmxnp_notice_mixin_id = m.nmxn_id join cn_notices n on n.not_id = m.nmxn_not_id where nmxnp_param_name = 'skipInitial' and nmxnp_param_value != 0; [02:46:07] Very nice--I guess you need the pageviews broken down the same way, now [02:49:29] I guess the waitdate vs waitimps thing is just a "whichever is less" rule [02:54:57] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: iframe not appearing for some people (confirmed modern browsers) - https://phabricator.wikimedia.org/T112181#1872052 (MBeat33) #186200 using Safari on an iPhone via a banner reporte... [03:29:27] (PS1) Awight: WIP Increasing impression maximums will affect readers in the "waitdate" state [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) [03:30:20] AndyRussG: ^ that needs more pairs of eyes. I'll take another look later tonight... [05:40:38] (CR) Ejegg: "Yep, confusing stuff for sure. +1 for the rename suggestions - could rename the counts.X variables too, especially waitSeenCount -> seenT" (2 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [05:44:03] (PS1) Ejegg: Fix text filters [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/258419 [05:44:39] (CR) Ejegg: "Text filters fixed in https://gerrit.wikimedia.org/r/258419" [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/257202 (owner: Ejegg) [06:33:13] (PS1) Ejegg: Update test data [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/258423 [06:38:44] (CR) Ejegg: "test data fixed in https://gerrit.wikimedia.org/r/258423. not sure of the best way to nuke existing contributions though. Could just ins" [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/257078 (https://phabricator.wikimedia.org/T86094) (owner: Ejegg) [06:40:29] (CR) Ejegg: "TODO: all kinds of error handling. In the meantime, this update to test data includes 1000 donations for 2014: https://gerrit.wikimedia.o" [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/257677 (https://phabricator.wikimedia.org/T94110) (owner: Ejegg) [07:02:09] (CR) Awight: WIP Increasing impression maximums will affect readers in the "waitdate" state (3 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [08:21:36] (CR) Awight: WIP Increasing impression maximums will affect readers in the "waitdate" state (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [08:46:57] (PS2) Awight: WIP Improve impression diet state machine [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) [08:48:20] (CR) jenkins-bot: [V: -1] WIP Improve impression diet state machine [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [10:45:30] (PS3) Awight: Improve impression diet state machine [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) [11:10:10] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: civicrm.wmo favicon missing - https://phabricator.wikimedia.org/T121211#1872697 (awight) NEW [11:11:45] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Admin menu and site name are obscuring menus - https://phabricator.wikimedia.org/T121212#1872709 (awight) NEW [11:29:08] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising, MediaWiki-extensions-CentralNotice, and 2 others: Invalidate "waitdate" status whenever the impression diet maximum impressions limit is increased - https://phabricator.wikimedia.org/T121178#1872775 (awight) [11:29:23] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising, MediaWiki-extensions-CentralNotice, and 2 others: Invalidate "waitdate" status whenever the impression diet maximum impressions limit is increased - https://phabricator.wikimedia.org/T121178#1872777 (awight) a:awight [12:18:25] Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Can we get the staging (CiviCRM & Drupal) DB updated from live? - https://phabricator.wikimedia.org/T119748#1872921 (Jgreen) >>! In T119748#1871715, @atgo wrote: > @jgreen we need this to debug multiple tasks now. Any chance it could happen Friday? done! [12:21:08] Fundraising Sprint Zapp, Fundraising-Backlog: Civi exports timing out - https://phabricator.wikimedia.org/T120892#1872930 (Jgreen) [12:21:09] Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Can we get the staging (CiviCRM & Drupal) DB updated from live? - https://phabricator.wikimedia.org/T119748#1872928 (Jgreen) Open>Resolved refreshed dumps for developer restores in lutetium:/srv/, and restored to dev_civicrm and /dev_drupal [14:28:57] (CR) Ejegg: [C: 1] "Definitely easier to follow. Looks like it should have the desired effect and no bad side effects." (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [14:52:15] Fundraising Sprint William Shatner, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 2 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#1873196 (CCogdill_WMF) @Ejegg awesome, looks like the data is coming in! Is... [16:03:16] * cwd regrets rebuilding drupal/civi dbs [16:42:16] (PS2) Ejegg: CRM-17675 set date AND time to 'now' when refunding contributions [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/257269 (https://phabricator.wikimedia.org/T116317) (owner: Eileen) [16:42:34] (CR) Ejegg: [C: 2] CRM-17675 set date AND time to 'now' when refunding contributions [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/257269 (https://phabricator.wikimedia.org/T116317) (owner: Eileen) [16:48:55] morning ejegg [16:49:16] hi cwd [16:49:38] I generated some improved (and expanded) test data [16:49:39] (Merged) jenkins-bot: CRM-17675 set date AND time to 'now' when refunding contributions [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/257269 (https://phabricator.wikimedia.org/T116317) (owner: Eileen) [16:49:56] saw that! just trying to get it to import...i nuked my civi install [16:50:10] oh no, just saw your regrets line above.... [16:50:16] i remember it being a lot easier to rebuild last time i tried :-\ [16:50:41] bummer - what's the current sticking point? [16:50:45] i can get as far as: CIVI_CONFIG Payment Instrument 'Credit Card: JCB' not found! [16:51:02] which table is payment instrument again? custom_group? [16:51:16] hmm, so you've turned on all our modules and run drush updatedb ? [16:51:37] yup [16:52:08] let me see, i was pretty sure one of our .install files creates that and the rest of em [16:52:16] though i should say i see a good bit of this: DB Error: syntax error in civicrm_api3() (line 45 of /srv/crm/civicrm/api/api.php) [16:52:17] i think it's an option group / value [16:52:32] ahh, i have a hard time keeping those straight [16:52:38] syntax error, huh? that sounds like a regression [16:54:02] cwd yeah, the very first db update in wmf_civicrm should have created all the payment instruments [16:55:18] cwd ah, that like 45 thing sounds familiar [16:55:27] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, and 3 others: Fix "Notice: Undefined index: next_time_24h in SpecialHideBanners.php" - https://phabricator.wikimedia.org/T120890#1873391 (demon) While you're in this part of the code,... [16:55:36] I think the result doesn't have an error_message key sometimes [16:56:27] hmmmm [16:57:46] it's some sort of regression, or, transgression (ha ha) [16:58:02] i have a little shell script i used to rebuild civi with [16:58:06] but it don't work no mo [16:59:40] (PS3) Ejegg: CRM-17641 backport Civi class in order to support other backports [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/256162 (https://phabricator.wikimedia.org/T116724) (owner: Eileen) [17:00:40] well, let me point my settings at some new dbs and see what happens... [17:04:55] (CR) Ejegg: "The '\Civi\Core\' vs 'Civi\Core\' discrepancy makes me think I'm missing something, but if awight likes it..." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/256162 (https://phabricator.wikimedia.org/T116724) (owner: Eileen) [17:15:43] aargh, what did I do to apache! [17:16:16] https://xkcd.com/349/ [17:19:08] haha yep [17:19:11] been my morning also [17:19:39] now i have NO sites :( [17:19:57] even if I remove the new civi from sites-enabled and restart [17:35:27] ejegg: i really should have deployed DI last night, spaced today being friday [17:35:58] a deploy would mostly get rid of the adyen scrollbars, which would be nice since they're going to do 1hr test today [17:36:05] but...there's quite a bit of other stuff that would roll too [17:36:56] durn, what else is in there? [17:37:05] I didn't think we'd merged much lately [17:38:00] cwd got latest deploy? All I see is your css + an l10n patch [17:38:11] I think you're safe to go [17:38:41] also, I haz sites again. turns out apache barfs if your alias has a dash in it [17:40:01] ah you're right, i got myself in a weird git situation [17:40:09] and that's ridiculous [17:40:19] ok i'm gonna deploy then [17:43:24] (PS1) Cdentinger: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/258478 [17:44:14] (CR) Cdentinger: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/258478 (owner: Cdentinger) [17:44:50] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/258478 (owner: Cdentinger) [17:45:11] aaaaaargh!! [17:45:14] coffeeeeeeeeeeee!! [17:45:24] braaaaaiins [17:48:28] i really, really hope i get a chance to build out the deploy script soon [17:48:42] (PS1) Cdentinger: update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/258479 [17:50:24] (CR) Cdentinger: [C: 2] update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/258479 (owner: Cdentinger) [17:51:16] (CR) Pcoombe: [C: 1] "This looks okay to me, and clearer." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [17:52:15] AndyRussG: looks like we're all waiting for you to weigh in decisively on awight's latest [17:52:29] ack, didn't mean to sound pressure-y [17:52:49] just noticed we were deferring to you on this delicate bit o code! [17:56:16] phooey, neither of our civi gurus are present today [17:59:27] https://integration.wikimedia.org/zuul/ [17:59:32] this is not the zuul i am looking for [18:00:04] ejegg: cool, yeah I did see!! I'll check it out after FR standup, thanks! [18:00:19] cwd is your patch not merging? [18:00:30] whoa, gerrit temporarily unavailable! [18:00:36] k back [18:00:38] yikes [18:01:07] cwd looks like it merged [18:01:19] * cwd tries again [18:02:43] Is it FR standup or retrospective? [18:03:15] retro AndyRussG [18:03:32] ccogdill: cool thanks! Yeah I'm there [18:06:25] ejegg: awight: hi! I think deploying the patch would go until Monday in any case, no? [18:06:35] AndyRussG: morning! [18:06:39] hi! [18:06:51] Are you talking about the CN patch? [18:06:56] yep [18:07:00] I'd say this is a valid emergency... [18:07:13] awight: a K. [18:07:18] It's ultimately up to whoever would be babysitting, though [18:07:42] awight: if you miss work, there's FR retro [18:07:51] haarrrr [18:08:01] * AndyRussG hides union rep under desk [18:08:33] :D [18:21:38] !log updated payments from 74143e43eb36f93be7881f626443182d3bb58cef to a1be1ad134d06464e98de180227554fceddc91d4 [18:21:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:38:46] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Impressions are looking weird for mobile campaign - https://phabricator.wikimedia.org/T121042#1867922 (AndyRussG) [19:13:15] heads up cwd, PPena: I'm just prepping the adyen test now. should go up in a couple of minutes unless there's any objections? [19:13:44] the-wub: sounds good! i'm going to head to lunch soon but i'll have irc on my phone [19:14:14] i got the new iframe markup live btw, shouldn't see the scroll bar anymore [19:15:01] the-wub go for ir [19:15:04] it [19:19:15] ok, adyen banner is up [19:19:50] watchin the logs [19:23:28] the-wub: not seeing any clicks! [19:23:32] oops [19:23:34] there's one [19:25:05] cwd: I see 13 clicks for adyen. no donations through yet, but there's none for GC either [19:25:23] probably just the queue backed up [19:25:58] cool yeah, it's picking up now [19:30:55] the-wub cwd i see 8 donations in the Adyen console [19:31:09] cwd: actually, I still don't see any adyen cc donations in civi [19:31:35] scratch that, just changed. 7 now! [19:31:38] the-wub: yeah, that queue might be backed up [19:31:53] good! i think i saw one that misfired (will get caught by the audit) [19:32:39] few of them alreayd in CIVI [19:32:52] wohooo [19:32:52] it said "Could not find a processable message for PSP Reference" [19:33:30] i think i saw that same thing when mine hung, essentially means adyen told us to process the payment before the person got back from the iframe [19:33:39] but it will get caught by the audit [19:38:53] ok everything looks good, i need to eat, the-wub are you comfortable taking it down if anything comes up? [19:39:12] we should see smashpig failmail if anything goes wrong [19:39:26] cwd: sure, no problem. thanks cwd! [19:39:33] :) [19:57:15] hey the-wub cwd ejegg XenoRyet AndyRussG|bassoo did you see that this was in review? https://phabricator.wikimedia.org/T121178 [19:57:43] atgomez: yep! the-wub and I have each +1ed it [20:00:01] thanks atgomez. I'm not comfortable +2ing it myself, but it does look good to me [20:05:50] atgomez: ejegg: the-wub: yeah I'm looking at it. Adam suggested it might be even urgent enough for a Friday deploy, but I'm a bit wary, might be OK [20:07:16] atgomez: I think in fact no one knows exactly how much it would impact. How important is today and this weekend for getting any of those missing impressions, vs. a deploy on Monday around 8 am Pacific Time? [20:07:30] AndyRussG: I'm happy to help test if you go ahead with it [20:13:54] ejegg: right... I mean, I think testing it is fine... WRT deploys, the thing is there are no SWAT deploys today. On those, you're allowed to register a patch at the last minute. For non-SWAT, you're supposed to book your own slot much futher in advance [20:14:10] ah, ok [20:14:34] well, I can be around for monday 11am eastern! [20:20:06] cwd: okay, adyen test is down. all looked good on my end! [20:20:44] great! i didn't notice anything out of ordinary [20:20:53] ejegg: K... I mean, I'm willing to ask the about a deploy today if people think it's important enough. The other thing is it's not a completely trivial patch, maybe only 55% trivial [20:20:56] thanks for all your work on this! [20:21:04] atgomez: ^ ? [20:24:02] Maybe greg-g'd opine? ^ there's a patch that may or may not be important for getting some missing banner impressions over the weekend. It's 55% (+/- 5%) trivial. We don't know for sure how useful it might be... I imagine booking a Friday deploy is only for absolute emergencies, eh? [20:37:59] atgomez: cwd: XenoRyet: ejegg: asking about a CN deploy now on -operations channel [20:40:29] (greg-g is on vacation, ostriches is his delegate.) [20:40:41] OK, cool [20:51:34] ejegg: ostriches wants K4's opinion [20:52:04] Sounds reasonable. [20:52:24] Hmm, not online now... hope she's not feeling too blah [20:52:37] I think she just finishd a meeting [20:52:59] ejegg: the path to organizational insanity is paved with reasonable deferments of decisions :) [20:53:04] hehe [20:53:37] I guess 'be bold' is some kind of antidote for that [20:54:22] aaaarg and the "I broke Wikipedia" shirts are the antidote to the emotional fallout from such boldness? [20:55:02] sounds right :) [20:56:09] ejegg: BTW Roan offered to deploy if we go ahead with that [20:56:35] Oh cool, that's very helpful [20:56:53] Yeah he's an ace at that [20:58:38] <_< [20:58:44] Still considering a CN party? [21:00:12] awight: yeah I'm reviewing the patch and we've studied the legal issues [21:01:33] ejegg: awight: cwd: XenoRyet: atgomez: I've obtained a check-in w/ Ellery 1:30 - 2 PST, in case anyone is interested :) [21:16:35] legal, huh? That sure does sound like a party ;) [21:43:19] awight: well just joking it was "legal" in the sense of "rule-based" [21:49:57] hey guys… silly question [21:50:34] hehe [21:50:43] AndyRussG: seriously. [21:51:39] awight: sorryin a meeting w/ ellery :) [21:51:42] Join if u like :) [21:52:00] ohai [22:01:57] awight: ejegg is banging on the patch moar :) [22:05:48] or trying to at least... keep getting cn undefined errors.... [22:06:04] mw.centralNotice is undefined, rather [22:06:30] must be a localsettings thing... [22:08:58] aaahgh [22:09:33] oddly, it's only loading .startUp, .geoIP, and .kvStoreMaintenance [22:10:00] and the choiceData, which is trying to set properties of as-yet-undefined mw.centralNotice [22:12:50] ejegg: wut? [22:13:19] on this patch? [22:13:26] AndyRussG: haven't done much cn locally on this machine lately, trying to figure out why .display isn't loading [22:13:33] btw just finished meeting with Ellery [22:13:35] ejegg: ah K [22:14:50] so odd.. CNChoiceDataResourceLoaderModule definitely adds it as a dependency [22:15:03] Ah yeah hmmm [22:15:15] Ur campaigns are config'd correctly? [22:15:24] ohwaitaminit, is there something new that means CN doesn't work on 1.25? [22:15:35] ejegg: hmmm dunno [22:15:40] maaaaaybe [22:15:41] I feel like something RL-related had a breaking change [22:15:49] lemme just look on my trunkwiki [22:16:01] ejegg: atgomez: awight: cwd: XenoRyet: main take-away from meeting w/ Ellery: there is a discrepancy on /beacon/impressions vs. pageviews on desktop too, though it's less. One general explanation possible is that a lot of spiders and bots are not properly detected and such and that can bloat pageview numbers [22:16:16] hm [22:16:32] So far looking ath the details of the pageview definition finds no issue [22:16:47] ah, trunkwiki's showing banners like a champ [22:17:15] NEway now the thing to figure out on that point would be why there's a greater discrepancy of pageviews vs /beacon/impression calls on mobile [22:21:27] AndyRussG: neither ios safari nor pre-chrome android browser support beacon api [22:23:24] AndyRussG: but we got the numbers to match almost exactly--if anything, impressions higher than pageviews, I think [22:23:25] AndyRussG: Do we have ratios of /beacon/impression calls by beacon support, to see how well our fallback is working? [22:24:23] awight: we didn't filter on countries correctly [22:24:33] ejegg: interesting point! [22:24:37] ooh/ [22:25:00] > and country_code in ('AU', 'CA', 'IE', 'NZ', 'UK', 'US') [22:25:01] no good? [22:25:19] awight: that's fine for pageviews. Also gotta filter on the impression side [22:25:24] awight: GB not UK? [22:25:49] and ( parse_url(concat('http://bla.org/woo/', uri_query), 'QUERY', 'country') in ('AU', 'CA', 'IE', 'NZ', 'UK', 'US') ) [22:26:02] ejegg: woooooooo!!! :) [22:26:08] That's gotta be it!!!!!! [22:26:30] * AndyRussG rings FR-tech million-dollar-answer bell [22:26:48] hehe. we should add Ukraine to big english just to make things more confusing [22:26:50] I can't believe how many times that gotcha has bitten us [22:27:01] * AndyRussG turns to CentralNotice [22:28:14] ejegg: thanks so much, yea way to go :) [22:28:55] glad to help with trivia! [22:29:22] AndyRussG: oooooh [22:29:45] wowowow [22:29:50] trivia ain't trivia when it bites us in the butt [22:29:53] that was awesome to watch [22:29:56] GB is so imperial [22:30:29] Hahahah /me checks images on local currency [22:47:54] ejegg: seen any issues with the patch? [22:48:22] AndyRussG: nope, not yet! [22:48:27] cool! [22:48:28] I think we're almost ready to call in K4 [22:48:32] just trying a couple more runs through [22:49:16] verified that the existing code does indeed reset the counts as soon as it hits the cycle limit, so we will be adding an *additional* whatever-the-new-limit-is [22:50:38] AndyRussG: any plans to move the code that increments counts to the post-banner handler? [22:50:58] ejegg: exactly yes that's where this should go [22:51:17] What if you haven't filled up your impressions, but the cycle delay passes? [22:51:21] ah cool, sorry to bring that up now! [22:51:37] ejegg: au contraire, great point :) [22:51:56] ah, let me see... [22:52:34] I think the '&& counts.waitSeenCount >= mixinParams.maximumSeen' makes sure we don't reset in that case [22:53:13] so users who aren't yet in waitdate will have all of their views to date count towards the new max [22:54:32] Rrrrg should be, but I just got hid unexpectedly :( lemme try again [22:54:56] Hmm, let me set up that scenario [22:55:11] Set a 60-second cycle restart, and some number of impressions, then do only one pageview, and wait [22:56:40] Hmmm something wrong w/ my Chromium or something... [22:56:49] working on iceweasel? [22:58:32] weird now it's just not showing, I wonder what I did [22:58:51] ok, my 60sec has passed, lemme reload [22:59:45] Hahaha my campaign expired [22:59:46] Sorry. I suddenly got interested [22:59:54] heh [23:00:20] I think old code was correct to reset all the counts once pastDate [23:00:35] You do want another maxImpressions banners delivered at that point. [23:01:12] awight: why do you say? [23:01:30] But the bad was when the waitUntil date had not arrived yet, and the configured maximum was increased. [23:01:34] awight sure, but it's nicer now when you wait to reset 'em till the next cycle [23:01:55] ejegg: yeah works for me now, it was just that my campaign expired exactly at the wrong instnat ;[ [23:01:57] ah--right, the old code reset once maxImpressions was reached, rather than on pastDate [23:02:10] ejegg: awight: K this looks fine I think :) [23:02:16] I think I misunderstood when you were talking about "existing code" [23:02:20] * AndyRussG prepares to pull trigger [23:02:32] * awight pretends to be invisible [23:02:42] * cwd makes popcorn [23:02:44] AndyRussG: yep, looking good here too [23:02:51] (CR) AndyRussG: [C: 2] "Great job!!!!! :D /me dances bannerous though impression-thin jig" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [23:03:39] it was sort of hilarious how many intermediate versions of the logic I had produced, which were utterly unintelligible [23:03:57] It's not as easy as one might think! [23:04:07] heh, definitely twisty [23:04:09] awight: ejegg: atgomez: cwd: XenoRyet: K unless there are objections, I'm going to look for K4 for rubberstampiness [23:04:14] please do! [23:04:18] the incorrect variable names really helped with obfuscation [23:04:39] I don't think K4-713 is around? [23:04:59] (Merged) jenkins-bot: Improve impression diet state machine [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/258414 (https://phabricator.wikimedia.org/T121178) (owner: Awight) [23:05:04] Nope [23:05:09] We'll be summoning her [23:05:19] * ejegg draws pentagram [23:06:14] * ejegg waves charizard carcass over lightsaber [23:06:27] I'm happy to rubberstamp--like I was mentioning, it's the deployer who should decide whether they feel like standing in the rain for the rest of the day [23:11:49] awight: greg-g is out and ostriches is standing in, and he said he'd like K4's approval for a Friday deploy [23:11:53] awight, AndyRussG - is there a reason we store total seen count? Does in-banner logic still use that? [23:12:38] ejegg: it was meant to but it no longer works. But there's still snippets of code that try in vain to support it [23:12:47] ah [23:12:58] I think it might be useful for some debugging, so I'm tending towards fixing (one date) [23:13:00] day [23:13:21] +1, I think that's important info [23:13:53] K I'm going to text K4 [23:16:37] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874602 (LeanneS) NEW [23:16:54] AndyRussG: really not necessary... I'm trying to flag down Chad in -operations... [23:17:10] awight: eh? OK... [23:17:12] fwiw, if k4 is out shouldn't approval fall back to awight? [23:17:19] just a thought... [23:17:23] atgomez: yeah I think so... [23:17:27] sure--but more importantly, I don't think we even need such a mechanism. [23:17:49] well let's try to solve the problem of the day and then we can deal with the mechanism later [23:17:49] :) [23:17:56] awight BTW the issue is not that it's freeze, but that it's Friday [23:18:10] We're each the expert in our own domains, so I would much prefer that e.g. anyone trying to deploy CentralNotice would ask AndyRussG's opinion and not mine or K4-713's [23:18:32] Second opinions are always great to get... but that different than "approval" [23:18:45] awight: have you been watching movies about workplace democracy again? [23:19:00] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874615 (atgo) @leannes can you provide some information to help us prioritize this work? [23:19:15] lol. AndyRussG: totally. I'm glad to be the scapegoat if anything goes wrong, since it is my bad code, my patch, and me pressuring for a deployment [23:19:52] awight: heh not bad code, no guilt, all good :) [23:21:10] I feel pretty good about it ;) [23:22:46] i forget every time: which table are payment_instruments in? option_group? [23:23:13] cwd: it's devilish. civicrm_option_group to find the civicrm_option_value.option_group_id [23:23:24] 10, IIRC [23:23:52] something in option group should say "instrument" ? [23:24:01] i swear i've scanned every field in both tables [23:30:18] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874649 (CaitVirtue) 1) this is the appeal landing page: https://donate.wikimedia.org/w/index.php?title=Special:FundraiserLandingPage&appeal=Appeal-MGAnnual&form-co... [23:33:41] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874668 (atgo) Thanks @caitvirtue and @leannes. I'm hearing that this is high priority. Right now, Tech is working on some critical issues for the online campaign. W... [23:33:55] (PS1) AndyRussG: Improve impression diet state machine [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/258641 (https://phabricator.wikimedia.org/T121178) [23:34:00] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874670 (atgo) p:Unbreak!>High [23:34:13] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874602 (atgo) [23:34:30] (CR) AndyRussG: [C: 2] Improve impression diet state machine [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/258641 (https://phabricator.wikimedia.org/T121178) (owner: AndyRussG) [23:36:03] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874676 (CaitVirtue) If we're going to wait until Monday, we should add in a solution to bulk correct all the... [23:36:28] (Merged) jenkins-bot: Improve impression diet state machine [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/258641 (https://phabricator.wikimedia.org/T121178) (owner: AndyRussG) [23:37:29] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874680 (atgo) @caitvirtue they are being noted in the database with the utm_campaign (I just checked using my... [23:37:56] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874681 (CCogdill_WMF) I can help @LeanneS get a list of all the donations from this utm_campaign, if tech can... [23:38:09] hey atgo [23:38:33] I may be wrong, but I don’t think utm_campaign is in the civicrm database. I always use drupal when I’m looking at utm tracking [23:38:50] so that’s why I’ve wondered if this is less than trivial [23:39:01] re T121284 [23:39:06] ccogdill: It's called something really crazy, but it's there [23:39:09] haha [23:39:12] oh okay [23:39:20] For historical reasons, I think it's called Direct Mail Appeal [23:39:25] I’d love to know what it is! [23:39:30] -- but it gets weirder, the actual database field is... [23:39:31] is [23:39:54] hmm okay do you know what table? [23:40:29] ohhh i see [23:40:32] grumble [23:40:32] yeah you're right [23:40:50] :D [23:41:01] ejegg: don't want to bug you mid-deploy, but where should all those payment instruments come from? i've done all the updates but have none still [23:41:04] anyway, i talked to leanne and this doesn't need to be dealt with today, so i've retriaged as high priority ininstead of unbreak now. [23:41:08] if you are too busy to answer np! [23:41:15] sounds good [23:41:18] let me know if I can help [23:41:23] cwd: ah, we used to jam everything into the "Appeal" column, but the new Civi doesn't support that [23:41:32] cwd the very first update in wmf_civicrm.install should do it [23:41:32] so you'll have to find utm_campaign in contribution_tracking [23:41:39] cwd: sorry, I meant ccogdill ^ [23:41:46] got it [23:41:53] ccogdill: Is that gonna work for you? [23:41:54] well contribution_tracking works for me [23:42:06] but not sure what that means for the task [23:42:16] cwd did you get any errors on drush en wmf_civicrm? [23:42:24] thanks for the answer! :) [23:42:59] cwd also drupal.system should have a schema version number for wmf_civicrm corresponding to the last successful db update [23:43:04] ejegg: nope, and i dropped and recreated the drupal db so i think the schema version should be correct.. [23:43:12] * awight peers into T leaves [23:43:21] what's the version # for that module? [23:43:58] also, cwd, what did it take to finally get civi installed? [23:44:46] ejegg: well the scripts in bin report successful install but i'm not super convinced [23:45:00] i see schema version 7063 [23:45:04] let me try reverting that [23:45:07] phooey, I still get duplicate tag errors with the bin scripts... [23:45:40] well, lemme try nuking both and starting over [23:46:56] ack, it's got to be this: Failed: CiviCRM_API3_Exception: DB Error: syntax error in civicrm_api3() (line 45 of /srv/crm/civicrm/api/api.php). [23:47:21] aha, or at least that's a cascaded error [23:47:46] turn on mysql full query log? [23:47:58] good idea [23:49:44] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874699 (awight) > Timing: We need this today or tomorrow. Geez. Okay, there are complications here. We use... [23:50:32] cwd: Usually there's more. Have you looked in sites/default/files/civicrm/ConfigAndLog ? [23:50:37] (obviously :p) [23:50:57] awight: nope! /me checks [23:52:08] the plot thickens [23:53:46] ejegg: awight: cwd: XenoRyet: atgo: K CentralNotice patch going out momentarily [23:55:02] good luck! [23:55:15] AndyRussG: ok, operators are standing by [23:56:10] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Need to see UTM Source/UTM Campaign in Civi - https://phabricator.wikimedia.org/T121284#1874710 (atgo) @caitvirtue does the ease of backfilling address your concerns? [23:56:36] cwd: ejegg: thx! [23:58:35] * cwd flail [23:59:22] ejegg: ah sad news! the GB vs UK issue wasn't the cause of the pageview vs. impression. It caused us to omit Great Britain from the queries, but it was the same (GB not UK) on both ends, so same old same old... [23:59:26] ejegg: great catch in any case! [23:59:37] (sorry to disappoint...)