[00:02:18] awight: back! [00:02:34] LMK what I can do, deploywise [01:17:14] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, and 2 others: Translation tool showing zero translations in MessageGroupStats for CentralNotice banner - https://phabricator.wikimedia.org/T122251#1907573 (awight) @JAlexander Please k... [01:24:11] awight: new JS code is being served, currently banners seem OK [01:32:38] AndyRussG: Thanks! [01:32:48] awight: likewise! [01:33:40] awight: what changes are you expecting on the translation interface side? I still see all 0%'s up and down here: https://meta.wikimedia.org/w/index.php?title=Special%3AMessageGroupStats&x=D&group=Centralnotice-tgroup-WP15BannerTranslations#sortable:5=asc [02:04:52] awight: erranding, back in a bit [02:45:33] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, and 2 others: Translation tool showing zero translations in MessageGroupStats for CentralNotice banner - https://phabricator.wikimedia.org/T122251#1907649 (jrbs) >>! In T122251#1907573... [02:54:11] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, and 2 others: Translation tool showing zero translations in MessageGroupStats for CentralNotice banner - https://phabricator.wikimedia.org/T122251#1907651 (awight) @jrbs Thank you! I'... [08:24:38] moin leszek_wmde [11:37:13] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, and 2 others: Translation tool showing zero translations in MessageGroupStats for CentralNotice banner - https://phabricator.wikimedia.org/T122251#1907945 (awight) Open>Resolved... [14:49:18] Fundraising-Backlog, fundraising-tech-ops: fr-log-announcer should not hang out in our private channel - https://phabricator.wikimedia.org/T122104#1908135 (Jgreen) My vote is for a persistent bot with better use of IRC's basic security framework. We can register a nick and have the bot authenticate by... [16:58:45] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work: banner data and missing alterImpressionData(): Determine exact impact on data and coping mechanisms - https://phabricator.wikimedia.org/T120017#1908261 (AndyRussG) Replied, thanks!! I also think we... [17:04:49] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Impression and donation numbers are looking weird for mobile campaign - https://phabricator.wikimedia.org/T121042#1908267 (AndyRussG) The figures were improved substantially following the IPv6... [17:07:25] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Analyze options for limiting banners following close button clicks causing incorrect impression counts in imp... - https://phabricator.wikimedia.org/T121031#1908269 [17:07:47] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Analyze options for limiting banners following close button clicks causing incorrect impression counts in imp... - https://phabricator.wikimedia.org/T121031#1908271 [17:08:01] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Analyze options for limiting banners following close button clicks causing incorrect impression counts in imp... - https://phabricator.wikimedia.org/T121031#1908273 [17:10:42] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: [Epic] Create tools to monitor the operation of CentralNotice for targeted groups of users - https://phabricator.wikimedia.org/T122589#1908274 (AndyRussG) NEW [17:10:58] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Create tools to monitor the operation of CentralNotice for targeted groups of users - https://phabricator.wikimedia.org/T122589#1908282 (AndyRussG) [17:14:09] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Study more details of the impressions-to-pageviews correlations - https://phabricator.wikimedia.org/T122590#1908285 (AndyRussG) NEW [17:34:33] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Temp tables for silverpop_export don't need to be temporary - https://phabricator.wikimedia.org/T120899#1908320 (Ejegg) Open>Resolved [17:34:50] Fundraising-Backlog, Epic: [epic] Adyen campaign ready - https://phabricator.wikimedia.org/T118202#1908324 (Ejegg) [17:34:52] Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Smashpig: Don't delete Adyen audit files - https://phabricator.wikimedia.org/T121695#1908321 (Ejegg) Open>Resolved [17:36:11] Fundraising Sprint Zapp, Fundraising-Backlog, Patch-For-Review, WMF-deploy-2016-01-12_(1.27.0-wmf.10): Advise BPay donors not to reuse reference #s - https://phabricator.wikimedia.org/T119912#1908327 (Ejegg) Open>Resolved [17:36:57] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work, Patch-For-Review: Add configurable sample rate for creating CiviMail activities from Thank You emails - https://phabricator.wikimedia.org/T119560#1908329 (Ejegg) p:Triage>Low [17:42:08] Fundraising Sprint William Shatner, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 2 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#1908348 (CCogdill_WMF) @Ejegg we sent emails yesterday which all should hav... [17:44:07] Fundraising-Backlog, Wikimedia-Fundraising: PayPal donation link sometimes unresponsive - https://phabricator.wikimedia.org/T120995#1908353 (MBeat33) @atgo, you're right - would it be better to expand this task or create a new one? [18:02:53] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Create CiviCRM naming convention rule: In general, records should not begin with "The" - https://phabricator.wikimedia.org/T122592#1908371 (CaitVirtue) NEW [18:03:37] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: "The" issue: Contacts with dup records (with and without "The") - https://phabricator.wikimedia.org/T122264#1899850 (CaitVirtue) OK, this task is a scan of current org records for duplicates (The Merck Foundation and Merck Foundations) and clean up. Cr... [18:05:07] Fundraising Sprint William Shatner, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 2 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#1908382 (Ejegg) @JGreen, I replaced table pgehres.donatewiki_counts with a... [18:23:13] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: Impression and donation numbers are looking weird for mobile campaign - https://phabricator.wikimedia.org/T121042#1908397 (atgo) +1 @andyrussg, let's do what you suggested and close this out +... [18:33:01] dstrine-away: brt [18:33:29] cool ... I was just going to ask [18:34:09] atgomez: You sound busy--if not, I was hoping you could point me to our official Q3 goals. [18:36:48] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Create CiviCRM naming convention rule: In general, records should not begin with "The" - https://phabricator.wikimedia.org/T122592#1908409 (atgo) a:CaitVirtue [18:37:23] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Create CiviCRM naming convention rule: In general, records should not begin with "The" - https://phabricator.wikimedia.org/T122592#1908371 (atgo) assigning to you @caitvirtue because we'll need to come up with the convention before this moves to tech (i... [18:38:07] atgomez: dstrine: I've only found https://docs.google.com/document/d/1syqZcWXm1EovZLc8OeeDvMUJN1Ghr61sIqEu6TfzB88/edit but this doesn't look like our final decision [18:38:46] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Create tools to monitor the operation of CentralNotice for targeted groups of users - https://phabricator.wikimedia.org/T122589#1908417 (atgo) @andyrussg how do you see this interacting with the Dash T121121 ? [18:39:28] atgomez: dstrine: sorry I cannot read today. "What we will communicate" is exactly what I was looking for. [18:40:06] ahh got you [18:40:14] yeah… there's a mediawiki page, too [18:40:49] https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q3_Goals#Fundraising_Tech [18:40:51] awight dstrine ^ [18:41:53] word! [18:41:54] thank you [18:42:14] ooh, those are slightly different. [18:42:23] no. [18:42:28] I should take a nap. [19:01:25] Fundraising Sprint William Shatner, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 2 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#1908483 (Ejegg) @CCogdill_WMF, looks like @Jgreen just fixed it for us. Yo... [19:14:37] (PS1) Legoktm: build: Updating development dependencies [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/261409 [19:16:08] (CR) Awight: [C: 2] "Thanks!" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/261409 (owner: Legoktm) [19:19:34] Fundraising-Backlog, Wikimedia-Fundraising: PayPal donation link sometimes unresponsive - https://phabricator.wikimedia.org/T120995#1908529 (awight) @MBeat33 Please create a new one, cos it seems unrelated. You can mention "see also T120995" in the new task if you want to point to information here, etc... [19:20:33] (Merged) jenkins-bot: build: Updating development dependencies [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/261409 (owner: Legoktm) [19:26:21] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, FR-Smashpig: Rename GlobalCollect to Ingenico - https://phabricator.wikimedia.org/T122420#1908539 (awight) I agree that we need to make this change eventually. I've verified that we never show the GC/Ingen... [19:28:20] (PS1) Legoktm: build: Updating development dependencies [extensions/FundraisingTranslateWorkflow] - https://gerrit.wikimedia.org/r/261445 [19:31:16] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Change any donor-facing GlobalCollect branding to read "Ingenico". - https://phabricator.wikimedia.org/T122597#1908549 (awight) NEW [19:37:22] (PS1) Legoktm: build: Updating development dependencies [extensions/LandingCheck] - https://gerrit.wikimedia.org/r/261472 [19:51:07] (CR) Addshore: [C: 2] build: Updating development dependencies [extensions/FundraisingTranslateWorkflow] - https://gerrit.wikimedia.org/r/261445 (owner: Legoktm) [20:01:34] Fundraising-Backlog: PayPal 404 error: mobile and desktop - https://phabricator.wikimedia.org/T122603#1908628 (MBeat33) NEW [20:02:22] Fundraising-Backlog, Wikimedia-Fundraising: PayPal donation link sometimes unresponsive - https://phabricator.wikimedia.org/T120995#1908638 (MBeat33) Thanks, @awight, I created T122603 for this. [20:05:12] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MediaWiki-extensions-Translate, and 2 others: Translation tool showing zero translations in MessageGroupStats for CentralNotice banner - https://phabricator.wikimedia.org/T122251#1908645 (DStrine) [20:07:27] Fundraising Sprint Zapp, Fundraising-Backlog, Wikimedia-Fundraising, MediaWiki-extensions-CentralNotice, and 2 others: Invalidate "waitdate" status whenever the impression diet maximum impressions limit is increased - https://phabricator.wikimedia.org/T121178#1908660 (DStrine) [20:08:16] Fundraising Sprint Zapp, Fundraising-Backlog, FR-Adyen: Adyen donation messages missing payment submethod - https://phabricator.wikimedia.org/T117820#1908663 (DStrine) [20:08:35] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work: Silverpop data inaccurate - two donors with the same contact ID - https://phabricator.wikimedia.org/T120897#1908665 (Ejegg) [20:09:10] Fundraising Sprint Zapp, Fundraising-Backlog, Patch-For-Review: Deploy thank you letter to update postal address - https://phabricator.wikimedia.org/T120877#1908667 (DStrine) [20:09:55] dstrine almost there [20:10:01] (Merged) jenkins-bot: build: Updating development dependencies [extensions/LandingCheck] - https://gerrit.wikimedia.org/r/261472 (owner: Legoktm) [20:10:37] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: iframe not appearing for some people (confirmed modern browsers) - https://phabricator.wikimedia.org/T112181#1908670 (DStrine) a:Ejegg [20:13:45] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work: Silverpop data inaccurate - two donors with the same contact ID - https://phabricator.wikimedia.org/T120897#1908688 (DStrine) a:Ejegg [20:15:33] Fundraising Sprint Zapp, Fundraising-Backlog, Traffic, Unplanned-Sprint-Work, operations: Firefox SPDY-coalesces requests to geoiplookup over text-lb, causing GeoIP IPv6 failures - https://phabricator.wikimedia.org/T121922#1908701 (DStrine) [20:16:41] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, and 2 others: Globalcollect Status 25/404 errors - https://phabricator.wikimedia.org/T120030#1908702 (DStrine) [20:17:07] Fundraising Sprint Zapp, Fundraising-Backlog: Spike: Is it a problem that the Silverpop export is still not updating entire database? - https://phabricator.wikimedia.org/T120880#1908703 (DStrine) [20:18:50] Fundraising Sprint William Shatner, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 4 others: [EPIC] Build Frig: tools to help update our deployment branches - https://phabricator.wikimedia.org/T116820#1908715 (DStrine) [20:20:30] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Change any donor-facing GlobalCollect branding to read "Ingenico". - https://phabricator.wikimedia.org/T122597#1908718 (DStrine) [20:21:15] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Permissions issue? "You do not have permission to access this page" - https://phabricator.wikimedia.org/T122519#1908728 (DStrine) [20:24:46] Fundraising-Backlog: Adyen needs GB form - https://phabricator.wikimedia.org/T121036#1908751 (awight) [20:25:18] Fundraising-Backlog: Adyen needs GB form - https://phabricator.wikimedia.org/T121036#1908756 (DStrine) [20:26:36] Fundraising Sprint Yo La Tengo, Fundraising-Backlog, FR-GlobalCollect: WR1 audit parser emitting multiple bad refund lines - https://phabricator.wikimedia.org/T119927#1908757 (DStrine) [20:26:43] Fundraising-Backlog, FR-Adyen, FR-Smashpig: Process refunds in Adyen audit file - https://phabricator.wikimedia.org/T121129#1908759 (DStrine) [20:28:21] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM bounce import job is failing - https://phabricator.wikimedia.org/T122265#1908761 (DStrine) [20:29:43] Fundraising-Backlog, FR-Amazon: Clear up Amazon donations that got disconnected from CT ID - https://phabricator.wikimedia.org/T118296#1908763 (DStrine) [20:29:58] Fundraising-Backlog, FR-Paypal: Scope PayPal express checkout - https://phabricator.wikimedia.org/T113625#1908764 (DStrine) [20:31:46] (Merged) jenkins-bot: build: Updating development dependencies [extensions/FundraisingTranslateWorkflow] - https://gerrit.wikimedia.org/r/261445 (owner: Legoktm) [20:31:46] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Fix remaining uses of "contribution_type"* - https://phabricator.wikimedia.org/T120424#1908765 (DStrine) [20:33:39] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, FR-Smashpig: [Epic] Reusable log parsing mechanisms - https://phabricator.wikimedia.org/T122611#1908767 (awight) NEW [20:33:41] Fundraising-Backlog, FR-Amazon: Clear up Amazon donations that got disconnected from CT ID - https://phabricator.wikimedia.org/T118296#1908776 (awight) [20:33:44] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, FR-Smashpig: [Epic] Reusable log parsing mechanisms - https://phabricator.wikimedia.org/T122611#1908775 (awight) [20:33:46] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-GlobalCollect: GlobalCollect audit: lots of "impossible" errors for refunds - https://phabricator.wikimedia.org/T120430#1908778 (DStrine) [20:59:35] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising Tech Backlog, Fundraising-Backlog, and 2 others: Spike [2 hours]: QuickSurveys deployment process - https://phabricator.wikimedia.org/T119632#1908817 (Jdlrobson) Have asked @jhobs to create a task to track this work. I want to u... [21:28:43] Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work: Silverpop data inaccurate - two donors with the same contact ID - https://phabricator.wikimedia.org/T120897#1908890 (Ejegg) @CCogdill_WMF, the only duplicated contact IDs I see in the export now ar... [21:31:44] dstrine: gotta run a quick errand--maybe we could IRC chat about what you mentioned in the meeting in about 30-40 min? [21:31:59] AndyRussG: sure [21:33:38] hey ejegg - we're making a move to switch to a different fully hosted thing for cc by astropay - they say it's no change on our side, just config on theirs [21:33:57] oh really? [21:34:03] anything we need to worry about or is it fine if we just let them pull the trigger? it's not an active campaign so i'm not suuuper worried about it even if it goes a bit weirdly [21:34:10] yeah let me fwd from fr-tech [21:34:24] oh sorry, I can look for that [21:34:39] dstrine: K thx! [21:38:08] atgomez: sorry, don't see it - did you say it went to fr-tech? [22:01:40] should we be concerned about the smashpig/failmail emails? [22:02:36] awight: cwd|afk K4-713 ejegg XenoRyet ^? [22:02:46] looking [22:02:50] kk [22:03:09] yeah, that's bad... [22:03:23] if the queue's dead we'll need to take down campaigns [22:04:40] oh god the emails [22:05:03] ejegg: sorry forgot to send [22:05:04] I am at lunch sans computer [22:06:31] ejegg: are you able to help or do we need others? awight perhaps? [22:08:06] dstrine: I think I can handle it, just confirming the issue [22:08:25] cool let me know if I can help call anyone [22:08:28] holy failmail batman [22:09:16] ejegg: Let me know if i can help [22:09:27] should we send a heads up to fr-online? [22:09:46] yeah, please do! Just taking down campaigns [22:11:27] my donation is stalling [22:12:54] ejegg: sent a very barebones email to fr-all [22:13:35] ty [22:14:49] Fundraising Sprint Zapp, Fundraising-Backlog, Unplanned-Sprint-Work: 1 million failmails (fill this out more when we know more) - https://phabricator.wikimedia.org/T122619#1908985 (atgo) NEW a:Ejegg [22:14:57] !log shut down campaigns and scheduled jobs due to dead ActiveMQ [22:15:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:15:02] thanks for taking campaigns down, I was just coming online to do that [22:15:10] my phone went crazy :/ [22:15:12] MBeat: heads up, campaigns are down and processing is broken [22:15:28] saw that, many thx for the alert [22:18:10] fwiw donors at front of queue report the 25/404 error from T122603 [22:21:16] just texted jeff, he's driving but will be online to help revive the queue in 15 [22:27:15] cmjohnson1: are you familiar with / comfortable with restarting activemq? Jeff will be back in 10 min, so no worries if not [22:27:35] ejegg: probably best to wait for jeff...sorry [22:27:39] k, cool [22:30:52] oh, world is on fire! [22:30:55] dstrine: hi! drat that took much longer than I thought 8p Still around? [22:31:21] AndyRussG: I was just going to suggest looking through your tasks inside the sprint and adding esmations [22:31:40] we had passed by them during the meeting [22:32:12] hi awight .. welcome back :) [22:33:07] dstrine: ah K you mean then the past ones? I don't see anything pending right now... It's Zapp still, right? [22:33:52] make sure you are using the sprint link: https://phabricator.wikimedia.org/project/sprint/board/1661/ [22:34:10] dstrine: ah k yeah I was gonna ask [22:47:52] did banners get taken down? [22:47:56] Jeff_Green: looks like the admin interface is back up - want me to try restarting a job or two? [22:48:05] ejegg: sure [22:48:12] Jeff_Green: yep, I took em down about 5 min after the queue died [22:48:23] you can put em back online [22:48:37] it looked like last time [22:48:40] Cool, will do [22:48:58] do you know if there was a large job running or something? [22:49:16] Nothing that stands out [22:50:10] !log restarted fundraising scheduled jobs [22:50:34] something must have changed for this year, crashing has never been a problem before [22:50:34] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:51:07] The new Amazon integration is definitely harder on the queue than before [22:51:38] how much harder? [22:52:50] wow, ganglia graphs are rather telling [22:53:22] what about banner history? [22:54:10] banner history graph skyrocketed leading up to the outage [22:54:40] I'll write up the outage in a bit, gotta go afk [23:00:00] ooh, maybe that too [23:00:01] see ya [23:00:22] !log re-enabled fundraising banner campaigns [23:00:26] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [23:00:38] Thanks ejegg and Jeff_Green ! [23:00:45] +1 [23:04:23] thanks Jeff_Green [23:04:33] and everyone :) [23:11:48] If nothing else happens, I'm going to head out in like... 30 minutes [23:16:53] word [23:21:51] i'm running out for a bit - but will be back online later probably [23:42:14] Argh. [23:42:27] So many things happen when one is stuck in a meeting. [23:47:33] Fundraising Sprint Zapp, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work: Donors still don't know payments succeeded when queue is dead. - https://phabricator.wikimedia.org/T122627#1909335 (Ejegg) NEW