[00:25:14] Jeff_Green: modsec difference between precise and trusty...sounds fun to track down [00:30:37] blargh [00:30:59] this day (and part of yesterday) was a bust becuase of that [00:31:56] cwd: any traction on slaying referrer= ? [00:32:01] i just discovered some code that i can't understand how it's working in production [00:32:13] where? [00:32:28] it's deep in the php mess [00:32:39] https://github.com/wikimedia/mediawiki/blob/master/includes/Title.php#L1632 [00:33:17] in production it's (apparently) working as if that was set to PROTO_CURRENT instead of RELATIVE [00:33:39] ugh [00:34:01] anyway, i did bring up referrer again but i didn't get any traction [00:34:15] is PROTO_RELATIVE interpreted there as a string or variable? [00:34:28] it's a constant [00:34:32] set to // [00:34:35] i see [00:34:52] something funny is going on and it fried my brain [00:35:04] i know how that feels :-) [00:35:18] anyway regarding referrer, i would personally like to push for not saving it at all [00:35:51] unless we *really* need it for something, it seems like a liability to have around [00:36:34] maybe we could like, a/b test it and see if anything explodes [00:36:52] i can probably make the proxy remove it on the way in :-) [00:37:06] ya [00:37:18] even stripping the query string off would help immensely [00:37:29] i mean they remove it on the main cluster so i think the precedent is there [00:37:35] at least then it would be sort of feasible to write a regex [00:37:56] the only thing that still jumps out to me is article titles with weird characters [00:38:05] jp wiki for instance [00:38:19] i can work with that I think, sorta [00:38:31] oh right, they were categorized as unicode word chars? [00:38:53] yeah [00:39:08] that's really helpful [00:39:14] thanks unicode consortium [00:39:16] I'm not sure what other characters are valid for article titles [00:40:02] parens come to mind [00:40:03] definitely preferable to get rid of it entirely, or just use the site domain name or something [00:40:14] yeah, and spaces [00:40:59] basically if I can write a regex that is a convincing test for non-maliciousness, i can whitelist [00:43:39] it seems like they want article title [00:43:55] it seems a little far fetched to me [00:44:14] i bet i a lot of people click donate on the hitler article [00:44:24] but i am no statistician [00:44:58] i can see why it would be useful to collect stats on which articles people donate from [00:45:35] but it seems like we should do that with a subrequest or something that doesn't flow through the payments pipeline [00:46:04] that's an interesting idea [00:46:42] short term i can't imagine how stripping the query string would hurt anything [00:46:42] it goes banner->donatewiki->paymentswiki right? could we collect it at donatewiki? [00:47:16] is that feasible? [00:47:19] Jeff_Green: i think the problem is some banners go right to payments and skip donate [00:47:24] there was a task about this... [00:47:43] i see [00:47:56] https://phabricator.wikimedia.org/T131792 [00:48:38] i guess we don't record stuff there because the servers that hold this data are pci scope? [00:48:51] and donate is not [00:49:26] thwarted by 2FA: my phone is at the other end of the house, too lazy to deal [00:49:52] we don't record stuff on donate because of PCI? [00:50:23] hehe [00:50:33] isn't contribution_tracking in a pci scope server? [00:50:41] and if so, isn't donate on the main cluster? [00:50:47] so i thought they weren't allowed to talk [00:50:47] donate is on the main cluster [00:51:41] donate-wiki traffic should be collectible on the prod analytics cluster, maybe the issue is that we don't want to risk collecting PII there [00:54:47] yeah, i'm guessing the issue is all the machinery that looks for it in the drupal db [00:55:07] i don't really know the scope of that [00:55:20] oic [00:55:40] but if stripping the query string and using a unicode regex is viable, that sounds pretty low overhead [00:56:13] that would certainly help [00:56:34] just to validate my assumptions, donate cannot write to drupal.contribution_tracking right? [00:56:43] right [00:57:04] i wonder how many things actually look for referrer there [00:57:05] we would have to collect logs via analytics, and consume them [00:57:09] might not be that many [00:57:30] there's already a whole pipeline for dragging production logs into fundraising that way [00:58:13] hmm, i wonder who would know the extent to which code would have to change if we relocated that value [00:58:17] ellery? [00:58:27] honestly I don't understand how having referrer would be useful at the payments server [00:58:55] i guess one data point we lose is if the donation was completed [00:58:58] or not [00:59:19] true [01:00:52] removing the query string does seem like the simplest approach. i can make a patch for that right now. [01:01:11] _but_ i also don't know what valid referrers would contain a query string [01:02:40] i can mine that data [01:03:32] for instance, you posted one that had all the args from the banner url. is that because someone clicked a banner, then clicked the url bar and hit enter? [01:04:28] ah, good question, i have no idea [01:04:33] i don't understand why someone on payments would have a referrer of payments [01:05:00] especially because it's not even the real referrer we're talking about but the get param [01:05:05] right [01:05:29] maybe the payments code has a default if there's no argument? [01:05:43] and then it would populate the argument and keep dragging it through the process? [01:06:19] that could be, it's a rats nest [01:06:48] if might re-harvest it from the header if it's not donatewiki [01:07:14] yawp [01:07:14] but in any case, whether the data is junk or not stripping the query should be fine [01:07:55] barbaric [01:09:32] upon a cursory inspection i see 4 places it is getting set [01:09:35] * cwd sigh [01:10:05] referrer=www.google.com%2Furl%3Fq%3Dhttps%3C3A-uy%3E2F%3C2Fdonate%3E2F%3C3Futm_campaign-uy%3E3... [01:10:27] wat [01:10:39] i know :-) [01:10:50] did they...index a banner? [01:11:07] i dunno [01:11:15] baffling [01:12:58] utm_campaign-uy [01:13:06] that's not even our url [01:14:04] i think that's an artifact of encoding horrors [01:14:23] brutal [01:14:34] well if google accidentally indexed WP donate links...great! [01:15:25] jfyi to gauge the impact of this, I blindly whitelisted referrer= for most of the rules, without that the firehose [01:20:41] i'm heading out, have a good night! [01:20:42] Jeff_Green: so what i see in email right now is with referrer whitelisted? [01:20:47] yeah [01:20:53] word [01:20:56] me too! [01:20:58] have a good one [01:20:59] mostly whitelisted, there are some rules I didn't bother with [15:17:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: is it possible to correct obvious email address typos before deduping exact matches? - https://phabricator.wikimedia.org/T135042#2305213 (MBeat33) [15:38:49] Fundraising-Backlog, FR-Adyen: Adyen refunds: look at reconciliation / auditing to make sure they get to Civi - https://phabricator.wikimedia.org/T135641#2305421 (MBeat33) [16:02:48] hi cwd! [16:03:08] mornin! [16:05:05] hey, would you mind filling in for me at Scrum of Scrums? [16:05:26] I'm curious about those bias workshops, but I missed the last two [16:06:13] sure i can do that [16:06:24] thanks! [16:06:29] you have the links? [16:06:37] np! yeah i have them somewhere... [16:06:50] awesome [16:07:12] for now, want to get back to that CR swap? [16:07:53] i was just checking on gerrit [16:08:11] i sent you an email last night about that url issue, i am totally baffled [16:08:44] oh, let me read that... [16:10:23] cwd yeah, so $proto is still defaulting to // for me [16:10:53] but due to quirks of wfExpandUrl, it never actually uses the $proto argument in some cases [16:11:22] when getLocalURL returns something starting with a single slash, it prepends $wgServer instead [16:11:49] hmm, I should get debugging working with vagrant, lemme see if that [16:11:58] 's not too big a time sink [16:18:20] (CR) Cdentinger: [C: 2] More 'token' cleanup [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289096 (owner: Ejegg) [16:35:43] cwd found it [16:36:03] wgServer is getting overriden with relative protocol [16:36:07] /vagrant/settings.d/wikis/paymentswiki/wgConf.php:$wgConf->settings['wgServer']['paymentswiki'] = '//payments.wiki.local.wmftest.net:8007'; [16:36:35] omg [16:36:39] WHY [16:37:18] That whole 'crap buried in settings.d overrides what you thought you set in LocalSettings' thing has bitten this team a bunch of times now... [16:38:42] what even is that settings.d directory about? [16:39:11] partly so vagrant can manage some files and let you mess with others [16:39:21] and partly so you can split on conf files for different roles [16:39:26] *split out [16:39:51] what applies these settings? [16:40:09] at the end of mediawiki-fr/LocalSettings.php [16:40:16] it includes /vagrant/LocalSettings.php [16:40:51] brutal [16:41:11] yeah, that setup needs some big warning signs [16:42:57] But still... I think your patch has merit. For URLs we might be sending off to third parties, we never want to use // [16:43:25] no matter what wgServer is set to [16:43:43] that's true [16:43:52] Wanna un-WIP it? [16:44:02] do you know what the rationale is for resetting wgserver on vagrant? [16:44:04] yeah sure [16:44:09] no freaking clue [16:44:17] oh, you know what? [16:44:37] I bet the mediawiki-fr/LocalSettings was just copied in there first [16:45:12] then someone figured out more of the settings stuff in the multiwiki role config and added things there [16:45:21] without removing them from the LocalSettings template [16:45:36] I'll add a phab task [16:48:08] sounds plausible [16:49:17] Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, Hovercards: CentralNotice: test registration - https://phabricator.wikimedia.org/T134286#2305799 (jrobell) Quick question @DStrine and @AndyRussG : do we have an idea on when this task will be done? The reading team should have their deploy... [16:49:19] Fundraising-Backlog, MediaWiki-Vagrant, MediaWiki-extensions-DonationInterface: Payments vagrant role has duplicated settings, needs more documentation - https://phabricator.wikimedia.org/T135653#2305801 (Ejegg) [16:49:37] gonna grab a quick bit before that workshop... [16:49:46] (PS3) Cdentinger: Use PROTO_CURRENT for paypal urls [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289102 (https://phabricator.wikimedia.org/T131811) [16:58:11] fundraising-tech-ops: test jessie for payments servers, upgrade servers to jessie - https://phabricator.wikimedia.org/T135562#2305857 (Jgreen) [16:58:14] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#2305855 (Jgreen) [16:59:32] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#1572640 (Jgreen) [17:00:30] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#2305865 (Jgreen) [17:02:01] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#1572640 (Jgreen) [17:11:14] Jeff_Green: https://phabricator.wikimedia.org/T135439 [17:11:20] want me to merge that into the new one? [17:11:36] fundraising-tech-ops: Frack (boron and bismuth) access for Darian Patrick - https://phabricator.wikimedia.org/T135165#2305912 (Jgreen) Darian can you please: 1) create a new/separate SSH keypair for use in fundraising, and send me the public key 2) send me your Yubikey public ID (first 12 characters of any... [17:25:35] cwd I've got no news for scrum of scrums btw [17:26:00] ejegg|mtg: word...guess you're the only one to ask! [17:30:48] ejegg|mtg: can i get that sprint of sprints link from you? [17:30:59] nobody is in the meeting i just joined, wondering if it changed [17:31:09] oh, one sec [17:31:13] i think it did [17:35:39] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#2306050 (Jgreen) >>! In T110227#2303224, @Jgreen wrote: > After a whole lot of testing and tweaking, I'm pretty much at the conclusion that owasp-cr... [17:49:46] Fundraising-Backlog: Change paymentswiki session cookie name - https://phabricator.wikimedia.org/T135439#2306122 (cwdent) [18:17:40] (CR) Ejegg: "Looking pretty good! Just a few comments/suggestions" (6 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286117 (https://phabricator.wikimedia.org/T131811) (owner: Awight) [18:24:39] (Merged) jenkins-bot: More 'token' cleanup [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289096 (owner: Ejegg) [18:25:05] (CR) Ejegg: "staging it, but still not sending it - should this patch also add PAYMENTINFO_0_ORDERTIME to one or more of the txn structures?" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/287171 (owner: Awight) [18:26:17] Aaarg! Finally got to a usb-based system with network access and where I'm able to access my disks, but now my old system (where I was gonna copy the disks w/ ddrescue) got hosed due to upgrade... [18:27:02] aww man [18:27:44] So near and yet so far!! [18:28:12] how hosed is the old system? [18:28:34] Yeah I had to upgrae that old bugger 'cause it had an old scsi driver which was apparently (not sure that was it) not writing to the usb pen drive correctly [18:28:49] No network card, dpkg errors, upgrade incomplete [18:28:56] I mean, network card not recognized [18:29:17] wow, weird, must have been a pretty darn old driver not to work w usb [18:29:42] It worked, but some low-level thing of removing partitions on this particular type of usb key was borked [18:29:51] I dunno [18:29:55] It might not have been that [18:30:37] huh, i'm kinda unclear where responsibility for partitions lay [18:30:38] If I can just get eth0 going now in recovery mode on the old system, at least I'll have a way to scp disk images before fsck'ing them [18:31:02] i guess i thought the scsi/whatever driver just provided blocks [18:31:13] Hmmm [18:31:15] No idea really [18:31:45] I kept being unable to write a new image to the usb key, the old image just wouldn't go away [18:32:11] So I downloade a tool especially for manipulating proprietary SanDisk whatever, called u3 [18:32:14] have you tried unetbootin? [18:32:22] I did try that.... [18:32:30] no dice, huh? [18:32:41] u3-tool complained that the scsi driver was too old, that's why I upgraded the old system [18:32:49] ah, gotcha [18:33:14] Mmm I did get past that point, finally now back on the Asus running off Debian live on the USB dongle [18:33:44] And the commands that I had used with the Debian installer rescue CD to assemble/decrypt etc the disks worked [18:33:55] And here I can get ssh, scp [18:34:04] I just need someplace to copy stuff to [18:45:11] (CR) Ejegg: "Found one more place that could use this: globalcollect.adapter.php line 67" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289102 (https://phabricator.wikimedia.org/T131811) (owner: Cdentinger) [18:45:48] (PS2) Ejegg: Prevent HTTP continuations [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/288256 (owner: Awight) [18:47:30] (CR) Ejegg: "might want to put this in the base function, in case any other adapter starts sending >1k requests and getting confused at the extra respo" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/288256 (owner: Awight) [19:01:56] (PS1) Ejegg: Avoid 'insert ignore' warning with no-op [wikimedia/fundraising/tools/DjangoBannerStats] - https://gerrit.wikimedia.org/r/289499 [19:15:53] woohoo got networking back on my old box... [19:38:10] congrats AndyRussG ! [19:41:50] (PS1) Ejegg: Detect email typos, suggest correction [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) [19:42:35] cwd, there's a pretty trivial email export patch to review: https://gerrit.wikimedia.org/r/289247 [19:43:09] If that merges, I can deploy & test whenever, and cross one off for this sprint [19:44:10] Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, Patch-For-Review, Unplanned-Sprint-Work: Catch email address typos early - https://phabricator.wikimedia.org/T135567#2306634 (Ejegg) p:Triage>Low a:Ejegg [19:46:34] sure! [19:51:25] (CR) Cdentinger: [C: 2] Fix encoding and paging on unsubscribe table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/289247 (https://phabricator.wikimedia.org/T135204) (owner: Ejegg) [19:51:32] (CR) Cdentinger: [V: 2] Fix encoding and paging on unsubscribe table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/289247 (https://phabricator.wikimedia.org/T135204) (owner: Ejegg) [19:53:30] thanks! [19:54:51] np! [19:55:11] again i have to admit i don't know enough about that repo to have a real opinion on the change [19:55:27] but i doubt it will break anything [19:56:03] yeah, it ran ok on my test data [20:00:50] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, Patch-For-Review: Clicktracking data not matching up with donation totals - https://phabricator.wikimedia.org/T132500#2306781 (Ejegg) Deployed this yest... [20:04:31] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, and 2 others: Only subscribe primary emails, secondary addresses should be suppressed - https://phabricator.wikimedia.org/T131979#2306785 (Ejegg... [20:24:29] Fundraising Tech Backlog, Fundraising-Backlog: another round of activemq queue decrufting needed - https://phabricator.wikimedia.org/T134916#2306838 (DStrine) [20:29:15] (CR) Cdentinger: [C: 2 V: 2] Avoid 'insert ignore' warning with no-op [wikimedia/fundraising/tools/DjangoBannerStats] - https://gerrit.wikimedia.org/r/289499 (owner: Ejegg) [20:49:51] (CR) Anomie: [C: 1] Replace usage of wfSetupSession() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/282479 (https://phabricator.wikimedia.org/T132251) (owner: Florianschmidtwelzow) [20:50:36] (CR) Cdentinger: "Since Ibed8c432bb879a2583c0bc95a4f6d7d09d55ae62 merged we shouldn't have to worry about this. Curl's behavior is RFC compliant so I tend " [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/288256 (owner: Awight) [21:25:16] !log updated DjangoBannerStats from 8172c4176ea2b78df0dccbfa052064f2739c64bd to 220f80ece976aecbfbfbe1cb476d79910eb48ff4 [21:25:25] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:30:27] AndyRussG: the banner history log ids are 16 hex digits [21:30:55] I figure I'll allow 16-20 hex digits in case they expand that stuff a little bit [21:31:30] (PS1) Cdentinger: remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 [21:31:53] (PS2) Cdentinger: remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) [21:33:35] (CR) jenkins-bot: [V: -1] remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) (owner: Cdentinger) [21:36:17] (PS3) Cdentinger: remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) [21:38:39] (CR) jenkins-bot: [V: -1] remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) (owner: Cdentinger) [21:42:22] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, Patch-For-Review: Clicktracking data not matching up with donation totals - https://phabricator.wikimedia.org/T132500#2307244 (CCogdill_WMF) Thanks for... [21:42:50] (PS4) Cdentinger: remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) [21:42:59] this is really annoying [21:43:17] Fundraising Sprint Ghostbusting , Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, and 2 others: Only subscribe primary emails, secondary addresses should be suppressed - https://phabricator.wikimedia.org/T131979#2307246 (CCogd... [21:43:49] (CR) jenkins-bot: [V: -1] remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) (owner: Cdentinger) [21:44:15] go me [21:44:54] (PS5) Cdentinger: remove query string from referrer [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289563 (https://phabricator.wikimedia.org/T110564) [21:45:17] (PS1) Ejegg: Validate banner history messages before insert [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289566 (https://phabricator.wikimedia.org/T134916) [22:10:22] (PS2) Ejegg: Validate banner history messages before insert [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289566 (https://phabricator.wikimedia.org/T134916) [22:10:39] good thing i added those tests... [22:12:22] (CR) jenkins-bot: [V: -1] Validate banner history messages before insert [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289566 (https://phabricator.wikimedia.org/T134916) (owner: Ejegg) [22:13:08] ejegg: yay to past-you [22:13:41] hmm, tests don't seem to be including the expected files tho [22:16:25] oh, i bet the test harness doesn't turn on the banner history module [22:19:25] (CR) Cdentinger: "Lib is a little weird, does not complain about some values, e.g. @@@@@ggggg, I guess because it has no suggestions? Also suggests some in" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) (owner: Ejegg) [22:19:53] (PS1) Ejegg: Add banner history module to wmff setup [wikimedia/fundraising/civicrm-buildkit] - https://gerrit.wikimedia.org/r/289571 [22:20:28] ^^ blocking the thing that will let us clear out the banner history log queue [22:20:37] AndyRussG: got a couple patches: [22:20:55] https://gerrit.wikimedia.org/r/289571 just lets us run tests on your module [22:21:33] and after that merges, https://gerrit.wikimedia.org/r/289566 banner message validation should pass [22:21:59] Do you think we should set the BANNER_HISTORY exception to no_email ? [22:23:20] (CR) AndyRussG: "Looks good so far, thx much!!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289566 (https://phabricator.wikimedia.org/T134916) (owner: Ejegg) [22:23:53] ejegg: hmm intersting question [22:24:18] Can you make it only e-mail after a certain non-emailing quota is surpassed? [22:26:38] (CR) Ejegg: "This is more to fix almost-valid things than to stop anybody from intentionally entering malformed data. There's some light validation on" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) (owner: Ejegg) [22:27:06] AndyRussG: not yet, but that might be a nice feature [22:32:35] Hmm :) K, well, not sure then... So far as I know, this is the only time we've seen something like this. Maybe something Jeff G would like to weigh in on? [22:34:27] (CR) Ejegg: "blocked on I68cdbfd6e950aad7c7bb9 (turn on banner history module in crm test setup)" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289566 (https://phabricator.wikimedia.org/T134916) (owner: Ejegg) [22:40:21] (CR) Ejegg: [C: -1] "If you're moving session setup out of the constructor, please also move the call to getCNSessionVar that follows it." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/282479 (https://phabricator.wikimedia.org/T132251) (owner: Florianschmidtwelzow) [22:41:51] have you ever removed yourself as a reviewer from a patch because it messed with the column widths on your gerrit homepage? [22:42:44] no I have not... ;p [22:44:23] hehe... long gerrit username, that ^ [22:45:46] (PS2) Cdentinger: Detect email typos, suggest correction [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) (owner: Ejegg) [22:48:59] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, Patch-For-Review: Clicktracking data not matching up with donation totals - https://phabricator.wikimedia.org/T132500#2307444 (Ejegg) @CCogdill_WMF : yo... [22:52:26] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising-Backlog, Patch-For-Review: Clicktracking data not matching up with donation totals - https://phabricator.wikimedia.org/T132500#2307448 (CCogdill_WMF) @Ejegg see... [22:52:32] laterz.... [22:52:46] (CR) Cdentinger: [C: 2] "That does seem valid then and this should help. Also, TIL asdf@ca *is* a valid email address, albeit not on the public internet: http://t" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) (owner: Ejegg) [22:54:41] (Merged) jenkins-bot: Detect email typos, suggest correction [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/289505 (https://phabricator.wikimedia.org/T135567) (owner: Ejegg) [22:58:20] gotta bounce, back later! [22:58:29] cya [22:59:52] actually, me too...