[00:00:11] <mwalker>	 aye; but I'm not sure if there's a good way to prevent that
[00:00:18] <awight>	 Sure, global counters...
[00:00:38] <awight>	 switch to digest mode at some threshold...
[00:00:58] <mwalker>	 in php?
[00:01:07] <mwalker>	 then you're logging to memcache or something
[00:01:24] <awight>	 right, something horrible
[00:01:33] <awight>	 but probably marginally better than 10,000 failmails
[00:01:49] <mwalker>	 I can filter failmail though
[00:01:56] <awight>	 ona phone :p?
[00:02:09] <awight>	 so, lemme revisit your SP approach
[00:02:34] <mwalker>	 heh; what would you be able to do on a phone anyways?
[00:02:41] <awight>	 run away!
[00:02:50] <mwalker>	 filter on phone == turn it off
[00:03:29] <mwalker>	 anyways, smashpig sends failmail (with context) at the end of the session if it saw any errors, or immediately if it was a critical
[00:03:37] <awight>	 yes that's very nice
[00:03:52] <mwalker>	 and I thought that the python logger could do that too
[00:04:06] <mwalker>	 it was either monolog or python logging that I took inspiration from
[00:04:07] <awight>	 this is... exactly why it sucks to reinvent for every codebase
[00:04:48] <awight>	 How about: I am in 90% agreement, but this is a step up from what we have now: crash catastrophically if any exceptions are thrown
[00:05:32] <awight>	 My goal right now is, get the paypal audit script running again, no matter what trash they throw under our wheels
[00:05:43] <mwalker>	 fair nuf
[00:06:56] <awight>	 I think we're pretty close to defining the best-practice answer for https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/1644
[00:18:36] <grrrit-wm>	 (PS2) Adamw: Failmail module [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275
[00:23:56] <mwalker>	 awight, yaml?
[00:24:03] <mwalker>	 why that over json?
[00:24:10] <mwalker>	 *asking because we use json a lot of other places
[00:27:18] <awight>	 mwalker: I played with json, pformat, and yaml
[00:27:29] <awight>	 what u see gave the clearest output
[00:27:35] <awight>	 alphabetized the keys
[00:27:44] <awight>	 and had minimal punctuation drain bamage
[00:27:57] <awight>	 and, it's still a format that could be piped back into code if we need to
[00:50:06] <grrrit-wm>	 (CR) Mwalker: Failmail module (2 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[00:50:13] <grrrit-wm>	 (CR) Mwalker: [C: -1] Failmail module [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[00:51:01] <grrrit-wm>	 (CR) Mwalker: [C: 2] Failmail on recoverable errors during PayPal audit [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134276 (owner: Adamw)
[00:52:44] <grrrit-wm>	 (PS1) Adamw: use stompy rather than stomp.py :'-( [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134286
[00:52:48] <awight>	 mwalker: ^^
[00:55:07] <grrrit-wm>	 (CR) Mwalker: [C: -1] "I'm confused; you got rid of the serialization/deserialization -- does something do that for you?" (2 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134286 (owner: Adamw)
[00:55:21] <awight>	 oh dear
[00:55:23] <awight>	 no, thx
[00:56:21] <grrrit-wm>	 (PS2) Adamw: use stompy rather than stomp.py :'-( [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134286
[00:59:50] <grrrit-wm>	 (CR) Mwalker: [C: 2] "yay; thanks for doing this" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134286 (owner: Adamw)
[01:05:43] <grrrit-wm>	 (CR) Adamw: Failmail module (2 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[01:08:10] <grrrit-wm>	 (PS3) Adamw: Failmail module [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275
[01:09:29] <grrrit-wm>	 (CR) Adamw: "oh heh: untested." [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134286 (owner: Adamw)
[01:09:57] <grrrit-wm>	 (CR) Mwalker: Failmail module (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[01:11:16] <grrrit-wm>	 (CR) Adamw: Failmail module (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[01:12:34] <grrrit-wm>	 (CR) Mwalker: Failmail module (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[01:12:41] <grrrit-wm>	 (CR) Mwalker: [C: 2] Failmail module [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/134275 (owner: Adamw)
[18:21:40] <atgomez>	 ccogdill for the email called "DO_PAYMENT APIs for order IDs 5287274942, 6911151466 and 9976813120" were those all DD?
[18:22:01] <ccogdill>	 ummm let me remind myself
[18:30:08] <ccogdill>	 hey atgomez, sorry something came up!
[18:30:08] <ccogdill>	 but no, not all DD
[18:30:08] <ccogdill>	 5287274942
[18:30:08] <ccogdill>	 is mastercard
[18:30:08] <atgomez>	 ok
[18:30:22] <ccogdill>	 6911151466 is visa
[18:30:33] <ccogdill>	 and 9976813120 visa again
[18:30:36] <atgomez>	 are any dd?
[18:31:03] <ccogdill>	 not of those 3. do you want to know if *any* rejected recurring are?
[18:31:18] <ccogdill>	 fwiw, we think this is gc’s issue
[18:31:26] <ccogdill>	 they’re just taking forever to get back to us
[18:31:30] <atgomez>	 we do? i thought that it looked like we were retrying the hard fails?
[18:31:45] <ccogdill>	 oh, maybe that part is us
[18:31:54] <ccogdill>	 it’s just we don’t think these should be failing in the first place
[20:18:05] <sumanah>	 mwalker: again offering my availability offer my availability today, or on Thurs, or Fri of this week to work on improving https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy .