[00:06:28] I thought it was amusing that the CS's "flock has dwindled"... I was imagining all kinds of mild, unmedicated headaches that might have wreaked devastation on them [00:06:49] ejegg: U have a chance to look at the horrific silverpop hack? [00:08:24] yeah! [00:09:02] is there anything we can do to make sure the old query isn't still running after the exception [00:14:19] ejegg: hmm, I don't even know if there can be more than one query per db connection [00:14:43] what's our current approach for killing that dangling thread? [00:15:02] we've got a shutdown handler registered [00:15:06] I suppose we could run http://dev.mysql.com/doc/refman/5.0/en/kill.html [00:15:06] ah damn [00:15:09] oh [00:15:15] i just realized your thing won't actually work [00:15:20] mm? [00:15:26] because it's jenkins killing the job [00:15:29] after a timeout [00:15:37] not a python-level db timeout [00:15:37] yeah I was gonna boost the timeout [00:15:45] oh. the timeout. we're screwed [00:16:07] does the python mysql driver support query timeouts? [00:16:17] probably, if we set a session variable [00:16:30] but that seems like a destabilizing change [00:16:41] it could lead to much more frequent fail and stuff [00:16:51] how about we make the query timeout the length of the jenkins timeout? [00:16:58] and then triple the jenkins timeout? [00:17:02] Well, nah, yeah we could just make it 3x the longest query [00:17:06] jinx [00:17:09] heh [00:19:41] OK lemme try a session timeout [01:07:56] Wikimedia-Fundraising, Fundraising Tech Backlog, Fundraising-Backlog, Fundraising Sprint Flaming Lips, Patch-For-Review: Silverpop export is still broken - https://phabricator.wikimedia.org/T92537#1118559 (awight) OK, we still don't have a good solution. @ejegg pointed out that the error is ac... [01:22:43] Wikimedia-Fundraising, Fundraising Tech Backlog: Regression: interstitial page with PayPal even when the amount is good. - https://phabricator.wikimedia.org/T92705#1118572 (awight) NEW [10:15:49] (CR) Awight: [C: -1] TEMPORARY: retry Silverpop export job 3 times [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/196508 (https://phabricator.wikimedia.org/T92537) (owner: Awight) [10:16:48] (CR) Awight: [C: 2] Correct language code for Norwegian aggregation [wikimedia/fundraising/tools/DjangoBannerStats] - https://gerrit.wikimedia.org/r/195612 (owner: Pcoombe) [10:17:49] (CR) Awight: "Thanks! But I wish I wasn't sitting here wondering what special case "detail_languages" might be about..." [wikimedia/fundraising/tools/DjangoBannerStats] - https://gerrit.wikimedia.org/r/195612 (owner: Pcoombe) [10:18:20] (CR) Awight: [V: 2] Correct language code for Norwegian aggregation [wikimedia/fundraising/tools/DjangoBannerStats] - https://gerrit.wikimedia.org/r/195612 (owner: Pcoombe)