[01:13:53] let's go, Cleveland! (@maxbinder) :baseb [01:13:59] ⚾️ [02:16:45] jaufrecht: I used to be willing to root for the dodgers you know [02:52:05] RECIPROCITY IS NOT PART OF FANHOOD [04:18:47] jaufrecht: yea, well, phlogiston's new feature doesn't work, so you're still down 1 [04:19:04] it worked fine when I shipped it [18:41:02] jaufrecht: should I file a bug? Or do you wanna sit down and look at it? [18:53:28] file a bug please so I can track and remember it [18:53:34] does it work on dev? [19:27:47] jaufrecht: no, dev has the same issue [19:28:00] jaufrecht: just looks like it was never deployed (or my config is wrong) [19:28:31] checking [19:29:37] jaufrecht: actually, it does seem to work on the burnup charts, but the point was for it to work on status report [19:30:58] jaufrecht: hard to tell on the visualization tho [19:31:03] The category table in the database for both dev and prod is the new table [19:31:29] what's your example case? [19:33:57] side note: the explanation (http://phlogiston.wmflabs.org/red_rules.html) hasn't been updated to handle omit [19:39:29] jaufrecht: example - red recat says omit tasks in "Upcoming" column as well as the intersection of #Category and #Readers-Web-Kanban-Board and tasks from both of those criteria still appear in the Status Report [19:39:47] jaufrecht: example task: https://phabricator.wikimedia.org/T166714 [19:40:49] and we know that the "omit" rule is catching it because it's categorized as Kanban Board (Other) [19:41:00] so it shouldn't be possible to both be in that category, and visible at all anywhere in the report [19:41:05] hmm [19:41:08] definitely file a bug [19:41:33] I think I fixed the rules.html generator to make it easier to debug these, so I kicked off a new report for red on dev [19:42:03] jaufrecht: cool, thx, filed https://phabricator.wikimedia.org/T177656 [19:45:22] the stored procedure is up to date on dev [19:45:28] so we have something more subtle going on [20:06:01] * meeple27 smugly ponders whether my approach would have avoided this subtle bug [20:11:08] random phlog thought of the day: Would it be faster to transform the input file into a parsed XML file, and bulk load that into the database, instead of performing lots of individual inserts? [20:32:55] that's not the slowest part [20:32:59] load takes 20 minutes [20:33:18] slowest part is the reconstruction, rebuilding 1 row per task per day [20:33:28] and then doing operations on those hundreds of thousands of rows [20:33:54] I had assumed that basic optimization would make all of that quick, but that hasn't been true yet [20:34:20] I'm not completely ignorant of indexing, but evidence suggests I'm ignorant enough [20:35:29] max, look at the latest http://phlogiston-dev.wmflabs.org/red_rules.html [20:35:44] read 50, 51, and 52 carefully [20:36:54] (maxbinder) https://phabricator.wikimedia.org/T166714 is caught by rule 50 before it gets to rule 51 [20:41:17] cutting 20 minutes to 5 would be...nice [20:41:58] and what if the reconstruction could be done in memory with python objects, rather than via a db? Then export that as xml, and load it up. Not sure it's possible, but if it were, that could help. [20:42:30] indexing speeds up retrieval, but slows down inserts, right? (as a sweeping over-generalization) [20:47:31] Right [20:48:06] Reconstructing in memory in python seems plausible [20:48:47] But I can't help feeling that changing some memory setting in postgresql would fix everything instantly [20:52:26] jaufrecht: That makes sense, but I recall that we said that "omit" should be valid whether or not it is before or after something else [20:52:32] jaufrecht: seems that isn't the case? [20:52:41] jaufrecht: not a terrible thing, if so [20:53:13] jaufrecht: "that" meaning "(maxbinder) https://phabricator.wikimedia.org/T166714 is caught by rule 50 before it gets to rule 51" [20:53:41] It's possible we said that. It's possible we were thinking that as we implemented. That doesn't seem to be the implementation, though. [20:54:38] Where "that" refers to omit overriding priority in the rules list. [21:07:22] if it depends on sequence, then a) it's consistent with other features, and b) allows "omit all except..." [21:07:54] I may have pushed for (b) as a feature, as with frt wanting to include * but exclude *x [21:08:50] hmmm. now that I'm playing back what I said, I'm not sure it's true [21:10:06] I want to include "fundraising*" but omit "*wmde". When both rules apply to a task, should it be always included, always omitted, or should it depend on sequence? (and if it depends on sequence, what would be the correct sequence?) [21:10:41] I think it would be: omit *wmde before categorizing fundraising*, to be consistent [21:22:26] maxbinder, do you want to flip around red_recategorization.csv and see if that fixes it? [21:22:50] jaufrecht: sure, needs a rerecon after for that right? [21:24:59] jaufrecht: I've changed the recat [21:25:08] jaufrecht: moved them all the way to the top, to make the omits go first [21:25:25] no rerecon, just reeport [21:25:27] ok [21:25:37] 17 minutes [21:25:54] yea, running [21:45:53] jaufrecht: no change, save for slight reordering on status report [21:45:58] git seems current [21:46:12] rules report didn't change [21:46:27] because category table in database didn't change, still shows the two omit rules at the bottom [21:46:35] shouldn't need a rerecon, though. checking ... [21:47:31] think I found it. rerecon and reconstruct both reload the category file into the database [21:47:38] but report doesn't, for no particular reason [21:47:43] certainly it's intended to [21:47:45] fixing ... [21:47:54] ok [21:48:49] fixed. try again. [21:49:00] go Cleveland ⚾️ [21:49:04] running [21:49:13] something something midges [21:49:26] Joba retired yesterday, I think [21:49:30] yup [22:02:24] not a lot of pitching duels yet this postseason [22:03:06] jaufrecht: seems to work on dev, no change on prod [22:03:22] did you re-run on prod? [22:03:28] fun fact: Luis Severino gave up the fewest runs of any WC starter [22:03:43] if it looks good, then commit and I'll push everything to prod [22:03:50] jaufrecht: I don't know how to run more that what I did [22:04:18] you're not supposed to run on prod, which is why I was surprised that you were surprised that nothing changed on prod [22:04:30] jaufrecht: not surprised, just stating what I see :) [22:04:40] thx for pushing it to prod [22:06:56] is everything checked in? [22:30:24] jaufrecht: checked in meaning git? [22:30:31] (Sorry, busy yelling at game) [22:31:30] jaufrecht: looks good on dev [22:31:34] jaufrecht: and prod [22:31:55] exciting, big feature for our usecase, very handy [22:32:44] * meeple27 feels less smug, now that this turned out not to be a subtle bug [22:36:50] https://www.irccloud.com/pastebin/n1w5yTfW/ [22:42:30] Cy Young AL ... [22:42:51] Cy Young NL is the good stuff 🏄‍♀️ [22:43:34] Although Kluber actually has a lower ERA than Kershaw [22:43:44] first time the AL has done that to the NL in a while