[16:34:14] isaacj maybe this helps: https://stackoverflow.com/questions/39914266/memory-consumption-of-a-list-and-set-in-python [16:34:40] oof yeah, so i should probably switch to list then [16:34:50] which would be totally fine -- i can just convert to a set when writing out to a file [16:34:56] thanks! [16:35:30] clearly it's been a while since i did coding interviews :) [16:37:06] haha [16:37:29] from the top of my head, I would said: dictionary of dictionaries... [16:38:47] like {row1:{targetx:'',targety:'',...target_n:''}} [16:39:35] it's slow, but i'm enjoying my "partition by last integer of the page ID and run the process 10 times with just a tenth of the data each time" approach. actually kinda interviewy type of answer in terms of how painfully but effectively hacky it is. [18:06:11] o/ isaacj [18:06:13] Are we meeting? [18:06:22] yes, sorry, hopping on