[01:07:15] TimStarling: just a random thought re: session store. how much is replication lag a thing in cassandra vs mysql? given these are indepedent key-value pairs without any transaction or interdependent queries afak (no snapshots or "slow" queries etc.), I would expect replication to have a much better p99/max than mysql. So.. maybe there's potential for maybe getting away with saying we don't "care" about the lag. Just write to local and [01:07:15] expect all is well on any subsequent request. [01:09:32] if time-wise that seems realistic, then the worst case scenario would be that in the event of a dc brainsplit, we would not know at the time of writing that the data may end up discarded by cassandra. I don't have a gut feeling on whether there are "bad" scenarios there with the kind of data we store in sessions (it'd definitely be a problem for priimary mysql data, but this isn't that). [01:09:55] just a thought, more to understand the problem space than anything else. Feel free to discard it as nonsense or unworkable [01:11:50] if a request writes to the session and returns a 302 redirecting the user to another page which then reads from the session, there is a race between the client and the server and it seems like it would be sensitive to replication lag on the order of tens of milliseconds [01:12:35] I guess as long as Setup.php is slow enough, replication will win [01:15:19] right, semi-automated sequences are a problem. both bots and redirects. and at that point you're back to either sync_writes or sprinkling stickiness throughout the journey. [01:16:46] ok, forget I said anything :) I'll put my attention back on looking at less statefull tokens. [01:17:10] that was petr's suggestion today in our tech planning meeting [01:17:27] just stop having stateful tokens, use an encrypted token in localStorage for CSRF [01:17:57] for preferences save confirmation, use the URL [01:18:08] for the use case of tokens on GET views for later writes (ajax user.tokens embed, edit page, html form etc) it seems like we mostly do that already with hmac'ed timestamps that we validate on receipt. [01:18:48] for stricter nonces in OAuth and such we could (and perhas already do) require POST requests that add and consume them in a stateful manner e.g. in the db, main stash or whever in the primary dc. [01:20:36] Hm.. I remember ApiTokens using POST. I guess not. [01:21:15] for edit confirmation we use a cookie afaik. [01:21:54] (setPostEditCookie) [01:22:10] but yeah, parameter could work as well so long as its hidden afterwards with JS to avoid sharing it that way. [01:22:15] That's why we went with a cookie there [01:22:20] though that's not a concern on preferences [01:22:54] and these use POST for the action so we're in the same DC for a few seconds after that anyway [01:23:00] so session could very well continue there afaik