16:20 | <Steve Hicks> |
This took me a while to understand your logic as to why this benefits from flowing through, rather than the other way around. It seems strange to me that you would expect to be able to use the id produced later - if you just rearrange them, then it works with flow-around, and that's the order I'd expect them to be in. But it seems like flow-through allows making it order-independent with a bit more care, at the cost of risking a catastrophic failure if somebody trashes your state, while flow-around insulates you from the risk (it's defensive programming by default, essentially), but then adds risk from structuring your middlewares incorrectly. |
17:11 | <Stephen Belanger> | That particular case is an obvious ordering problem, but when different systems are owned by different teams it's common to encounter scenarios like that where a team doesn't realize the specific implementation details of something means the data is not available where they expect it to be. With flow-through, anything that is logically after should reasonably expect to receive the context from something which came before it, whereas with around flow you have to deeply understand the full structure of the application to know exactly where context will reach, and refactors become problematic because if things get raised to different levels suddenly they have different contexts. As for risk of state being trashed, I don't see how that's possible unless you go around sharing stores with untrusted users, in which case it's kind of your own fault if it gets trashed. Generally a store should be owned and managed by the same code so you always fully know what's in the store. Even with around flow you can still get trashed stores if you give others access to the store as they can just |
17:13 | <Stephen Belanger> | To me, there's not really any "risk" of store state being corrupted in the through flow, and on the contrary there is lots of risk of it being corrupted by around flow as it cuts off branches all over the place, often in places not desired by the consuming code. |
17:17 | <Stephen Belanger> | Even for the intended case of only flowing inward, it's still possible to mess up the state by wrapping nested calls in additional unexpected store.run(...) calls, if you've passed around the store, and the global snapshot concept gives anyone a bunch of power to completely destroy the state of all stores by binding in strange places. I've always advocated for per-store binding with global/snapshot bind only used for very exceptional cases, generally managed by the runtime or some very specific userland module cases which should be very clearly communicated what the risks are to modifying the global flow graph. |
18:07 | <Steve Hicks> | To me, there's not really any "risk" of store state being corrupted in the through flow, and on the contrary there is lots of risk of it being corrupted by around flow as it cuts off branches all over the place, often in places not desired by the consuming code. The existence of global snapshotting means that (with flow-through) a subtask you call can trash your continuation context. Flow-around insulates you from this because it guarantees that no subtask can affect your state, but at the cost of making it more expensive to extract state you actually want to get out of a subtask. Again, it's a question of intention, assumptions, and defaults. If we get the defaults right, then it may be perfectly reasonable to say "just don't use global snapshot" because the defaults will make it so that nobody ever really needs to. If we get it even slightly wrong, we risk a cargo cult developing where less-informed developers think they need to use it everywhere and now you're stuck being ridiculously defensive about protecting your own context. |
18:08 | <Stephen Belanger> | It can't trash your state within local scope, but it definitely can trash your state in descending contexts. |
18:09 | <Stephen Belanger> | As I expressed previously, around flow is actually just through flow with some extra snapshot binds layered on top. You can interfere with it in exactly the same ways when messing with global snapshots. |
18:11 | <Stephen Belanger> | All the issues you are pointing out are issues with around flow too. |
18:11 | <Steve Hicks> | I see descending contexts as their own responsibility. I think this is reflecting our different perspectives. I'm inclined to trust functions that call me (insofar as I'll assume that whatever state they give me is what they intended to, at least) more than I am to trust functions that I call. It sounds like you're more inclined to trust the functions that you call, and less inclined to trust the functions that call you. |
18:12 | <Steve Hicks> | What I'm getting at is that we seem to be looking at something from two different directions - there's a lot of analogue/duality between them, and I think if we understand the bigger picture it might lead to a unified solution that makes sense from all angles. |
18:13 | <Stephen Belanger> | It's not about trust, it's about expectation. With through flow it is expected that intermediate things can mutate the context if you give them access to do so. Whereas with around context it's not generally expected, but still possible. |
18:17 | <Stephen Belanger> | Around context is actually two separate flows, which is why it's confusing. It flows inward as would be expected of context to flow to logically continuing code. But it also flows around branches which orphans branches all over the place, leaving you with a whole lot of paths to nowhere. In certain flows, mainly function/recursive, the inward-only flow can make sense. But in more complex code which branches and converges you get context changing unexpectedly all over the place when using around flow. |
18:18 | <Stephen Belanger> | Essentially the context splits every time an await happens, and then walks one path out to the leaf before switching back to the other half of the split when returning, so you get all this context fragmentation. |
18:19 | <Steve Hicks> | Essentially the context splits every time an await happens, and then walks one path out to the leaf before switching back to the other half of the split when returning, so you get all this context fragmentation. |
18:21 | <Steve Hicks> | I think splits (looking forward in time) comprise any sort of scheduling event - if you start an async task without awaiting it, you're scheduling a continuation that splits off from the sync flow. |
18:21 | <Stephen Belanger> | It's a split before the await. |
18:22 | <Stephen Belanger> | It splits into one copy of the context flowing into the branch and a separate copy capturing and restoring when the await resolves. |
18:22 | <Steve Hicks> | but then that scheduled task is basically a merge (possibly with a null context) to even start running its own continuation |
18:23 | <Stephen Belanger> | Why a null context? It inherits whatever context was present when the async function was called. |
18:23 | <Steve Hicks> | fair, I was being lazy about e.g. browser events |
18:23 | <Steve Hicks> | some potentially-older inherited context |
18:25 | <Stephen Belanger> | That inheriting is what I mean by it splitting context. The context goes off down that branch and can be modified there, but what comes out the other side of the await is what was captured before it did any of that, discarding the entire branch it just made and creating this pattern of continuously chopping of all branching code, which seems like a really odd thing for context to do. |
18:29 | <Steve Hicks> | You're coming at this from an APM perspective. When you look at how parameters flow through function calls, this "continuously chopping off" is exactly what happens all the time and is perfectly expected. |
18:39 | <Steve Hicks> | Suppose you have some abstract code like so:
The question of flow-around vs flow-through seems to be entirely about whether |
18:51 | <Steve Hicks> | I'm drawing some diagrams. I've identified then , await , addEventListener , etc, as corresponding to merge points. Instantiation of a closure seems to correspond to branch points (and note that every promise takes a closure in its ctor). Event listeners can actually cause multiple repeated rebranches and merges - so branching is apparently N-way - basically, the vertex is the instantiation and the separate edges coming off are each individual call. Merging tends to be 2-to-1 except for Promise.all. I don't know how helpful this is, but looking at the different shapes for different operations in the abstract might inform something? |
19:17 | <Stephen Belanger> | You're coming at this from an APM perspective. When you look at how parameters flow through function calls, this "continuously chopping off" is exactly what happens all the time and is perfectly expected. |
19:19 | <Stephen Belanger> |
doStuffWhileWaiting() is sync, so it's already done whatever it was going to do with the context. And even if it spins off async activity internally, it's not merging back and so the following code would not be in any way a continuation of that. |
19:22 | <Stephen Belanger> | Not closures. Callbacks. The promise executor is not a callback as it executes immediately so is not relevant to capture and restore around that boundary. It's only relevant to capture and restore around points where execution rearrangement occurs, which is between some promise resolving and its eventual continuation running, because of the microtask queue, or some libuv-backed thing in Node.js which takes a callback, because of the event loop moving on to other things until the libuv handle signals the work is complete. |
22:12 | <Steve Hicks> | https://docs.google.com/document/d/11f1GsjHfxsGO_r5cQZwfDwLbvALyr_TElzESDbiJTiU/edit Not sure how useful this is... it kind of lays out a framework. |
22:12 | <Steve Hicks> | might make it easier to reason about things |