The bad actor changing your value is only a problem if you explicitly give them the store and let them do that. If you just keep your stores private this is not a real problem.

So I would say the generally encouraged way to do binds should be instance-scoped by default and global bind should only ever be a "Are you sure you know what you're doing?" type of API for the power-user cases like module authors making sure their resource pool will not leak implementation details that would never be relevant to user code execution flow. Pool mechanisms I would say are almost universally okay to bind globally, but almost every other scenario is a matter of opinion and should (at least in my opinion) probably not bind at all by default and always follow that path through internals because otherwise you end up with these strange flows like with async/await not flowing through awaits the way most users seem to expect.

I don't think that's right. Simply restoring a global snapshot is enough. If the subtask you're awaiting resolves its promise in a context that didn't derive from the one that was active when you called it, then you've lost your state even though it didn't have access to your variable.

It's not just the timing of enabling PromiseHook. It is that the portion of an async function before the first await behaves differently from the rest of the function because the context capture will have already happened around the outer await before passing over the inner awaits. It's confusing to users that the context is not flowing out.

It happens to use enterWith(...) rather than run(...), but the confusion around expected flow is effectively the same.

why are global snapshots bad?

The proposal as it current stands is maintaining the lexical consistent value inside a async function body across await for AsyncContext.Variable, which is part of the approach aiming to address the dynamic scope concerns from the previous meeting's feedback.

Also, I'd really appreciate if people reconsidered much of what James M Snell was was saying in https://github.com/nodejs/node/issues/46262. As far as people that understand the issues of context flow, he's one of very few others I'd trust to understand this stuff, having done a bunch of work on the Cloudflare equivalent of AsyncLocalStorage.

What? I think "restore after await" would equally fix that bug (though agree that flows-out works here as well)

people have suggested things like "how about all callbacks just automatically close over the context" and this is broken and not what is proposed.

but also sometimes the registration context is the most relevant one (often it's the only possible one)

I have never found that with ALS. The appropriate place to do "resolve" path does get pushed a bit out sometimes though. Like if the resolve is in C++ internals somewhere then it gets pushed back to whatever call ran the C++ code which led to that resolve.

It's not that setTimeout needs to be registration time. It's that registration and scheduling time happen at the same time in some case. But it's not the registering of a callback which you actually care about, it's the path that led to it being called.

we should not be in the business of running anything with an empty context ever IMO. That would preclude many important context use cases.

because the initial execution will have an empty context, and that context might flow to places

Everything in the process has a cause and therefore should always have a parent to flow from. The top-level flows from the fact of the process starting, which maybe doesn't have a "value" per-se, but I think that is probably the clearest expression of never having an "empty" context.

I've asked two questions that I don't yet know the answer to:

• How are we handling merges, given that both contexts are often relevant (and in general this forms a big tree--there are more than two relevant things)?
• How could per-variable handling scale, when you have a lot of different libraries and a lot of different variables and they're all supposed to fit together? It's easier for me to understand "two classes of variables".

cool, sounds like we're all agreeing, "JS shouldn't create empty contexts". This doesn't resolve the "two causes" point which your document so clearly lays out.

wait, if we go with resolution context, and we don't change how the current API works otherwise, we'd run into this. Is this what we want?

await asyncVar.run("foo", async () => {
    await doSomethingHere();
});

console.log(asyncVar.get()) // "foo"

Yes. That is actually what users keep telling me they are expecting.

I know about the expense of PromiseHooks and patching all the libraries (which we all definitely want to resolve); what kind of expense is related to this issue that you're talking about now?

that is a very different behavior from the sync case

Just the general expense of all the stuff we have to do. Like I pointed out before that we presently need to keep a list of all spans in-memory from the lifetime of the whole request as we don't know how to do follows-from relationships otherwise.

It is, but it's identical to how the same code rewritten with callbacks flows, and that's the confusion users are running into. We keep seeing people rewriting their apps with promises and then being confused when the context doesn't flow the way they're used to anymore.

It is, but it's identical to how the same code rewritten with callbacks flows, and that's the confusion users are running into. We keep seeing people rewriting their apps with promises and then being confused when the context doesn't flow the way they're used to anymore.

It would require a change in the OTel spec which they've basically said unequivocally no to ever changing. 😐️

is this always the form of this sort of migration, or do people also have this confusion with greenfield promise apps?

Can you say more about the interactions with upstream OTel folks? Has anyone offered to make an efficient open-source collector to actually take advantage of this protocol improvement?

in web APIs there are cases where you can get a promise from somewhere else, that wasn't created in a data flow starting from the current function

Can't be done in the collector because the data is needed in-process for distributed tracing headers wherever outgoing activity may occur.

and all customer complaints are private? Can you talk with any of them and ask them if they'd be willing to be in touch with us?

Yes, I'm trying to talk with some of them right now. Getting time from enterprises to talk about things like this can be a bit time-consuming so we'll see how that goes...

maybe I'm using the wrong terms, but it sounds like you had some idea for an improvement which was rejected. What was the idea, and how was it proposed?

True, but then it wasn't caused by that function so probably should still be keeping the registration context. And in any case this is what bind is for--in exceptional cases a user (or possibly even the runtime sometimes) can decide to apply a bind to capture the context at a different point.

You pass around the contexts yourself rather than the system managing it for you.

Probably, but that's what bind is for. 🤷

did you ever explain this mismatch to them in writing?

The problem is in the mutable cloning thing. You're basically passing around many clones and you "activate" a particular clone at some point in time, so it's essentially like you're passing around a bunch of these snapshots all over the place manually, but still doing the same step of making it the "active" value it has stored globally.

we should anticipate that most people don't know how to use bind most of the time and just won't use it, IMO.

(that or only kind of know how to use bind, and end up using it defensively because they don't know which promises they can't await without breaking their context... meaning they break anything that relies on the causal-but-not-scoped propagation out of an await)

I don't see how that differs meaningfully from what we have

Sorry I missed a large chunk of this conversation but I can expand a bit more on the await/resolve/reject issue. The key problem is sometimes what you really want is *both*. That is, sometimes after a promise settles, what you want is whatever the context was before you started waiting for it (resolve or reject), and sometimes what you want is whatever the context was when it was settled (resolve or reject). And sometimes you might actually want both at the same time

I see. Where is this implemented?

So it's not critical that the API does the whole two paths thing. Like I've said before, what is in my doc is essentially the ideal from APM perspective, but quite possibly has things which shouldn't actually exist at language level and should just be done externally. But we're aiming for a balance that anything we need to do externally is as low-cost as possible, because what we have right now is very expensive.

so are there any usages of this API which are not "well-balanced"?

James M Snell: would be great to get your take on this comment, outlining a possible policy to deal with the two contexts

Flowing around is essentially just local variable scope which, to me, doesn't really seem to have much value. I've asked a few times for an explanation or some use case why anyone would want that flow but have not yet got any answer on that. 🤔

Flowing around is essentially just local variable scope which, to me, doesn't really seem to have much value. I've asked a few times for an explanation or some use case why anyone would want that flow but have not yet got any answer on that. 🤔

this is a straw man, no one is proposing fully lexically scoped semantics, but rather sticking with how AsyncLocalStorage currently works

it would be lexical scope within a function, but it propagates across function calls in a way that lexical scope doesn't

Got it. and yeah, that's what I suspected you meant :-) ... the answer is: whether it should be 1 or 2 depends on the storage cell is being used for. That is, In some cases what you want is 1 and others what you want is 2 . Even in your second example there using the callback, there are use cases where store.getStore() within the callback should not "clearly be 2".

by two storage cells I mean something like... (a) is the context value going in, (b) is the context value going out, both are accessible

that only works if you can run code in enough places (e.g., you'd want to inject logic into Promise.all, I think)

That or just building out the tools so we can not only reduce the graph with binds but expand the graph back to the original paths with the inverse. (We've called that callingContext so far, but it's still very much up for debate.)

I don't get it, where does the logic live to enable that?

In the case of AsyncLocalStorage, it just flows that way because that's just what we were actually able to do with the limited tools and resources we had available when building it.

Where the context is swapped by a snapshot.run(...), we can just capture the context that had already flowed to that point and make that accessible in some way.

It's been a bit, but there was some complications with it being layered over async_hooks and async_hooks not flowing that way.

async_hooks was created and then years later core finally accepted that context management was actually a reasonable thing to want.

I think the two types of variables makes sense... but maybe with two distinct behaviors.. For this example I'm going to change up the terms a bit from what is currently spec'd and known. What the spec currently calls AsyncContext.Variable, I'm going to call AsyncContext.Scope instead... it's a thing that can be entered with a specific value. Then, what I'm calling AsyncContext.Variable in this example is a thing whose value can be set (similar to enterWith but that cannot be explicitly entered... the value is just retained as the context is propagated.

const store1 = new AsyncContext.Scope
const store2 = new AsyncContext.Variable

async function doSomething() {
  store2.set(`abc${store1.get()}`);
}

await Promise.all([
  store1.run(1, async () => { await doSomething(); console.log(store2.get()); // 'abc1' }),
  store1.run(2, async () => { await doSomething(); console.log(store2.get()); // 'abc2' }),
]);
console.log(store2.get()); // undefined ?

The key point is store2 here is not a context that can be entered but only a value that can be set at any point during a flow such that the value propagates through the flow

so we ended up rediscovering that setting a dynamically scoped variable was an anti-pattern, and we moved towards doing .run instead

I've asked two questions that I don't yet know the answer to:

• How are we handling merges, given that both contexts are often relevant (and in general this forms a big tree--there are more than two relevant things)?
• How could per-variable handling scale, when you have a lot of different libraries and a lot of different variables and they're all supposed to fit together? It's easier for me to understand "two classes of variables".

The APIs I'm thinking of are things like https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamDefaultReader/closed, or https://developer.mozilla.org/en-US/docs/Web/API/FontFaceSet/ready. I don't see how those could possibly made to work with resolution time, unless every user knows to work around them

what if we designed both of these, but could be OK with them achieving different levels/timelines of standardization?

and i worry that either default (flow-around vs flow-through) is going to require opt-outs in normal code... so maybe per-store configurability is the only viable option

I think the two types of variables makes sense... but maybe with two distinct behaviors.. For this example I'm going to change up the terms a bit from what is currently spec'd and known. What the spec currently calls AsyncContext.Variable, I'm going to call AsyncContext.Scope instead... it's a thing that can be entered with a specific value. Then, what I'm calling AsyncContext.Variable in this example is a thing whose value can be set (similar to enterWith but that cannot be explicitly entered... the value is just retained as the context is propagated.

const store1 = new AsyncContext.Scope
const store2 = new AsyncContext.Variable

async function doSomething() {
  store2.set(`abc${store1.get()}`);
}

await Promise.all([
  store1.run(1, async () => { await doSomething(); console.log(store2.get()); // 'abc1' }),
  store1.run(2, async () => { await doSomething(); console.log(store2.get()); // 'abc2' }),
]);
console.log(store2.get()); // undefined ?

The key point is store2 here is not a context that can be entered but only a value that can be set at any point during a flow such that the value propagates through the flow