TC39 Module Harmony
2022-06-27
plaintext logs
prev next
source on github
14:40
<Kris Kowal>
Caridy and I had a conversation about paring down the compartments/loader proposal to avoid having to specify any maps at all, to avoid mentioning referrer specifiers at all, to further slim its profile against the wind.
14:40
<Kris Kowal>
The thought experiment is to return to the proposal Guy and Luca made at plenary and ask more questions.
14:41
<Kris Kowal>
Suppose new ModuleSource(source) and new ModuleInstance(source, importHook, importMeta?)
14:41
<Kris Kowal>
Such that source.bindings reflects imports/exports and instance.source refers to the original source.
14:44
<Kris Kowal>
Suppose also that import reflection gives us the means to obtain either of these through syntax. Recursively, that would mean that ModuleInstances and ModuleSources could be drawn out of ModuleInstance hooks.
14:45
<Kris Kowal>
Also, supposing that importHook’s signature is importHook(importSpecifier, moduleInstance), such that there is no referrer, but the importHook can associate instances with referrers and do whatever it must internally.
14:45
<Kris Kowal>
Caridy also proposes that we use dynamic import in the same way as blocks.
14:45
<Kris Kowal>
Which is to say, in this thought experiment, ModuleInstance corresponds to module {}
14:46
<Kris Kowal>
With that in mind, it’s sufficient to have a module instance to get the underlying unhooked Module Source, like (module {}).source, but Caridy suggests that would potentially mean the creation of throw-away module instances as in new ModuleInstance((module {}.source), importHook, importMeta)
14:47
<Kris Kowal>
So we go on to suggest (static module {}) instanceof ModuleSource, bypassing that step
14:47
<Kris Kowal>
That suggests that there be import syntax that gives us both of those options.
14:48
<Kris Kowal>
One thing I had not considered until this moment is that importHook would not be sufficient in order to support the ModuleSource import case.
14:50
<Kris Kowal>
But importHook is sufficient for deferred execution. For the deferred execution case, you’re importing a fully linked ModuleInstance that inherits the surrounding ModuleInstance’s importHook. You would then have the option of using it as a handle in a more elaborate import graph or just import(moduleInstance) at whatever time you deem appropriate to experience jank.
14:51
<Kris Kowal>
In any case, I sketched this, borrowing as much as possible from the Compartments proposal https://gist.github.com/kriskowal/288d38e62e55e09685bf62c3a3c25565
14:52
<Kris Kowal>
And note that it varies from Guy and Luca’s proposal in only one meaningful way: there’s no link method. Linking is a side-effect of kicking off the module system with a dynamic import of some leaf instance, which draws in its transitive dependencies.
14:52
<Kris Kowal>
So the difference is that the importHook gets used for both static and dynamic import.
14:53
<Kris Kowal>
I need to think more about the ramifications, but you’re all welcome to join me in the thought experiment.
15:15
<Kris Kowal>
One question ljharb asked on Github was whether it is ever desirable for dynamic import to provide different values for the same specifier (un-idem-potent?), or results inconsistent with import * as name. It would be surprising, which is undesirable. I’m not sure whether it’s always undesirable. In the context of this thought experiment, it would be possible to enforce idempotence on dynamic import and consistency with static import by having a locale memo as part of the ModuleInstance state. That comes at a cost we might prefer not to bear, especially if it is adequate to suggest that the responsibility to maintain this invariant be deferred to user code.
15:17
<Kris Kowal>
Maintaining import consistency at any granularity broader than module instance might be futile though. Since compartments as specified do and must allow inter-compartment linkage, it is possible in the degenerate case for user code to arrange a single compartment for every module instance and do whatever they want.
15:17
<ljharb>
to be clear, my default assumption is that it should be an unbreakable axiom modulo thenable modules, that static and dynamic imports are the same. I asked when it’d be desirable to figure out if that axiom should hold
15:18
<Kris Kowal>
Do you mean in the scope of an instance, cohort of instances, or realm?
15:19
<Kris Kowal>
to be clear, my default assumption is that it should be an unbreakable axiom modulo thenable modules, that static and dynamic imports are the same. I asked when it’d be desirable to figure out if that axiom should hold
That’s also my default assumption. I can think of no use for inconsistency in the scope of an instance.
15:23
<Kris Kowal>
This thought experiment would allow for inconsistency, and although inconsistency is not a desired effect, it does allow us to decouple the module loader from the realm’s module map, since it becomes the responsibility of the module instance hooks to maintain consistency.
15:24
<Kris Kowal>
The compartments proposal as written today would require module maps to be factored out of Realm so each Compartment can have its own maps. We’re anticipating that will meet resistance, but I don’t know for sure.
15:28
<Kris Kowal>
nicolo-ribaudo can correct me if I’m wrong, but I believe that module blocks and fragments as littledan has described them to me, would enforce consistency in the same realm by incorporating every module instance into the realm’s module map, using either a gensym or the module instance’s identity as an (ephemeral?) key. That would not allow for inconsistency. But, round-tripping a module instance through an intermediate realm would.
15:29
<Kris Kowal>
But I think the inconsistency would be limited to fragments themselves. Stringly-named dependencies would remain consistent.
15:30
<Kris Kowal>
I for one see no reason to expect consistency of fragment identity under any circumstances, but that is a separate matter.
15:34
<Kris Kowal>
Compartments as proposed do attempt to maintain consistency by having per-compartment maps and consistent-linkage to other compartments. I should try harder to come up with a consistency attack. I’m not sure a meaningful one exists.
15:35
<ljharb>
Do you mean in the scope of an instance, cohort of instances, or realm?
i mean in a realm, but I’m not paged in on all the new terminology here
16:10
<nicolo-ribaudo>
I need some time to digest all of this, but I like this direction.
For module blocks, having "module instances" and "module sources" easily explains why/how passing a module to a different realm (and back) looses the import() caching: you now have a new module, just with the same source (like having two identical files on the file system but in two different locations)
16:17
<nicolo-ribaudo>

The "import axioms" I think we shuold guarantee are (modulo export function then):

  • await import("x") and import * as _ from "x" in the same module must always evaluate to the same object
  • await import(fragment) and import * as _ from fragment in the same realm (compartment?) must always evaluate to the same object

And, with your proposed ModuleInstance/ModuleSource,

  • await import(fragment) and await import(new ModuleInstance(fragment.source)) must re-evaluate the module two times

The only thing I'm not sure about is if in module blocks the evaluation context should depend on where import() is called or where the module is defined. Given realm the globalThis of an iframe, I don't know if await import(fragment) and await realm.Function("s", "import(s)")(fragment) should return the same object (because they are importing the same ModuleInstance) or if they should evaluate the module twice (because they are in two different realms).
For string-based imports they would evaluate the module twice, but with this ModuleSource/ModuleInstance thing maye they should return the same object.

16:35
<nicolo-ribaudo>
Round-tripping to a worker would be similar to doing new ModuleInstance(fragment.source), and that can explain why you get two different modules.
17:02
<Kris Kowal>
I think that’s at least coherent, given that ModuleInstance would not have a referrer internal slot in this imagining.
17:17
<littledan>
to be clear, my default assumption is that it should be an unbreakable axiom modulo thenable modules, that static and dynamic imports are the same. I asked when it’d be desirable to figure out if that axiom should hold
I'm not a big fan of the "unbreakable axiom" framing but I definitely think that we should have a very good reason for mismatches, and cloning the module is a pretty huge mismatch to explain
17:17
<littledan>
OTOH it's common for transport to be not 100% perfect and in particular clone things
17:19
<littledan>
Round-tripping to a worker would be similar to doing new ModuleInstance(fragment.source), and that can explain why you get two different modules.
Well, almost. I think round-tripping would preserve the referrer base (and hosts would probably ensure that that tracks through import.meta)
17:20
<Kris Kowal>
One of Caridy’s premises is that ModuleInstance would not capture the referrer base. If it does, that should be a constructor argument.
17:20
<littledan>
(sorry I posted those comments before I read all of Kris's first comment that explains that)
17:20
<Kris Kowal>
And for that to work end to end, probably useful to surface import.referrer
17:21
<Kris Kowal>
Or really lean in on import.referer because chaos reigns.
17:21
<littledan>
hmm, it'd be unfortunate to have a JS-exposed duplicate of something derivable from import.meta.url, but at the same time that is funny because of layering
17:21
<nicolo-ribaudo>
Q: what is the referrer?
17:22
<littledan>
nicolo-ribaudo: The base for module specifier resolution
17:22
<Kris Kowal>
So, Caridy’s goal is to not enshrine an answer to that question. But, it exists in the shadows regardless.
17:40
<Kris Kowal>
The terminology I’m favoring in the compartments proposal is “import specifier” for strings that appear in source and “full specifier” or “referrer specifier” for strings that are keys in the module memo.
17:41
<Kris Kowal>
And for those terms to be fully orthogonal to the names “relative specifier” and “absolute specifier” which are specific to CommonJS.
20:28
<shu>
okay, so, for lack of a better place to post notes, i'm thinking i'll just put them in the incubator-agendas repo
20:28
<shu>
any objections?