00:49
<nicolo-ribaudo>
nicolo-ribaudo: Caridy is working on First-class Modules spec text. It might be good for you to connect.

Awesome! This week I started refactoring the HostResolveImportedModule and HostImportModuleDynamically host hooks, because I didn't want to go through host-defined behavior when importing module blocks/fragments (until they import external modules).

I now have a single HostLoadImportedModule hook, used both for static and dynamic imports, that "returns" an unlinked Module Record. ECMA-262 takes care of iterating through its dependencies, and it recursively calls HostLoadImportedModule to visit the whole graph (cc guybedford: this matches your intuition in https://github.com/tc39/proposal-js-module-blocks/pull/65#discussion_r912412376 !).

This is my proposed refactor: https://nicolo-ribaudo.github.io/modules-import-hooks-refactor/
If you are curious, this is how the module blocks spec looks like on top of that refactor: https://nicolo-ribaudo.github.io/modules-import-hooks-refactor/proposals/module-blocks - the updates to import() are relevant for "layer 0", because they show how to import a "module object".

I believe that the new logic closely resembles the loading logic introduced by "layer 0": we could probably just add a step tocheck to 2.d.ii of InnerModuleLoading that calls a module.[[ImportHook]] function, rather than always delegating to HostLoadImportedModule.

It would probably also help with module reflection: assuming that WebAssembly.Module will be a module source, the proposal could use HostLoadImportedModule(...).[[ModuleSource]] instead of introducing a new HostResolveModuleReflection AO. This would also guarantee that import module x from "x.wasm" and import * as ns from "x.wasm" import the same module, even if they do two different things with it (probably we could guarantee ns === await import(x)).

I would love to hear your thoughts on this 🙂

00:53
<Kris Kowal>
Your description matches my intuition.
00:54
<Kris Kowal>
Though, it would be nice if we could always route through module.[[ImportHook]], even when that is just a thunk for HostLoadImportedModule.
00:55
<littledan>
I reviewed this refactor earlier today and it looks great to me.
00:55
<Kris Kowal>
And I think this generally closes the gap with Caridy’s upcoming changes, which had no reasonable default importHook.
00:58
<littledan>
This refactor is great because it makes it clear exactly where importHook is called; it is not too often as currently
01:00
<littledan>
IMO it would make sense to land separately in HTML and JS already, since it is a significant cleanup, or at least put out for review as a separated item
01:05
<Kris Kowal>
Would it be reasonable for HostLoadImportedModule to be the default [[ImportHook]] of a Realm, such that Module instances and module blocks can pick it up from their execution context?
01:05
<Kris Kowal>
That should be behaviorally equivalent, I think.
01:06
<Kris Kowal>
Or rather, in the case of the Module constructor, picked up from the ModuleConstructor.[[Context]].[[Realm]].[[ImportHook]]
01:07
<nicolo-ribaudo>
Though, it would be nice if we could always route through module.[[ImportHook]], even when that is just a thunk for HostLoadImportedModule.
I thought about that, but it has a problem: I expect that we will always call Promise.resolve on [[ImportHook]]()'s result, because its an user-exposed API. This forces a microtask for every imported module, while the old HostResolveImportedModule allowed everything to be synchronous (the new .LoadRequiredModule always returns a promise, but hosts can implement a sync HostLoadImportedModule and then .LoadRequiredModule always returns an already resolved promise)
01:08
<Kris Kowal>
I see.
01:09
<Kris Kowal>
I think the ends I’m looking for could be accomplished another way, then…
01:09
<nicolo-ribaudo>
I know that Bun (a new experimental JS runtime) allows require() of ESM, and throws if it uses TLA. So there is at least one project that depends on modules loadnig being mostly synchroous (without TLA, .Evaluate() returns an already resolved promise)
01:09
<Kris Kowal>
That is, for there to be a slot that the HostLoadImportedModule fits into, which a VirtualHostLoadImportedModule would fit in.
01:10
<nicolo-ribaudo>
That is, for there to be a slot that the HostLoadImportedModule fits into, which a VirtualHostLoadImportedModule would fit in.
That would work, instead of the user function we would store an abstract closure that wraps the user function
01:10
<Kris Kowal>
Yeah, I’m familiar with Bun. It makes some similar architectural choices to Endo.
01:11
<Kris Kowal>
Or rather is born from similar requirements: All things must bundle.
01:15
<nicolo-ribaudo>
Kris Kowal Do you know what's the best way to share my refactor with Caridy? I see that he's not in this room.
03:36
<Jack Works>

Did a quick scan over the refactor spec:

  1. ContinueDynamicImport should not try to Link() / Evaluate() multiple times.
  2. Should [[LoadedModules]] be added in Abstract Module Record instead of Cyclic Module Record? A non-cyclic-able module type can also have dependencies.
  3. InnerModuleLoading is it a normative change? Say, today loading a module can be unordered (as I can recall)

Module block spec:

  1. Why disallow import a module block from another realm? (13.3.10.1, step 6.a)
  2. Looks like you're not using the building layers from the compartments? (Module and ModuleSource class)
  3. IIRC we may want to hide the source of a module. Source may not always be available on some platform, like embedding JS engine. (21.6.2.1, step 2.a)
05:02
<littledan>
To review these PRs, understand the ModuleBlock class to stand for Module/ModuleInstance
05:03
<littledan>
Since we are talking about instances, they exist just within one module graph and therefore one realm. You can make a different module instance (eg with structured clone) to get something similar but within another module graph
05:06
<littledan>
Yes, this change is normative, since it makes all environments follow some of the logic that is currently in HTML. I think this logic is pretty reasonable and makes sense to apply universally but we should be careful to examine the various environments that we have to make sure that is appropriate.
05:07
<littledan>
How does ContinueDynamicImport Link/Evaluate the same thing multiple times? I agree that it shouldn’t do that
05:08
<littledan>
I agree that the source might not always be visible to JS code. How does that relate to these patches?
05:10
<littledan>
About AbstractModuleRecord vs CyclicModuleRecord: currently, the notion of an import/dependency is all in CyclicModuleRecord. We could generalize this but I think it would help to have a driving use case where we don’t want to use CyclicModuleRecord, which I can’t think of. This refactoring seems orthogonal to the current PR.
07:15
<guybedford>
nicolo-ribaudo: great to see! Will review further over the weekend. Did you mean to also include the updates to HostResolveImportedModule calls in InnerModuleLinking and InnerModuleEvaluation? I didn't see those included.
10:45
<nicolo-ribaudo>

Thanks or the review!

  1. Both Link() and Evaluate() are idempotent operations: calling them multiple times has no effect. They are the "public interface" of module records, and we shold use them rather than checking the [[Status]] (which is an "implementation detail" of Cyclic Module Records). It already happens that they are called multiple times:
    • ECMA-262 always calls them for non-cyclic module records, even if they have already been linked/evaluated
    • HTML always calls them even for all module records imported either with a <script> tag or with dynamic import, even if they have already been linked/evaluated
  2. ECMA-262 never fiddles with the internals of non-cyclic module records, for example the .Link() and .Evaluate() implementations are only given for cyclic module records. From its point of view, that they are terminal nodes of the dependencies graph and they have to handle their own dependencies opaquely. However, it makes sense to move LoadRequestedModules to Abstract Module Record, where we have Link and Evaluate.
  3. Hosts can still load dependencies in the order they want: they can continue pre-fetching them as they currently do, and make HostLoadImportedModule behave just like HostLoadResolvedModule/HostImportModuleDynamically depending on where it's called from. I don't think this refatoring is necessarily normative.

Regarding module blocks: yes, the spec still needs to be updated after the harmony discussions in the last week, I only rewrote the current state of the proposal.

10:45
<nicolo-ribaudo>
nicolo-ribaudo: great to see! Will review further over the weekend. Did you mean to also include the updates to HostResolveImportedModule calls in InnerModuleLinking and InnerModuleEvaluation? I didn't see those included.
There is an editorial note that says "update all the examples of HostResolveImportedModules"
23:36
<guybedford>
Okay, I just would have thought that the InnerModuleLinking logic repeats a lot of the payload logic you have.