| 00:00 | <shu> | i'm not sure it's reasonable, more that it's the path of least resistance |
| 00:00 | <Kris Kowal> | And taking esm-integration at face value, and stated assumption, there’s already a reasonable interpretation. |
| 00:01 | <Kris Kowal> | So, the question is whether it can be simultaneously true that x instanceof ModuleSource and x instanceof WebAssembly.Module, such that the former could be fed back into the JS module graph, and the latter could be used for manual instantiation. |
| 00:02 | <shu> | i don't think that can be done for the existing WebAssembly.Module |
| 00:02 | <Kris Kowal> | That seems like a problem. |
| 00:03 | <shu> | there are solutions the acceptability of which i'm unsure |
| 00:03 | <shu> | WebAssembly.Module.fromModuleSource |
| 00:03 | <shu> | WebAssembly.Module2 |
| 00:04 | <shu> | WebAssembly.Instance.instantiateFromModuleSource |
| 00:04 | <guybedford> | this was where I was wondering if we might be able to define an internal slot on WebAssembly.Module that provided this relation in an internal way to 262 |
| 00:04 | <guybedford> | that could then be incorporated into the esm integration |
| 00:04 | <shu> | yeah, where does the instanceof requirement come from |
| 00:04 | <shu> | it could be a deep-ish sense of duck typing |
| 00:04 | <guybedford> | or the wasm js api |
| 00:05 | <Kris Kowal> | Yeah, I was not precise. I specifically mean that x be a “module source” if it could later be passed to new Module(source, ...rest) and the prototype chain is not material. |
| 00:06 | <shu> | ah, then i think that is possible |
| 00:06 | <Kris Kowal> | And I’ve got slides in the overflow that source should fall through to a protocol. |
| 00:07 | <Kris Kowal> | For user virtualized module sources. That’s the escape hatch for defining bindings for, say, CommonJS in userspace. |
| 00:07 | <Kris Kowal> | So, yeah, stuffing an internal slot on WebAssembly.Module such that it could be digested by new Module() is acceptable to me. |
| 00:08 | <Kris Kowal> | Not just acceptable, a condition of my acceptance. |
| 00:08 | <Kris Kowal> | And guybedford we can punt on how to link ESM into a JS graph. That’s answered https://github.com/WebAssembly/esm-integration/tree/main/proposals/esm-integration |
| 00:10 | <Kris Kowal> | At least, I trust it does. |
| 00:10 | <guybedford> | my hope was not to have to flesh out such a protocol as a stage 2 requirement here by treating it as somewhat under-specified for now |
| 00:10 | <guybedford> | but if the feedback is to more clearly work this through we can investigate options |
| 00:10 | <guybedford> | I think the protocol likely belongs in the spec for the module source object |
| 00:11 | <Kris Kowal> | For stage 2, I can be satisfied if we know we’re not painting ourself into a corner, regardless of what’s captured in the proposal text. |
| 00:12 | <Kris Kowal> | The protocol for virtual modules is orthogonal and I only point it out because I expect the Module constructor to have overloads on its source argument. |
| 00:12 | <Kris Kowal> | Which is to say that it can brand check for ModuleSource, then brand check for WebAssembly.Module, then fall through to a virtualized module protocol. |
| 00:14 | <Kris Kowal> | The other wrinkle is that implementing a bundler or file watcher for all this is substantially easier of all “module source” types have a uniform way to access the module source’s list of import specifiers. |
| 00:15 | <Kris Kowal> | As currently sketched, moduleSource.bindings is the same shape for ModuleSource and the protocol for virtual module sources. |
| 00:21 | <Kris Kowal> | Which is to say that it can brand check for |
| 00:21 | <guybedford> | Yeah so the WebAssembly way of doing this is quite unique - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Module/imports |
| 00:22 | <Kris Kowal> | Yeah, we’ve discussed a protocol oriented around source.constructor.imports(source), which we agreed would be Special™. |
| 00:26 | <Kris Kowal> | Is there a reason that WebAssembly.Module.prototype couldn’t have a bindings accessor to throw bundlers a bone? |
| 00:28 | <guybedford> | I don't know, if we're adding branding in theory we could add other things, Dan might know more about what's involved here since he did a lot of the JS API spec work I believe |
| 00:29 | <Kris Kowal> | Well, I owe you a review, and I’m at least oriented. I can leave a note. |
| 00:30 | <Kris Kowal> | Thanks for playing Wheel of Spec Fiction™. |
| 00:31 | <guybedford> | haha sure we've covered a lot! will take the winnings for now indeed |
| 00:33 | <Kris Kowal> | I have just noticed the pun in spec fiction. Who’s responsible for this!)/ |
| 00:37 | <guybedford> | well we just call it specification I guess to begin with :P |
| 01:00 | <Kris Kowal> | I mean, this is clearly a pun on SpecFic, the category containing both Fantasy and SciFi https://en.wikipedia.org/wiki/Speculative_fiction |
| 02:50 | <Kris Kowal> | guybedford Luca Casonato I provided a review. I think it would be good to settle the question before plenary, whether a reflected module captures a module instance or a module source instance. I am pretty sure it must be the latter and the front matter for the proposal states the former. https://github.com/tc39/proposal-import-reflection/pull/21#pullrequestreview-1030806802 |
| 04:28 | <guybedford> | Kris Kowal: I think it would align with Wasm better if the JS reflection can be treated as supporting arbitrary linkage as opposed to only host linkage |
| 04:29 | <guybedford> | Since that is effectively what you get with WebAssembly.Module so that would be the symmetric choice |
| 04:29 | <guybedford> | I'm open to the representation options though |
| 04:29 | <guybedford> | at the moment that object is still a big ? |
| 04:29 | <guybedford> | thanks for the review, I've pushed some updated, do take another look if you have a moment |
| 04:33 | <Kris Kowal> | Kris Kowal: I think it would align with Wasm better if the JS reflection can be treated as supporting arbitrary linkage as opposed to only host linkage |
| 04:34 | <guybedford> | Kris Kowal: a WebAssembly.Module has imports in its binary |
| 04:34 | <guybedford> | which are not yet satisfied and can be linked against arbitrary dependencies |
| 04:34 | <guybedford> | So the natural choice would be to aim for symmetry on that with the JS side |
| 04:35 | <guybedford> | That a JS reflection would not resolve its imports yet, but could still be instrumented against arbitrary dependencies |
| 04:35 | <guybedford> | depending on what hook is passed |
| 04:35 | <guybedford> | we could of course break that symmetry |
| 05:00 | <Kris Kowal> | That’s consistent with my interpretation of a module source. |
| 05:02 | <Kris Kowal> | That multiple Module instances can use a common source and link in different ways according to the provided importHook. |
| 05:03 | <Kris Kowal> | Caridy’s Module object is very similar to the ModuleInstance you proposed at the last plenary. The big difference is that dynamic import kicks off the loader / linker, and that uses the import hook to discover the linkage. It has the nice property that a single hook can be used for both static and dynamic import. |
| 05:04 | <Kris Kowal> | So instead of having to provide an importHook for dynamic import and a link method for static import, you just have importHook. |
| 05:04 | <Kris Kowal> | And the small difference is that each Module instance has an independent memo for the results of importHook. |
| 05:05 | <Kris Kowal> | It doesn’t account for assets and module reflection though, so I imagine we’ll have to revisit that. |
| 05:09 | <guybedford> | one step at a time I guess |
| 05:10 | <Kris Kowal> | Yeah, assets and import reflection imply a different design for virtualized host hooks on Module. More like fetchHook(specifier, importMeta) => Promise<TypedArray> followed by reflectHook(bytes, reflectionType) => unknown |
| 05:10 | <guybedford> | right, I want to put some more thought to the cache questions in due course as well around this for reflection |
| 05:10 | <guybedford> | for now it's explicitly flagged as a spec todo to define the exact cache behaviours |
| 05:11 | <Kris Kowal> | I must say in the quiet of my heart that I’d hoped that virtualized module sources and externalizing the concern of interpretation such that each module has exactly one reification would put the issue of assets to bed. |
| 05:13 | <Kris Kowal> | And this wakes the old MIME-type / extension mismatch issue that drove import assertions into its current shape. |
| 05:13 | <Kris Kowal> | So, perhaps reification of module source is special and not general. |
| 05:14 | <Kris Kowal> | But even then, the host hooks have to separate loading from instantiation. |
| 05:16 | <Kris Kowal> | e.g., loadHook(specifier, importMeta) => Promise<AbstractModuleSource> and importHook(source, importMeta) => Module. |
| 05:22 | <guybedford> | the hope is to leave the internal invariants and host hooks in a better state that we found them |
| 05:22 | <guybedford> | I think between the specs that is achievable |
| 05:22 | <guybedford> | but it will take some formalizing yet |
| 11:29 | <littledan> | @littledan thanks for the feedback, we see module reflection as laying down an approach that asset reflection and deferred execution can follow in, as opposed to the original more abstract view of a generical mechanic. Trying to be practical and use case focused while treating generalization as a possibility but not to over-generalize. We are keen to see asset references and deferred module loading move forward, we are just trying to focus our efforts in a way that keeps scope clear in the staging process. We currently are underspecifying what exactly the shape of the reflection object is though, and treating this as a detail to be refined as opposed to prescribing it at this point. The way we do this for now is to entirely delegate it to a host hook, then the hope for Stage 3 would be to follow-up with an exact definition for JS that will hopefully have progressed by then. We need the host hook regardless for Wasm though. |
| 11:32 | <littledan> | My feeling is that trusting the sender and receiver to have identical importHook behavior is possible but fragile. But, if we have blocks, static blocks, |
| 11:34 | <littledan> | That is to say, I predict that |
| 11:37 | <littledan> | Wasm itself prohibits module import cycles, but the Wasm ESM integration proposal is designed to permit them between JS and Wasm… but maybe it does so in too flaky of a way |
| 12:01 | <littledan> | I guess now that I hear more about how the caching for the import happens within the Module, rather than a single Module producing a separate instance somewhere else… I understand why you all have been calling this construct ModuleInstance over the years |
| 14:44 | <nicolo-ribaudo> | I'm re-reading the slides Kris Kowal presented yesterday, and I have a question. In the example with a custom module source, I see that the |
| 14:59 | <Kris Kowal> | I think the answer is yes. JavaScript module sources can be virtualized. It’s also possible to implement ModuleSource as an object that satisfies the virtual module source protocol. But, I imagine that would come at a performance cost that isn’t acceptable. |
| 15:01 | <Kris Kowal> | For JavaScript to JavaScript bindings, I think engines are free to implement linkage as literal pointers to shared slots. Through the virtual module source protocol, reading and writing bindings would have to pass through the module imports namespace exotic object, which would be slow on top of slow. |
| 15:02 | <nicolo-ribaudo> |
Yes, I was thinking about something like eshost rather than a production thing. |
| 15:02 | <Kris Kowal> | You’re right that initialize needs to receive a dynamic import function. It would also have to be the one bound to the module instance’s memo. |
| 15:03 | <Kris Kowal> | And providing either importMeta or dynamicImport can be contingent on whether the module declares that it needs them with needsImport and needsImportMeta on its module source object. |
| 15:03 | <Kris Kowal> | So, an options bag. |
| 15:03 | <Kris Kowal> | I believe they can also participate in cycles. |
| 15:04 | <Kris Kowal> | The module imports namespace object was an idea that the folks on XS came up with. |
| 15:04 | <Kris Kowal> | It’s quite clever. |
| 15:04 | <Kris Kowal> | And they implemented it, and it works, so my confidence in the idea is pretty high. |
| 15:04 | <nicolo-ribaudo> | I guess my custom module source would also have to know how the import hook works, because it might need to pass a custom importMeta object when calling importHook to make sure that the "base URL" is correct and not the one of the entrypoint module. |
| 15:05 | <nicolo-ribaudo> | Ok well no, because importHook would return a new module instance |
| 15:06 | <Kris Kowal> | Yes. |
| 15:14 | <littledan> | I'd like to invite Rob Palmer here, but I don't seem to have permission to invite people here |
| 15:15 | <nicolo-ribaudo> | Isn't this room public? So Rob just needs the link |
| 17:11 | <guybedford> | littledan: it's more about handling the consensus process itself, we don't want to try to do more than absolutely necessary to avoid unnecessary frictions. But yes asset references themselves should be relatively simple to spec and we are aiming to design host invariants that can work with that as well. The shape of the asset object may or may not be specified in ECMA-262, I'd personally like to see it in EMCA-262. |
| 17:18 | <littledan> | OK, I'd prefer to err on the side of trying to understand the full solution we're going after, even if having the design discussion risks some friction. I'm looking forward to hearing more of your thoughts on how asset references should work. |
| 17:22 | <littledan> | Once we understand that, it will help us prioritize what to push forward in the stage process and ship |
| 17:22 | <littledan> | I'm fine with Stage 2 for a bunch of this work while we're still in an undetermined state, though |
| 17:39 | <Kris Kowal> | Well… this is much of the stated motivation of the proposal that Surma and I presented to committee and that the committee agreed to Stage 2 on. I am surprised to see so much skepticism on this point and interesting in understanding more about why you feel this way; I wonder what went wrong in communication that we didn’t uncover this disagreement sooner. |
| 17:41 | <Kris Kowal> | However, consider the possibility that the receiver can create an importHook that calls back to the sender. |
| 17:59 | <Kris Kowal> | Daniel Ehrenberg: I don’t mean to communicate an absolutist stance. Even locally, there’s no guarantee that a module graph is consistent over time. But, on one extreme, sending a module to a worker is maximally fragile and on the other (sending a module with an RPC/IPC handle to the original loader) minimally fragile. I believe you and I agree that we’re converging on a foundation that is sufficient for implementing either extreme. And I think that we also agree that sending a worker and depending on out-of-band coordination of importHooks is a maximally fragile solution on that spectrum. And emphatically, I personally don’t object. |
| 18:01 | <Kris Kowal> | It’s also possible to have a sender and receiver with different but equivalent import maps, another case in which just sending the source and meta would work fine. |
| 18:08 | <shu> | what you call fragility i just think of "misconfiguration"? |
| 18:10 | <shu> | In the web, I’m not sure if we need any “trusting” here: the hooks simply are the same for native modules, and maybe we should just throw a DataCloneError if you try to serialize a Module with any other importHook |
| 18:11 | <shu> | maximal virtualization is an explicit non-goal for me |
| 18:12 | <shu> | experience has shown me here that those APIs are a world of pain for implementation complexity and security |
| 18:52 | <littledan> | It's a bit hard to think about this problem space while generalizing over all well-behaved importHooks since we don't really have much of a definition for well-behaved here, but I'd venture: any well-behaved mechanism for serializing a ModuleInstance will need to be somewhat tightly coupled with the importHook |
| 18:54 | <littledan> | Although we'll be able to enforce some invariants by construction or through checks: if we ship a loader API, it will be possible to use importHook badly/confusingly, just as it is possible to use Proxy and even Object.defineProperty badly/confusingly today. These misuses would lack some desired properties but still be deemed OK by the system. I don't think that's fatal. |
| 18:55 | <littledan> | for the import map case: yes, I think the import maps would need to be sent "as part of" importMeta conceptually |
| 18:56 | <Kris Kowal> | I agree about the limitations of enforcing good behavior of import hooks. For every behavior we would want to prevent there is an equal but opposite valid use. |
| 18:56 | <littledan> | (well, the other case for import maps is to consider that they are closed over by the importHooks, i.e. just interpreted from what's loaded on the other side, since importHooks would not be part of HTML serialization. This is also coherent but I don't like it as much.) |
| 18:57 | <Kris Kowal> | I think it’s reasonable to fall back to local consistency per-instance. |
| 18:58 | <Kris Kowal> | for the import map case: yes, I think the import maps would need to be sent "as part of" importMeta conceptually |
| 18:59 | <littledan> | well, it can be an internal slot... |
| 18:59 | <littledan> | I dunno how much introspection we want to provide |
| 18:59 | <Kris Kowal> | Same. |
| 19:55 | <littledan> | Given the light TC39 agenda, I wonder if we should have an hybrid modules breakout session |
| 20:10 | <Kris Kowal> | I have a 90' slot to talk about compartments that I was thinking to repurpose as a Module Harmony preso, in the same vein as the last two presentations, if folks here are alright with me representing us collectively. |
| 20:11 | <Kris Kowal> | That is, here’s a strawman / holistic vision and here are some of the open questions. |
| 20:12 | <Kris Kowal> | I think that leaves a decent chunk of time for discussion, but it’s not like I’ve ever finished a preso. |
| 20:13 | <littledan> | I think that, in this upcoming meeting, we aren't yet ready to present ourselves as a group beyond "we've been meeting and discussing these proposals"; it's premature to announce a holistic vision of the group, as opposed to just certain members of the group |
| 20:13 | <Kris Kowal> | Sure, so I should stick to, here’s where I think we’re headed with compartments. |
| 20:14 | <Kris Kowal> | I don’t think that precludes “how this might relate to blocks, deferred execution, and module reflection” but I agree that it’s too early to claim I’ve internalized all the champions’ perspectives. |
| 20:14 | <littledan> | yes, let's leave it to proposal champions to decide which open questions they want to bring to the committee. For example, I'd be a little uncomfortable having an all-committee debate on whether module blocks are ModuleSource or Module, at least just right now |
| 20:15 | <Kris Kowal> | I don’t feel like I’m far off, but I wouldn’t presume. |
| 20:15 | <Kris Kowal> | Oh, indeed. That would not be a fun conversation. |
| 20:16 | <littledan> | yeah I trust that you have a good understanding of at least Caridy's and my views on that particular question, but it's more that, we can let the individual champion decide when to encourage a large group discussion to draw a conclusion. |
| 20:18 | <Kris Kowal> | Caridy has a good intuition about what plays well at plenary, I think. I’ll listen harder for the next couple weeks. |
| 20:19 | <shu> | simply, there is no agreement on the foundations yet, so i agree with dan's intuition to not present as a group nor open the floor to questions like "is this (e.g. ModuleSource) a good foundation" |
| 20:19 | <Kris Kowal> | For example, Caridy’s intuition for this last presentation was that we should only cover the first half 😉 Timing worked out, there. |
| 20:20 | <shu> | IME committee, or any gathering of strong technical folks, will have a tendency to engage on technical grounds if you lay out a technical question in front of them |
| 20:21 | <shu> | you should do that if you think you'll get good signals from it |
| 20:21 | <shu> | but it feels premature in that the signals you want are from the other champions right now |
| 20:21 | <Kris Kowal> | I’m pretty confident that we don’t need to discuss that particular question at plenary. |
| 20:22 | <shu> | that said there are larger motivation questions i would like plenary feedback on |
| 20:23 | <Kris Kowal> | Pray tell, and I’d be glad to ask. |
| 20:24 | <Kris Kowal> | Also will circulate a deck among folks here. |
| 20:24 | <shu> |
|
| 20:24 | <shu> |
|
| 20:28 | <littledan> | this sounds more like a breakout session than a presentation |
| 20:28 | <shu> | fair |
| 20:29 | <shu> | for 1) especially, i only have a mild position |
| 20:29 | <littledan> | they do sound like very important questions to discuss, but it also sounds very complicated to run the discussion in a productive way |
| 20:30 | <littledan> | I think for 1) we could get a good read if someone presented on asset declarations and what they would mean for applications and tooling... |
| 20:30 | <shu> | i wish some of the MSFT folks would come |
| 20:30 | <shu> | they care so much about this i feel like |
| 20:30 | <littledan> | you mean the edge people working on html modules and such? |
| 20:30 | <littledan> | or which ones? |
| 20:30 | <shu> | yes, i mean those |
| 20:30 | <shu> | and constructable css whatevers |
| 20:30 | <littledan> | maybe we can get Dan Clark to come to this chatroom at least? |
| 20:31 | <shu> | indeed i'd love his opinions |
| 20:31 | <shu> | but it is also not clear to me who at MSFT is driving that programme |
| 20:31 | <littledan> |
|
| 20:31 | <shu> | is it dan? |
| 20:32 | <shu> | i think currently that is correct, no other proposals really get into virtualization. there are one-off discussions of hooking points here and there |
| 20:44 | <Kris Kowal> | The compartments proposal does have an answer for virtualization that could be served a la carte. |
| 20:45 | <Kris Kowal> | Caridy’s subset does not include virtualization at all, and I think we will want to lead with that. |
| 20:46 | <Kris Kowal> | But module source virtualization is the key to unlocking many experiments in user code. Currently, we use this feature in Endo to create a bridge back to CommonJS that is coherent with bundling and other tools. |
| 20:47 | <Kris Kowal> | Reifying execution contexts can also be served a la carte. |
| 20:51 | <littledan> | importHook is exactly what you might call "virtualization" |
| 20:51 | <littledan> | like, it's more virtualized than what some hosts provide now |
| 20:55 | <Kris Kowal> | Pardon, yes, that is one of the two kinds of virtualization on the table. |
| 20:56 | <Kris Kowal> | importHook virtualizes load and link. Virtual module sources virtualize initialization. |
| 20:56 | <littledan> | sorry, what is module source virtualization? |
| 20:56 | <Kris Kowal> | Module source virtualization is a module source protocol that allows a Module to be constructed from arbitrary other module types, e.g., CommonJS or assets. |
| 20:57 | <Kris Kowal> | new Module({ bindings: [{export: 'default'}], initialize(namespace) { namespace.default = 42 }}) |
| 20:58 | <Kris Kowal> | That is separable if there’s appetite for one kind of virtualization but not the other. |
| 21:00 | <Kris Kowal> | Probably should be separated simply because it is separable and has different motivating use cases. |
| 21:02 | <littledan> | hmm should that be new ModuleSource? |
| 21:03 | <Kris Kowal> | No. The argument is a module source protocol. |
| 21:03 | <littledan> | oh I see |
| 21:03 | <Kris Kowal> | But I can see that being a valid interpretation. |
| 21:03 | <littledan> | yeah I like the idea of separating these two types of virtualization |
| 21:03 | <Kris Kowal> | In Endo, we borrowed Guy Bedford’s heuristic module exports lexer which is part of Node.js and trained it to also analyze imports. It generates a bindings array and we construct a function body wrapper. |
| 21:04 | <littledan> | they should hopefully be orthogonal, right? |
| 21:04 | <Kris Kowal> | They are fully orthogonal. |
| 21:04 | <littledan> | well, I guess you need importHook to tie them together |
| 21:04 | <Kris Kowal> | And fully coherent. |
| 21:04 | <Kris Kowal> | Pardon, module source virtualization does depend on module virtualization. |
| 21:04 | <littledan> | I mean, this is fully "step 2", you can't start with module source virtualization |
| 21:04 | <Kris Kowal> | But not the other way around. |
| 21:04 | <littledan> | right OK I think I get it |
| 21:05 | <Kris Kowal> | And execution context depends on step 1 but not step 2. |
| 21:06 | <Kris Kowal> | With reification of execution context and module link and load virtualization, we can build an ESM-only Compartment in user code. |
| 21:07 | <Kris Kowal> | A reified execution context is just new ExecutionContext(globalThis, importHook) and has {eval, Function, Module}. |
| 21:09 | <Kris Kowal> | The importHook being for dynamic import in scripts. |