| 08:05 | <nicolo-ribaudo> | I will unfortunately skip the call today, I'll be traveling at that time. See you in two weeks! |
| 08:54 | <Mathieu Hofman> | @annevk wrote in General: > yulia | sick: littledan: I could make the Module Loading call tonight if that's helpful, please lmk https://matrix.to/#/!wbACpffbfxANskIFZq:matrix.org/$oeD0yH7tagpCoRrR4gbgusb0fzM1ydOzWLIasu1T2ng?via=matrix.org&via=mozilla.org&via=igalia.com |
| 12:52 | <Luca Casonato> | I will unfortunately skip the call today, I'll be traveling at that time. See you in two weeks! |
| 14:47 | <littledan> | Let's declare this time cancelled, if Yulia is also sick |
| 14:47 | <littledan> | also let's invite Anne tot his room |
| 16:19 | <Jack Works> | Do we have an agenda today |
| 17:01 | <Kris Kowal> | We do not. I concur with the overall sentiment. We’ll convene in two weeks. I’ll invite Anne to this room. |
| 17:02 | <Jack Works> | Ok (btw I'm also on a sick leave until Jan 16) |
| 17:05 | <Kris Kowal> | For agenda building for N*2 weeks from now, I’d like for us to plan to meet with bakkot to improve our understanding of the feedback from the previous plenary. That may be an opportunity to prepare and dry-run a module harmony presentation-of-presentations. |
| 17:06 | <Kris Kowal> | I still am on the hook to refactor the compartments proposal into epic-module-harmony and proposal-module-constructor etc. |
| 17:19 | <Kris Kowal> | Until then, @bakkot, your feedback from plenary, if memory serves is, “no new path to eval”, and when we convene again, we are going to want to break that down and examine whether import(new Module(new ModuleSource(text))) qualifies as a new path to eval in the sense that you mean, and whether it’s fatal given the mitigations we have in mind. Not looking to dig in now, since we’ve got a lot of folks away, but I’d like to prime the pump for the next meeting (or a meeting thereafter). |
| 17:20 | <littledan> | I also nominate following up on the import assertions design discussion, as an agenda item next fortnight. And I nominate peetk and nicolo-ribaudo to lead that discussion :) |
| 17:20 | <Kris Kowal> | We also have a standing invitation to give @annevk the floor to discuss the next steps for import assertions. |
| 17:21 | <littledan> | We also have a standing invitation to give @annevk the floor to discuss the next steps for import assertions. |
| 17:22 | <Kris Kowal> | @annevk are you available +1 fortnight? |
| 17:23 | <Kris Kowal> | Also, thank you littledan, I’m ecstatic to find company in which biweekly isn’t a word. |
| 18:26 | <bakkot> | Until then, @bakkot, your feedback from plenary, if memory serves is, “no new path to eval”, and when we convene again, we are going to want to break that down and examine whether eval - I would just want there to be a very strong reason for it, which I haven't heard yet. |
| 18:39 | <annevk> | Kris Kowal: yeah that should work |
| 18:39 | <littledan> | To be precise, I'm not dead set against having a new path to |
| 18:42 | <littledan> | (so, I think it should be treated as a post-Stage 2, pre-Stage 3 decision, having been fully scoped out) |
| 18:44 | <littledan> | (still fine to hear out bakkot more, but I think he was plenty clear in plenary?) |
| 18:46 | <littledan> | (I think the answer to the question is: clearly yes) |
| 18:47 | <Kris Kowal> | To be precise, I'm not dead set against having a new path to |
| 18:50 | <Kris Kowal> | That is, being able to parse a string’s static imports and exports is currently subsumed by the ModuleSource(text) constructor but does not necessarily need to be on the path to importing the text. That can be recovered either by a parse bindings function sitting somewhere else, or by rendering module source instances constructed by ModuleSource(text) unusable (which is consistent with what a CSP would do anyway) |
| 18:50 | <Kris Kowal> | And the motivation for parsing bindings is bundle, web archive, or import map generation, in which case, a path to eval is not needed. |
| 18:53 | <Kris Kowal> | I suspect that the strongest case for a path-to-eval is a debugging environment, with hot module replacement occurring client-side, such that the runtime behavior more closely resembles a production environment than is possible to achieve with the current tooling ecosystem. |
| 19:02 | <bakkot> |
Those seem like somewhat niche use cases which would be adequately met by userland parsers, to me? Doesn't seem like support for those things would need to be built into the language. |
| 19:02 | <bakkot> | We already have pure-JS bundlers, after all. |
| 19:04 | <bakkot> | (I'm not saying they're niche things for developers to do, just that they're niche things to do in code which is shipped to users; when building something into the language the latter is more relevant.) |
| 19:05 | <Kris Kowal> | I agree that the value is bounded in this way. |
| 19:07 | <Kris Kowal> | And other motivating use cases I can expound upon do not involve web browsers, so I expect them to be less convincing. |
| 19:13 | <Kris Kowal> | For example, at Agoric, we would like to ship programs to workers as zip files containing the original sources. The workers would use the ModuleSource constructor to build out the module graph. We’re also avoiding source-to-source transforms so that audits are more transparent. The auditor can review the code and not worry about holes that would be opened by generated code. We’re currently unable to do this because our shim has to funnel module code through eval, so we necessarily have to do a module-to-program transform, and since that is prohibitively expensive, our zip files currently contain a JSON blob that contains the program and the bindings. |
| 19:15 | <Kris Kowal> | And notably, Agoric/Endo execution environments do not employ CSP but do harden the worker realm and confine guest programs, so the trade-offs are different. It would be fair to call our case niche, but not unimportant. |
| 19:15 | <Kris Kowal> | Rather, I’d call it nascent. |
| 19:17 | <Kris Kowal> | (For example, the very common babel facebook/regenerator runtime introduces thawn-objects in a way that would not be obvious to an auditor and would open a program to interference that would otherwise not be possible.) |
| 19:19 | <Kris Kowal> | A consequence of the current language limitations is that a debugger attached to a production heap snapshot can’t match stack traces with local code artifacts. |
| 19:34 | <bakkot> | If you could import a reified-but-not-evaluated Module and postMessage it to a worker, would that eliminate your need for eval and therefore your need to bundle? |
| 19:35 | <bakkot> | (import or otherwise obtain in some manner other than constructing from a string, as would be provided some of these proposals IIUC) |
| 19:35 | <Kris Kowal> | postMessage is not available in the particular context. |
| 19:36 | <Kris Kowal> | Though a suitable postMessage could be implemented in terms of ModuleSource. |
| 19:36 | <bakkot> | Wait, so how are you getting stuff into a worker without postMessage? |
| 19:37 | <Kris Kowal> | I’m using the term Worker loosely. This is a Node.js or XS child process. |
| 19:37 | <bakkot> | Node has postMessage - though I guess not to actual child processes spawned with exec or whatever. |
| 19:38 | <Kris Kowal> | Aye, and process isolation is key. XS too. The application can resume from snapshot. |
| 19:43 | <bakkot> | Since you're targeting specific platforms, can you not hook into the runtime with native code and get at bindings that way? Or roll your own native ModuleSource-equivalent constructor, for that matter? |
| 19:43 | <bakkot> | I guess V8 might not expose the things you'd need, though that's in principle solvable. |
| 19:43 | <Kris Kowal> | XS implements ModuleSource effectively. We are working in that direction, yes. |
| 19:46 | <Kris Kowal> | There’s also a course to having these applications portable-to-web using XS on WASM. This is of course, not ideal, but certainly suitable for evidence of motivation :-P |
| 19:55 | <Kris Kowal> | And also the course of using a Service Worker to pop the zip open. My feeling is that’s got its own trade-offs. |
| 19:57 | <Kris Kowal> | And of these options, having ModuleSource provided by the language is the most portable. |
| 19:58 | <Kris Kowal> | The alternatives require assumptions of a shared foundation that does not exist. |
| 21:57 | <bakkot> | Spawning a new process is fundamentally not portable to begin with, so I'm not sure why portability matters here |
| 22:11 | <Kris Kowal> | Spawning, the conceit that all content everything is a URL backed by an ambient fetch capability, and module loading can (and I reckoon should) be made orthogonal. The portion of the problem pertaining to spawning is expressible in a hook that differs by platform and, with ModuleSource, nothing else is different. Drawing module sources out of an archive doesn’t require any I/O. |
| 22:12 | <bakkot> | Well, neither does parsing the bindings out of a string, in principle |
| 22:12 | <Kris Kowal> | Not requiring I/O is a pretty big deal. Dynamic import currently creates an exfiltration channel. |
| 22:13 | <Kris Kowal> | Well, neither does parsing the bindings out of a string, in principle |
| 22:15 | <bakkot> | Well, right - evaluating JS is a weird and dangerous thing to do, so it's best not to expose a language-level capability for doing that, but specialist applications can (and do) do it in userland if they really need to. Seems like this use of eval is pretty analogous, to me. |
| 22:15 | <Kris Kowal> | So, one of the intertwingled concerns is that Agoric/Endo prevents that exfiltration channel. Currently, we do that by censoring the word import, which has a number of inconvenient false positives and artificially limits how much extant code can participate on the platform. |
| 22:15 | <Kris Kowal> | Well, right - evaluating JS is a weird and dangerous thing to do, so it's best not to expose a language-level capability for doing that, but specialist applications can (and do) do it in userland if they really need to. Seems like this use of |
| 22:16 | <Kris Kowal> | And equally worthy of existence, in my opinion. |
| 22:17 | <bakkot> | ... Which is to say, not, or do you have a different impression of the net benefit of eval than I do? |
| 22:17 | <Kris Kowal> | Except direct eval. That can die in a fire. |
| 22:18 | <Kris Kowal> | ... Which is to say, not, or do you have a different impression of the net benefit of |
| 22:18 | <bakkot> | That... is not my impression. |
| 22:18 | <Kris Kowal> | CommonJS certainly would not have happened. |
| 22:19 | <Kris Kowal> | I know opinions vary on the point, but I have few regrets. |
| 22:21 | <Kris Kowal> | (ESM would not have happened if CJS hadn’t happened. TC39’s position in 2009 was to wait and see whether the ecosystem had an appetite for modules, which would not have been observable otherwise.) |
| 22:21 | <Kris Kowal> | Besides, the ecosystem didn’t really want it until they had it. |
| 22:22 | <bakkot> | I don't think it's really that plausible that we'd've never had a module system at all in the absence of eval, but in any case, we do now have such a system, so that can't be an argument for eval being something we would design into the language now. |
| 22:31 | <Kris Kowal> | How sympathetic are you to jsfiddle as a specialist application that would benefit from eval-but-modules? |
| 22:34 | <bakkot> | I don't think jsfiddle uses eval? |
| 22:34 | <bakkot> | So not particularly sympathetic. |
| 22:36 | <Kris Kowal> | At his point in the argument, would you say that the best way for Module constructor to Stage 2 would be to preserve the concept of a module source object (e.g., the source for a module block or a WebAssembly.Module) and omit the ModuleSource constructor? |
| 22:36 | <Kris Kowal> | I do not imagine finding better arguments for ModuleSource from text than the above. |
| 22:37 | <littledan> | bakkot: Do you consider this decision a Stage 2 blocker? IMO it seems OK to leave it an open question to resolve before Stage 3. |
| 22:37 | <littledan> | I think for Stage 2 we should sort of understand the decision space fairly well, and I think we do |
| 22:37 | <bakkot> | I think in general that for any proposal to advance all components of that proposal should be justified, and I am not yet convinced that the ModuleSource constructor is justified, so yes. |
| 22:38 | <bakkot> | As to whether it's a pre-Stage 2 or pre-Stage 3 concern, personally this decision feels like a fairly major one, so personally I would prefer it be made before Stage 2, but the distinction is pretty artificial given that no one is shipping before stage 3 anyway, so I don't feel strongly about that. |
| 22:38 | <Kris Kowal> | Do you also contend that surfacing the bindings parser is not sufficiently justified? (The performance hit for drawing in Babel into production is significant. I hope that justifies it.) |
| 22:39 | <bakkot> | Oh, you can do a lot better than pulling Babel into production |
| 22:39 | <bakkot> | You don't need a full parser just to get the bindings out |
| 22:39 | <littledan> | Stage 2 implies all sorts of other important qualities, like "is the committee sort of committed to doing something here" and "do we have a concrete first draft"; it is really common to have disagreements about major semantic points when something goes to Stage 2 |
| 22:39 | <bakkot> | If shipping a parser specialized to the problem of getting bindings is still not performant enough, then that's maybe something to talk about, though I'm not sure it would make sense as part of this proposal in the absence of a ModuleSource constructor |
| 22:40 | <Kris Kowal> | You don't need a full parser just to get the bindings out |
| 22:40 | <bakkot> | Yup, but those things are lot smaller and faster than babel's parser. |
| 22:40 | <bakkot> | In particular not having to construct trees saves a lot of time. |
| 22:41 | <Kris Kowal> | That’ll run you about 16kloc. We’ve got guybedford (Guy Bedford)’s for that. |
| 22:41 | <littledan> | https://github.com/guybedford/es-module-lexer |
| 22:41 | <littledan> | <1kloc |
| 22:42 | <bakkot> | Stage 2 implies all sorts of other important qualities, like "is the committee sort of committed to doing something here" and "do we have a concrete first draft"; it is really common to have disagreements about major semantic points when something goes to Stage 2 |
| 22:42 | <Kris Kowal> | That’s…much smaller than his other one. Surprising! |
| 22:43 | <bakkot> | our full JS parser isn't even 16k semantic LoC |
| 22:43 | <bakkot> | (it's like 3-5k I think) |
| 22:46 | <Kris Kowal> | Oh, pardon. I’m off by ten. I’m looking at Guy’s CJS lexer (which we use in prod at Agoric) and it’s 1.6K. |
| 22:48 | <Kris Kowal> | So, agreed, it’s consequently practical to statically analyze the bindings of a module in production without ModuleSource(text).bindings or a lower-powered Module.parse(text). So making a web page that generates bundles isn’t that much trouble. |
| 22:50 | <littledan> | Eh, well, depend on what you mean by "major semantic points". In theory stage 2 signifies "all major semantics, syntax and API are covered", and "does this add a new eval" is IMO a major semantic point. But like I said I don't feel that strongly about this, as long as it's understood that agreeing to stage 2 doesn't signify consensus on this question either way. |
| 22:50 | <littledan> | anyway sorry lawyering here is beside the point |
| 22:51 | <littledan> | not trying to deny your right to block this proposal (which has another sort of existential question, about whether we want the importHook to be hookable at all) |
| 22:54 | <Kris Kowal> | There are a number of such questions. The folks at Moddable discovered that implementing Module and ModuleSource made it possible for their fuzzer to discover bugs in their module system that were previously too impractical to cover. They’ve fixed the bugs, but it highlights that it’s hard to get ESM right. There’s certainly some philosophical wiggle-room whether composing ESM with callbacks creates more bugs than it reveals. |
| 22:54 | <littledan> | I guess I felt like the ordering should be more like, we focus first on whether we want importHook to be a thing at all (as Shu raised), and then we'd resolve lower-order things like whether the ModuleSource constructor should exist |
| 22:55 | <Kris Kowal> | That is certainly the most efficient way to prune the tree. |
| 22:55 | <littledan> | There are a number of such questions. The folks at Moddable discovered that implementing Module and ModuleSource made it possible for their fuzzer to discover bugs in their module system that were previously too impractical to cover. They’ve fixed the bugs, but it highlights that it’s hard to get ESM right. There’s certainly some philosophical wiggle-room whether composing ESM with callbacks creates more bugs than it reveals. |
| 22:56 | <littledan> | I guess I felt like the ordering should be more like, we focus first on whether we want importHook to be a thing at all (as Shu raised), and then we'd resolve lower-order things like whether the ModuleSource constructor should exist |
| 23:11 | <Kris Kowal> | I assume this is sufficient to accept exactly and only grammatically valid sources. That requires block matching, but not AST generation. The other motivation for a full parse is that eval('module {' + text + '}') is an inevitable work-around for the lack of ModuleSource, which is much worse if text is not constrained to be a valid module source. |