01:17
<shu>

planned agenda for tomorrow's meeting:

  • method sharing and prototype lookup
  • property redefinition (and freezing)
  • R&T interaction
01:25
<asumu>
Unfortunately I don't think I will be able to attend tomorrow due to a conflicting appointment. I don't have much to report, Wasm GC is moving towards getting a formal spec (& JS API spec) so if anything relevant comes out of that I'll report in future meetings.
01:28
<shu>
thanks asumu
02:14
<Ashley Claymore>
Also feel free to have at the original time next week (10am PT), I'm getting an earlier train than I would usually today :)
^^ if missed before. Feel free to keep the 10am time, if that works better for asumu . My clash last week was not reoccurring
15:54
<rbuckton>

planned agenda for tomorrow's meeting:

  • method sharing and prototype lookup
  • property redefinition (and freezing)
  • R&T interaction
I'm also curious if we can leverage using with Mutex/ConditionVariable, since that was one of the reasons I focused on getting that proposal to Stage 3.
16:50
<shu>
sure, sounds good
17:04
<shu>
Ashley Claymore: the call is now, btw
17:05
<Ashley Claymore>
omw
17:05
<Ashley Claymore>
sorry, actually 1 min
17:05
<shu>
np
18:39
<rbuckton>

Regarding the "shared modules" suggestion. I may have jumped a few steps ahead in my reasoning without explaining how I got there, so I'll take a few steps back.

If we imagine an implementation of shared structs that contains some form of methods, what can those methods close over? Only globals? What about functions that are siblings to the shared struct? What about imports?

If we do not close over these things, the methods of shared structs won't be able to reuse useful utilities such as common vector math operations you might use in a 3D graphics library, that aren't somehow patched into globalThis.

If we do close over these things, how do we ensure the module graph has been instantiated on the worker thread? What about initialization logic that might be needed that wires together some of these modules, applies polyfills, etc.? What if these modules contain top-level await or must otherwise be loaded asynchronously?

How and when do we instantiate the thread-local (or per-realm?) prototype for each struct, especially if doing so might kick off this kind of module loading in the worker?

The leap in logic I took to the "shared module" approach was to address these concerns.

  • "Shared modules" would have an identity in the module cache that could be used as part of the type identity.
  • "Shared modules" would promote code reuse.
  • "Shared modules" would be restricted to containing only those things that can safely be shared, i.e. references to globals (which would be re-bound per realm), local functions, variables, shared structs, and imports/exports from other "shared modules".
  • Due to the nature of the above restrictions, "shared modules" avoid accidental references to non-shared code.

It could be that the idea of isolating shared code to its own file is overkill, but many developers are already used to doing this with things like protobuf today (i.e., maintaining their protobuf schema in a separate file).

18:52
<littledan>

My answer to this closing over/loading question was, the other side which receives the object has three options for handling module loading:

  • The receiving side already expected that the object would come, so the module where the shared struct is defined has already been loaded, and the receiving module can start using it immediately.
  • [I honestly can't think of a use case for this, but ] The receiving side sets itself up to handle objects dynamically, so it queries the object it received for the module specifier, await import()s that, and then can use the methods.
  • The receiving side just wants to use the plain old data, and can do so without importing anything.

In all cases, there are no particular restrictions in what is closed over (just by construction because we do this whole dance per module map). And there just is no such thing as shared code, no limitations on mutating the local copy of the shared classes, or on TLA (because no synchronous module loading is ever used, just normal async). It does depend on one or other type of identity (which could be URL, or module block, or symbol if we have a global mapping).

What do you see as the downsides of this option?

18:53
<littledan>

What about initialization logic that might be needed that wires together some of these modules, applies polyfills, etc.?

I don't have a solution to this; I was assuming that you could somehow bake this into the module.

18:53
<littledan>
(which doesn't mean necessarily bundling all recursive dependencies! it can have import statements like normal.)
18:57
<littledan>
I honestly don't understand how "shared modules" would work in detail--how they would differ from this, beyond being a subset requiring only recursive use of shared modules
19:53
<rbuckton>

The receiving side already expected that the object would come, so the module where the shared struct is defined has already been loaded, and the receiving module can start using it immediately.

This is probably reasonable, as long as you can reliably correlate a shared struct type in both realms by module id and export name. However, that would potentially restrict shared struct definitions to only be at the top level of a Module, since returning them from a function call might not necessarily result in the same identity being valid.

19:54
<rbuckton>

[I honestly can't think of a use case for this, but ] The receiving side sets itself up to handle objects dynamically, so it queries the object it received for the module specifier, await import()s that, and then can use the methods.

This seems like a poor developer experience.

20:04
<rbuckton>
The "shared module" I was imagining would be fairly restrictive so as to have the declarations only really be resident in memory once, and not reparsed/linked/evaluated per-realm. No per-realm initialization, variables limited to constant, primitive values (and trivially reduceable expressions containing primitives), only top-level declarations: structs, functions, vars, imports/exports, maybe enums (if we can find a version of that proposal that might be accepted).
Such a module could be accessed via module id, and reachable from any worker/realm. Evaluating functions/methods/constructors/etc. from a "shared module" would use the current realm.
Struct type identity would be trivially resolvable via module id+export name, producing the correct prototype in each realm.
20:05
<rbuckton>
A fourth approach, which I'm trying to enable with this design, is that the receiving end doesn't need to worry about running code to support the struct since its easily reachable.
20:10
<rbuckton>
Yes, the setup is more restrictive due to the limitations imposed by a "shared module", but it also avoids many pitfalls like developers inadvertently depending on thread-local or realm-local state in shared code. The benefit being that consuming shared structs is simple and intuitive. You just send the value via postMessage and can use it immediately in the worker without any added fuss.
20:12
<littledan>
OK, so this is trying to solve the stronger version of the problem that you explained
20:13
<littledan>

The receiving side already expected that the object would come, so the module where the shared struct is defined has already been loaded, and the receiving module can start using it immediately.

This is probably reasonable, as long as you can reliably correlate a shared struct type in both realms by module id and export name. However, that would potentially restrict shared struct definitions to only be at the top level of a Module, since returning them from a function call might not necessarily result in the same identity being valid.

Yes, shared structs which define methods that are supposed to be accessible from other agents need to be defined at the top level of a module. I agree that this is a singificant restriction.
20:13
<littledan>

[I honestly can't think of a use case for this, but ] The receiving side sets itself up to handle objects dynamically, so it queries the object it received for the module specifier, await import()s that, and then can use the methods.

This seems like a poor developer experience.

It's hard for me to evaluate how bad it is without understanding the use cases for this scenario.
20:13
<rbuckton>
Yes. It's trying to impose restrictions on what a shared struct can reference so as to make the rest of the system simple and intuitive.
20:14
<littledan>
I just can't construct the scenario in my head where it wouldn't be natural to directly import the module defining the shared struct, when you expect to receive it in postMessage
20:15
<rbuckton>
I came at this from the perspective of: "Lets say we wanted to implement Number as a shared struct, from the ground up, what would we need to do?" (excl. operator overloading)
20:16
<rbuckton>
Would you want everyone to need to write import "std:number"; in their module to receive a number via postMessage?
20:17
<littledan>
While that's an interesting lens, I like to think of those things being in an implicit "prelude". (The same logic applies for the operator overloading usage declarations, for example)
20:18
<littledan>
so I guess I would go for, "let's see if we can implement Number except for that specific import statement"
20:18
<rbuckton>
You'd need to use a side-effecting import "structModule" if you never access the constructor yourself, otherwise minifiers will tree shake it away.
20:19
<rbuckton>
Which just adds one more source of potential confusion when things don't work in your bundled, minified release build.
20:19
<littledan>
Ah, I hadn't really considered tree shaking
20:21
<littledan>
are there any other problems that come to mind for you besides tree shaking?
20:21
<rbuckton>
As I mentioned in the thread above, depending on an import is one more shaky foundation to build on that is a potential pit of failure for developers. A tree shaking minifier might remove the import, or would need to perform additional static analysis to know whether its actually safe to remove the import.
20:22
<littledan>
I mean, I think we could teach tree shakers this particular thing: You can't just eliminate running an export of a shared struct, since executing that has a side effect. (We'd have to teach the tree shaker about that syntactic construct anyway!)
20:23
<littledan>
OK, thanks for explaining; I hadn't considered that
20:23
<rbuckton>
  • Remembering to include the import
  • The main process changing the data it sends to the worker (depending on how the app is structured)
  • Middleware that might run before application code is loaded.
20:24
<rbuckton>
Its not the tree shaking of the export, its the tree shaking of the import. That requires looking across files to say "oh, this import is from a module that transitively imports a module containing a shared struct that I might potentially receive", which is far more complicated.
20:25
<littledan>
Its not the tree shaking of the export, its the tree shaking of the import. That requires looking across files to say "oh, this import is from a module that transitively imports a module containing a shared struct that I might potentially receive", which is far more complicated.
oh, I guess I assumed that this was normal stuff for tree shakers
20:25
<littledan>
that a module execution may be known to have a side effect and that that shouldn't be removed
20:25
<rbuckton>
If your app/package contains a single shared struct definition, imports becomes un-tree-shakable.
20:26
<littledan>
right
20:26
<littledan>
if this is an issue you could break up the module
20:26
<rbuckton>
That seems bad.
20:27
<rbuckton>
if this is an issue you could break up the module
Which is basically what a "shared module" enforces.
20:27
<littledan>
  • Remembering to include the import
  • The main process changing the data it sends to the worker (depending on how the app is structured)
  • Middleware that might run before application code is loaded.
OK, I guess the badness of that, together with the badness of this comment I'm replying to, is something which I don't have sufficient intuition into.
20:27
<littledan>
or sufficient practical experience of the negative consequences
20:29
<rbuckton>
Due to their restrictions, "shared modules" have no Evaluation step when the module is loaded. Dependency order becomes far less important (excluding decorators, which I'd have to think more on), so you could just surface the shared module imports in place of whatever other import you might have used that was otherwise removed.
20:44
<littledan>
computed property names also do stuff when evaluated, as do, you know, the RHS of an export const... I'm pretty skeptical that it'd be practical to articulate a usable-enough subset of JS which doesn't have side effects when loaded. This would be very useful if possible, of course! It'd handle the lazy module loading issue
23:04
<shu>
unrelated sidebar: matrix has threads??
23:04
<shu>
is this a new feature?
23:05
<shu>
i was wondering why it was showing the channel as having 25 unread messages until i found the thread above
23:43
<asumu>
It’s had threads experimentally for a while (opt in) but they recently turned it on by default.