00:00
<Mathieu Hofman>
I just don't think there is a way in JS to express linear types, which would pretty much be required for any sender opt-in of exclusive use
00:00
<Kris Kowal>
True.
00:01
<Kris Kowal>
Are you arguing that detachment is a composition hazard we shouldn’t tolerate?
00:01
<Mathieu Hofman>
Ah you'd need to add an explicit "detach" for CoW to make sense, so that the program can express release of the copy before GC can figure that out
00:02
<Kris Kowal>
And the sender and receiver would need separate views so the sender’s detachment doesn’t break the receiver.
00:02
<Mathieu Hofman>
Are you arguing that detachment is a composition hazard we shouldn’t tolerate?
Correct. The sender would need to somehow opt-in it expect the receiver to temporarily claim a lock
00:02
<Kris Kowal>
Thus take()
00:03
<Kris Kowal>
Or rather, give()
00:03
<Kris Kowal>
Or rather, transfer()? I think I see where this is headed.
00:03
<Mathieu Hofman>
And the sender and receiver would need separate views so the sender’s detachment doesn’t break the receiver.
If by view you mean the array buffer, then there's already an API for that: slice()
00:04
<Kris Kowal>
Fair.
00:04
<Mathieu Hofman>
(for CoW)
00:04
<Mathieu Hofman>
Sorry I'm jumping back and forth between the 2 approaches.
00:05
<Mathieu Hofman>
Anyway, it seems that CoW is not the way, but I really don't know how to solve all uses cases without it.
00:06
<Kris Kowal>
Sorry I'm jumping back and forth between the 2 approaches.
Yeah, I got lost with slice() for CoW because it doesn’t express detachment to the original.
00:20
<Mathieu Hofman>

Ok to summarize:

For CoW, you'd do: writer.write(ab); with async write (ab) { const copy = ab.slice(0); try { await doWrite(copy); } finally { copy.detach() } }

For explicit take you'd have to do something like: write.write(ab.getTaker()) with async write (taker) { const [ab, release] = taker(); try { await doWrite(ab); } finally { release(); } }

taker() could only be called once (throws after) and returns both the ab and a release function, which once called restore the state of the original array buffer instance. I guess we could imagine different modes for the taker. For example readonly would yield a readonly buffer and put the original buffer in read-only mode, which means they could be the same object. A write mode would yield a mutable buffer, putting the original buffer in read-only mode?

There's always the risk that the receiver to which you provide the taker will never call release.

00:22
<Mathieu Hofman>
That's the only pattern I can imagine right now providing for the explicit/opt-in and guaranteed transfer of use, while still allowing the buffer to be reused.
01:27
<Domenic>
I think the two-stage getTaker/take pattern works. It could be implemented today if you don't care about releasing it back.
01:37
<Domenic>
https://gist.github.com/domenic/a9343fa787ba54b4ba3a60882c49cc32#file-zero-copy-mjs
01:45
<Domenic>
I just fixed a crucial bug, please reload :)
01:46
<Mathieu Hofman>
You'd still need a mechanism to guarantee the uniqueness of a taker for a given array buffer, and for the take to guarantee the original array buffer gets detached (at least temporarily). For these reasons it cannot currently be done in userland
01:46
<Domenic>
transfer() is that mechanism
01:46
<Domenic>
Or structuredClone(ab, { transfer: [ab] }) if you want it literally today
01:47
<Mathieu Hofman>
Oh that's the refresh needed, yes
01:47
<Mathieu Hofman>
yeah if we give up on the reusable aspect, transfer would do
01:48
<Mathieu Hofman>
technically we'd still need some kind of brand check to ensure the taker function is actually the thing that implements the right transfer semantics
01:49
<Mathieu Hofman>
oh takeOrCopy does that
01:49
<Domenic>
Yeah this needs to be reimplemented in C++ or non-idiomatic first-running JS to actually give the guarantee it claims.
01:50
<Mathieu Hofman>
right, I had read too quickly and missed the take() was actually done through takeOrCopy which brand checks the taker
01:51
<Mathieu Hofman>
of course that assumes ArrayBufferTaker.takeOrCopy is the original, but that's the same constraint for every other JS API
01:53
<Mathieu Hofman>
I think we should still look at a release pattern, which would require changes to the AB internals, as to keep the re-use use cases
01:55
<Domenic>

Streams would definitely use that. Right now users have to do

const { value: newUint8Array } = reader.read(inputUint8Array);

where newUint8Array.buffer !== inputUint8Array.buffer but they point to the same backing memory. If people didn't have to juggle multiple variables for the same backing memory, that would be nicer.

01:56
<Domenic>
(inputUint8Array.buffer gets detached, with newUint8Array.buffer pointing to the same memory, with data read into it from the stream.)
02:00
<Mathieu Hofman>
ha I didn't realize the reader did a double transfer
02:01
<Mathieu Hofman>
honestly I haven't had the opportunity to use the Web streams API yet, but that makes sense to implement the exclusivity guarantees
04:26
<jmdyck>
If I go to a PR's discussion page and see an entry saying "jmdyck started a review" and then "Pending" (because apparently I never submitted it), how do I do anything with it now (e.g., submit, cancel, modify)?
04:29
<jmdyck>
Never mind, it looks like I can Delete individual comments.
09:03
<annevk>
Domenic: it would be nice if that gist had a sentence or two about why it's not as simple as detaching the input
10:37
<Domenic>
Done
14:54
<Justin Ridgewell>
Wait, do Web APIs that process the buffer sync also make slices of the buffer?
14:54
<Justin Ridgewell>
Is it only APIs that hold on to the buffer past the current execution?
14:58
<annevk>
Justin Ridgewell: it depends, if the processing happens in a method call a copy is typically made if the API has [AllowShared] or if the API needs to manipulate the buffer for some reason
14:59
<annevk>
Justin Ridgewell: I recommend looking at Encoding and Fetch for some representative examples
15:38
<littledan>
There could be an even fancier version of this: many APIs just need a read-only view of the buffer, and only to borrow it for a limited amount of time. We could allow others to also take a simultaneous read-only view as well, or even to do direct read operations if we want to get super fancy.
16:18
<Mathieu Hofman>
One issue is that a take of a read-only version in this case shouldn't allow to detach the array buffer, as it'd prevent the sender to get it back
16:25
<James M Snell>

I definitely like the idea of this API. I think I'd prefer to generalize it a bit so it's not tied specifically to ArrayBuffer/ArrayBuffer views only. If a generic concept of a "transferable" object were introduced to in the language (of which only ArrayBuffer and TypedArrays would initially be included), then this taker api would work with any transferable that is introduced later or by any objects that other specifications separately define as transferable (https://html.spec.whatwg.org/multipage/structured-data.html#transferable).

To modify Domenic s example a bit.. it would be something like... const holder = new TransferableHolder(anyTransferable); const thing = holder.take();

The question then becomes what objects are Transferables? We can introduce two new well-known Symbols to address that and the relevant use cases: Symbol.transfer and Symbol.clone.

16:44
<littledan>
I think more things are transferrable/shareable within an Agent than between Agents
16:44
<littledan>
the Symbol approach is a lot easier if it's within the same Agent
20:49
<tolmasky>
From a terminology perspective, given a well-known intrinsic object "%name%", if the spec references "%name.a.b%", it that ALSO considered to be a "well-known intrinsic object" (just like %name%), or it just an "intrinsic object"?
20:49
<bakkot>
Is there an observable difference?
20:51
<tolmasky>
I guess that is part of my question, they also are unique per realm, but don't show up in the well-known intrinsic objects table, so if I were to make an "exploded table" of all the referenced items, just want to know whether to refer to them as well-known or not. Perhaps, for example, there is an observable difference somewhere else in the spec that I haven't put together
20:57
<ljharb>
tolmasky: technically only the things in the table are well-known, but because of the way the notation is specified now, that distinction doesn't make a difference anymore
20:58
<ljharb>
i'd just call them all "intrinsics"
20:59
<tolmasky>
Gotcha, so just to clarify then, "intrinsics" are "built-ins" referenced in the spec. If there is a built-in in the particular engine that is not referenced in the spec (but otherwise behaves identically, unique copy per realm, etc. etc.), it is not an intrinsic
20:59
<tolmasky>
In other words well-known-intrinsic=intrinsic, but intrinsic!=built-in
20:59
<ljharb>
what'd you have in mind as an example of one that's not?
21:00
<ljharb>
every intrinsic is indeed built in
21:00
<tolmasky>
but not all built-ins are intrinsic is what i mean
21:00
<tolmasky>
like, if I just write a custom engine and make available global.Francisco = { }, its built-in but not intrinsic
21:01
<tolmasky>
Or, if Function.prototype happens to have some custom added property in my custom engine, it also is not intrinsic, despite being built in, and getting a fresh copy in every realm
21:04
<tolmasky>
In other words, is "intrinsicness" meant to represent any properties beyond built-inness, like a "ECMA seal of official spec approval" vs. just "its a thing thats around at the beginning, that gets a custom copy in every realm"
21:39
<Kris Kowal>
I usually take “intrinsic” to be short for “intrinsic to a realm”.
21:40
<Kris Kowal>
MarkM begs a distinction between intrinsics and his own invention of “primordial” that I cannot explain, but the distinction might be germane here.
22:07
<ljharb>
tolmasky: i would just say those are different kinds of intrinsics - one's a language intrinsic, and the other is a platform intrinsic
22:08
<tolmasky>
gotcha
22:08
<tolmasky>
and, not to complicate this further, but intrinsics can be values, in the case of the well-known Symbols, which are not "Objects"
22:09
<ljharb>
of course, Number.MAX_SAFE_INTEGER is an intrinsic
22:10
<tolmasky>
but, there isnt a different "copy" of MAX_SAFE_INTEGER across realms (they all have value identity), however, there are different copies of the well-known symbols per realm, realm1.Symbol.iterator !== realm2.Symbol.iterator (or ARE they value identical?)
22:13
<tolmasky>
I guess it is the same in every realm
22:37
<ljharb>
correct, they're the same in every realm
23:34
<jmdyck>
there are built-ins that aren't intrinsic, but they're a bit obscure
23:36
<jmdyck>
If you look for explicit calls to CreateBuiltinFunction (i.e., not the generalized one in CreateIntrinsics), those are creating built-in functions that aren't intrinsics.
23:47
<jmdyck>
The spec normally only uses "intrinsic" to refer to objects, but Number.MAX_SAFE_INTEGER is a Number value, so I don't think the spec supports calling it an intrinsic. But I don't think it matters much if you want to call it one.