TC39 General
2023-10-05
plaintext logs
prev next
source on github
00:43
<Mathieu Hofman>

Hello all. I've got a proposal that I'd like to surface for consideration. I put this together after speaking a bit with Matteo Collina and @ljharb... The fundamental idea is to introduce a mechanism for zero-copy concatenation of ArrayBuffer in a way that allows the result to still be an ArrayBuffer that can be wrapped with a TypedArray. The explainer is here: https://github.com/jasnell/proposal-zero-copy-arraybuffer-list/blob/main/README.md

For a quick example:

const ab1 = new ArrayBuffer(10);
const ab2 = new ArrayBuffer(20);
const combined = ArrayBuffer.of(ab1, ab2);
const u8 = new Uint8Array(combined);

Here, combined is effectively a list of the component ArrayBuffer instances that is itself an ArrayBuffer.

The idea here is adapted from the very popular npm module bl which implements a similar idea around Node.js Buffer interface but in a way that still has a number of warts.

There is a more detailed example in the explainer. @littledan and ljharb have already graciously provided some extremely helpful feedback.

Oh I've been wanting this for years. I think I wrote an issue somewhere!
00:50
<Mathieu Hofman>
I also still really want CoW ArrayBuffer slices. I still do not understand how it would introduce much more complexity than the existing detached checks already required.
01:03
<Mathieu Hofman>

Basically I want to be able to do

const chunks = [];
chunks.push(chunk1.slice(10));
chunks.push(chunk2);
chunks.push(chunk3.slice(0, 5));
return ArrayBuffer.of(...chunks);

Obviously each chunk is received in separate events / iterator yields

01:04
<Mathieu Hofman>
That said I do expect the new combined buffer to itself be a CoW
01:04
<Mathieu Hofman>
And not a passthrough to the underlying buffer
01:06
<James M Snell>
The proposal currently does not include CoW but I can't see a reason why it couldn't be. Will give that some thought
01:09
<Mathieu Hofman>
Btw, in that case it really become a concat and the fact that the buffer is in fact a list of smaller buffers is just an unobservable implementation detail
01:10
<James M Snell>
as long as we're able to preserve the zero-copy concat and zero-copy subarray, then I'm fine with that
01:10
<Mathieu Hofman>
Basically I'm really concerned about having multiple ArrayBuffer instances backed by the same underlying data. That's more in the realm of SharedArrayBuffer semantics
01:12
<James M Snell>
True, but to be fair host implementations already give us that ability
01:13
<James M Snell>
(obviously that doesn't mean we should make it easier :-) ...)
01:16
<Mathieu Hofman>
I don't believe any host APIs currently expose that ability, right? I know that JS APIs don't
01:18
<Mathieu Hofman>
Anyway, my motivation for CoW is that I believe it would increase the performance of a ton of existing applications without requiring any code changes on their end
01:19
<Mathieu Hofman>
I think Luca Casonato shares that belief
01:19
<James M Snell>
With v8, it's fairly trivial to extract a std::shared_ptr<v8::BackingStore> and have it shared by multiple v8::ArrayBuffer instances
01:19
<James M Snell>
Not ideal, but trivial :-)
01:25
<Luca Casonato>
I think Luca Casonato shares that belief
Yes, a general purpose CoW optimization for AB.slice would enable many host APIs to become significantly faster
01:27
<Luca Casonato>
I don’t think we necessarily need a new API here (I view concat as a related, but separate API - it can make sense with or without the CoW optimization)
01:27
<Luca Casonato>
The nice thing about CoW is that it’s completely unobservable
01:28
<James M Snell>
Yeah, I'd view the proposal for ArrayBuffer.of(...) and CoW as orthogonal. Both nice to have but distinct
01:29
<James M Snell>
If we could get CoW slice, however, there would be no need at all for the ArrayBuffer.prototype.subarray(...) that I suggest in the proposal
01:31
<James M Snell>
Also if we had CoW, an argument could be absolutely made also that ArrayBuffer.of(...) should automatically slice(0, len) to ensure that the new ArrayBuffer is composed entirely of CoW slices
01:41
<Luca Casonato>
I share Mathieu Hofman’s view that we probably should not support having two AB objects backed by the same (or a sub array of the same) backing store
01:42
<ljharb>
if it’s unobservable it doesn’t need to be in the spec tho, and arguably couldn’t be
01:42
<ljharb>
impls can just do it
02:02
<Luca Casonato>
I agree
03:15
<Mathieu Hofman>
Yes that's the difficulty I've been having with this. CoW is technically an unobservable optimizing engines could make today. But we'll only get it if there is sufficient feedback from the community it's needed. Proposals like this IMO show the need.
03:17
<Mathieu Hofman>
In that world , a concat feature is orthogonal. But it mostly makes sense if implemented as a list of CoW buffers because otherwise it's equivalent to the program creating a new contiguous buffer and copying into it.
09:49
<ljharb>
sooo it sounds like this proposal might have a beneficial side effect of nudging engines towards Just Doing CoW, and nothing further need be done?
11:18
<rkirsling>
need a shu deck titled Have a CoW Man
12:54
<James M Snell>
Well, having the CoW bit would be great but I don't want to lose track of the zero-copy concat use case, which is what motivated the proposal
13:46
<Mathieu Hofman>
I understand the performance motivation, but is there any reason zero copy should be something observable directly by the program?
13:49
<ljharb>
it'd be observable if you mutated one and wanted to see the result in the other, which seems like kind of the main purpose of wanting zero-copy? (other than perf)
13:50
<James M Snell>
Well, the vast majority of my zero-copy use cases are for read. Specifically, around things like optimizing Stream API implementations due to the fact that WHATWG streams have no concept of writev the way Node.js streams do
13:52
<James M Snell>
For instance, I just had a case where a streams impl was resulting in many small writes that needed to be aggregated into larger buffers before being passed down a transform pipeline... unfortunately it's currently not possible to do without copying or modifying the various stream impls down the pipeline which is... difficult
13:54
<James M Snell>
in the implementation here I would likely be calling transfer() anyway so I don't really care so much about mutations being visible as allowing zero-copy concat for reading
13:59
<Mathieu Hofman>
Yeah that's basically my question, is there any use cases requiring observing the mutation through an aggregation or subarray. I can't think of any, and if those are in fact needed, I'd prefer we add that set of features to SAB which has the shared backing store semantics.
14:03
<Mathieu Hofman>
For instance, I just had a case where a streams impl was resulting in many small writes that needed to be aggregated into larger buffers before being passed down a transform pipeline... unfortunately it's currently not possible to do without copying or modifying the various stream impls down the pipeline which is... difficult
Btw, it was the exact same experience of stitching together smaller chunks that led me to wanting CoW and an internally optimized concat.