TC39 Delegates on 2022-09-05

01:13	<ljharb>	i didn’t realize that. It shouldn’t use instanceof - like everything else in the language, it should use a brand.
01:17	<bakkot>	no, it's an interface
01:17	<bakkot>	it's not a class
01:17	<bakkot>	I mean, it is also a class
01:18	<bakkot>	but there is no particular reason a userland instance needs to have the brand
02:14	<snek>	i didn’t realize that. It shouldn’t use instanceof - like everything else in the language, it should use a brand. how do you brand something whose definition is "has a next method that returns { value, done }"
02:15	<snek>	or if you subscribe to the other idea, "an object inheriting Iterator.prototype"
03:35	<ljharb>	objects in the language that inherit from a prototype also have a brand, thats checked in places.
03:38	<snek>	no i like
03:38	<snek>	i do not understand the mechanic by which this would work
17:17	<ljharb>	anything that's `instanceof Iterator` in the language would have an internal slot, which is what Iterator.from would check
17:28	<bakkot>	in the language or, presumably, the web platform?
17:28	<bakkot>	I suppose that would be possible but... why?
18:22	<snek>	anything that's `instanceof Iterator` in the language would have an internal slot, which is what Iterator.from would check how do they get the internal slot? they're just random objects
18:23	<snek>	oh are you saying Iterator.from would be the thing that produces objects with that slot
18:25	<snek>	technically the new object it returns has `[[Iterated]]`
18:25	<snek>	but
18:25	<snek>	it doesn't have to return the new object
18:26	<snek>	like if i run `Iterator.from({ next() {}, __proto__: Iterator.prototype })`, it should return that argument unchanged
19:26	<ljharb>	that doesn’t seem like what it should be doing at all.
19:27	<ljharb>	Array.from doesnt pass through return an arraylike object that has a proto of Array.prototype - it returns a proper array. So should Iterator.from.
19:28	<bakkot>	the definition of "proper iterator" is "inherits from Iterator.prototype", in this context
19:44	<ljharb>	i don't think that is or should be the definition
20:00	<bakkot>	why?
21:08	<snek>	Iterator is not a concrete thing, I'd rather rename the method if that's the problem
21:15	<ljharb>	the whole point of Iterator.from is that it's becoming a concrete thing
21:15	<snek>	no its just that its giving it the prototype
21:16	<snek>	its a convenience for the methods
21:16	<ljharb>	… and the methods do a wrap (to something with the slot) if they're `.call`ed on a random object?
21:17	<snek>	i don't understand what this means
21:17	<ljharb>	the `[[Iterated]]` slot you mentioned
21:18	<snek>	`[[Iterated]]` is just used for %WrapForValidIteratorPrototype%.next/return
21:18	<ljharb>	how can the next method retain internal state unless it has a place to put it?
21:19	<snek>	WrapForValidIteratorPrototype objects don't have any internal state
21:19	<snek>	[[Iterated]] is just the object they wrap
21:19	<snek>	well, the iterator record for it
21:20	<ljharb>	ok so like `map`. it has to hold on to the original iterator, and to the callback
21:21	<ljharb>	so presumably the iterator object returned by `map` holds that, in an internal slot
21:21	<snek>	they use closures
21:22	<ljharb>	so the iterator object doesn't use a shared `next` method, it makes a new one each time `map` is called?
21:22	<snek>	uhh
21:22	<snek>	it uses %IteratorHelperPrototype%.next
21:22	<ljharb>	right. but that's a shared method.
21:22	<snek>	which is just %GeneratorPrototype%.next but fancy
21:22	<ljharb>	so it can't close over something that's unique to a given `map` call
21:23	<ljharb>	because it exists before `map` is ever called.
21:23	<ljharb>	so where is that state held so that `next` can access it?
21:23	<snek>	same place that normal generators store it
21:23	<snek>	the only thing is
21:24	<snek>	you can grep for `[[GeneratorBrand]]` in the spec
21:24	<ljharb>	right but that's in a slot on the generator instance, no?
21:24	<snek>	https://tc39.es/ecma262/#sec-generatorvalidate
21:24	<snek>	this is the only way it tells them apart
21:24	<snek>	but that just tells you if it was a generator helper method
21:24	<snek>	tbh i'm not sure why we separate them at all, they just call `GeneratorResume`
21:25	<ljharb>	right. and [[GeneratorContext]] and [[GeneratorState]] are slots on the iterator object that hold the state
21:25	<snek>	yeah its a generator
21:25	<ljharb>	right
21:26	<ljharb>	and GeneratorValidate brand-checks the iterator. it doesn't matter what its [[Prototype]] is, it matters if it has the expected slots.
21:26	<snek>	ye
21:26	<ljharb>	so i assume that Iterator Helper methods all do the same validation
21:26	<snek>	iterator helper methods do not know about those brand checks, they perform them via calling stuff like `GeneratorResume`
21:26	<ljharb>	right but they still exist
21:26	<snek>	sure
21:27	<ljharb>	which means that `{ __proto__: Iterator.prototype }` would fail that check, and not be a usable receiver with the Iterator Helper methods. which means Iterator.from musn't ever return it.
21:27	<ljharb>	which means that Iterator.from always returns a brand-checkable object, one that GeneratorValidate will accept.
21:27	<snek>	uhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
21:27	<snek>	no
21:27	<ljharb>	what am i missing there?
21:28	<snek>	assuming this `let iterWithGeneratorSlots = [].values().map(x => x);`
21:28	<snek>	`Iterator.from(iterWithGeneratorSlots)` passes it through unchanged because it inherits from %Iterator.prototype%
21:28	<snek>	but
21:28	<snek>	if you do `Iterator.from({ next() {}, __proto__: Iterator.prototype })` it also passes it through unchanged
21:29	<snek>	that's not doing any generator stuff
21:29	<snek>	if you do `Iterator.from({ next() {} })` that creates a %WrapForValidIteratorPrototype% object and returns it
21:29	<ljharb>	ok but what happens with `Iterator.from({ next() {}, __proto__: Iterator.prototype }).map(x => x)`?
21:29	<snek>	but that still doesn't have any generator slots
21:29	<snek>	cuz its not a generator
21:29	<snek>	its just a stupid wrapper
21:29	<snek>	ok but what happens with `Iterator.from({ next() {}, __proto__: Iterator.prototype }).map(x => x)`? that has generator slots cuz you called map
21:30	<ljharb>	ok, and `.map` doesn't care if its receiver has the slots?
21:30	<snek>	no it does not
21:30	<ljharb>	oof
21:30	<snek>	all it cares is that the receiver is an iterator
21:30	<snek>	https://gc.gy/130118458.png
21:32	<snek>	this is why `{ next() {}, __proto__: Iterator.prototype }` is valid
21:32	<snek>	with protocols you have to assume that the object is already created as some other shape, you're augmenting them with the functionality, implementing the protocol on top of something else
21:33	<ljharb>	i don't understand why, if everyone's going to `Iterator.from()` anyways to get the methods, we wouldn't want to unconditionally wrap - ie, to unconditionally `.map(x => x)`
21:33	<snek>	i think its pretty rare that you would need to use Iterator.from
21:33	<ljharb>	that way we have a consistent "real iterator" definition
21:34	<ljharb>	i think you'll want to do it any time you're dealing with a user-supplied iterator. which will be frequent.
21:34	<snek>	i mean, those are rare lol
21:34	<ljharb>	you'd want to maximally accept and normalize all possible iterators
21:34	<ljharb>	that's what `Promise.resolve` is good for too
21:34	<snek>	yeah i mean
21:34	<ljharb>	it normalizes thenables into Real Promises, which are always preferred
21:34	<snek>	if you're you, writing some sort of complex fancy library that has to deal with the world
21:34	<ljharb>	similarly i'd always want a Real Iterator, not just an "iterator"
21:34	<snek>	go for it
21:35	<ljharb>	this isn't just a "my unique coding style" thing tho, this is any package
21:35	<ljharb>	anything that accepts an iterator. just like anything that accepts a thenable always uses Promise.resolve or `await` to normalize it first into a real promise
21:35	<ljharb>	Iterator.from will be exceedingly common imo.
21:35	<snek>	its a nice convenient method yeah
21:35	<snek>	i was just pointing out
21:35	<snek>	its superfluous in 99% of cases
21:36	<snek>	so we made it not add extra overhead
21:36	<ljharb>	i don't think that's true
21:36	<ljharb>	i think 99% of cases will be dealing with untrusted iterators
21:36	<ljharb>	the case where you're iterating your own iterator are likely to be far rarer. (where "your own" doesn't deal with who the author is, but which chunk of code owns it)
21:37	<ljharb>	"extra overhead" isn't a real problem (it can be optimized away), but a footgun like "sometimes it doesn't wrap" imo would be
21:37	<snek>	the genesis of this proposal was the observation that all web and js platform iterators already inherit from a shared %IteratorPrototype%
21:37	<snek>	and the remaining weird ones often use the `(function*(){})().next().__proto__` trick or whatever it is
21:37	<ljharb>	then whose don't need to wrap because they'll have the slots to begin with, so it doesn't matter what `Iterator.from` does for them
21:38	<ljharb>	why are we concerned with "overhead" for the highly unlikely case of someone doing `{ __proto__: Iterator.prototype }`?
21:38	<ljharb>	unlikely still means nonzero, and having to deal with that complexity (in the current case) is far more dangerous than "it's a bit slower if you do something super weird" (in the "always wrap" case)
21:39	<snek>	i am lost now
21:40	<snek>	the overhead is when you wrap something that is already matching the "iterator with prototype" definition, which is most iterator objects flying around
21:44	<ljharb>	anything created from the language or a generator or the web would/could already have the slots it needs. It’d only be a custom iterator implementation that’d need wrapping. How common are those?
21:44	<snek>	most custom ones already inherit from Iterator.prototype
21:44	<snek>	or they did, i haven't looked in several years
21:44	<snek>	i would assume they didn't delete that though lol
21:45	<snek>	the pattern is `[][Symbol.iterator]().__proto__.__proto__`
21:45	<snek>	or getPrototypeOf instead
21:47	<snek>	so anyway to handle the case where they don't, we have this `Iterator.from` helper, and you can throw that in random places as needed
21:47	<snek>	but it exists purely to paper over the prototype
21:48	<snek>	like just thinking about the code i write for my day-to-day work, i would basically never use `Iterator.from`
22:14	<bakkot>	i think 99% of cases will be dealing with untrusted iterators i would guess well over 99% of code is internal, not in libraries, and in internal code you are not dealing with untrusted stuff, as a rule
22:19	<bakkot>	also I guess I am missing what the benefit of the internal slot is supposed to be
22:28	<rbuckton>	I voiced an opinion, several years ago now, that I believed the iterator helper methods should always just do `this[Symbol.iterator]()` so that they would work for both `Iterator` and Iterables. Then, the presence of `Symbol.iterator` would be enough of a brand check (insomuch as it is the same amount of brand checking that `yield*` and `for..of` care about).
22:30	<rbuckton>	most custom ones already inherit from Iterator.prototype Most custom "Iterators" I've seen are just `{ next() { ... }, [Symbol.iterator]() { return this; } }`.
22:40	<bakkot>	rbuckton: why would you ever be invoking an iterator helper method on something which was an iterable-and-not-iterator?
22:41	<bakkot>	what is an example of some code where that might come up?
22:51	<rbuckton>	My original argument stems from my belief that Iterator is still the wrong level of abstraction. But aside from that, if the iterator helper methods were to only care about whether the object had a `next()` then we would have the same kind of duck typing we get with `.then()` on Promises. As a result, we end up needing a brand check of some kind which a custom iterator like `{ next() { ... } }` wouldn't have.
22:51	<ljharb>	that wouldn't be any different than the current proposal which just checks the [[Prototype]] - it's not a brand check if it's based on a public property.
22:52	<ljharb>	Most custom "Iterators" I've seen are just `{ next() { ... }, [Symbol.iterator]() { return this; } }`. i agree, but why would those use Iterator Helper methods without being passed through `Iterator.from` first?
22:52	<rbuckton>	Honestly I wish we'd left IteratorPrototype alone and had adopted a chainable wrapper object over Iterable, or just bare functions to work with `\|>`, but we're probably already too far gone.
22:53	<snek>	i definitely do not want to argue about iterable vs iterator more
22:55	<rbuckton>	i definitely do not want to argue about iterable vs iterator more I've never really changed my opinion, and the fact we're introducing an `Iterator.from()` that could have been an `Iterable.from()` (or some other name) that works over Iterables really reinforces that belief.
22:55	<ljharb>	as has been said many times, iterable isn't a thing. We don't have `Thenable.from` either.
22:56	<ljharb>	it's an adjective to describe the presence of a method.
22:56	<rbuckton>	Iterable is a thing in the spec. Its an object with a `[Symbol.iterator]()` method.
22:56	<snek>	thats what an iterator is too
22:56	<ljharb>	iterators are kind of both, in that they're a thing and also a protocol
22:56	<snek>	its all a huge mess
22:56	<ljharb>	and yes, iterators are a mess
22:56	<ljharb>	Iterator.from always wrapping into a "Real" Iterator will help clean it up.
22:57	<ljharb>	(that built-in iterators all happen to be iterable isn't a part of the iterator contract, it's just a convenience these happen to have)
22:57	<rbuckton>	According to the spec, a "Real" iterator is an object with a `next()` method that returns a `{ value, done }` object.
22:57	<rbuckton>	The fact that built-in iterators happen to have a shared prototype doesn't matter to that definition.
22:57	<rbuckton>	And the fact they have a shared prototype was always strange to me.
22:57	<snek>	js is indeed strange
22:58	<ljharb>	a real Promise is more than just a thenable. what i thought this proposal was doing was reifying a concept of a real iterator.
22:59	<rbuckton>	TBH, I will almost never reach for iterator helpers when I could use a more fully-capable third-party library, which is the only reason I haven't pushed back harder against it.
23:00	<rbuckton>	I don't think reifying `Iterator` will have anywhere near the impact that `Promise` did. At least, I certainly hope it doesn't.
23:00	<ljharb>	i think that it wouldn't on its own, but the iterator helpers themselves are why it will.
23:01	<ljharb>	i believe this proposal advancing will trigger a sea change in the way APIs are designed - everyone will start preferring lazy computation and passing around iterators instead of collections, which is a big shift from the current world where people tend to pass arrays.
23:05	<rbuckton>	i believe this proposal advancing will trigger a sea change in the way APIs are designed - everyone will start preferring lazy computation and passing around iterators instead of collections, which is a big shift from the current world where people tend to pass arrays. I'm hoping it doesn't. It's the wrong abstraction. I hope it's relegated to the "I need to do this basic thing", and developers reach for the right abstraction for more complex scenarios.
23:08	<rbuckton>	I honestly think that if `\|>` had advanced more quickly we wouldn't be pursuing this at all.
23:08	<snek>	i really wish we could add things to the stdlib without every delegate being involved in every single decision of every function
23:09	<snek>	temporal achieved this by being large and complex
23:11	<rbuckton>	i really wish we could add things to the stdlib without every delegate being involved in every single decision of every function This is a little more impactful than adding a single function though.
23:11	<snek>	everything is impactful in some way 🤷
23:12	<snek>	and to be clear i didn't mean you specifically, i was just thinking about how this has been a proposal for like 3 years
23:12	<snek>	its depressing 😔
23:17	<bakkot>	in fairness it's mostly just that people pick it up and put it down
23:17	<bakkot>	it's advancing now because michael and I found time to get back to it
23:17	<bakkot>	not so much because of there being an insurmountable number of details
23:20	<bakkot>	Iterator helper methods do only care about whether their receiver has a `next` method. But this is not at all the same kind of duck typing as with `.then` on promises, because it's on the receiver, not an arbitrary other thing; also nothing switches on its presence, just unconditionally attempts to call it. So it lacks the big two problems that thenables have, which are a.) consuming a string-named property on an argument, not just the receiver, and b.) changing behavior based on the presence or absence of the property instead of simply throwing in its absence
23:23	<bakkot>	on that note, ljharb should we maybe merge https://github.com/tc39/agendas/pull/1233 now, so delegates get a chance to review? we can always back it out if approval doesn't come through
23:24	<bakkot>	anyway, I do want to strongly affirm that iterator helpers are precisely the correct abstraction for what they are
23:26	<bakkot>	if the helpers consumed iterables they either a.) produce single-shot iterables, which would be weird and confusing or b.) produce reusable iterables, which would be weird and confusing for other reasons, i.e. https://stackoverflow.com/a/28513908
23:27	<bakkot>	"iterator helpers" or some equivalent are a solution common to most mainstream languages - java, rust, scala, c++ (boost), scala - and they work well
23:28	<bakkot>	this is not uncharted territory here
23:29	<bakkot>	and I do expect them to get wide adoption, for the same reason they get wide adoption in other languages: they are the right abstraction to express what you want in a wide variety of scenarios, namely single-shot deferred computation
23:45	<rbuckton>	ES2015 established syntactic support for Iterable with `for..of` and `yield`, even "one-shot" Iterables with generators. The only reason iterator helpers can be used with these constructs is that they are also "one-shot" Iterables. The languages you've referenced with a preference for iterator's are all statically typed. A function can receive an iterator and know* its an iterator.
23:47	<rbuckton>	In JS, we are left with either "duck typing" or checking @@iterator. If @ljharb is correct, an ecosystem will be built around receiving iterators as well as producing them. And those functions may also need to check the input type.
23:48	<rbuckton>	Since the ecosystem likes to write overloads, they need some way to check if the input is an Iterator. `next()` is not sufficient. It is Promise.p.then() all over again, even if it's a smaller case.
23:50	<rbuckton>	Today the best way to do that overload check is to check for @@iterator, making the check for an Iterable. That at least still works for iterator helpers, but not for user defined `{ next() {}}`.
23:51	<snek>	all iterators are also iterables
23:51	<snek>	i think checking for `@@iterator` is fine
23:51	<rbuckton>	Maybe a reified Iterator helps, but it is still just a "one-shot" Iterable as far as `for-of`, `yield*`, `[...iter]`, etc. are concerned.
23:52	<bakkot>	I do not expect there to be an ecosystem built around receiving iterators. why would there be? you can receive an iterable and consume it only once, and that works just as well.
23:52	<snek>	as for "passing iterators around" that would be interesting i guess, though i don't imagine it becoming that common
23:52	<snek>	looking at other languages like rust
23:52	<rbuckton>	all iterators are also iterables This is false. All built-in iterators and generators, maybe. But not "all".
23:53	<snek>	yes you're right, all the ones that inherit from the prototype
23:53	<rbuckton>	as for "passing iterators around" that would be interesting i guess, though i don't imagine it becoming that common This is exactly what @ljharb was describing earlier
23:53	<snek>	yeah i don't agree with the prediction
23:55	<snek>	but either way i don't think it is too problematic
23:57	<ljharb>	if the only way to know if something is an iterator is to consume it, then everyone will have to Iterator.from(x).map(x=>x) to be sure they’re dealing with one. so why not build that into Iterator.from?
23:57	<bakkot>	the `.map` is not necessary
23:57	<bakkot>	it does nothing