| 00:01 | <ljharb> | oof, that is a super gross user experience :-/ |
| 00:02 | <ljharb> | so everyone basically has to wrap their return in [], ugh |
| 00:04 | <littledan> | point 2 of https://github.com/tc39/proposal-iterator-helpers/issues/55 sold me on those semantics, personally |
| 00:05 | <ljharb> | yes - but since the time that decision was made in 2019, iterator helpers now (in from i think) only treats Object iterables as iterable |
| 00:05 | <ljharb> | meaning, strings would not be auto-flattened, which resolves point 2 |
| 00:05 | <ljharb> | iow, if flatMap only considered Object as potential iterables, and all primitives were allowed, i think it'd maximize usability without causing any of the footguns mentioned |
| 00:06 | <bakkot> | I am still confused about, why would you be returning something which is not already iterable from flatMap? |
| 00:06 | <bakkot> | like |
| 00:06 | <bakkot> | if you don't do that, you don't need to wrap |
| 00:07 | <bakkot> | and I don't know why you would do that |
| 00:08 | <bakkot> | in any case I consider the third item in the issue linked above (i.e., auto-boxing means adding Symbol.iterator is basically always a breaking change) to be sufficient on its own |
| 00:08 | <ljharb> | specifically because i don't want to have to care if the item i'm returning is iterable or not |
| 00:08 | <bakkot> | I guess we already had this conversation in this thread |
| 00:08 | <ljharb> | that is also addressed by disallowing a non-iterable Object |
| 00:08 | <bakkot> | don't need to repeat it |
| 00:08 | <ljharb> | iow, currently you're allowed "only iterable objects", i'd like to see "primitive, or iterable objects" |
| 00:08 | <bakkot> | but... why |
| 00:08 | <bakkot> | that's even weirder |
| 00:09 | <ljharb> | why? a common case of a mapper is to produce a primitive |
| 00:09 | <bakkot> | yeah but if you are mapping you use .map, not .flatMap |
| 00:09 | <ljharb> | not if i sometimes want to return an array |
| 00:09 | <bakkot> | flatMap is for when you are producing a sequence |
| 00:09 | <ljharb> | map and flatMap on arrays already work this way |
| 00:09 | <ljharb> | yes, flatMap is for when i want to produce a sequence at the end |
| 00:09 | <ljharb> | it is NOT for "each callback produces a sequence" |
| 00:09 | <bakkot> | yes it is |
| 00:10 | <bakkot> | that is what it is for |
| 00:10 | <bakkot> | that is, very definitively, what it is for |
| 00:10 | <ljharb> | it's for "each callback might produce a sequence", which is why the one on arrays works that way |
| 00:10 | <bakkot> | we did not invent flatMap |
| 00:10 | <ljharb> | i realize that. but this makes the iterator flatMap different from the array one in a way that harms usability - it means i can't transparently refactor between map and flatMap without also changing the mapper - something i can already do on arrays, and very very often do. |
| 00:11 | <bakkot> | the only reason the one on arrays works that way is for symmetry with .flat, which does not exist on iterator helpers |
| 00:11 | <ljharb> | ok but flat and flatMap now exist on Arrays. the reason for array flatMap's behavior isn't really relevant - the two flatMaps are now inconsistent |
| 00:11 | <bakkot> | "primitives or iterables" would not restore the consistency with Array that you're looking for |
| 00:12 | <ljharb> | true. it would bring it closer tho. |
| 00:12 | <bakkot> | that's not always a good thing |
| 00:12 | <littledan> | sorry what was the change in how non-objects were handled? |
| 00:12 | <ljharb> | agreed. i fail to see why this wouldn't be a good thing tho |
| 00:12 | <ljharb> | littledan: in array flatMap, if the callback return is an array, it's flattened; if not, it's just used as-is |
| 00:13 | <ljharb> | in iterator flatMap, if the callback return is a non-iterable, it always throws - instead of just being used as-is |
| 00:13 | <bakkot> | littledan: right now, flatMap throws if you return anything other than an iterable object; ljharb proposes that instead it throw if you return anything other than an iterable object or a primitive, and that in the latter case it auto-box |
| 00:13 | <ljharb> | in particular this introduces a refactoring hazard for arrays |
| 00:14 | <ljharb> | arr.map(x).filter(y).flatMap(z) to Iterator.from(arr).map(x).filter(y).flatMap(z) won't work as intended |
| 00:14 | <bakkot> | the refactoring hazard is there either way, though; making the hazard more subtle (i.e. only relevant for non-primitives) is not an improvement IMO |
| 00:14 | <bakkot> | also Array.prototype.flatMap flattens arrays, and not iterables |
| 00:14 | <bakkot> | they're just fundamentally different operations |
| 00:14 | <ljharb> | i suppose if you were returning a non-array iterable in the array case, the hazard is indeed always there |
| 00:15 | <ljharb> | but the only non-array iterable i would think is commonly returned from a mapper is primitive strings |
| 00:15 | <bakkot> | I think that Set and Map are not unusual data structures to be working with, personally |
| 00:15 | <ljharb> | it's not like, surprising or bad code to have a mapper that returns those, ofc. but i doubt it's at all common. |
| 00:17 | <bakkot> | ok but I think that having a flatMap function that returns a primitive is also not common |
| 00:17 | <bakkot> | I know you do this but I do not expect that most people reaching for flatMap would do that |
| 00:18 | <bakkot> | and of the people who would, it's not going to be that unusual to sometimes return an object instead |
| 00:18 | <ljharb> | i do expect most flatMap functions are also map functions |
| 00:18 | <bakkot> | I really think we should just expect people to learn that X.prototype.flatMap is for returning and flattening Xs, and not try to guess what they meant if they return a non-X |
| 00:18 | <bakkot> | that's, uh. |
| 00:18 | <bakkot> | not an expectation I share |
| 00:19 | <ljharb> | i agree with the X.prototype.flatMap flattens X's argument |
| 00:19 | <bakkot> | again, we did not invent flatMap |
| 00:19 | <ljharb> | but the precedent we already have is that if you return a non-X, it's automatically somethinged into an X containing that non-X |
| 00:19 | <ljharb> | i forget the haskelly term for "something" |
| 00:19 | <ljharb> | auto-somethinged maybe |
| 00:20 | <bakkot> | I agree that Arrays work this way, and you are increasingly convincing me that it's a mistake, but we are agreed that iterators cannot work that way |
| 00:20 | <ljharb> | i don't understand why not |
| 00:20 | <ljharb> | adding Symbol.iterator to anything is already a breaking change |
| 00:20 | <ljharb> | adding a protocol to anything that didn't previously have it is always a breaking change, and must always be considered as one |
| 00:20 | <bakkot> | well |
| 00:21 | <bakkot> | that is an opinion you can have but I am talking about, like, actually breaking in practice |
| 00:21 | <bakkot> | and in real life people add Symbol.iterator to existing classes all the time without breaking anything |
| 00:21 | <bakkot> | web platform just did that with streams and it was fine |
| 00:21 | <ljharb> | i'm saying that "i can't use a map callback on iterator flatmap" has already broken me in practice, today |
| 00:21 | <ljharb> | which is why i brought it up |
| 00:21 | <ljharb> | and if adding Symbol.iterator to things is fine, then it would still be fine even if it changed how flatMap worked |
| 00:21 | <bakkot> | "I did not know how this API worked" is not at all as serious of a problem as "I am permanently prevented from adding a feature to my library" |
| 00:22 | <ljharb> | agreed. why would it be a permanent obstacle? |
| 00:22 | <ljharb> | that's what semver is for |
| 00:22 | <ljharb> | the language and platform already are hopefully risk-averse enough to be very cautious adding existing protocols to anything |
| 00:22 | <bakkot> | I think you're the only person I have ever met who is of the opinion that adding a protocol to a thing that didn't previously have it is always a breaking change |
| 00:23 | <ljharb> | i mean, potentially. obv not in practice, just like any breaking change might not be |
| 00:23 | <bakkot> | literally every change is potentially brekaing |
| 00:23 | <bakkot> | I have had things depend on the literal source text of a function and break because I changed that |
| 00:23 | <bakkot> | this is not what is normally meant by "breaking change" |
| 00:23 | <ljharb> | i don't think that's true, but that's not really a useful debate |
| 00:24 | <ljharb> | oh sure, there's a line like "changing source text" isn't breaking |
| 00:25 | <ljharb> | i would love to see a concrete example of an object type, that someone is returning from a flatMap callback, that makes sense to suddenly make iterable later, and where the breakage caused is difficult to debug or fix. |
| 00:25 | <bakkot> | streams! |
| 00:25 | <bakkot> | streams literally just did this! |
| 00:25 | <bakkot> | and by "just" I mean "sometime in the last few years" but still |
| 00:25 | <ljharb> | did streams predate iteration, or was it a mistake to not have them iterable in the first place? |
| 00:26 | <bakkot> | neither? they just didn't start out being iterable, and then evolved, as is very often the case |
| 00:26 | <bakkot> | https://github.com/whatwg/streams/pull/980 |
| 00:26 | <ljharb> | oh well i'm sure web streams predated async iteration |
| 00:26 | <ljharb> | i meant normal iteration |
| 00:26 | <bakkot> | flatMap also exists on async iterables so I don't understand why you are drawing a distinction |
| 00:27 | <ljharb> | i'm not really thinking about async rn, because the usability problem i'm running into is primarily on sync |
| 00:27 | <ljharb> | so why would auto-whatevering primitives be a problem for flatMap? |
| 00:28 | <ljharb> | "wrap" i guess but there's another term i'm forgetting |
| 00:28 | <bakkot> | "box", possibly |
| 00:28 | <bakkot> | or "pure" |
| 00:29 | <ljharb> | pure! i think that's the term michael used |
| 00:29 | <bakkot> | anyway: because once we get to the point that we're not being exactly consistent with Arrays, we need to choose different semantics anyway, and I think the only coherent option is "reject things you can't flatten" |
| 00:29 | <bakkot> | "reject things you can't flatten, except primitives for some reason" is not very coherent |
| 00:29 | <ljharb> | "can't flatten, but allow things that are already flat" |
| 00:29 | <bakkot> | {} is already flat |
| 00:29 | <bakkot> | arguably |
| 00:30 | <ljharb> | except that the existing semantics say that only Objects are flattenable, so {} is potentially flat or not, depending on what Object.prototype has on it. primitives are always flat, by the current semantics of the proposal. |
| 00:31 | <bakkot> | I don't really see why a user would be thinking about whether something is "potentially" flat |
| 00:31 | <ljharb> | well sure, but i also don't think they're going to be thinking about whether something is iterable or not. they're going to assume it works like array concat/flatMap and "just work" with non-iterables |
| 00:32 | <bakkot> | and they will be wrong for at least objects, in the "we are special-casing primitives" world, so they are going to have to correct their misconception at some point anyway |
| 00:32 | <bakkot> | why not make the misconception easier to correct, by making the rule simpler and more obvious? |
| 00:33 | <ljharb> | i don't think the current thing is more obvious |
| 00:33 | <ljharb> | or i'd have brought it up during plenary |
| 00:33 | <ljharb> | it literally took me implementing it to discover this confusing behavvior |
| 00:34 | <bakkot> | you don't think it's more obvious than special-casing primitives? |
| 00:34 | <bakkot> | I think it is definitely more obvious than that; we are unlikely to reconcile on that point |
| 00:34 | <ljharb> | i totally accept that it's a simpler thing to explain as-is |
| 00:34 | <ljharb> | but that's not the same thing as which is more surprising and frustrating |
| 00:34 | <bakkot> | I think it's both less surprising and less frustrating as-is |
| 00:36 | <bakkot> | the only way you can run into the special case is if you return a primitive, and while I accept that you personally do that on purpose, I think that for a typical user that's going to be a bug which they would prefer to be notified of |
| 00:36 | <bakkot> | moreover I think even the person who does want the auto-boxing behavior is going to want to know as early as possible that their assumption about the behavior is wrong, and they will learn that faster if primitives are not special-cased |
| 00:41 | <littledan> | yes - but since the time that decision was made in 2019, iterator helpers now (in from i think) only treats Object iterables as iterable |
| 00:43 | <bakkot> | littledan: ah, specifically, Iterator.prototype.flatMap throws if your mapper function returns a string, instead of spreading its code points |
| 00:43 | <bakkot> | no one was particularly happy about this decision but it seems pragmatic |
| 00:43 | <littledan> | oh, I thought ljharb was saying that there was some other decision made which would have some impact on that decision |
| 00:44 | <littledan> | and so the point 2 I referenced is no longer relevant |
| 00:45 | <ljharb> | i'd be opposed to any argument that would make strings auto-spread, for the hopefully obvious reasons :-) |
| 00:45 | <littledan> | as would I, but I was just having trouble understanding what you meant by that comment I quoted |
| 00:46 | <littledan> | special-casing primitives seems weird--wouldn't it be natural to want pojos to have this sort of behavior as well? |
| 00:47 | <littledan> | from there, I'm kinda synpathetic to the argument that it'd be weird if adding an implementation of a protocol were a compat hazard |
| 00:49 | <ljharb> | my claim is that adding a pre-existing protocol to an object is a potential breaking change, since it could cause the object to travel a different code path than previously |
| 00:50 | <ljharb> | (this is the same argument browsers used to say that it would not be web compatible to have a predicate that changed its response to an input over time, ftr) |
| 00:50 | <bakkot> | it's true that it's a potential breaking change, but by making flatMap auto-flatten we would radically increase the likelihood of it being breaking in practice |
| 00:51 | <bakkot> | in particular I strongly suspect it would have meant that Streams could never have been made iterable |
| 00:52 | <bakkot> | (had things been sequenced in that order) |
| 01:55 | <Michael Ficarra> | pure! i think that's the term michael used pure is a way to "wrap" a thing without caring what kind of "wrapper" you need (inferring from the type system), NOT a way to "wrap" only things that are not already "wrapped" |
| 01:55 | <Michael Ficarra> | so pure ∘ pure is not the same as pure |
| 01:57 | <Michael Ficarra> | additionally, I find it to be a much bigger refactoring hazard that I could not change a.flatMap(b => 0) to a.flatMap(b => c) where c is a non-primitive |
| 01:59 | <littledan> | I guess the word breaking is used different ways in different contexts, but observable change != webcompat issue |
| 01:59 | <littledan> | The implication only goes <= |
| 15:02 | <Jack Works> | tmLanguage is so |
| 19:05 | <ljharb> | rickbutton: is there a R&T call today? i'm getting "invalid meeting ID" on the zoom |
| 19:05 | <ljharb> | ah nvm, i see the doc |
| 19:05 | <Ashley Claymore> | Sorry yes, the link died |
| 19:05 | <Ashley Claymore> | yeah, new link in the doc |
| 19:05 | <ljharb> | i'll update the invite |
| 19:05 | <Ashley Claymore> | thank you |
| 19:06 | <littledan> | also feel free to DM to ask for the new zoom link for the R&T call |
| 21:52 | <TabAtkins> | bakkot: I'm finally getting back to specifying map/list iteration in Infra (https://github.com/whatwg/infra/pull/451), and I'm looking over Array iteration behavior. It looks like forEach snapshots the list length at the beginning, so items adding mid-iteration won't be visited (https://tc39.es/ecma262/#sec-array.prototype.foreach), but an array iterator recalculates the length each iteration, so it will see mid-iteration additions (https://tc39.es/ecma262/#sec-createarrayiterator). Is this a correct reading? If so, do you have an opinion on which behavior I should prefer? Maps do the "recalculate on each iteration" thing for both forEach and iteration so I'm leaning that way as well. |
| 23:16 | <bakkot> | TabAtkins: my reading matches yours (and also engines), and I also share your preference for matching the behavior of the newer APIs i.e. recalculating length. that said I am not super worried about trying to pick the "right" semantics for code which is mutating a collection while iterating over that collection, as long as it is specified and implemented consistently, so if there's some other reason (e.g. implementability) to prefer the "cache length up front" semantics I wouldn't object to that either |
| 23:27 | <TabAtkins> | Cool 🙏 |