00:36 | <sirisian> | Here's an example:
|
00:51 | <sirisian> | Handled in-order one must wait 300ms to see the first result. All the operations are handled sequentially taking a total 300 * 2 + 30 * 4 = 720ms. An out of order first come with a task limit of 5 for each step (assume these are network or single worker thread delays) would take 300ms. Even a task limit of 3 would take 300ms. in that example. |
02:14 | <bakkot> | sirisian: yeah it looks like you're trying to do things with sync iterables of promises, which isn't a concept the language ever really works with, right now. except in like Promise.all I guess. |
02:45 | <sirisian> | bakkot: https://jsfiddle.net/9xsyev61/ This isn't cursed at all. |
02:48 | <bakkot> | sirisian: lol |
02:48 | <bakkot> | looks like it dropped some though? I only see a single 1 in the console |
02:48 | <bakkot> | oh, wait, that's just the wrapping, sigh |
02:49 | <bakkot> | that still only works with finite iterators though |
02:49 | <bakkot> | finite underlying iterators, that is |
02:49 | <bakkot> | well... maybe it could be made to work with infinite ones, hmm |
02:50 | <sirisian> | yes it should be able to. Just pass in the iterator. |
02:50 | <sirisian> | take(5) kind of thing then next to get the next work. |
02:54 | <bakkot> | oh, but it doesn't compose, because it produces an async iterator rather than a sync iterator of promises |
02:54 | <bakkot> | so you can't do it twice with different mapping functions |
03:26 | <sirisian> | bakkot: https://jsfiddle.net/xs1brkL8/ Wait, I don't use these much. Can you detect the break in code like that to print "map done"? |
03:27 | <bakkot> | sirisian: i don't understand the question |
03:32 | <bakkot> | oh, do you mean, the break in the bottom loop? yes: it will cause the yield to be a return , and you can use a try/finally to do cleanup |
03:32 | <sirisian> | oh it throws. I see. |
03:32 | <bakkot> | it doesn't throw |
03:32 | <bakkot> | it puts a return |
03:32 | <bakkot> | a finally will trigger, but not a catch |
03:33 | <bakkot> |
|
03:34 | <sirisian> | https://jsfiddle.net/xs1brkL8/1/ neat, is this composable? |
03:35 | <sirisian> | In the actual helper setup it would need to be on the iterator, but that should be fine. I could rewrite it do that I think. |
03:37 | <sirisian> | Oh wait, can you modify all iterators to add functions to them. (I know this would be bad). |
03:38 | <bakkot> | that's what the iterator helpers proposal is |
03:38 | <bakkot> | i.e. adding functions to all iterators (all iterators which inherit from iterator.prototype, anyway) |
03:39 | <bakkot> | I can't quite tell if your thing actually composes properly just from staring at it |
03:40 | <bakkot> | oh, !tasks.done isn't a thing, though |
03:41 | <sirisian> | whoops forgot that one. ~~https://jsfiddle.net/xs1brkL8/2/~~ https://jsfiddle.net/xs1brkL8/3/ |
03:41 | <sirisian> | I mean can you edit the iterator prototype right now for testing polyfills? |
03:41 | <bakkot> | yes |
03:42 | <bakkot> | re^, you have to check the done property of the result every time you call .next() |
03:42 | <bakkot> | you have to check done before reading value |
03:43 | <bakkot> | (to obey the iterator contract, anyway; you don't have to but it's going to behave weirdly if you don't) |
03:56 | <sirisian> | Wait, how do you edit the iterator prototype? What is it called? |
03:57 | <bakkot> | it's called [].values().__proto__.__proto__ |
03:57 | <sirisian> | That's how I know this is a good idea. |
04:06 | <sirisian> | Is the async iterator a different one then? |
04:07 | <bakkot> | yes |
04:11 | <bakkot> | (it's (async function*(){})().__proto__.__proto__.__proto__ ) |
04:11 | <sirisian> | ah, I was off one proto |
04:24 | <bakkot> | sirisian: ok I looked again and I am pretty sure this implementation will never start the third phase until at least 5 items from the second phase have finished, which I think is not what you intend |
04:25 | <bakkot> | where you have await tasks.next() you want to instead have a race with that and the existing work queue items, and if one from the work queue finishes you yield it right away |
04:26 | <sirisian> | https://jsfiddle.net/xs1brkL8/4/ My current one for reference. Thinking more. Updated: https://jsfiddle.net/xs1brkL8/10/ |
04:32 | <sirisian> | ah, I think I see what you mean since the iterator can take a while to yield. I'll put a sleep in my makeALotOfWork just to make it more clear also and handle it. |
05:29 | <sirisian> | There's no way to get the Promise from Promise.race along with the value without creating another promise right? Like it returns the value, but if you have say 10 promises you don't know what promise that value corresponds to. |
12:19 | <Justin Ridgewell> | I made a few concurrent async iterable helpers at https://github.com/jridgewell/minx/blob/main/src/async-iterable-concurrent.mjs |
12:40 | <annevk> | Why does tc39.es not host ECMA-404? |
12:47 | <yulia> | That is probably just an oversight.. |
13:45 | <annevk> | yulia: ta, filed https://github.com/tc39/tc39.github.io/issues/283 to track |
13:46 | <yulia> | yulia: ta, filed https://github.com/tc39/tc39.github.io/issues/283 to track |
13:46 | <yulia> | we don't seem to have an html source anywhere -- i think it predates that |
13:46 | <yulia> | so we likely need to write that up |
13:46 | <yulia> | We will get in touch with Chip |
13:47 | <yulia> | for now we might just host the pdf under the usual url |
13:47 | <annevk> | I figured that might be part of the problem, but PDF could be linked until hosting is sorted in the future |
13:47 | <yulia> | yeah |
15:05 | <bakkot> | sirisian: you can get substantially simpler, I think: https://gist.github.com/bakkot/6bee327466c06887d96a65f88f4cf728 |
15:14 | <bakkot> | though I notice there's another way this could be configurable, which is, there is both "number of tasks running in the mapping function" and "number of unsettled promises read from the underlying iterator", and those don't actually need to be the same number |
15:15 | <bakkot> | in my gist I have it so that the sum of those two numbers is n , but they could reasonably be independently configured |
15:23 | <bakkot> | Justin Ridgewell: hmm, I wonder if we should make iterator helpers work like your mapper... |
15:23 | <bakkot> | probably not I guess? it would maybe be surprising. |
15:25 | <bakkot> | though I guess you're only exposed to the difference if you're manually pumping the iterator rather than just using for await |
15:33 | <Justin Ridgewell> | It’s really difficult to reason about my way. |
15:34 | <Justin Ridgewell> | You can get a done message before all the previous messages have completed |
15:34 | <Justin Ridgewell> | I had to write the cap iterator to help me reason about the end states when using the map and interleave |
15:35 | <Justin Ridgewell> | Even map, you have to be aware of reentrancy during any async portions, which was difficult |
15:37 | <Justin Ridgewell> | I do not recommend we make standard iterators that behave this way |
15:37 | <Justin Ridgewell> | Backpressure may be slow but it has a very predictable ordering |
15:38 | <bakkot> | Yeah, sounds good. |
15:39 | <bakkot> | Staring at this async code above has been making my head hurt also. Which says there's maybe a place for the language to help out, but as a power tool, not one of the simple helpers. |
17:14 | <sirisian> | bakkot: Very nice and compact. Also your idea about separating n above is excellent. These problems are definitely fun little puzzles to think about. Your use of first is nice. I asked a question yesterday, but deleted it, about getting the promise instance from a Promise.race when was thinking about things. ( https://jsfiddle.net/yt5n4gLd/ Pasted your solution into a polyfill kind of implementation) |