| 11:11 | <annevk> | JakeA: I'm cleaning up the SW issue list a bit |
| 11:12 | <annevk> | JakeA: in particular those issues referenced from Fetch that should probably move elsewhere |
| 11:30 | <JakeA> | annevk: cool, will have a crawl through them on Tuesday too |
| 12:29 | annevk | is trying to figure out IRC urls... irc://freenode.net:6697/whatwg look good? |
| 12:30 | <annevk> | I sort of want to encourage SSL, but there doesn't seem to be ircs... |
| 12:32 | <annevk> | Hmm, there's a draft https://tools.ietf.org/html/draft-butcher-irc-url-04#section-2.1 that does define "ircs", but nothing that's an RFC afaict |
| 12:51 | <Krinkle> | annevk: Freenode has SSL on port 7000 |
| 12:54 | <Krinkle> | https://en.wikipedia.org/wiki/Template:Freenode#Example |
| 12:54 | <Krinkle> | irc://irc.freenode.net/#wikipedia-en |
| 12:54 | <Krinkle> | Looks right yeah |
| 14:10 | <annevk> | Krinkle: yeah, except ircs:// doesn't work :-( |
| 14:11 | <annevk> | Krinkle: btw, did my reply to your storage email help? |
| 14:11 | <Krinkle> | Right. When opening it in an IRC client you'd need a way to tell the app to use SSL. |
| 14:11 | <Krinkle> | In LimeChat there's a checkbox for it. |
| 14:11 | <Krinkle> | I don't think there's a way to programmatically set that based on a url. |
| 14:12 | <Krinkle> | ircs makes sense. |
| 14:12 | <Krinkle> | annevk: Yep. |
| 14:12 | <annevk> | Krinkle: https://wiki.whatwg.org/wiki/Storage has some v2 markers now, planning on elaborating a bit in the next couple weeks |
| 14:13 | <annevk> | The current situation is indeed rather messy and if we want to be competitive we should address that |
| 14:13 | <Krinkle> | Yeah, our native app doesn't have these issues for instance. |
| 14:13 | <Krinkle> | Not that that's really a solution either. But from a dev perspective it's great |
| 14:13 | <Krinkle> | not have to deal with a limit |
| 14:15 | <annevk> | Krinkle: another thing I was interested in is how Wikipedia plans on addressing its issues with having multiple origins for locales |
| 14:15 | <Krinkle> | At the moment that's not really a concern to us. |
| 14:15 | <annevk> | Krinkle: it seems like if e.g. Wikipedia gets a location-based service of sorts you'd want users to have to opt into that only once |
| 14:15 | <Krinkle> | They're independent websites. |
| 14:16 | <Krinkle> | separate content, communities and user priviledges. |
| 14:16 | <Krinkle> | not translations. |
| 14:16 | <annevk> | Krinkle: nothing is automated between them? |
| 14:16 | <annevk> | okay |
| 14:16 | <Krinkle> | Of course, they're all hosted by the same foundation. And we share the source code (differnet configuration loads based on http hostname). |
| 14:17 | <Krinkle> | But lots of version and plugin differences do exist, so we currently dont' share anything client-side. |
| 14:17 | <Krinkle> | Many modules (e.g. jquery ui) can vary based on language and skin. |
| 14:17 | <annevk> | Interesting |
| 14:17 | <Krinkle> | The cookieless domain bits.wikimedia.org is what serves most static resources from a hostname subdirectory. |
| 14:18 | <Krinkle> | virtual, not on disk. |
| 14:18 | <Krinkle> | But the localStorage system is indeed occopying the actual userfacing website origin (e.g. en.wikipedia.org, not bits.wm.o) |
| 14:19 | <Krinkle> | Which causes other plugins and power-user apps to fail sometimes due to space conflicts. |
| 14:19 | <Krinkle> | We have not solved this as of yet. |
| 14:20 | <Krinkle> | Other than to basically reject anything that uses localStorage and defer to other storage areas instead, which sucks. |
| 14:20 | <Krinkle> | We even had to revert some code that switches from cookies to localstorage where it made sense. But didn't work (e.g. hide-banner and boolean stuff like that) |
| 14:20 | <Krinkle> | so those are back in cookies at the moment. |
| 14:21 | <Krinkle> | We'll probably move a the js/css module storage to IndexDB which nobody uses. |
| 14:21 | <Krinkle> | :-) |
| 14:22 | <annevk> | Krinkle: so we are planning on putting all storage APIs into the same bucket so that might not help you much |
| 14:22 | <annevk> | Krinkle: but maybe having multiple buckets per site will help |
| 14:22 | <Krinkle> | There would have to be some way to limit the size or priority of a sub-bucket. |
| 14:23 | <Krinkle> | like a namespace of sorts |
| 14:23 | <Krinkle> | storage = Storage.instance('<rfee key>', { prio: -2 }); |
| 14:23 | <Krinkle> | just thinking out loud |
| 14:23 | <annevk> | Yeah, the idea is to have names that you can use with IDB and the Cache API (not sure it can work for localStorage) |
| 14:23 | <Krinkle> | it would create it ondemand, and re-configure if pre-existing. |
| 14:24 | <annevk> | And yeah, then use LRU across all buckets meaning often used buckets get cleared less |
| 14:25 | <Krinkle> | ideally we'd have LRU within a bucket as well. |
| 14:25 | <annevk> | And the other idea is having an LRU variant of the Cache API. Allowing you to store static resources there. And then the browser could do global reasoning about all LRU Cache APIs... |
| 14:25 | <Krinkle> | So that if a user visited the patrol app once but never comes back, it'll drop off eventually |
| 14:25 | <annevk> | Yeah, basically that |
| 14:26 | <Krinkle> | is there actually structure between origins recognised? I forget. |
| 14:26 | <Krinkle> | e.g. could we hack up a modules.en.wikipedia.org and access that from en.wikipedia.org |
| 14:26 | <Krinkle> | :P |
| 14:27 | <Krinkle> | for storage only |
| 14:27 | <annevk> | Through an <iframe> |
| 14:27 | <Krinkle> | Right |
| 14:27 | <annevk> | If the user doesn't disable third-party cookies... |
| 14:27 | <Krinkle> | "cookies" |
| 14:28 | <annevk> | Well yeah, again that's something we don't do well currently, but cookies need to be grouped together with all the other storage bits for these settings to have the effect the user expects |
| 14:28 | <Krinkle> | pumping 4-10 MB of serialised js/css/images through an iframe doesn't sound great though :D |
| 14:28 | <Krinkle> | Yeah |
| 14:28 | <Krinkle> | cookies is what users know |
| 14:28 | <Krinkle> | at least the few users that are aware of it |
| 14:28 | <Krinkle> | that's the customer facing term |
| 14:28 | <annevk> | The one other thing that's a bit unclear still is what the scope of storage is. eTLD+1 or origin. And we might want to make it different for "best effort" vs "persistent"... |
| 14:29 | <annevk> | Yup |
| 14:30 | <annevk> | Yeah, so tentative plan is persistent, multiple buckets, and a LRU API, and then see if that works... |
| 14:30 | <annevk> | an LRU I guess? |
| 14:33 | <Krinkle> | LRU cache is what I tend to say |
| 14:34 | <Krinkle> | Though it'd be interesting to see if there is usecase for multiple cache algorythms. |
| 14:34 | <Krinkle> | But I guess it's best to not bloat |
| 14:34 | <Krinkle> | instead provide a way for users to build their own if needed. |
| 14:35 | <Krinkle> | perhaps service worker can be authorised to routinely kick in and evaluate cache content based on some logic. |
| 14:35 | <Krinkle> | with native being told it's "persistent" but it will loop through and clear things based on its own logic |
| 14:35 | <annevk> | There's a proposal for an eviction event of sorts, but based on other systems that tried you probably want something like /tmp instead |
| 14:36 | <annevk> | At the moment you need to clear having to ask a dozen apps is not deal |
| 14:36 | <annevk> | ideal* |
| 14:38 | <Krinkle> | yeah |
| 14:38 | <Krinkle> | definitely not on the main thread synchronous |
| 15:07 | <zewt> | also evictable storage might be managed by something outside of the browser, eg. on mobile |
| 17:09 | <wanderview> | annevk: instead of a storage pressure event, what do you think about at least exposing a "systemStoragePressure" boolean attribute on StorageManager. |
| 17:10 | <annevk> | wanderview: what does that mean? |
| 17:10 | <wanderview> | annevk: instead of firing storage pressure events at origins... allow the origin to opt-in, check that storage pressure state, and manually free stuff when its running anyway |
| 17:11 | <wanderview> | I dunno... it was just an idea I had the other day |
| 17:11 | <wanderview> | not sure it makes sense |
| 17:11 | <wanderview> | the app might just want to do something different if the overall system disk is almost full to avoid having itself get deleted |
| 17:12 | <wanderview> | but we don't spin up the app automatically to handle it, etc |
| 17:12 | <wanderview> | its probably a bad idea |
| 17:14 | <annevk> | wanderview: I would imagine a sensible application would already do everything it can to keep storage low |
| 17:15 | <annevk> | wanderview: that's another reason I'm somewhat skeptical about this event |
| 17:15 | <annevk> | wanderview: gigantic apps will be at the top of the list for the user to remove, you'd try hard not to end up there |
| 17:15 | <wanderview> | annevk: I guess I was more observing that the current "getEstimate" API we have only provides info on the origin's quota/storage vs overall system state... might be nice to include something about if the system is under pressure there |
| 17:16 | <annevk> | wanderview: telling the app about the user's hard drive might be tricky, but maybe if the user opts into persistent mode we could expose a bit more stuff? |
| 17:17 | <wanderview> | annevk: I'm not talking about exact amounts for the system... just a "system low on disk space" signal |
| 17:17 | <wanderview> | I'm happy to leave that off for now |
| 17:18 | <annevk> | wanderview: yeah might be interesting to have a 80%-90% full indication |
| 17:18 | <annevk> | wanderview: though with persistent you'd probably get to see that amount anyway unless the user agent caps it, we haven't really discussed the finer details of that |
| 17:18 | <wanderview> | annevk: if nothing else, this could be a compromise with people who want a storage pressure event |
| 17:19 | <annevk> | wanderview: my plan is to write a slightly more detailed model of the existing and proposed architecture somewhere next week |
| 17:20 | <wanderview> | cool |
| 17:20 | <wanderview> | I'm going to be focusing on streams this week, I think |
| 17:22 | <annevk> | yay |
| 17:22 | <annevk> | wanderview: quite happy you're reviewing the proposed Fetch integration |
| 17:22 | <wanderview> | annevk: wish I had time to do it earlier! |
| 17:23 | <wanderview> | annevk: are there existing cases of that GC behavior that is proposed? |
| 17:23 | <wanderview> | make some DOM call, and then system resources are potentially leaked forever |
| 17:24 | <annevk> | wanderview: I missed that, does forever mean until you close the tab? |
| 17:24 | <annevk> | (there's a lot of that) |
| 17:24 | <wanderview> | annevk: it explicitly says if you do response.body.getReadable() then the UA cannot terminate the fetch |
| 17:24 | <wanderview> | cannot terminate the fetch due to GC |
| 17:25 | <annevk> | wanderview: oh that makes sense, but closing the tab would not be GC |
| 17:25 | <annevk> | wanderview: closing the tab is akin to "end-user abort" |
| 17:26 | <wanderview> | annevk: I guess I'm worried on the stream just blocking on back pressure (because no one is actually reading) and then it just doesn't close until the tab closes |
| 17:26 | <annevk> | wanderview: I think WebSocket has a similar thing about not allowing the connection to be closed due to GC |
| 17:26 | <annevk> | wanderview: and actually, even XMLHttpRequest has that: https://xhr.spec.whatwg.org/#garbage-collection |
| 17:27 | <annevk> | wanderview: anything to avoid exposing GC |
| 17:28 | <annevk> | Granted that this case is a little different, but still similar enough |
| 17:29 | <wanderview> | annevk: hmm... but what if there was no close handler registered.... seems like it should be GC'able then |
| 17:29 | annevk | was afraid momentarily that they were exposing GC |
| 17:29 | <wanderview> | annevk: good to know that there is precedent, though |
| 17:30 | <annevk> | wanderview: if there are no handlers that might be okay, though I think jsbell has a trick for that case |
| 17:34 | <wanderview> | annevk: it seems the XHR spec says it must not be GC'd only if it has an event handler set |
| 17:35 | <annevk> | wanderview: yeah, I think jsbell's argument was that you could detect GC in such a case by having a side-channel open and logging the TCP closing time on the server |
| 17:35 | <annevk> | wanderview: but I don't think any specification is concerned with that trick at the moment, not even WebSocket |
| 17:35 | <wanderview> | annevk: uh... that sounds akin to saying "you can detect GC by attaching with a debugger" |
| 17:35 | <annevk> | wanderview: so presumably streams shouldn't either |
| 17:35 | <annevk> | wanderview: well except there's no debugger |
| 17:37 | <wanderview> | annevk: do you know what Domenic meant by: It's the same as if you allocate a large array buffer and then don't clean it up. You've manually said "I want this potentially-large resource to stick around." |
| 17:37 | <wanderview> | do ArrayBuffers not GC?? |
| 17:37 | <annevk> | wanderview: I would expect them to GC just fine |
| 17:38 | <annevk> | wanderview: I've never heard anyone talk about magic GC abilities of ArrayBuffer |
| 18:01 | <wanderview> | annevk: I don't understand the opaque response stuff well enough to comment on https://github.com/slightlyoff/ServiceWorker/issues/590#issuecomment-89757518 |
| 18:52 | <wanderview> | Domenic: ping |
| 18:52 | <Domenic> | wanderview: pong |
| 18:53 | <wanderview> | Domenic: hi! I was reading this, but have a question https://github.com/yutakahirano/fetch-with-streams/issues/15 |
| 18:54 | <wanderview> | there you wrote: "The graph in this scenario is: resolve -> foo.closedPromise. So even if author code stops using foo and we GC foo, foo.closedPromise is not GCed until the UA code frees resolve." |
| 18:54 | <annevk> | wanderview: do you know who else I could ping? |
| 18:54 | <wanderview> | Domenic: doesn't the graph go the other way? foo.closePromise -> resolve? I don't see how resolve holds a ref to the promise |
| 18:54 | <wanderview> | annevk: nsm is probably good |
| 18:54 | <Domenic> | wanderview: no, resolve holds the ability to change the state of closePromise |
| 18:54 | <annevk> | ta |
| 18:55 | <Domenic> | assume a function ResolvePromise(p, value) |
| 18:55 | <Domenic> | then resolve = ResolvePromise.bind(foo.closePromise) |
| 18:55 | <Domenic> | or, resolve = function (value) { ResolvePromise(foo.closePromise, value); } |
| 18:55 | <wanderview> | oh... resolve is set to a function, ok |
| 18:55 | <Domenic> | right, yeah |
| 18:55 | <wanderview> | Domenic: this is pretty different from the c++ :-) |
| 18:55 | <Domenic> | welp :P |
| 18:55 | <Domenic> | In C++ I guess it would be a PromiseResolver object? |
| 18:56 | <Domenic> | with resolve and reject methods, and a promise private field? |
| 18:56 | <wanderview> | Domenic: I think what you are saying is we can GC the stream, but hold the promise alive |
| 18:56 | <wanderview> | Domenic: is that accurate? |
| 18:56 | <Domenic> | wanderview: right, I think that is what I am saying :P |
| 18:56 | <Domenic> | this stuff gets confusing... |
| 18:57 | wanderview | tries to find the point in the streams spec that says to resolve one of these promises. |
| 18:57 | <Domenic> | Ctrl+F "Resolve reader@[[closedPromise]]" |
| 18:59 | <Domenic> | Oh hey, found some dead code -_- |
| 18:59 | <wanderview> | Domenic: yea... CloseReadableStreamReader is not used any more? |
| 18:59 | <Domenic> | yep X_X |
| 18:59 | <Domenic> | killing it with fire |
| 18:59 | <Domenic> | wow it's even gone from the reference impl |
| 19:00 | <Domenic> | I was like ... this code looks pretty similar ... couldn't I refactor it out into a shared operation? And then I saw why it was similar ... I already factored it elsewhere and forgot to delete the old one. |
| 19:01 | <wanderview> | Domenic: so... it seems to me the spec requires doing other things with the stream when that promise is resolved... I don't see anything that lets us handle it separately from the stream object for when the socket closes |
| 19:02 | <Domenic> | wanderview: not sure I understand the question/dillema... |
| 19:03 | <wanderview> | Domenic: I don't see anything that says "when the socket closes, resolve the promise"... the streams spec has us modifying state variables and other stuff on the stream object... which implies we cannot GC the stream object |
| 19:03 | <Domenic> | wanderview: Right. You can close the socket and not call close(), i.e. not signal to the stream implementation that the stream itself should start acting closed |
| 19:04 | <wanderview> | Domenic: maybe its also just a conceptual issue for me... feels weird to say "here is an object that represents the stream", but that object can be destroyed without actually closing the underlying stream |
| 19:04 | <Domenic> | hmm |
| 19:04 | <Domenic> | So my take is that this underlying source object is adapting the socket or whatever into a stream object that reflects the state of the socket |
| 19:05 | <Domenic> | If the object goes away that doesn't necessarily mean the socket does, but it could if you want to set up a weak reference thingy to do so |
| 19:05 | <wanderview> | Domenic: but the fetch-with-streams doesn't define this "socket or whatever", but implies we have to give the promise to it directly |
| 19:05 | <Domenic> | right ... i'm getting confused now :( |
| 19:05 | <wanderview> | I have to go in a few minutes |
| 19:06 | <Domenic> | ok, i probably should too. (why are we working on sunday? :P) |
| 19:06 | <Domenic> | we can pick this up later ... i agree it's confusing and probably can be made clearer in fetch-with-streams |
| 19:06 | <wanderview> | Domenic: this is why I find it easier to define it as a ref holding the stream alive... and GC causes abort |
| 19:06 | <wanderview> | ok, have a good Easter! |
| 19:06 | <wanderview> | byte |
| 19:06 | <wanderview> | bye |
| 19:07 | <Domenic> | later! |
| 19:39 | <annevk> | Wait, we're allowed breaks on Sunday? |
| 23:22 | <wanderview> | annevk: only easter sunday |
| 23:41 | <gsnedders> | A properly defined grammar for the html5lib test format would be great, because dear god trying to parse this |
| 23:42 | <gsnedders> | Pretty sure we're LR(2) |
| 23:42 | <gsnedders> | At least in the simple definition |