| 03:09 | <Hixie> | I have strings of numbers that I must match against patterns. The patterns are represented as nested lists of numbers. |
| 03:09 | <Hixie> | Each list of which requires one, zero or more, or one or more numbers to be matched from a set of numbers, either in sequence or in any order depending on the list. |
| 03:09 | <Hixie> | So for example, a pattern could be "sequence(1, one-of(2, sequence(20, 21)), zero-or-one-of(3, 4), one-or-more-of(5, 6, one-of(7, 8), 9)". |
| 03:09 | <Hixie> | The sequence 1,2,4,8,9 would match it, as would 1,20,21,5,6,7,9, but 1,2,7,8,9 would not, and nor would 1,20,3,4,9. |
| 03:09 | <Hixie> | The question is, what's a good way to represent these patterns in memory that is both fast to evaluate and reasonably memory efficient? |
| 03:09 | <Hixie> | The quickest way to evaluate them seems to be to compute every match and then form a state machine tree to walk down, but that is pathalogical in some common cases. |
| 03:09 | <Hixie> | For example one-or-more-of(one-or-more-of(1,2,3),one-or-more-of(4,5,6),one-or-more-of(7,8,9)) has 495 permutations if I worked it right. |
| 03:09 | <Hixie> | I tried looking up things like "how to compile regular expressions" but it's all about how to use them, not how to compile them. |
| 03:28 | <roc> | maybe just keep a set of positions within your tree |
| 03:28 | <roc> | and scan the input updating that set after each number |
| 03:31 | <roc> | depending on your workload, you could compile it to a nondeterministic finite state machine and minimize the number of states using standard algorithms, but that adds quite a lot of complexity and may not perform any better |
| 03:35 | <Hixie> | fair enough |
| 03:36 | <Hixie> | thanks |
| 03:38 | <doublec> | Hixie, did you come across Russ Cox's articles on compiling regular expressions? |
| 03:38 | <doublec> | eg: http://swtch.com/~rsc/regexp/regexp3.html |
| 03:38 | <Hixie> | i did not, will read, thanks |
| 03:52 | <othermaciej> | Hixie: isn't that effectively a regular expression? |
| 03:52 | <Hixie> | yes |
| 03:52 | <othermaciej> | I'm pretty sure your language is a regular language |
| 03:52 | <Hixie> | hence my looking up stuff on compiling regexps |
| 03:52 | <othermaciej> | in which case, my suggestion would be to express the problem in a form in which you can use an out-of-the-box regexp engine |
| 03:52 | <Hixie> | doublec's url got me something more useful though |
| 03:52 | <Hixie> | yeah, that might be a good idea |
| 03:53 | <Hixie> | part of the goal here is to learn more about how to write this kind of thing, though |
| 03:53 | <othermaciej> | if you really want to hand-code something, a DFA would be most efficient for matching |
| 03:53 | <Hixie> | cool |
| 03:53 | <othermaciej> | though it could be costly to compute up front |
| 03:53 | <othermaciej> | I've seen implementations that do lazy NFA-to-DFA conversion, but that is complicated |
| 03:53 | <Hixie> | in this case, that's turns out to not be an issue |
| 03:54 | <Hixie> | since the compilation step is far removed from the matching step |
| 03:54 | <Hixie> | and only the latter is time-sensitie |
| 03:54 | <Hixie> | ve |
| 03:54 | <othermaciej> | a DFA would be faster to match against than an NFA |
| 03:56 | <othermaciej> | you don't have to compute every match to make a state machine |
| 03:56 | <roc> | yes |
| 03:56 | <roc> | the only other thing you have to worry about is DFA size explosion |
| 03:57 | <othermaciej> | (otherwise you'd have a hell of a time compiling any regexp that uses the * operator) |
| 04:02 | <Hixie> | the case i'm having the most trouble with when converting this to a state machine is the "zero or more of the following, in any order: ..." expression |
| 04:02 | <Hixie> | (which i notice regular expressions don't really support) |
| 04:02 | <Hixie> | (but which is quite important for my application) |
| 04:03 | <roc> | isn't that just (A | B | C | D)* ? |
| 04:03 | <Hixie> | sorry, i forgot to clarify: no duplicates |
| 04:03 | <roc> | ah right |
| 04:04 | <Hixie> | in my original approach earlier today i was considering just mutating my state machine as i walked it |
| 04:04 | <Hixie> | but that doesn't work so well with either the back-tracking or "following all states at once" approaches |
| 04:05 | <roc> | yeah you need 2^N states for that |
| 04:05 | <roc> | so a DFA is not going to work too well if you have more than a small number of alternatives |
| 04:05 | <Hixie> | yeah, that's been my conclusion too |
| 04:09 | <othermaciej> | zero or more with no duplicates shouldn't require 2^N states |
| 04:09 | <othermaciej> | it would be O(N^2) states |
| 04:11 | <Hixie> | my N will be around 10 for many of these patterns |
| 04:11 | <Hixie> | and many patterns will have several of these and/or nest them |
| 04:12 | <othermaciej> | 10^2 is not that big |
| 04:14 | <othermaciej> | actually I guess I am oversimplifying things, since it's variable length, the number of states depends on the following operator |
| 04:14 | <othermaciej> | in an NFA it would be O(N^2) for sure |
| 04:18 | <roc> | I'm confused |
| 04:19 | <roc> | if you have N possible objects and you have to avoid duplicates, then at any given point in matching a string, you have to remember which subset of the N objects you have seen so far |
| 04:19 | <roc> | that's 2^N states |
| 04:19 | <roc> | for a DFA |
| 04:30 | <Hixie> | for zero-or-more (no duplicates) and N=3, i count 10 states |
| 04:30 | <Hixie> | (not counting the terminal state) |
| 04:31 | <Hixie> | for N=2 i count 3 states |
| 04:32 | <Hixie> | N=3 is just three N=2s with a state on the front |
| 04:33 | <Hixie> | assuming N=4 is four N=3s with a state on the front, that'd be 41 states for N=4 |
| 04:34 | <Hixie> | (this is for an NFA, obviously, since I'm ignoring whatever comes next) |
| 04:34 | <Hixie> | (so one of the transitions at each state is to just move on to the next part of the NFA) |
| 04:37 | <Hixie> | http://www.research.att.com/~njas/sequences/A002627 |
| 04:39 | <Hixie> | that list gets way out of hand far too quickly |
| 04:39 | <Hixie> | N=8 -> 69281 states |
| 04:39 | <Hixie> | N=13, which is plausible for my application, would have 10699776686 states |
| 04:42 | <Hixie> | what i need is a kind of NFA where i synthesis new states as i am walking it |
| 04:42 | <Hixie> | that might work |
| 04:54 | <Hixie> | yes... |
| 04:54 | <TabAtkins__> | Hixie: How fast does this have to be? Shouldn't be too hard to make an actual greedy backtracer out of what you've got. |
| 04:56 | <Hixie> | well i have to convert it to a serialisable form anyway, might as well convert it to a state machine while i'm atit |
| 04:56 | <TabAtkins__> | Seems easier to just serialize something like you have above, and package it with your matcher. |
| 04:56 | <Hixie> | *shrug* |
| 04:57 | <Hixie> | it's just for fun |
| 04:58 | <TabAtkins__> | All right. I think that actually serializing it as a state machine won't be fun, though. ^_^ |
| 04:58 | <TabAtkins__> | But writing a matcher could be. |
| 04:59 | <Hixie> | writing a state machine would be near-trivial if it wasn't for this particularly weird case |
| 06:41 | <theMadness> | Wait a minute, the specs are written in html4.01 |
| 06:47 | <jstar-taiwan> | hi, is there a way to get canvas fallback working when JS is disable in Firefox ? |
| 07:24 | <TabAtkins___> | theMadness: Yeah, w3c doesn't yet allow their specs to be written in HTML5, and it's too much effort to write the WHATWG version in HTML5 and down-convert to HTML4 for the w3c version. |
| 07:45 | <jstar-taiwan> | any tutorial explaining how to add link to a canvas text ? |
| 07:53 | <Hixie> | TabAtkins__: actually the whatwg one is written in HTML5 and I have a script to down-convert it to HTML4 for the W3C |
| 07:53 | <Hixie> | it takes out some of the examples that use HTML5 |
| 07:53 | <TabAtkins__> | Hm. I thought I'd heard you saying the opposite. Shrug. |
| 07:54 | <micheil> | Hixie: I've finally got the draft75 websocket server working in node, and it's written in a way to also be able to easily work with draft76 |
| 07:54 | <micheil> | (it's just a matter of sorting out the handshaking code) |
| 07:56 | <micheil> | Hixie: is draft76 going to become draft76 on the ieft site? |
| 08:05 | <Hixie> | micheil: the ietf asked me to stop sending them updates because they couldn't cope with the volume of updates, so no idea |
| 08:05 | <Hixie> | as far as i'm concerned, -75 is long dead |
| 08:05 | <micheil> | oh, righteo |
| 08:05 | <Hixie> | if i'd still been sending updates, -76 would actually be like -90 or so by now |
| 08:05 | <micheil> | yeah, -75 is still the one supported by chrome (and other clients), so it's the one I must have work |
| 08:06 | <micheil> | is there a way to see the changes made to the spec (like a git diff) |
| 08:06 | <Hixie> | svn diffs of all the changes made to the whatwg specs are at http://html5.org/tools/web-apps-tracker |
| 08:07 | <Hixie> | there's no per-section breakdown |
| 08:07 | <micheil> | there's just three sources to read the spec, so it sometimes gets confusing as to which one to conform to |
| 08:07 | <Hixie> | well right now it's too early to be doing anything but experimental work, so it's not a big deal |
| 08:08 | <micheil> | well, true, although, I'd like to be able to make my websocket server ready for when we do see a final version of the spec |
| 08:10 | <micheil> | in node, we've just added in support for things like http upgrade, so that when a client requests an upgrade (eg a websocket server), it can be handled appropriately, while still allowing a http server to function normally |
| 08:12 | <hsivonen> | http://www.w3.org/Bugs/Public/show_bug.cgi?id=8979 interesting WONTFIX |
| 08:15 | <Traveler> | hi |
| 08:58 | <hsivonen> | Is there anything is the spec that disassociates form-associated elements from their forms when the form-associated elements are re-inserted into a document? |
| 08:58 | <hsivonen> | in particular, when the fragment created by the fragment parsing algorithm is inserted by the innerHTML setter |
| 08:59 | <Hixie> | yes |
| 08:59 | <hsivonen> | this? http://www.whatwg.org/specs/web-apps/current-work/#dfnReturnLink-11 |
| 08:59 | <Hixie> | "When a form-associated element's ancestor chain changes, e.g. because it or one of its ancestors was inserted or removed from a Document, then the user agent must reset the form owner of that element." |
| 09:00 | <Hixie> | (dfnReturnLink are dynamic and not portable) |
| 09:00 | <hsivonen> | Hixie: ok. thanks |
| 09:00 | <Hixie> | see http://www.whatwg.org/specs/web-apps/current-work/complete.html#concept-form-association |
| 09:00 | <Hixie> | there's various other conditions that reset it |
| 09:01 | <hsivonen> | next question: can the parser-created associations ever be observed in the fragment case? Maybe when createContextualFragement has been called but the fragment hasn't been inserted? |
| 09:02 | <hsivonen> | I wonder if the parser-created associations should simply be turned off in the fragment case |
| 09:02 | <Hixie> | unless i made a pretty serious mistake, there's no way to observe innerHTML until after it's in the document |
| 09:03 | <hsivonen> | createContextualFragment in anyone's spec yet? |
| 09:03 | <hsivonen> | that is, who should I bother about it? |
| 09:03 | <Hixie> | what's createContextualFragment? |
| 09:04 | <hsivonen> | Hixie: it's a Mozilla/Netscape ad hoc API that got cloned by WebKit and (IIRC) Presto |
| 09:04 | <Hixie> | oh jeez |
| 09:04 | <zcorpan> | http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-March/018892.html |
| 09:04 | <Hixie> | will you people stop making up new apis and implementing them widely |
| 09:04 | <Hixie> | i have enough trouble keeping track of the ones _i_ make up |
| 09:04 | <Hixie> | oh, it's on DOMRange |
| 09:04 | <Hixie> | ok |
| 09:04 | <Hixie> | well i expect i'll have to spec that one day |
| 09:04 | <Hixie> | but not any time soon |
| 09:05 | <hsivonen> | hmm. can a form be submitted if it hasn't been inserted into a document? |
| 09:05 | <hsivonen> | and if it can, do people do it? |
| 09:05 | <Hixie> | dunno, see the spec |
| 09:05 | <Hixie> | what does it say? |
| 09:06 | <hsivonen> | it doesn't work if there's no associated browsing context |
| 09:07 | <zcorpan> | hsivonen: in all browsers? |
| 09:07 | <Hixie> | so yes? |
| 09:07 | <hsivonen> | but a fragment that hasn't been inserted still has an associated document which has a browsing context |
| 09:07 | <hsivonen> | Hixie: looks like it can be submitted per spec |
| 09:07 | <Hixie> | cool |
| 09:07 | <Hixie> | makes sense i guess |
| 09:08 | <hsivonen> | which means that it theory there can be someone somewhere calling createContextualFragment with a malformed form and calling .submit() on it |
| 09:08 | <hsivonen> | s/it/in/ |
| 09:08 | <Hixie> | when i spec createContextFragment, it'll still be "inserted into a document" |
| 09:09 | <hsivonen> | Hixie: I don't follow. Surely the fragment has an owner document but it's not inserted |
| 09:10 | <Hixie> | "inserted into a document" doesn't mean what it sounds like |
| 09:10 | <Hixie> | it's named that way because in most cases that's what happens |
| 09:10 | <Hixie> | hm actually i'm wrong |
| 09:10 | <Hixie> | nevermind |
| 09:10 | <Hixie> | i was thinking of xbl |
| 09:11 | <Hixie> | i think the spec for form controls should be changed to just refer to the home subtree changing |
| 09:11 | <Hixie> | or something |
| 09:11 | <Hixie> | but i've not really paged any of this stuff in |
| 09:11 | <Hixie> | so i could be talking nonsense |
| 09:23 | <hsivonen> | shepazu: any news on the SVG load event? |
| 09:24 | <jstar-taiwan> | how am I supposed to add link to a <canvas> ? |
| 09:24 | <hsivonen> | shepazu: I just realized that one thing to consider is whether to fire the SVG load event when <svg> is parsed in an innerHTML setter |
| 09:39 | <jstar-taiwan> | how am I supposed to add link to a <canvas> ? |
| 09:41 | <Hixie> | jstar-taiwan: use an onclick handler |
| 09:41 | <jstar-taiwan> | Hixie, there is no native way to do this o_O ? |
| 09:45 | <Hixie> | no, intentionally |
| 09:45 | <webben> | jstar-taiwan: Note also http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#focus-management-0 |
| 09:45 | <Hixie> | if you want to have interactive graphics, use svg |
| 09:45 | <Hixie> | or html |
| 09:45 | <Hixie> | <canvas> is meant for scripted graphics |
| 09:45 | <jstar-taiwan> | Hixie, but SVG is not supported correctly |
| 09:46 | <webben> | jstar-taiwan: Where? |
| 09:46 | <jstar-taiwan> | webben, IE |
| 09:46 | <Hixie> | IE doesn't do canvas either |
| 09:46 | <webben> | jstar-taiwan: http://code.google.com/p/svgweb/ |
| 09:49 | <jstar-taiwan> | I understand there is way to get SVG or other w3c standards to work on IE, but it's not native |
| 09:49 | <Hixie> | IE doesn't support <canvas> either |
| 09:52 | <theMadness> | Uh, my mail made it through to www-style. |
| 09:53 | <jstar-taiwan> | Hixie, oh~ indeed I thought that IE8 add already a bit of support for it (i'm on linux) |
| 09:53 | <webben> | Nope. |
| 09:54 | <webben> | jstar-taiwan: Both svgweb and excanvas fake IE support with VML. |
| 09:54 | <hsivonen> | webben: doesn't svgweb use Flash? |
| 09:55 | <webben> | oh, yeah, sorry |
| 09:56 | <jstar-taiwan> | webben, Hixie does inline SVG work on IE8 ?? |
| 09:56 | <Hixie> | no |
| 09:56 | <Hixie> | SVG and canvas only work in firefox, webkit browsers, and opera |
| 09:57 | <zcorpan> | the ie9 preview has partial support for svg but no canvas |
| 09:57 | <zcorpan> | i speculate that some future ie9 preview will also have partial support for canvas |
| 09:57 | <theMadness> | Which is weird considering all the effort they are putting into making it a sort of directx thingie. |
| 09:58 | <roc> | and considering canvas is a lot easier to implement |
| 10:00 | <jstar-taiwan> | argh~ why do IE team take so much time to implement basic ~.~ |
| 10:01 | <jstar-taiwan> | anyway do you have a sample on how to add links to a <canvas> ? |
| 10:02 | <Hixie> | if you're needing to add links to canvas you're almost certainly misusing canvas |
| 10:03 | <jstar-taiwan> | I'm willing to provide an alternative to a flash diagram |
| 10:04 | <Hixie> | why not use svg? |
| 10:04 | <jstar-taiwan> | Hixie, it's a kind of pie chart with label which display text when clicked |
| 10:05 | <hsivonen> | jstar-taiwan: svg works for that use case better |
| 10:05 | <jstar-taiwan> | I wanted to try <canvas> and thought it was better supported |
| 10:07 | <hsivonen> | canvas is more of a buzzword than svg, but for almost all non-game use cases svg is more appropriate |
| 10:11 | <jstar-taiwan> | yeap I tried it to know a bit more and it seem to bit another blob technology |
| 10:12 | <jgraham> | hsivonen: I'm pretty sure I have written code that depends on submitting non-inserted forms |
| 10:12 | <jgraham> | (unless it doesn't work in which case I haven't, obviously) |
| 10:14 | <zcorpan> | wonder why http://software.hixie.ch/utilities/js/live-dom-viewer/saved/471 throws in opera |
| 10:15 | <zcorpan> | also throws in firefox |
| 10:17 | <zcorpan> | seems annoying to have to create a new event and copy all properties |
| 10:19 | <hsivonen> | jgraham: did you also use createContextualFragment with a malformed form? |
| 10:20 | <zcorpan> | of course he did |
| 10:20 | <zcorpan> | and he put it in a library that was reused all over the place |
| 10:21 | <jgraham> | hsivonen: No :p |
| 10:21 | <hsivonen> | jgraham: good :-) |
| 10:21 | jgraham | still doesn't know what createContextualFragment does |
| 10:22 | <jgraham> | I would say that prevents me from using it but the web is empirical evidence against that line of thought |
| 10:22 | <hsivonen> | jgraham: it invokes the fragment parsing algorithm with a context node and a string to parse and returns a DocumentFragment |
| 10:22 | <hsivonen> | jgraham: it's like an innerHTML setter that gives you the fragment |
| 10:23 | <jgraham> | Doesn't it work to createElement a context element .innerHTML it and read the children? |
| 10:23 | <jgraham> | Might not be such a clean API though |
| 10:23 | <hsivonen> | jgraham: createContextualFragment predates innerHTML in Gecko, IIRC |
| 10:24 | <hsivonen> | jgraham: from the era when IEism were bad but creating own vendor-specific ad hoc APIs was OK |
| 10:24 | <hsivonen> | *IEisms |
| 10:24 | <jgraham> | Oh |
| 10:26 | <hsivonen> | I'd have to reread the CVS logs, but IIRC the use case was something in Netscape Composer and the API was exposed to the Web as a side effect |
| 10:28 | <zcorpan> | how disappointing, i thought the wine IE in crossover would use trident, but it uses gecko 1.8 |
| 10:28 | <zcorpan> | does ie throw if you click the border in http://software.hixie.ch/utilities/js/live-dom-viewer/saved/471 ? |
| 10:37 | <zcorpan> | oh ie doesn't have dispatchEvent at all |
| 11:48 | <hsivonen> | The Key Points slide at http://www.robglidden.com/2009/09/how-to-fix-dtv-patent-pools/ is interesting |
| 11:48 | <hsivonen> | looks like the TV people aren't too happy about MPEG encumberances, either |
| 11:52 | <roc> | Rob Glidden sounds like an interesting person |
| 12:01 | <hsivonen> | hmm. the fake Gtk menus in Opera 10.52 have the ancient bug that prevents diagonal mouse movement to a submenu |
| 12:02 | <hsivonen> | for Firefox and Chrome get this right in their fake Gtk menus |
| 12:02 | <hsivonen> | s/for/both/ |
| 12:03 | <roc> | doesn't Chrome use real Gtk menus? |
| 12:08 | <hsivonen> | roc: possibly. I thought it used fake Gtk stuff, because the other controls don't look right at all and the title bar is in the uncanny valley |
| 12:08 | <hsivonen> | (specifically, the buttons in the title bar) |
| 12:10 | <Lachy> | hsivonen, that's not really surprising. The fake UI approach causes problems on many platforms, but is unfortunately how things are being done for cross platform development. |
| 12:11 | <Lachy> | I've unsuccessfully argued against the fake UI approach before |
| 12:12 | <roc> | the thing about browsers is that they have to go for the fake UI approach for Web content |
| 12:12 | <hsivonen> | looks like OO.o has the same bug |
| 12:12 | <roc> | so given you have to tackle a lot of the hard fake UI problems anyway, it makes it that much more attractive for your actual browser UI |
| 12:13 | <Lachy> | roc, yes, for web content, fake UI is essential. But for browser chrome, I believe that only native UI will give optimal results. |
| 12:13 | <roc> | maybe |
| 12:14 | <Philip`> | Lachy: It's better for the chrome to be consistent with the desktop than to be consistent with the browser content? |
| 12:14 | <roc> | another thing is that on Windows and X, the toolkits that give you "real UI" are pretty lame |
| 12:15 | <Philip`> | I guess people who don't use any applications other than their browser would prefer the latter |
| 12:15 | <Lachy> | there are a whole bunch of Mac bugs in both Firefox and Opera that seem to be caused by the fake UI, especially in the nightly builds. |
| 12:15 | <roc> | at least, the ones you can access from C/C++ |
| 12:16 | <Lachy> | e.g. After having the browser open for a while, it becomes impossible to resize the window or to click and drag the window from any area of the chrome overlayed with the fake UI. |
| 12:17 | <roc> | I haven't seen that one |
| 12:18 | <Philip`> | roc: Just recompile Gecko in C++/CLI and then use WPF |
| 12:18 | <Lachy> | drop down, auto complete menus (e.g. address bar, search box, text fields) can start to only appear in a fixed position, and moving the window leaves them in place |
| 12:18 | <Lachy> | those 2 bugs require browser restarts to fix |
| 12:18 | <roc> | Philip`: ho ho ho |
| 12:18 | <Lachy> | they happen all the time on Snow Leopard |
| 12:18 | <roc> | I've seen that one |
| 12:18 | <roc> | but dropdowns are an example where using "real UI" is super-problematic in Web content |
| 12:19 | <roc> | Safari's "real UI" version of dropdowns is quite unusable for Web content with a large number of options |
| 12:19 | <Lachy> | roc, they usually occur at the same time. So when you see the drop downs appear in the wrong place, try resizing the window or moving the window by dragging on status bar. |
| 12:20 | <hsivonen> | the checkboxes in fake Gtk menus in Opera look wrong, too |
| 12:21 | <hsivonen> | I wonder how the Opera 10.52 fake Gtk widgets are drawn |
| 12:21 | <Lachy> | hsivonen, please file bugs about those issues |
| 12:22 | <hsivonen> | the pref window in Chrome looks like real Gtk |
| 12:22 | <hsivonen> | but the same window on Chrome OS looks generally terrible and Windows 95esque |
| 12:23 | <hsivonen> | I wonder how the Chrome pref window is implemented |
| 13:20 | <gsnedders> | "When content loads in an iframe, after any load events are fired within the content itself, the user agent must queue a task to fire a simple event named load at the iframe element." |
| 13:20 | <gsnedders> | What does it mean for content to load in an iframe? |
| 13:21 | <gsnedders> | I presume any URL change apart from a same-document reference will cause content to load |
| 13:31 | <gsnedders> | If you set location.href with a fragment reference, should it be done sync or async (esp. wrt reading back the URI from script)? |
| 15:51 | <AryehGregor> | IEBlog is really hit-and-miss, isn't it? The last post was very informative. |
| 15:51 | <AryehGregor> | I was particularly interested by: "Microsoft receives back from MPEG-LA less than half the amount for the patent rights that it contributes because there are many other companies that provide the licensed functionality in content and products that sell in high volume. Microsoft pledged its patent rights to this neutral organization in order to make its rights broadly available under clear terms, not because it thought this might be a good rev |
| 15:51 | <AryehGregor> | enue stream. We do not foresee this patent pool ever producing a material revenue stream, and revenue plays no part in our decision here. " |
| 15:54 | <AryehGregor> | I guess my working theory at this point is that Microsoft and Apple are probably just following the advice of their lawyers, as they claim, not engaging in some corporate strategy. Maybe Google supports Theora because Google is still ambitious and risk-taking at this point, and hasn't yet lapsed into megacorporate conservatism despite its size. |
| 15:55 | <tabatkins> | We do try, AryehGregor. |
| 15:57 | <jgraham> | AryehGregor: I don't think the "they're making vast profits off the patent licensing" theory was ever the most credible one for possible corporate strategy reasons for wanting MPEG-LA to win |
| 15:57 | <AryehGregor> | What's more credible? That open-source can't pay the fees? Open-source projects are either backed by a company, which can pay the fees; or aren't, in which case they make no money and MPEG-LA doesn't care. |
| 15:58 | <AryehGregor> | s/open-source/open source/ |
| 15:58 | <jgraham> | Small players in general (and new entrants to the market) can't pay the fees |
| 15:58 | <AryehGregor> | I thought they were scaled somewhat reasonably. Surely it's not in MPEG-LA's interest to discourage anyone from licensing? |
| 15:59 | <AryehGregor> | It's obviously easier for big companies due to the cap, of course. |
| 15:59 | <rektide> | i dont really understand why browsers dont just use whatever codecs are on the system? or do they, and its just a matter of how the browser supplements the codec collection? |
| 15:59 | <jgraham> | Plus it is a real problem for open source becuase it menas that they have to distribute non-open-source components |
| 15:59 | <AryehGregor> | What do you mean? There are open-source implementations of H.264, no? Chrome uses ffmpeg, for example. |
| 15:59 | <jgraham> | AryehGregor: My understanding is that any web browser would effectively have to pay the same amount |
| 16:00 | <hsivonen> | rektide: see roc's blog and roc's comment on the IE blog |
| 16:00 | <jgraham> | Although that is not based on anything much |
| 16:00 | <AryehGregor> | Yeah, maybe with a web browser you'd hit the cap very quickly . . . |
| 16:00 | <rektide> | why not use gstreamer, and support everything? there's a gstreamer-ffmpeg, for example. |
| 16:00 | <AryehGregor> | IIRC it's a fee per unit distributed, and if you give copies away you'd have to pay a lot. |
| 16:00 | <AryehGregor> | rektide, we don't want to support "everything", because in practice that means different browsers would support different things and we'd lose interoperability. |
| 16:01 | <AryehGregor> | Plus it means more poorly-tested code paths, larger executable size, etc. |
| 16:01 | <AryehGregor> | jgraham, well, all major platforms except XP and Vista have H.264 codecs installed by default or readily available, AFAIK, so the browser might not have to pay anything at all. |
| 16:01 | <jgraham> | AryehGregor: FWIW I don't consider source-avaliable components that you are prevented from redistributing to be "open source" |
| 16:02 | <AryehGregor> | (probably installed illegally on Linux, but again, nobody cares very much) |
| 16:02 | <AryehGregor> | Well, no, they technically aren't open-source. But in practice open-source people are resigned to having some not-fully-open-source components, unless you're rms. |
| 16:02 | <hsivonen> | AryehGregor: not caring is not a viable solution if you want linux to be successful |
| 16:02 | <AryehGregor> | Even Debian ships binary blobs in the kernel. |
| 16:03 | <AryehGregor> | hsivonen, distributions backed by a company with deep pockets can pay the fees. Others will get ignored indefinitely. |
| 16:03 | <AryehGregor> | Is MPEG-LA going to sue Debian? (Not sure if Debian actually distributes H.264 codecs, to be fair.) |
| 16:03 | <TabAtkins> | I am loathe to depend on the kindness of patent trolls to ignore people who can't legitimately pay. |
| 16:03 | <AryehGregor> | Oh, so am I. |
| 16:03 | <AryehGregor> | I'm talking purely pragmatically here. |
| 16:04 | <TabAtkins> | In addition, you're ignoring the middle case where people are profitable, but having to pay the licensing fees would cut their margins to thin to survive. |
| 16:04 | <AryehGregor> | This discussion started with the suggestion that Microsoft had ulterior motives for supporting H.264 over Theora, beyond patent risk. |
| 16:04 | <Dashiva> | How about just plain financial motives |
| 16:04 | <AryehGregor> | I don't think that trying to hurt open source is a plausible motive, because open source doesn't have much of a problem with H.264 *in practice*. |
| 16:04 | <AryehGregor> | Dashiva, those being? |
| 16:05 | <hsivonen> | binary blobs are about 1st party copyright or trade secret. h.264 is about 3rd party patents. totally different |
| 16:05 | <Dashiva> | Keeping small players out of the game |
| 16:05 | <AryehGregor> | TabAtkins, I don't think MPEG-LA would go after anyone in that scenario, because it wouldn't be in their interest. |
| 16:05 | <TabAtkins> | Though, hurting Firefox in particular could be a valid reason, given their hardline stance on the matter. Whereas supporting Theora would just give Apple decent reason to switch over too. |
| 16:05 | <AryehGregor> | Dashiva, how does it keep small players out of the game? OS X, Windows 7, and most Linux versions already include H.264 somehow, so browsers can just use those. |
| 16:06 | <TabAtkins> | AryehGregor: Again, I think you're assuming far too much from a patent-troll organization. |
| 16:06 | <AryehGregor> | MPEG-LA is not a patent troll. Patent trolls are organizations that file for patents that are probably frivolous and that they won't ever use. |
| 16:06 | <AryehGregor> | MPEG-LA organizes patents that are definitely not all frivolous, and which its members do use. |
| 16:07 | <Dashiva> | AryehGregor: Apparently that's not a viable solution according to anti-h264 arguments. |
| 16:07 | <AryehGregor> | Dashiva, I'm pretty sure I saw someone from Mozilla saying that if they supported H.264, they would indeed just use the system codec where available, and their reasons for not doing that were mainly idealistic. |
| 16:08 | <AryehGregor> | (not that I disagree with them, although at this point it seems like a lost cause) |
| 16:08 | <TabAtkins> | AryehGregor: The entire codec arena is so fraught with patent peril that the situation is much more ambiguous than you describe. Virtually *anything* you can write in the codec space is covered by a patent somewhere. |
| 16:08 | <AryehGregor> | TabAtkins, Gregory Maxwell of Xiph just wrote a fairly lengthy e-mail saying that that is exactly not the case. It's only what MPEG-LA wants you to think. |
| 16:08 | <AryehGregor> | http://lists.xiph.org/pipermail/theora/2010-April/003769.html |
| 16:09 | TabAtkins | reads. |
| 16:09 | <AryehGregor> | In practice, H.264 has been used so widely for so many years by so many companies that any patent-holders would have been flushed out by now. |
| 16:09 | <AryehGregor> | Even if they haven't been, then you'd expect them to join the MPEG-LA, not sue random companies licensing H.264. |
| 16:09 | <AryehGregor> | So it's pretty safe. |
| 16:10 | <AryehGregor> | Theora is hopefully safe, but it hasn't had the same level of exposure. |
| 16:10 | <AryehGregor> | If Google goes a few more years without getting sued, it will look a lot more promising. |
| 16:11 | <TabAtkins> | The email makes a convincing case. The fact that he framed it in terms of incentives makes it more believable to me. |
| 16:12 | <AryehGregor> | That's typical of him. The "being extremely convincing and well-thought-out" thing, I mean. |
| 16:12 | <AryehGregor> | I can tell you from personal experience that it is a terrible idea to argue with Greg Maxwell about anything. You will not only lose, you will lose *horribly*. |
| 16:12 | <AryehGregor> | (he's involved in Wikipedia too) |
| 16:12 | <Lachy> | AryehGregor, the MPEG-LA focusses on software patents, which are frivolous, and should never be patented. |
| 16:12 | <TabAtkins> | Hehe. |
| 16:13 | <Lachy> | it just sucks that we have such broken patent systems around the world that permit software patents, either explicitly or through loop holes |
| 16:13 | <TabAtkins> | Lachy: Yeah, but Aryeh's point about them not being a "patent troll" per se is still valid. That's generally reserved for non-practicing patent-holding entities. |
| 16:13 | <Lachy> | TabAtkins, I'm not arguing against that. I know MPEG-LA aren't patent trolls |
| 16:13 | <AryehGregor> | Lachy, it's not just software patents, it's lots of types of patents. Most if not all are believed to be legitimate under current law. That law should be changed, yes, but that doesn't make the patent suits frivolous. A frivolous lawsuit is one that clearly has no basis in law, not one that has a basis in law you don't like. |
| 16:13 | <TabAtkins> | Kk. Well, I agree with you, then. |
| 16:14 | <Lachy> | ok, if that's what you meant by frivolous, then fair enough |
| 16:15 | <AryehGregor> | Greg Maxwell is still technically Chief Research Coordinator at Wikimedia: http://wikimediafoundation.org/wiki/Resolution:Chief_Research_Coordinator |
| 16:15 | <AryehGregor> | Although I think the role is meaningless. |
| 16:16 | <AryehGregor> | (predating the time when Wikimedia had a substantial number of actual employees) |
| 18:44 | <paul_irish> | Philip`: how did you come across the obfuscation methods in your font optimizer? |
| 18:45 | <paul_irish> | me and ethan of fontsquirrel were just discussing them. really excellent. |
| 18:48 | <Philip`> | paul_irish: I just tried deleting random stuff until finding that browsers rejected them and then went back a step and tried again |
| 18:49 | <paul_irish> | that's excellent. can you summarize what the POST table changes you make are? |
| 18:49 | <paul_irish> | ethan seemed to think this would conflict with the OTS sanitizing/security stuff that Chrome is doing. |
| 18:50 | <Philip`> | paul_irish: They're just http://bitbucket.org/philip/font-optimizer/src/tip/obfuscate-font.pl#cl-76 |
| 18:51 | <Philip`> | i.e. keeping some minimum length, setting everything to 0, but setting the version field to 1 because otherwise Chrome didn't like it |
| 18:52 | <Philip`> | (It's quite possible that Chrome might become stricter and reject it) |
| 18:53 | <paul_irish> | cool. i'm going to attempt to write up these obfuscations up as a spec of sorts.. with the goal of foundries agreeing to license their work if implementers agree to this sort of level of protection. |
| 18:54 | <Philip`> | (since it's probably totally invalid - the goal was to be invalid enough that e.g. Windows wouldn't let you install or view the font, but that it would still work in all the browsers I could test) |
| 18:54 | <Philip`> | (If I remember correctly, it does break the font installation on Windows and OS X) |
| 18:55 | <Philip`> | (though obviously it's totally trivial for a tool to fix the font and make it readable again) |
| 18:55 | <JonathanNeal> | That's a really groovy idea Philip`, paul_irish nice work :) |
| 18:55 | <paul_irish> | totally. Ethan actually had a variation of the name table technique, where he uses a unicode smiley face as the Name (instead of empty string).. this prevents installation on Mac |
| 18:55 | <Philip`> | (It's a disgusting evil standards-violating hack, but that's okay) |
| 18:58 | <JonathanNeal> | I like evil hacks. |
| 19:06 | <Dashiva> | I guess you'd need some kind of statement from microsoft that they won't make their font importer start supporting those files |
| 21:05 | <jgraham> | Yeah, the font thing sounds bad (corrupting the files and assuming the OS will never support the coruppted file but browsers always will) |
| 21:09 | <zcorpan> | http://www.ietf.org/mail-archive/web/hybi/current/msg01830.html - hmmm |
| 21:10 | <zcorpan> | wonder if i should send an email saying "hi there, we're implementing complete.html#websocket over here" |
| 21:21 | <franksalim> | zcorpan, I think that information would be appreciated |
| 21:26 | <zcorpan> | done |
| 21:27 | <zcorpan> | i hope webkit and mozilla are also tracking the latest version |
| 21:28 | <jgraham> | zcorpan: Mozilla said they were holding off shipping anything untill it stabilised |
| 21:29 | <zcorpan> | ok |
| 21:29 | <zcorpan> | but they're not holding off implementation work, or are they? |
| 21:30 | <othermaciej> | zcorpan: Chrome has already shipped WebSocket, for the next Safari I am not sure whether we will disable it, and we're actively updating trunk to the latest version |
| 21:30 | <othermaciej> | as in, there's patches in progress |
| 21:31 | <zcorpan> | othermaciej: ok, thanks |
| 21:32 | <zcorpan> | othermaciej: any news on URL/url? |
| 21:32 | <othermaciej> | zcorpan: I have not done anything related to it |
| 21:32 | <zcorpan> | ok |
| 21:34 | <jgraham> | zcorpan: http://hacks.mozilla.org/2010/04/websockets-in-firefox/ |
| 21:38 | <othermaciej> | jgraham, zcorpan: in any case I think -76 is a big improvement over -75 in terms of security and ease of implementation for combo http/websocket servers |
| 21:40 | <jgraham> | othermaciej: Yeah. I'm not sure what the big problems with the WHATWG version are supposed to be, and whether they are real or not |
| 21:40 | <gregw> | I think -76 has some good ease of implementation improvements, but I'm very dubious about the new handshake |
| 21:40 | <gregw> | it does not work well with HTTP servers |
| 21:40 | <zcorpan> | gregw: what part doesn't work well? |
| 21:40 | <othermaciej> | gregw: how is it a problem for HTTP servers? or rather, how is it more of a problem than the -75 version? |
| 21:41 | <gregw> | because of the non content data after the requests |
| 21:41 | <jgraham> | gregw: The random bytes? |
| 21:42 | <gregw> | the IETF WG really wants the handshake to be HTTP compliant until the 101 is sent |
| 21:42 | <gregw> | those bytes break that |
| 21:42 | <othermaciej> | gregw: aren't those bytes effectively just a message body? |
| 21:42 | <jgraham> | Shouldn't the server have done the handoff to the WebSockets specific code by that point? (or treat them like a body) |
| 21:42 | <gregw> | they would be if there was a content length |
| 21:43 | <othermaciej> | I don't think Content-Length is required to send a request body |
| 21:43 | <gregw> | It is in a persistent connection. |
| 21:43 | <zcorpan> | iirc Hixie argued that the random bytes are content after the upgrade |
| 21:43 | <gregw> | and if we are HTTP complient other status codes might get sent back - like a 401 for authentication |
| 21:44 | <gregw> | zcorpan: no he wants them to not be content |
| 21:44 | <gregw> | so they break HTTP servers... so they can't be "tricked" into doing a handshake |
| 21:44 | <jgraham> | gregw: Are there examples of HTTP servers that cannot implement WebSockets due to this design |
| 21:44 | <othermaciej> | the WebSocket connection can't be used as a persistent HTTP connection |
| 21:45 | <othermaciej> | so Content-Length is not required |
| 21:45 | <othermaciej> | that being said, if it was added, would that address your objection? |
| 21:45 | <gregw> | sure - but then you might as well make it a header |
| 21:45 | <gregw> | simpler to implement |
| 21:45 | <gregw> | and I'm not sure that 101 can have a body |
| 21:46 | <othermaciej> | these bytes are in the request, not the response |
| 21:46 | <othermaciej> | any request can have a body (other than GET or HEAD) IIRC |
| 21:46 | <gregw> | there are more in the response |
| 21:46 | <gregw> | but actually the response ones are OK |
| 21:46 | <gregw> | as they are after the 101 |
| 21:46 | <gregw> | so yeh - if the request ones can be made legal HTTP then I'm happy |
| 21:46 | <othermaciej> | so it sounds to me like nothing here breaks HTTP before the 101 response is sent |
| 21:47 | <gregw> | but I still think they are a little bit overkill |
| 21:47 | <gregw> | it does if you send something other than the 101 |
| 21:47 | <gregw> | like a 401 |
| 21:47 | <gregw> | or a 500 |
| 21:47 | <gregw> | etc |
| 21:47 | <othermaciej> | what breaks http in that case? |
| 21:47 | <gregw> | the random bytes will be a bad request |
| 21:48 | <gregw> | unless they are content of the request |
| 21:48 | <othermaciej> | the client doesn't send a 401 though |
| 21:48 | <othermaciej> | the client doesn't know whether it will get a 401 |
| 21:48 | <othermaciej> | so that can't possibly affect what is a valid http request |
| 21:48 | <gregw> | OK - let's flip this around... what's the problem with making it a completely legal HTTP request? |
| 21:48 | <othermaciej> | I believe it already is a completely legal HTTP request |
| 21:49 | <gregw> | It is, but the random bytes break the next request in the connection |
| 21:49 | <othermaciej> | if it isn't one, I would consider that a bug |
| 21:49 | <othermaciej> | but there won't be a next request in the connection |
| 21:49 | <gregw> | there can be if a 401 is sent |
| 21:49 | <zcorpan> | i thought the random bytes were there to make it harder to do a cross protocol attack or so |
| 21:49 | <Hixie> | the whole point of the first 8 bytes from the client after the upgrade is to make intermediaries who aren't goign to support websocket fail early |
| 21:49 | <gregw> | the random bytes work just as well in a header |
| 21:49 | <Hixie> | not for the purpose of breaking intermediaries |
| 21:50 | <jgraham> | Are clients already epected to deal with arbitary HTTP responses? |
| 21:50 | <othermaciej> | gregw: in the case of a 401, the client has to close the connection |
| 21:50 | <gregw> | no |
| 21:50 | <Hixie> | yes |
| 21:50 | <gregw> | my no was to jgraham |
| 21:51 | <othermaciej> | the client is not allowed per spec to send another http request down the same connection that was used for an attempted WebSocket upgrade |
| 21:51 | <gregw> | but there is a reasonable level of support in the IETF WG to have that as an option |
| 21:51 | <gregw> | and those bytes make that impossible for little clear benefit |
| 21:51 | <gregw> | othermaciej: why not - that is not compliant HTTP |
| 21:52 | <othermaciej> | what do you mean? |
| 21:52 | <gregw> | the whole point is that before the 101, the connection is-a HTTP request |
| 21:52 | <othermaciej> | the client can close its connection at any time |
| 21:52 | <gregw> | so if client and server agree to use it as a persistent HTTP connection, they can do so |
| 21:52 | <Hixie> | the client is never an http client, it's a websocket client. From the client's perspective, it's always a WebSocket connection. |
| 21:52 | <othermaciej> | tell me what HTTP conformance requirement is violated by closing the connection in response to a 401 (or any non-101 response) |
| 21:52 | <Hixie> | It's only the server who thinks it's HTTP |
| 21:52 | <gregw> | well you client might be, but there are other clients |
| 21:53 | <Hixie> | an HTTP client isn't going to try to upgrade to WebSocket |
| 21:53 | <Hixie> | and so there's no problem there either |
| 21:53 | <gregw> | there is the use-case that you want to try to do the upgrade, but also request some real content |
| 21:53 | <gregw> | so that if you can upgrade you do, if you can't you send some real content |
| 21:53 | <Hixie> | no, there isn't |
| 21:53 | <gregw> | and avoid another RTT |
| 21:53 | <Hixie> | no browser is ever going to do that |
| 21:54 | <othermaciej> | neither the client protocol nor the JS cl ient API support that use case |
| 21:54 | <gregw> | that is what the SPDY guys are interested in |
| 21:54 | <Hixie> | the SPDY guys aren't using websocket |
| 21:54 | <othermaciej> | if you want to propose it, go ahead, but "I want a new feature" is very different from "this breaks HTTP" |
| 21:54 | <gregw> | not yet |
| 21:54 | <Hixie> | the SPDY guys are never going to use websocket -- websocket is a completely inappropriate protocol for their use case |
| 21:54 | <gregw> | anyway, we've had this debate before... I really am just saying that a lot of -76 is well accepted, but some parts are still being debated |
| 21:54 | <othermaciej> | running SPDY over WebSocket would be silly |
| 21:55 | <othermaciej> | gregw: do you believe -76 has less consensus than -75? |
| 21:55 | <gregw> | I think that lots of -76 has consensus, but that parts do not |
| 21:55 | <othermaciej> | that doesn't answer my question |
| 21:55 | <gregw> | why would running SPDY over websocket be silly? there is interest on the EITF WG list |
| 21:56 | <gregw> | -75 has consensus in as much as it reflects the implementations currently shipping |
| 21:56 | <othermaciej> | because SPDY doesn't need or want an additional framing layer |
| 21:56 | <gregw> | I think we all accept there are problems |
| 21:56 | <gregw> | but it has a framing layer |
| 21:56 | <othermaciej> | the only implementation shipping -75 is Chrome and it's in the process of being rewritten to -76 |
| 21:56 | <gregw> | so why are we inventing 2 new framing layers that go over HTTP intermediaries |
| 21:56 | <gregw> | surely it would be best to come up with only 1 |
| 21:56 | <franksalim> | SPDY is not intended to go over HTTP intermediaries |
| 21:56 | <gregw> | othermaciej: have a read of the list of impls on wikipedia |
| 21:57 | <gregw> | there are 10s |
| 21:57 | <othermaciej> | gregw: the non-browser impls can be upgraded much more readily than browsers, but that being said, has any of those implementors said they prefer -75 to -76? |
| 21:57 | <jgraham> | gregw: There are many server side implementations. But they are worthless without clients |
| 21:58 | <gregw> | well the IETF WG is pretty clear that HTTP compliance is a requirement - I guess it is debatable if -76 meets that or not |
| 21:58 | <Hixie> | given that it takes about 4 hours to write a server-side implementation, i'm not surprised there are a lot of them |
| 21:58 | <Hixie> | :-) |
| 21:58 | <othermaciej> | I'm pretty sure all the client implementations listed in Wikipedia are going to upgrade to -76 |
| 21:59 | <othermaciej> | -76 definitely does not meet the HTTP compliance requirement any *less* than -75 |
| 21:59 | <Hixie> | (i mean, i've written at least 3 myself) |
| 21:59 | <othermaciej> | it definitely meets it more, and arguably meets it completely |
| 21:59 | <gregw> | Hixie: so are impls less important that clients? |
| 21:59 | <Hixie> | the HTTP compliance requirement isn't even a goal, IMHO. |
| 21:59 | <Hixie> | gregw: server impls are far less important than clients, yes |
| 22:00 | <gregw> | obviously not here... but it is in the IETF WG |
| 22:00 | <Hixie> | i'm in the IETF WG |
| 22:00 | <Hixie> | as is maciej |
| 22:00 | <gregw> | anyway... I'm obviously not making my point here |
| 22:00 | <gregw> | so I'll leave you be |
| 22:00 | <othermaciej> | I'm not sure what's better about -75 than -76 |
| 22:00 | <gregw> | no random bytes outside of the request |
| 22:00 | <gregw> | put them in a header and it would be a lot better |
| 22:01 | <othermaciej> | if it contains an actual regression on any requirement relative to -75, then I would see why you might not want to start with it |
| 22:01 | <othermaciej> | I don't recall a requirement that the handshake request must not have a body |
| 22:02 | <othermaciej> | certainly that's not required for the "http compliance" requirement |
| 22:02 | <gregw> | I'm cool if they are a legal body |
| 22:02 | <gregw> | but they are not |
| 22:02 | <othermaciej> | -75 also blatantly doesn't meet that requirement |
| 22:02 | jgraham | would still like examples of actual deployed HTTP servers that cannot cope with -76 |
| 22:02 | <othermaciej> | why are they not a legal body? |
| 22:02 | <Hixie> | per HTTP, the eight bytes the client send are the first eight bytes of a second pipelined request |
| 22:02 | <othermaciej> | can you point to a specific conformance requirement in the HTTP RFC that they violate? |
| 22:02 | <Hixie> | that's intentional, the whole point is to break intermediaries who don't know websocket |
| 22:03 | <gregw> | jgraham: it is more about being able to handle non 101 responses. OK currently implemented, but there is interest in 401, 302's etc |
| 22:03 | <othermaciej> | Hixie: so HTTP doesn't allow a request body without Content-Length? |
| 22:03 | <Hixie> | not for GET iirc |
| 22:03 | <gregw> | Hixie: +1 |
| 22:03 | <othermaciej> | the WebSocket request is a GET? |
| 22:03 | <othermaciej> | (it doesn't allow a body at all for GET) |
| 22:03 | <Hixie> | yes |
| 22:03 | <Hixie> | yes it does |
| 22:03 | <Hixie> | you can set Content-LEngth with a GET |
| 22:03 | <gregw> | and thus needs a content-length or chunking to have a body |
| 22:03 | <Hixie> | (but nobody does it) |
| 22:03 | <othermaciej> | I see |
| 22:03 | <othermaciej> | would sending a Content-Length be a problem? |
| 22:03 | <Hixie> | gregw: no the whole POINT is to cause the intermediaries to fail |
| 22:03 | <Hixie> | othermaciej: yes, cos then intermediaries wouldn't fail |
| 22:04 | <gregw> | it might cause some intermediaries to fail sometimes |
| 22:04 | <Hixie> | from the point of view of the client, it's not HTTP at all, it's websocket the whole time |
| 22:04 | <othermaciej> | Hixie: do we have data showing that this makes unaware intermediaries fail early? |
| 22:04 | <gregw> | they will fail anyway becuause Upgrade is hop by hop |
| 22:04 | <Hixie> | from the point of view of an HTTP server, it's an HTTP request followed by a bogus request |
| 22:04 | <gregw> | if the intermediary does not know about websocket, it will not forward the upgrade |
| 22:04 | <othermaciej> | (more so than anything else in the handshake)? |
| 22:04 | <Hixie> | from the point of view of a WebSocket server, it's a WebSocket request |
| 22:04 | <gregw> | unless it is a dumb byte copier |
| 22:05 | <jgraham> | I'm not sure what the advantage is of adding the full complexity of HTTP to WebSockets rather than doing it at the application layer |
| 22:05 | <gregw> | in which case the random bytes will be copied |
| 22:05 | <gregw> | jgraham: I'm not advocating that |
| 22:05 | <Hixie> | from the point of view of an HTTP+WebSocket server, it's an HTTP request followed by the remainder of the data of what was actually a WebSocket request |
| 22:05 | <Hixie> | othermaciej: i'm aware of at least one intermediary that failed because of this, but i don't have non-anecdotal data yet |
| 22:05 | <gregw> | Hixie: but if you send weboscket data before the 101, then you are not HTTP compliant |
| 22:06 | <Hixie> | gregw: intermediaries don't follow the spec, they pass upgrades through unmodified |
| 22:06 | <gregw> | it is a HTTP connection until the 101 is sent |
| 22:06 | <othermaciej> | Hixie: I guess non-anecdotal data would be what we need to determine if this mechanism is effective for its purpose |
| 22:06 | <gregw> | if intermediaries pass the upgrade unchanged, then they will probaby copy the bytes as well |
| 22:06 | <Hixie> | othermaciej: we have data showing that without this, the handshake "succeeds" in some double-digit number of cases but the first frame sent fails to make it through |
| 22:06 | <jgraham> | gregw: Allowing arbitary response codes seems like more complexity. Maybe not the full comlexity of HTTP |
| 22:06 | <Hixie> | othermaciej: which is basically what this handshake does now |
| 22:06 | <gregw> | or just make it legal HTTP as the IETF wants |
| 22:07 | <gregw> | jgraham: that's only for consenting clients/servers. it is not a MUST requirement |
| 22:07 | <john_fallows> | if you want HTTP intermediaries to fail during the handshake request, why not use POST instead of GET and omit the Content-Length header, that would very likely trigger a 411 Length Required |
| 22:07 | <othermaciej> | Hixie: would it cause a problem to send the random bytes after the "successful" 101 response? |
| 22:07 | <gregw> | jgraham: but if a client and server want to use BASIC or DIGEST auth, then why not let them use it? |
| 22:07 | <jgraham> | gregw: Consenting serves, yes. I don't see how any client could avoid it |
| 22:08 | <othermaciej> | john_fallows: that's a neat idea |
| 22:08 | <Hixie> | gregw: argument from authority has no effect here (invoking the IETF's name won't win you the argument) |
| 22:08 | <gregw> | jgraham: there is no need to follow a 401 |
| 22:08 | <Hixie> | gregw: especially since we are part of the IETF WG |
| 22:08 | <gregw> | it's just that the server will only allow clients that do connect |
| 22:08 | <othermaciej> | argument from authority shouldn't win in the IETF either |
| 22:08 | <Hixie> | othermaciej: it doubles the RTT for the handshake |
| 22:08 | <jgraham> | gregw: That's the same as saying "all clients need to implement it" |
| 22:08 | <othermaciej> | Hixie: what about john_fallows's POST idea? |
| 22:09 | <gregw> | jgraham: well all the browsers have it already... so it's not a big deal |
| 22:09 | <gregw> | othermaciej: that is still illegal HTTP |
| 22:09 | <Hixie> | othermaciej: i don't think it would make a difference -- intermediaries appear to be pretty HTTP-stupid. But I'm happy to change it to POST if that makes people happier. |
| 22:09 | <Hixie> | othermaciej: but i don't think it'd make gregw happier |
| 22:10 | <othermaciej> | Hixie: would it be technically legal HTTP? |
| 22:10 | <jgraham> | gregw: I am unconvinced it is that simple, but have no experience with implementing a client |
| 22:10 | <gregw> | Hixie: true. I think it should be HTTP legal until the 101 is received |
| 22:10 | <Hixie> | othermaciej: off-hand, no idea |
| 22:11 | <othermaciej> | I can't find any requirement to send Content-Length in a request |
| 22:11 | <othermaciej> | I guess I need to read the HTTP RFC more carefully |
| 22:11 | <gregw> | without something to indicate the content length, it is not legal |
| 22:11 | <othermaciej> | gregw: or if you have a cite for what conformance requirement is violated, that would help |
| 22:11 | <zcorpan> | are there other ways to make intermediaries fail while being legal http? |
| 22:11 | <othermaciej> | ah: |
| 22:11 | <gregw> | othermaciej: you need to look at RFC2616 in the section about marking body ends |
| 22:11 | <othermaciej> | "For compatibility with HTTP/1.0 applications, HTTP/1.1 requests containing a message-body MUST include a valid Content-Length header field unless the server is known to be HTTP/1.1 compliant." |
| 22:12 | <Hixie> | "The presence of a message-body in a request is signaled by the inclusion of a Content-Length or Transfer-Encoding header field in the request's message-headers." |
| 22:13 | <Hixie> | gregw: part of the reason you're not convincing me is that you're just saying "you can't do X" without giving me an alternative way of achieving the effect I'm attempting to achieve. |
| 22:13 | <othermaciej> | Hixie: that's not a conformance requirement (and doesn't seem to quite match the server requirements) |
| 22:13 | <Hixie> | gregw: so you just feel like stop energy, you don't seem to be helping. |
| 22:13 | <gregw> | Hixie: true, but I don't think your solution works anyway, and I don't think it is actually possible |
| 22:13 | <Hixie> | othermaciej: oh, good point |
| 22:13 | <gregw> | intermediairies will either be good or they will be stupid byte copiers |
| 22:14 | <othermaciej> | however, what I quoted is a MUST-level requirement for HTTP 1.1 clients |
| 22:14 | <gregw> | your proposal does not help with the later and is not needed for the former |
| 22:14 | <zcorpan> | othermaciej: so if the server is known to be compliant, that MUST doesn't apply |
| 22:14 | <othermaciej> | Hixie: could the random content bytes piggyback on the packet for the first client message? |
| 22:14 | <gregw> | zcorpan: but then it is either a content length or chunking |
| 22:14 | <othermaciej> | zcorpan: right - though connecting to a random server, you have no way to know |
| 22:14 | <gregw> | let me find the section.... |
| 22:14 | <Hixie> | gregw: evidence does not bear this out |
| 22:15 | <gregw> | Hixie: then please publish the evidence |
| 22:15 | <othermaciej> | gregw: at least in section 4.4 Message Length, there doesn't seem to be a requirement to that effect |
| 22:15 | <Hixie> | gregw: we have double-digit-percentage numbers of connections in the test who passed through the Upgrade, but did not pass through the first frame |
| 22:15 | <Hixie> | gregw: the data was sent to the hybi list |
| 22:15 | <zcorpan> | othermaciej: right, but it seems reasonable to assume that a server that supports websockets isn't going to be an http/1.0 application |
| 22:15 | <othermaciej> | http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 |
| 22:16 | <othermaciej> | zcorpan: yes, but when JS in the browser initiates the connection, the browser doesn't know if the server actually supports websockets |
| 22:16 | <othermaciej> | zcorpan: in fact the design of the handshake is set up so the browser can find out with good confidence whether the server does in fact support webscocket |
| 22:16 | <Hixie> | othermaciej: (the client in that case is a websocket client, so it isn't bound to HTTP rules) |
| 22:17 | <othermaciej> | Hixie: indeed, except to the extent one accepts the requirement that servers should observe what looks like valid HTTP until they respond with a 101 |
| 22:17 | <gregw> | Hixie: so you can make the first message of the websocket connect a check message |
| 22:17 | <gregw> | if a message is not quickly received, then the upgrade did not work |
| 22:17 | <gregw> | and you have fail fast |
| 22:18 | <gregw> | the first message can be an application message if one is available, or some kind of noop ping |
| 22:18 | <gregw> | there is already talk about keep-alives and pings |
| 22:18 | <gregw> | so just send one immediately if there is no application message |
| 22:19 | <gregw> | that also separates out the attack protection mechanism (the random bytes) from the fast fail mechanism |
| 22:19 | <Hixie> | othermaciej: well HTTP 1.1 servers can assume they are HTTP 1.1, so the 1.0 requirement doesn't apply |
| 22:19 | <Hixie> | gregw: how can you do that without requiring an extra RTT? |
| 22:20 | <gregw> | well if you have an application message, you send that just as you would anyway |
| 22:20 | <gregw> | if you don't then send a ping |
| 22:20 | <gregw> | ok so the fail fast is not as fast |
| 22:20 | <gregw> | the failure has one more RTT |
| 22:20 | <gregw> | but that's only for failures |
| 22:20 | <Hixie> | that makes the client behaviour depnd on JS execution speed, which is an interop nightmare |
| 22:21 | <Hixie> | i'd much rather do what we're doing now |
| 22:21 | <gregw> | no - the browser can send a ping immediately the 101 is received |
| 22:21 | <Hixie> | wait, that won't work anyway |
| 22:21 | <gregw> | but what you are doing now is breaking HTTP and will cause all sorts of future problems |
| 22:21 | <Hixie> | the whole point is the API doesn't say it's connected until it's connected |
| 22:22 | <Hixie> | if we require a ping first, then there's an extra RTT before we're connected |
| 22:22 | <zcorpan> | i don't like adding more latency to the handshake |
| 22:22 | <Hixie> | that's a terrible solution |
| 22:22 | <Hixie> | we're |
| 22:22 | <Hixie> | not |
| 22:22 | <Hixie> | breaking |
| 22:22 | <Hixie> | HTTP |
| 22:22 | <Hixie> | this isn't HTTP |
| 22:22 | <gregw> | it is until the 101 is sent |
| 22:22 | <Hixie> | no, it's not |
| 22:22 | <Hixie> | that's BS |
| 22:22 | <Hixie> | plus, even if it was, as maciej has pointed out, it's not actually invalid |
| 22:23 | <gregw> | you need to have a content length or chunking or one of the self limiting content types to have a content body |
| 22:23 | <gregw> | or you can close the connection... but that's not useful for a request |
| 22:24 | <Hixie> | quote the spec that says that |
| 22:24 | <Hixie> | please |
| 22:24 | <Hixie> | cite the MUST requirement |
| 22:24 | <gregw> | in section 4.4 |
| 22:24 | <gregw> | there are only 5 ways |
| 22:24 | <Hixie> | paste the sentence |
| 22:24 | <gregw> | and 1 does not apply, nor does 5 |
| 22:24 | <gregw> | so you are left with 2, 3 or 4 |
| 22:25 | <Hixie> | 4.4 is all server requirements |
| 22:25 | <Hixie> | except for the HTTP 1.0 compat requirement maciej pasted |
| 22:25 | <Hixie> | which as noted isn't relevant here |
| 22:25 | <gregw> | no it is for any HTTP message |
| 22:26 | <Hixie> | it's about how to receive the message |
| 22:26 | <Hixie> | which in the case of a client-sent message, is a server-side requirement |
| 22:26 | <gregw> | it's a HTTP message requirement |
| 22:27 | <Hixie> | no, it's not |
| 22:27 | <gregw> | and the random bytes will break a HTTP connection if the upgrade is not accepted |
| 22:27 | <Hixie> | there's no HTTP connection in that case |
| 22:27 | <Hixie> | the client will abort regardless of the response |
| 22:27 | <gregw> | why? |
| 22:27 | <Hixie> | because the websocket spec requires it to |
| 22:28 | <gregw> | but you are not in websockets! the upgrade faile |
| 22:28 | <gregw> | but you are not in websockets! the upgrade failed |
| 22:28 | <Hixie> | if you're not a websocket client, then you didn't send an upgrade request or the 8 random bytes |
| 22:28 | <zcorpan> | gregw: the client is always in websockets |
| 22:28 | <gregw> | so you have a valid HTTP connection that should be able to be used without requirement for another RTT |
| 22:29 | <gregw> | you might be a client that can be websocket or can be something else |
| 22:29 | <Hixie> | the connection itself isn't HTTP or WebSockets or whatnot. That's a meaningless abstraction. |
| 22:29 | <Hixie> | it's just bytes |
| 22:29 | <gregw> | so it tries the upgrade, and if that does not work, keeps using HTTP |
| 22:29 | <Hixie> | what matters is what the peers think is going on |
| 22:29 | <Hixie> | there is no way to try an upgrade and continue using HTTP |
| 22:29 | <gregw> | so you are going to require and extra RTT for all the apps that can't use Websocket |
| 22:29 | <Hixie> | that's a violation of the websocket protocol |
| 22:29 | <gregw> | just so they can try to use it |
| 22:29 | <gregw> | but until the 101, you are not in websockets |
| 22:29 | <Hixie> | there's no other way to use the API |
| 22:29 | <gregw> | you are in HTTP |
| 22:30 | <Hixie> | the client is NEVER in HTTP |
| 22:30 | <gregw> | there are other clients than JS |
| 22:30 | <Hixie> | those clients should use TCP |
| 22:30 | <Hixie> | websockets is irrelevant to those clients |
| 22:30 | <gregw> | well that's not what is happening out there |
| 22:30 | <Hixie> | those clients do not need to send the 8 bytes |
| 22:30 | <Hixie> | they can do whatever they want to upgrade the server |
| 22:30 | <gregw> | some of them are called browsers! |
| 22:31 | <Hixie> | you are not making any sense here |
| 22:31 | <gregw> | well that's a winning argument |
| 22:31 | <gregw> | l8r |
| 22:35 | <jgraham> | FWIW it seems likely to me that non-browser clients for websockets will be used |
| 22:36 | <Dashiva> | Wouldn't they just connect directly with tcp without going via http? |
| 22:36 | <jgraham> | Because there is value in interacting with the websocket ecosystem outside the browser |
| 22:36 | <jgraham> | Dashiva: Connect to what? |
| 22:36 | <jgraham> | Assume some service is provided over websockets for browsers |
| 22:37 | <jgraham> | And someone wants to develop a custom non-browser app that connets to exactly the same service |
| 22:44 | <othermaciej> | then they couldn't use a general-purpose http client to talk to that service |
| 22:45 | <gregw> | but they want to be able to tunnel bidirectional communication through a HTTP infrastructure (eg firewalls, proxies and intermediaries) |
| 22:45 | <gregw> | plus browsers themselves might want to do websocket extensions |
| 22:46 | <gregw> | so the "can't do it in JS, so can't do it at all" argument is not that valid |
| 22:46 | <jgraham> | I'm not suggesting a non-browser-client would be a general purpose HTTP client |
| 22:46 | <jgraham> | It would be custom websockets code |
| 22:46 | <othermaciej> | I agree that this seems likely |
| 22:47 | <jgraham> | (this is why I would like the client side to be relatively simple to implement) |
| 22:47 | <gregw> | jgraham: I don't anybody disagrees with that |
| 22:48 | <gregw> | but that is not to say that we cannot have optional less simple solutions as well |
| 22:48 | <othermaciej> | I'm somewhat more willing to impose client-side complexity, because at least browser-hosted clients have to do what it takes to ensure security against cross-protocol attacks |
| 22:49 | <jgraham> | gregw: I don't really believe in optional features |
| 22:49 | <jgraham> | othermaciej: agreed that complexity for security is well justified |
| 22:50 | <gregw> | jgraham: but that says that websocket has to have an authentication system that will keep everybody happy for all the time |
| 22:50 | <gregw> | and compression that will last forever |
| 22:50 | <gregw> | jgraham: but I agree that options have to be carefully thought out so they are not by default mandatory |
| 22:52 | <gregw> | but as a lot of websocket connections are going to be established from well featured HTTP clients to well featured HTTP servers, it seams very very strange to not let them use features like authentication |
| 22:52 | <gregw> | if they both wish |
| 22:56 | <jgraham> | I don't think it is bad to allow for the possibility of future expansion |
| 22:56 | <jgraham> | I do think it is bad to have explicitly optional features |
| 22:56 | <jgraham> | Anyway bedtime |
| 22:56 | <jgraham> | gn |
| 22:56 | <zcorpan> | nn jgraham |
| 22:56 | <gregw> | jgraham: guess I can agree with that.... which is why keeping the HTTP legal is good for future proofing |
| 23:58 | <Hixie> | does anyone know of any places that depend on the components for html5 bugs? |
| 23:59 | <Hixie> | so far i have: |
| 23:59 | <Hixie> | - bugs link on http://xn--74h.damowmow.com/ |
| 23:59 | <Hixie> | - script that updates http://www.whatwg.org/issues/data.html?period=1 |
| 23:59 | <Hixie> | - bug report form on the spec (file-bug.cgi) |
| 23:59 | <Hixie> | - various spec headers: |
| 23:59 | <Hixie> | (6 files) |
| 23:59 | <Hixie> | and some stuff i use personally |