| 07:15 | <rkirsling> | umm I think however test262 report is running JSC needs to be updated, lol |
| 07:15 | <rkirsling> | https://test262.report/?date=2020-04-07 |
| 07:15 | <rkirsling> | I assure you our overall conformance did not drop to 45% lol |
| 07:20 | <rkirsling> | (that V8 jumped from 89% to 96% in one day also seems quite dubious...the test count didn't even change on that day) |
| 07:27 | <rkirsling> | (hmm, I mean it is possible that a bunch work to perfect the implementation of classes was landed all at once, I guess? but if so, damn, I tip my hat) |
| 07:29 | <rkirsling> | anyway the JSC situation is almost certainly due to our ICU upgrade but I have no idea what test262 infra is doing wrong there |
| 14:19 | <devsnek> | https://test262.report/browse/language/module-code/namespace/internals/set.js |
| 14:19 | <devsnek> | everything is failing this test |
| 14:20 | <devsnek> | probably because it expects the assignments to throw |
| 14:25 | <devsnek> | oh wait its a module the assignments would throw |
| 14:27 | <devsnek> | weird it passes locally |
| 14:38 | <devsnek> | doesn't pass when using test262-harness https://gc.gy/54234503.png |
| 14:42 | <devsnek> | hah https://github.com/tc39/test262/pull/2574/files |
| 15:50 | <gsathya> | rkirsling, v8 shipped private methods/accessors last week |
| 16:14 | <devsnek> | rkirsling: all the strict tests are failing, must have something to do with how they're being concatenated or something |
| 16:52 | <rkirsling> | gsathya: oho |
| 17:55 | <shu> | really, private methods and accessors is 7% of all test262? |
| 17:57 | <devsnek> | it's the main reason i'm in the 80s instead of the 90s in test262.report |
| 17:58 | <Bakkot> | my guess is that a lot of those tests are generated, so there's more of them. test generation is relatively new to test262 so the generated tests tend to correspond to new features, which means new features have more tests. |
| 17:58 | <Bakkot> | but that's just a guess. |
| 17:58 | <rkirsling> | probably true |
| 17:59 | <rkirsling> | it is a pervasive feature |
| 17:59 | <rkirsling> | and yet 7% is such an enormous figure here |
| 17:59 | <rkirsling> | I remain floored |
| 22:28 | <leobalter> | shu wrt test262 on private methods and accessors: too much for syntax variation and too many details to cover |
| 22:30 | <leobalter> | also, one thing that is distributed in many Test262 parts: destructuring assignment |
| 22:30 | <leobalter> | and binding |
| 22:30 | <leobalter> | you'll get a lot of that in any function form. |
| 22:31 | <rkirsling> | leobalter: you saw about the test262.report infra choking on JSC though? |
| 22:32 | <rkirsling> | not sure what ICU assumptions are being made but it must be related |
| 22:33 | <devsnek> | all the strict tests are failing |
| 22:33 | <leobalter> | rkirsling I have no more access to the test262.report repo |
| 22:33 | <rkirsling> | oh ack |
| 22:33 | <rkirsling> | I guess rwaldron then |
| 22:33 | <leobalter> | and I can't even say what's going on with the JSC tests. There are CI reports for each one |
| 22:34 | <leobalter> | yes, and I'm not sure if IRC would be the best way to reach out to Rick. |
| 22:34 | <leobalter> | I'd recommend the https://github.com/bocoup/test262-report-issue-tracker |
| 22:34 | <rkirsling> | can do |
| 22:34 | <devsnek> | it would be cool if the error messages were made public |
| 22:35 | <devsnek> | rkirsling: https://github.com/tc39/test262/pull/2574#issuecomment-612248008 |
| 22:35 | <rkirsling> | ohh wow |
| 22:35 | <rkirsling> | it wasn't ICU after all |
| 22:39 | <rkirsling> | so it's a bug in https://github.com/bterlson/test262-harness then? |
| 22:42 | <leobalter> | rkirsling it might be? let me update everything here |
| 22:44 | <leobalter> | wow, everything is wrong in my env, lol. |
| 22:44 | <leobalter> | 404 on installing JSC |
| 22:46 | <rkirsling> | :oh_no: |
| 22:47 | <leobalter> | ok, I had a wrong version of jsvu and my PATH was messed up linking to it rather than the new version |
| 22:47 | <devsnek> | 404 webkit not found |
| 22:52 | <leobalter> | rkirsling confirmed, it's test262-harness |
| 22:52 | <leobalter> | probably eshost, but they are in the same family |
| 22:53 | <ljharb> | i looked at the test262-harness code, the error message implies it's something about a string receiver being turned into an object in sloppy mode, but the code path doesn't seem related at all |
| 22:53 | <leobalter> | "Expected no error, got TypeError: Attempted to assign to readonly property" |
| 22:53 | <ljharb> | node's `path.relative` is throwing saying it needs a string, but `argv._` should only ever contain strings |
| 22:54 | <ljharb> | (referring to the stack trace in the github comment linked earlier) |
| 22:54 | <leobalter> | it seems like the previous option for JSC's strict mode is now for some sort of Frozen Realms? |
| 22:55 | <ljharb> | what is "jsc's strict mode" and why would test262 be invoking it? |
| 22:55 | <ljharb> | eg node's strict mode, via v8, breaks tons of things and shouldn't ever be used |
| 22:55 | <ljharb> | (it's not "use strict", it's something wonky) |
| 22:56 | <devsnek> | it's use strict but literally everywhere |
| 22:56 | <devsnek> | i think there was some talk of removing it |
| 22:56 | <rkirsling> | the patch was just enforcing strict mode in all classes: https://bugs.webkit.org/show_bug.cgi?id=205578 |
| 22:56 | <ljharb> | right but that breaks things |
| 22:56 | <ljharb> | sloppy code must be sloppy |
| 22:57 | <devsnek> | class bodies are always strictg |
| 22:57 | <devsnek> | strict |
| 22:57 | <ljharb> | true |
| 22:57 | <rkirsling> | it is not meant to be breaking anything |
| 22:58 | <rkirsling> | in particular, it did not cause any test262 failures |
| 22:58 | <leobalter> | ljharb on what's strict mode: most of engines have a strict mode by default from the cli as an argument |
| 22:59 | <ljharb> | leobalter: sure, but why on earth would we want to use a mode that breaks javascript, which is sloppy by default? |
| 22:59 | <ljharb> | like what's the value |
| 22:59 | <devsnek> | if your intention is to put use strict at the top of every script you run |
| 22:59 | <devsnek> | might as well save some labor and use a flag |
| 22:59 | <leobalter> | sloppy by default != sloppy by req |
| 22:59 | <ljharb> | lots of people tried to use it with node because they thought it'd be good to have everything be in `'use strict'` mode by default, but a) that's not actually what it does, and b) third party code relies on being in sloppy mode at time |
| 23:00 | <ljharb> | *times |
| 23:00 | <devsnek> | i don't like those flags |
| 23:00 | <ljharb> | it'd be reasonable if it meant "do what the strict pragma does by default" but that's not what v8's flag does. |
| 23:00 | <devsnek> | what's it do |
| 23:00 | <ljharb> | so i assume that's not what the strict flag does on other runtimes too |
| 23:00 | <ljharb> | it's been years, i'd have to check. it had subtly different semantics in various places |
| 23:00 | <leobalter> | ljharb back in 2015 (or close) we found syntax tests that were just passing but then we found many valid and useful errors for duplicating test execution both on sloppy and strict mode |
| 23:01 | <leobalter> | IIRC |
| 23:01 | <devsnek> | so then the test would be run again with a strict flag, though nowadays the better way to do that is to concat the struct pragma to the test source |
| 23:01 | <leobalter> | test262 tests everything in both modes unless it's explicitly said otherwise in the metadata |
| 23:01 | <ljharb> | right, what devsnek said. run the test once normally and once with "use strict" prepended |
| 23:02 | <leobalter> | devsnek if we do that, we duplicate literally all the tests |
| 23:02 | <ljharb> | you can do it on the fly |
| 23:02 | <devsnek> | test262-stream does it on the fly |
| 23:02 | <leobalter> | wait |
| 23:02 | <devsnek> | i think all implementer's test runners do it on the fly |
| 23:02 | <devsnek> | or most |
| 23:02 | <leobalter> | I'm confused. Test262-harness uses test262-stream, so it uses the pragma on the fly |
| 23:02 | <ljharb> | (tbh there's no point in keeping generated tests committed/on disk, if the test generation is fast enough) |
| 23:03 | <ljharb> | leobalter: ok then it wouldn't be using jsc's strict mode |
| 23:03 | <devsnek> | yes it does |
| 23:03 | <leobalter> | I worked more on other test262 runners than test262-harness (and family) |
| 23:03 | <leobalter> | so I'm confused rn on what each one does |
| 23:03 | <leobalter> | lol, I believe that WebKit's runner uses the cli |
| 23:04 | <leobalter> | I'm trying to remember. so if test262-stream is already doing it the correct way, I wonder what JSC is actually doing |
| 23:04 | <leobalter> | let me get a streamed file from test262 and run it with JSC directly |
| 23:07 | <devsnek> | oh wow i just found a major slow area in my test262 runner |
| 23:08 | <devsnek> | doubled me test speed with this one easy trick (CPUs hate me) |
| 23:08 | <devsnek> | https://gc.gy/54265119.png |
| 23:10 | <leobalter> | that's pretty neat, devsnek! are you reporting that in an issue? |
| 23:10 | <leobalter> | like, I can use it |
| 23:10 | <leobalter> | oh wait |
| 23:10 | <devsnek> | oh that's my personal runner |
| 23:10 | <leobalter> | that's your runner, right? |
| 23:10 | <leobalter> | oh |
| 23:10 | <devsnek> | not test262-harness |
| 23:10 | <leobalter> | I'd love to do some performance work on test262-harness |
| 23:10 | <rkirsling> | did I just see a use case for logical assignment? :) |
| 23:10 | <devsnek> | yes |
| 23:10 | <leobalter> | I also wonder if I'll ever write a new runner for test262, lol |
| 23:11 | <devsnek> | i was thinking of going and rewriting all the agents in eshost at some point |
| 23:11 | <devsnek> | rkirsling: when i can do logical assignment in node i'll use it :P |
| 23:12 | <rkirsling> | hehehe |
| 23:12 | <rkirsling> | wao, `BigInt()` throws instead of defaulting to 0n, eh |
| 23:12 | <rkirsling> | (not that I'd ever do that I'm just writing tests) |
| 23:13 | <leobalter> | devsnek: `realm.evaluateScript(includeCache[include] || fs.readFileSync(p, 'utf8'), { specifier: p });` |
| 23:13 | <devsnek> | leobalter: that doesn't fill the cache though |
| 23:13 | <leobalter> | oh I see |
| 23:14 | <leobalter> | It was my next question if you actually used that cache. This seems to save a big chunk of time due to io |
| 23:14 | <devsnek> | i run multiple tests inside the same process |
| 23:15 | <devsnek> | so yeah only loading the file once is good |
| 23:15 | <devsnek> | also gotta appreciate this https://gc.gy/54265476.png |
| 23:16 | <leobalter> | well, my child skipped his nap time and now my planned time for open source today is over. |
| 23:23 | <devsnek> | oh boy anonymous names were merged i can unskip so many tests |
