| 12:04 | <canadahonk> | If anyone might find it useful, I just made https://test262.fyi which publishes results of test262 in most engines daily (like wpt.fyi or test262.report but up to date with more engines) (please give feedback/wants, sorry if not for this room) |
| 12:52 | <jmdyck> | @canadahonk: engine262 is checked in the repo's README, but doesn't appear on the website. |
| 13:51 | <canadahonk> | @canadahonk: engine262 is checked in the repo's README, but doesn't appear on the website. |
| 14:36 | <voidhedron> | If anyone might find it useful, I just made https://test262.fyi which publishes results of test262 in most engines daily (like wpt.fyi or test262.report but up to date with more engines) |
| 14:38 | <canadahonk> | not very good UI atm, in some cases you cant even click some engines because the hover animations overlap and shift your hover into a different engine, but definitely interesting info to see |
| 14:39 | <voidhedron> | yea im aware its new no worries, just giving feedback 👍️ |
| 14:39 | <canadahonk> | very welcome to it :) |
| 14:40 | <voidhedron> | its also pretty disorienting how if an engine fails a test it completely disappears from the bars list |
| 14:40 | <voidhedron> | well actually thats not really related to failing the test |
| 14:40 | <voidhedron> | in general how the engines shift each other around unless theyre all 100% |
| 14:41 | <canadahonk> | yeah I might change it so each one is in a container for 100% |
| 14:41 | <voidhedron> | it'd be cleaner if each engine's % bar column had a fixed start |
| 14:41 | <canadahonk> | yeah like that |
| 14:42 | <canadahonk> | also if anyone sees any (niche/obscure) engines missing I am very open to adding ;) |
| 14:44 | <canadahonk> | @canadahonk: engine262 is checked in the repo's README, but doesn't appear on the website. |
| 14:44 | <voidhedron> | it could be interesting to add runs for the versions of engines currently used in the latest versions of backend runtimes |
| 14:45 | <canadahonk> | I had that idea too, like V8 nightly (now) and V8 stable (chrome/node/deno/etc) |
| 14:45 | <voidhedron> | ie the V8 version of Node LTS and Latest, Deno's current V8 version, Bun's current JSC version, etc, yeah |
| 14:45 | <canadahonk> | yeah planning that for sure |
| 14:45 | <canadahonk> | also going to do a graph like WPT showing "interop" between chosen maybe |
| 14:47 | <voidhedron> | something like compare engines A and B and show tests where they differ so you can see what to avoid/account for between the two could be useful |
| 14:49 | <canadahonk> | yeah going to do diffs for sure |
| 14:49 | <voidhedron> | a small UI detail too is you could have the test filenames actually link to the files themselves on the tc39/test262 repository (with the exact commit used for the test results on the side at the time of course) |
| 14:49 | <canadahonk> | good idea, didn't think of that |
| 14:49 | <nicolo-ribaudo> | canadahonk We currently use test262 to test Babel+core-js, running them in a very old version of Node.js (0.8 iirc). It's not really an engine, but would you be open to also adding that? 👀 |
| 14:49 | <canadahonk> | for sure! |
| 14:50 | <nicolo-ribaudo> | Expect a PR at some point then :) |
| 14:50 | <canadahonk> | I'm also happy to do if you want btw |
| 14:50 | <canadahonk> | code is mostly bash scripts and forks of utils for testing so a bit messy to add a new one |
| 14:51 | <nicolo-ribaudo> | I will open an issue to talk about it, so that I can at least give info regarding our current test262 infra |
| 14:51 | <canadahonk> | thanks! |
| 14:52 | <voidhedron> | should perhaps code in some way of having static engines that only get tested once and then never again unless manually, so that you can add older engines like that which are unlikely to have their test results change anytime soon but without the extra cost of rerunning them every day or so |
| 14:53 | <canadahonk> | well test262 and Babel/core-js/etc would be updated but yeah I plan on main ones updating daily, some weekly |
| 14:53 | <canadahonk> | eg V8 stable weekly, V8 canary daily |
| 14:54 | <canadahonk> | (although there isn't that much cost as all engines are tested in parallel) |
| 14:54 | <nicolo-ribaudo> | Btw to save on running time — Babel runs test262 on every commit and we generate tap output, maybe it makes sense to just use that instead of running the tests again |
| 14:54 | <canadahonk> | oh neat |
| 14:54 | <voidhedron> | pretty much what i was typing out yeah |
| 14:55 | <voidhedron> | it would actually be smarter to just listen for updates on engines and test262 to trigger re-runs |
| 14:56 | <canadahonk> | I want most engines to use their canary/nightly/CI/etc counterparts so |
| 14:56 | <voidhedron> | well some of the engines may not be released on a platform that can be subscribed to like a webhook and might need to be polled on an interval for updates still, but still a lot cheaper than blindly doing re-runs of the same version |
| 14:56 | <canadahonk> | (hermes took a lot of effort to get working lol) |
| 14:57 | <voidhedron> | I want most engines to use their canary/nightly/CI/etc counterparts so |
| 14:58 | <voidhedron> | well for the engines on github at least |
| 14:58 | <voidhedron> | like v8 |
| 15:00 | <canadahonk> | unfortunately you need ownership of the repo to make webhooks on other repos so won't be that easy probably |
| 15:01 | <canadahonk> | Btw to save on running time — Babel runs test262 on every commit and we generate tap output, maybe it makes sense to just use that instead of running the tests again |