| 00:18 | <Domenic> | My experience in other open source projects is that it's easier and nicer for contributors if I just have my AI fix the review issues, instead of having my AI talk to them (or often to their AI) and do a round of iteration. Probably the right balance here would be to have the reviewer auto-push any AI-generated editorial fixes, and leave substantive reviews as comments. |
| 00:18 | <Domenic> | Oh I didn't know about this repo, I would have self-run it on my recent PRs if I had known. |
| 00:19 | <zcorpan> | It's pretty new |
| 00:24 | <Domenic> | https://github.com/whatwg/spec-ai/blob/main/skills/html-spec-review/html-style-guide.md is so cool |
| 00:25 | <zcorpan> | Domenic: agreed on editorial fixes. Substantive comments can still be many issues. Unclear if it's better to let the AI push fixes for those too (unless something clearly needs a design decision), or have a strict editorial/substantive split where the latter is posted as a comment (with proposed fix or fixes). |
| 00:58 | <zcorpan> | After testing the flow in #9 I think defects with 1 clear fix should be fixed directly, and a comment summarizing what changed and why (with short shas). At least for local, where the reviewer can edit locally before pushing and posting. Questions with options can have fixes on other branches. |
| 01:35 | <asuth> | zcorpan: I'm definitely interested in improving what searchfox is doing there and it's great if it can be helpful. I can definitely crank up searchfox proactively importing pull requests/etc. and could support a web hook or other mechanism if whatever robot is already doing stuff wanted to be able to confidently generate searchfox links that definitely exist and are already populated (or will be imminently populated if it doesn't want to wait for the async ingestion). Searchfox actually is already doing some patch stack-aware stuff under the hood, but it's not really materialized in the rendered view, although at least overlapping PRs will have the redunant bits suppressed. (That is, https://github.com/whatwg/fetch/pull/1941 contains a push from https://github.com/whatwg/fetch/pull/1940 in it and searchfox then tries to supress showing the duplicated content that's in 1941.) |
| 01:37 | <asuth> | The review comments situation looks like it is a little tricky in the sense that it sounds like searchfox or any other tool on a different origin can't just generate a POST at github with a pre-computed "please have the user confirm they want to make the following review comments", but instead need something like:
|
| 01:39 | <asuth> | I'm definitely interested in your thoughts on the review aspect in terms of what you'd be comfortable with or you think people would be comfortable with. Also, I'm not saying any infra needs to directly involve searchfox. Or it could involve searchfox but searchfox wouldn't necessarily need to be the front-end. It would be fine if searchfox could help some other tooling leverage searchfox's token-centric blame history (which is the main potentially non-trivial source of pre-computed data searchfox has that might be expensive to run elsewhere and needn't run elsewhere). |