07:19
<MikeSmith>
https://bugs.webkit.org/show_bug.cgi?id=175115
07:19
<MikeSmith>
https://lists.webkit.org/pipermail/webkit-dev/2017-August/029386.html
07:25
<Ms2ger>
Keen eye
07:40
<MikeSmith>
if you don’t know where you’re going, any road will take you there
09:56
<JakeA>
annevk: how do I terminate an "in parallel" series of steps? I'm within "Run these steps until fetch is terminated" so "abort these steps" is kinda ambiguous. "Abort these in-parallel steps"?
10:14
<annevk>
JakeA: not sure, might be good to raise against Infra, I think your proposal is reasonable though as a first go
10:14
<JakeA>
ta!
11:03
<JakeA>
annevk: "Run these steps, breaking if fetch is terminated" – does this sound less loopy?
11:04
<JakeA>
I think it was the "until" that made it sound like it might loop
11:05
<annevk>
JakeA: unfortunately, break is reserved for loops atm, https://infra.spec.whatwg.org/#algorithm-iteration
11:05
<JakeA>
hah
11:10
<JakeA>
annevk: I know you don't like substeps, but "Run these substeps, but abort the substeps at any point if fetch is terminated:"
11:15
<annevk>
JakeA: s/at any point// and an issue against Infra to figure out if this is one time we need them?
11:16
<JakeA>
wfm
11:20
<JakeA>
https://github.com/whatwg/infra/issues/143
11:26
<tobie>
JakeA: fwiw, the issue is missing your "in parallel" fetch use case described here. Think it's worth adding it for context (I understood the issue all wrong until I came back to read about it here).
12:05
<JakeA>
tobie: I think that's a different issue, and infra doesn't seem to want anything to do with in parallel https://github.com/whatwg/infra/issues/120
12:41
<tobie>
JakeA: oh. Because without that it just sounds like "if x or y or z, then..." would do.
12:43
<gsnedders>
so what specs outside of the WHATWG require tests alongside all normative changes now?
12:44
<annevk>
gsnedders: foolip_ would know, if around
12:44
<annevk>
gsnedders: I think there was one example at least
12:45
<gsnedders>
Payment Requests does, and I'm sure there was something else that rbyers had pushed it on ages ago
12:46
<annevk>
gsnedders: Web Performance (as a WG?) https://github.com/w3c/web-performance/blob/gh-pages/CONTRIBUTING.md#test-driven
12:46
<annevk>
gsnedders: via foolip_'s Twitter
12:46
<tobie>
I'm sort of informally moving towards that with Web IDL
12:46
<tobie>
It's a bit of a more complex problem, though.
12:47
<gsnedders>
ah, Web Perf.
12:47
<gsnedders>
annevk: do we have any examples of very early-stage specs that are changing massively that have done it?
12:48
<tobie>
gsnedders: ^ that was a response to your question
12:48
<gsnedders>
that wasn't meant to be really at annevk
12:48
<gsnedders>
I don't know why I did that :)
12:50
<annevk>
it's more of an idea if it still changes a lot
12:51
<gsnedders>
because at least here there seems to be some concern about time spent writing tests that nobody ever runs
12:52
<annevk>
gsnedders: I generally see it as two phases where phase one can be anything from a PR or separate document; phase one is the idea that you flush out, get commitments on, etc. then it's time for phase 2 where it lands in a proper spec inc tests
12:52
<annevk>
gsnedders: sometimes tests come first, as with cancelable fetch
12:53
<gsnedders>
annevk: so, e.g., one question is when you move to a proper spec with tests for something like CSS Grid
12:54
<annevk>
gsnedders: when two UAs are in principle happy to impl and ship it
12:55
<annevk>
gsnedders: in principle CSS grid is just a very large change, so https://whatwg.org/working-mode#changes applies
12:57
<gsnedders>
annevk: tbf for years simply applying that literally would be "you fail this new test because you still don't support grid at all" :)
12:58
<annevk>
gsnedders: that sounds like the implementer commitments were not as serious as assumed?
12:58
<annevk>
gsnedders: also note that writing decent tests will improve the change/addition tremendously as well due to lots of bugs being uncovered
12:59
<gsnedders>
annevk: I mean I guess there's also the question of what an implementer commitment is; whether it's "we want to implement some similar Grid system in CSS" or "we want to implement this exact spec"
12:59
<gsnedders>
which the former makes sense in the incubation case more than the latter
12:59
<annevk>
gsnedders: it's the latter
12:59
<annevk>
gsnedders: the former is whether you start working on phase 1 at all
13:00
<gsnedders>
so one question is when you start writing tests for every change, because you probably want to do that once you have any implementation, rather than waiting until you have two underway
13:01
<gsnedders>
annevk: see also https://www.w3.org/2017/08/03-css-minutes.html#item06
13:01
<gsnedders>
annevk: though a lot of that boils down to "nobody reviews tests"
13:02
<gsnedders>
and to a lesser degree "editors don't have time", but mostly "we'd never get any spec changes landed because nobody reviews tests"
13:02
<Ms2ger>
In other words, "we don't care"
13:03
<annevk>
So then you don't change specs for a while and work on interop first
13:03
<annevk>
I don't think many web developers would be displeased
13:03
<JakeA>
gsnedders: service worker
13:04
<jgraham>
gsnedders: You bake reviewing tests into how you write specs
13:04
<gsnedders>
jgraham: and how do you get anyone to review is the problem?
13:04
<jgraham>
Like you do for code. Pretty sure that "we don't write tests because no one reviews them" wouldn't wash in a software project
13:04
<JakeA>
tobie: It's more of a shortcut between putting "if <condition> then break" before every step of a set of steps
13:04
<jgraham>
gsnedders: You tell them they can't land spec changes without review
13:04
<jgraham>
of accompanying tests
13:04
<JakeA>
tobie: "shortcut for putting" I mean
13:05
<jgraham>
Like like software changes
13:05
<gsnedders>
jgraham: and how do you convince the WG of this?
13:05
<annevk>
gsnedders: if you don't take testing seriously you basically end up revisiting a whole bunch of things down the line
13:05
<jgraham>
gsnedders: Ask them if they're allowed to make changes to their code projects without tests
13:05
<botie>
will do
13:05
<jgraham>
See if they can justify the difference
13:06
<gsnedders>
jgraham: you've seen how many changes TabAtkins makes to BS without tests, right? ;
13:06
<gsnedders>
* ;P
13:06
<jgraham>
Sure, but I bet he wouldn't be allowed to ship chromium changes without tests
13:07
<TabAtkins>
You say, as I'm currently reviewing the delta a new change would make to tests to make sure rebasing them is safe. ^_^
13:07
<TabAtkins>
(I definitely don't test *enough*, tho.)
13:07
<gsnedders>
TabAtkins: were you here during the earlier discussion, BTW
13:07
<jgraham>
So the question is whether the spec is more like Chromium where bugs get shipped to millions of people, or bs where they get shipped to a few close collaborators
13:07
<TabAtkins>
Earlier CSS discussion, or earlier #whatwg discussion?
13:07
<jgraham>
(the answer is the former)
13:07
<gsnedders>
TabAtkins: earlier CSS
13:08
<gsnedders>
jgraham: the alternative answer is the latter because the spec gets shipped to vendors who are in the WG
13:08
<gsnedders>
that argument is that the spec doesn't affect users, impls do
13:08
<jgraham>
gsnedders: And how well has that approach worked for not shipping bugs to millions of users?
13:09
<gsnedders>
TabAtkins: mostly given you were mentioned at several points during it per the minutes
13:09
<jgraham>
The current state of the web suggests that interop approaches we have taken to date are not good enough
13:09
<gsnedders>
TabAtkins: oh you did say a few things
13:09
<TabAtkins>
Yes, I was there. ^_^
13:09
<annevk>
There's other reasons too, but I'm not sure how much I want to rehash everything
13:09
<annevk>
E.g., if a spec changes and there's no test changes, no implementer might notice
13:09
<annevk>
This happens a lot
13:10
<jgraham>
And then we ship bugs to millions of people…
13:10
<annevk>
Changing tests and filing bugs gets everyone in line
13:10
<TabAtkins>
But yeah, "stop working on new features and just improve quality" always sounds good, but is very rarely actually possible.
13:10
<TabAtkins>
Which sucks, but hey.
13:10
<annevk>
It's a little bit more work, but due to the due diligence it'll also last much longer since it's actually cemented
13:10
<jgraham>
FWIW I haven't said "stop working on new stuff"
13:11
<gsnedders>
jgraham: annevk did on the basis of being unable to land new spec changes given no test review
13:11
<annevk>
Yeah, I was assuming that if the new thing was important enough the review would come
13:11
<annevk>
Which it will, since the tests would save implementers time
13:11
<jgraham>
gsnedders: Right, but that's hyperbole, really. In practice the WG has to find a way to ship new stuff
13:11
<jgraham>
Otherwise it stops existing
13:11
<annevk>
But with CSS Grid it might not, if implementers are happy to wait years
13:12
<gsnedders>
annevk: is that really the case, though, given we have hundreds of open PRs on wpt?
13:12
<gsnedders>
or maybe that just implies they're all tests for unimportant things, but I'm unconvinced that's true
13:12
<annevk>
gsnedders: in my experience, yes
13:13
<annevk>
gsnedders: I went through those old PRs and they're not really in a great state
13:13
<annevk>
gsnedders: and if you compare the backlog to what gets merged it's tiny
13:13
<Ms2ger>
gsnedders, importance relative to the amount of work to get the PR in a landable state?
13:14
<annevk>
gsnedders: backlog will always grow a bit, since maintenance is not valued as much, but that doesn't mean there's not a lot that gets done
13:15
<jgraham>
gsnedders: Per GH we are landing 40-150 commits a week, every week at the moment
13:19
<jgraham>
2300 commits so far this year compared to 2600 all last year. The idea that nothing gets reviewed is frankly bollocks.
13:20
<foolip_>
annevk, gsnedders: search for "Existing policies at beginning of Q3" in https://docs.google.com/document/d/1aRpnNQ7Zz_-N9ngdcjQXNjlE1NblppN7lCQwGdxRLlc/edit?usp=sharing for the non-WHATWG testing policies
13:21
<gsnedders>
jgraham: nothing gets reviewed for CSS tests, that's not bollocks.
13:21
<foolip_>
gsnedders: is the Payment Requests policy documented?
13:21
<gsnedders>
foolip_: https://github.com/w3c/payment-request/blob/gh-pages/test-plan.md
13:22
<foolip_>
gsnedders: sweet! thanks MikeSmith!
13:23
<gsnedders>
jgraham: is there any way to search for only merged PRs?
13:23
<jgraham>
gsnedders: 30 open PRs with label wg-css and 448 closed.
13:23
<gsnedders>
jgraham: tbf, many haven't been reviewed in wpt
13:24
<gsnedders>
jgraham: 153 merged excluding those labelled with chromium-export or authored by you
13:24
<jgraham>
gsnedders: Anyway this is a silly conversation
13:24
<jgraham>
The CSS WG is observing that they aren't doing their job, and instead of working out how to fix that are trying to work out how to keep not doing it without any consequence
13:25
<jgraham>
Except making the web worse
13:25
<gsnedders>
that's not an argument that will change the WG's view, however
13:25
<jgraham>
I am saddened that the working gorup doesn't consider interop to be a compelling argument
13:26
<annevk>
gsnedders: if nobody in the CSS WG takes responsibility to review, you won't get reviews, no...
13:26
<jgraham>
Maybe they should consider going and working on something proprietary instead
13:26
<annevk>
gsnedders: it's not somebody else's problem though, it's the CSS WG's problem
13:26
<annevk>
gsnedders: it's the quality of their work
13:27
<jgraham>
Maybe Google could use some people on Flutter, for example
13:27
<annevk>
gsnedders: and it won't be solved by trying to find QA folks, editors actually do need to step up
13:28
<annevk>
gsnedders: and if there's not enough editors, they should figure out how to empower more folks to do that kind of thing
13:29
<gsnedders>
annevk: it's not clear to me they are any less empowered than in other groups that have more editors, though
13:30
<gsnedders>
there definitely seem to be cultural differences within the vendors between dom teams and layout/style teams.
13:31
<foolip_>
jgraham: how much of a credibility problem do you think it is with all of these open PRs? if it's plausible that it reduces vendors' commitment to wpt, then including some kind of maintainance in the ecosystem infra rotation would be an option, or setting up a shared think like was discussed in MTV, idea by patket I think?
13:31
<tobie>
foolip_: Do those open PRs have assigned reviewers?
13:32
<jgraham>
foolip_: I think it's a credibility problem if there are people looking for excuses
13:32
<jgraham>
foolip_: I have no confidence that an infra rotation system would actually work
13:32
<tobie>
jgraham: true, but if it's easy to fix, it might make finding excuses more difficult
13:33
<foolip_>
tobie: I assume it's a mixed bag, but most of them could presumbably be mapped to a group of people via OWNERS files
13:34
<jgraham>
So some PRs are legitimate tests but need considerable work to land. Lots of that is old stuff. Some are tests with feedback where the PR author never did the work. Some the spec is unclear. Some could probably land if someone reviewed them
13:34
<jgraham>
But getting people to review random stuff in wpt when it isn't part of their job is really hard
13:34
<tobie>
foolip_: combine that with a bot that closes the PR after a certain amount of non-activity from the reviewers, unless the reviewers tag them with a special tag and the problem should be largely solved.
13:35
<jgraham>
That's why out most effective strategies for getting contributions have all been of the form "make it part of people's job"
13:35
<annevk>
gsnedders: as far as I could tell dbaron supported tests there, but maybe other UAs have more differing ideas
13:35
<Ms2ger>
If someone has budget they'd like to throw at it...
13:35
<jgraham>
e.g. upstreaming from vendor repos where review is management enforced
13:35
<jgraham>
or making it part of the process to land spec changes
13:36
<dbaron>
?
13:36
<annevk>
dbaron: sorry, didn't mean to actually drag you into this
13:36
<jgraham>
tobie: Certainly we could just close all the old PRs. That seems like a pretty pyrrhic victory.
13:36
<annevk>
dbaron: we're discussing reasons to do tests and how to convince the CSS WG and https://www.w3.org/2017/08/03-css-minutes.html#item06 came up
13:37
<annevk>
jgraham: I agree with you that the real problem is people looking for excuses
13:37
<tobie>
jgraham: it might be a rational thing to do, if 100 poor old PRs prevent you from getting 10 good new ones a week…
13:37
<annevk>
jgraham: it's like saying w3c/html is higher quality than whatwg/html because it has less reported issues
13:37
<foolip>
If we have a mostly complete spec->wpt directory mapping, via links like at the top of https://dom.spec.whatwg.org/, and OWNERS files, then I think at least all old PRs could be put in a bucket. But it's only if the spec+test people care about maintaining the test suite well that the bucket will get tended of course.
13:37
<annevk>
jgraham: complete nonsense
13:37
<tobie>
jgraham: (or more realistically a month)
13:38
<dbaron>
I think the average Gecko developer working on a feature probably knows to check web-platform-tests for tests, but probably doesn't know to search through the PRs for more unlanded tests...
13:38
<Ms2ger>
tobie, it's not clear that that's something that happens, though
13:38
<foolip>
Like jgraham, I don't think a rotation to look at *all* PRs would make sense.
13:38
<dbaron>
I don't think that's "looking for excuses". Maybe it's an education problem or maybe it's a process problem.
13:38
<annevk>
dbaron: so maybe you don't know this, but each WHATWG specs links to tests and open test issues using the WPT labeling system
13:38
<dbaron>
I didn't
13:38
<annevk>
dbaron: e.g., https://github.com/w3c/web-platform-tests/labels/fetch captures all ongoing work for fetch
13:39
<dbaron>
but there are more specs than whatwg specs
13:39
<annevk>
dbaron: and is linked from the top of the Fetch Standard
13:39
<annevk>
dbaron: the hope is that folks copy our example since it seems pretty good (though open to better ideas)
13:39
<dbaron>
is it clear that "open test issues" might include pull requests with 75% of the tests?
13:40
<annevk>
dbaron: about as clear as open spec issues potentially including security bugs and such
13:40
<jgraham>
dbaron: I don't really expect people to review random open PRs, although they can certainly sign up to be notified using the OWNERS file mechanism
13:40
<jgraham>
I epxect people working on implementations to write tests that get upstreamed, and for those to get internal review
13:41
<dbaron>
does wpt have labels for indiviual css specs?
13:41
<jgraham>
And I expect WGs to have a process that ensures relevant tests are written and reviewed as specs are written
13:41
<tobie>
dbaron: yes
13:41
<jgraham>
CSSWG seems to be lacking the latter, because any such process would of course involve ensuring the tests have a credible means of landing
13:42
<tobie>
dbaron: e.g.: https://github.com/w3c/web-platform-tests/labels/css-position-3
13:42
<jgraham>
"hoping that a random person on the internet does it" is not very credible
13:42
<dbaron>
ok, but then there's https://github.com/w3c/web-platform-tests/labels/css-fonts which is unleveled
13:43
<tobie>
dbaron: labels are given depending on which directories the tests go in.
13:44
<tobie>
dbaron: if you want this to be labelled differently, just rename the directory.
13:44
<Ms2ger>
dbaron, there's a bunch of css tests outside the top-level css/ directory that are unversioned
13:45
<tobie>
Ms2ger, dbaron: wasn't aware of this. These also don't get tagged with the csswg label it seems
13:46
<Ms2ger>
They predate the merge
13:46
<tobie>
dbaron: for ref all css-wg related stuff should be here: https://github.com/w3c/web-platform-tests/labels/wg-css
13:46
<jgraham>
Well some of them do
13:46
<Ms2ger>
It's probably worth merging them into the relevant css/css-foo-N dirs
13:46
<Ms2ger>
Or going the other way around
13:46
<tobie>
Ms2ger: agreed.
13:48
<jgraham>
css-fonts seems to be rather new, and based on moving css-fonts-4 stuff out of the css-fonts directory
13:48
<jgraham>
*out of the css/css-fonts-3 directory
13:49
<tobie>
well, if these want an wg-css label, we'll need to fix the bot
13:51
<annevk>
dbaron: presumably you don't want the tests to be leveled? You'd end up maintaining many more tests than if you just tracked whatever everyone is implementing
13:51
<dbaron>
yeah, dunno, but lots of the css dirs are leveled
13:55
<gsnedders>
dbaron:
13:56
<gsnedders>
annevk: all the UAs agree, but nobody's willing to do any work
13:57
<annevk>
gsnedders: film at 11?
13:58
<jgraham>
I mean I might not discourage people who are removing the leveling by re-moving tests back to a levelled directory
14:01
<gsnedders>
dbaron: so the mix of levelled and unlevelled is essentially just down to what we had agreement to do in the WG
14:01
<gsnedders>
dbaron: all tests for existing specs remained in their levelled dirs; all tests for new tests are unlevelled per standard wpt policy
14:02
<gsnedders>
I would like to move it so everything is unlevelled (but that gets a bit messy given the modularisation of CSS 2), but I'd expect massive pushback from the WG if I tried
14:03
<gsnedders>
and I don't currently think it's worth spending the political capital on
14:03
<Ms2ger>
*for new specs
14:04
<Ms2ger>
+ cssom
14:04
<Ms2ger>
Anyway, as long as there's a unique directory for each spec... *shrug*
14:04
<gsnedders>
yeah, I think there's a few level 1 specs
14:05
<gsnedders>
Ms2ger: the current pain point is the mixture across css/CSS2 and various css/*-3 directories
14:05
<gsnedders>
and there's no clear way to come up with the complete testsuite for the level 3 spec
14:05
<Ms2ger>
Yeah
14:05
<Ms2ger>
Ceterum censeo css2 delenda est
14:06
<gsnedders>
:)
14:06
<gsnedders>
Ms2ger: well, s/C/c/
14:06
<gsnedders>
(why am I correcting Latin now?)
14:07
<Ms2ger>
s/est/esse/, apparently
14:07
Ms2ger
merged two versions
14:09
<gsnedders>
uh, yeah, that would also make more sense
15:25
<Domenic>
Reading the above was depressing.
15:25
<jgraham>
Which part?
15:25
<botie>
i think Which part is old in this case?
15:26
<Domenic>
Just the fact that there are still people pushing back against tests with every spec change. Especially if those people are editors.
15:27
<nox>
How surprising.
16:10
<TabAtkins>
In a theoretical world where I wasn't already 200% committed on specs, *plus* another big chunk of commitment on Bikeshed, I'd love to do tests-with-every-commit.
16:11
<TabAtkins>
We theoretically are doing this now with the specs Shane and I edit in Houdini, tho I did a large rewrite of Typed OM over several weeks and didn't slowly change tests along with it; I need to catch myself back up with the now-final state.
16:23
<annevk>
TabAtkins: I think you'll find that you might write less, but what you end up writing ends up getting shipped and ends up being higher quality
16:24
<annevk>
TabAtkins: and you should probably look into trying to scale the editing somehow if you're 200% committed since that's not ok
16:24
<TabAtkins>
I agree, but, uh, doesn't seem likely to be fixed any time soon.
16:24
<annevk>
TabAtkins: for instance, delegate by getting implementers to write PRs for new features (and the tests) that you then review
16:25
<TabAtkins>
Implementors aren't particularly good at, or enjoy, writing spec features.
16:25
<annevk>
TabAtkins: I think if you're willing to change your priorities it should be possible
16:26
<Domenic>
+1
16:26
<annevk>
TabAtkins: if they want to ship something you can at least tell them they have to write tests because you can't merge the spec PR otherwise
16:26
<annevk>
TabAtkins: but I also think that if you ask you'll find that some people might want to try their hand at writing stuff
16:27
<TabAtkins>
I'm doing that for several Houdini features.
16:27
<TabAtkins>
Bringing impls up for simpler issues does indeed work.
17:04
<wanderview>
jugglinmike: ping
17:18
<wanderview>
jugglinmike: I just made a PR: https://github.com/w3c/web-platform-tests/pull/6745
17:40
<smaug>
there was some way to run wpt on any browser
17:40
<smaug>
what would be the url?
17:40
smaug
would like to run workers/data-url.html
17:52
<jgraham>
smaug: http://w3c-test.org/workers/data-url.html perhaps
17:53
<jgraham>
Or you can run it from m-c with ./mcah wpt --product chrome|edge workers/data-url.html
17:53
<smaug>
ah, that too
19:36
<Domenic>
Is there any way to unsubscribe from a Mozilla bug that I reported?
19:36
<Domenic>
I really don't need to see all the code review comments and patches
19:42
<wanderview>
Domenic: yes
19:42
<wanderview>
Domenic: there is a "never email me about this bug again" somewhere on the page
19:43
<wanderview>
Domenic: at the bottom of the page immediately above the submit button
19:44
<Domenic>
Hmm all I see there is "Add me to CC list (follow this bug)" (which is unchecked)
19:45
<Domenic>
Oh I needed to hit "edit bug" first O_o
19:45
<Domenic>
Thanks!
19:45
<wanderview>
Domenic: sorry... bugzilla is a twisty mazy of corridors and you are likely to be eaten by a grue
19:46
<gsnedders>
s/likely/certain/
21:14
<Domenic>
Any historians may enjoy helping with https://github.com/whatwg/html/issues/2891
21:14
<Domenic>
Especially media folks, it seems
21:27
MikeSmith
looks