| 00:02 | <bakkot> | I am very confused by this |
| 00:03 | <bakkot> | what is the CoC if not the thing which sets the rules for what behavior is reasonable in our interactions with each other |
| 00:31 | <Chris de Almeida> | I can't speak for Jordan, but I would offer that the use of tools/methods does not directly violate the CoC tenets |
| 00:41 | <Chris de Almeida> | the issue is perhaps not so much that an LLM was used -- it was that low-quality, unreviewed nonsense was sent for consideration and ended up wasting peoples' time -- which is not respectful, and arguably, a violation of the CoC. but it's not that it was inherently the use of an LLM that was a violation, but how it was used |
| 00:44 | <bakkot> | I don't really care whether they reviewed the output or not, and whether it's "low quality" isn't something random contributors can effectively judge - I assume that they would not have posted it if they thought it was low quality |
| 00:45 | <bakkot> | the problem is that I do not care to spend my time reading or responding to LLM outputs |
| 00:45 | <bakkot> | (not including "outputs" which are just translation or whatever) |
| 00:45 | <bakkot> | so to me the problem is in fact use of the tool to author comments |
| 00:46 | <bakkot> | and to the extent that this is not currently captured by the CoC, I know, and that's why I am proposing to update the CoC |
| 00:51 | <Chris de Almeida> | what I am trying to say is that it's not necessarily always to be considered misuse. if someone uses an LLM to help them write something, and they believe it's accurate and useful, it's difficult for me to interpret that as a violation of the code of conduct. I agree with you on not wanting to read "LLM outputs", but it's tricky to totally define what that is, and to nail down what a prohibitive clause would look like |
| 00:51 | <Chris de Almeida> | that said, unless something has changed, you are proposing only that a disclaimer be attached to such comments, stating that an LLM was used to help generate it, in part, or in whole. is that accurate? |
| 00:56 | <bakkot> | My current proposal is:
|
| 00:57 | <bakkot> | I would also be ok with saying that you must disclose any use of LLMs, but I'd just immediately ignore anything where they said they used the LLM to write the post, so I think this is better |
| 00:57 | <bakkot> | I agree we don't want to forbid "use of LLMs" in general |
| 01:04 | <Chris de Almeida> | I think I agree. the proposed text could use a little wordsmithing but LGTM. still, I have doubts of whether that is most suited for CoC vs contributing guide / how-we-work |
| 01:23 | <bakkot> | this seems like a pretty central case of what the CoC is for, to me? |
| 01:23 | <bakkot> | it's defining what kinds of conduct are acceptable in any TC39 space |
| 01:23 | <bakkot> | that's what the CoC is for |
| 03:00 | <Chris de Almeida> | the CoC is not the place for banning tools; it's for establishing expectations for interpersonal conduct |
| 03:08 | <bakkot> | This seems like an expectation for interpersonal conduct to me |
| 03:14 | <bakkot> | The goal isn't to ban any particular tool - I don't want to read the output of Markov chains either |
| 03:15 | <bakkot> | The goal is to establish the expectation that the comments you post were written by you, not by some other system |
| 03:17 | <Chris de Almeida> | I agree the CoC is for defining acceptable conduct. it seems we disagree on what a conduct issue is. I suggest that the use of a tool, like an LLM, on its own, does not equate to behavior the CoC is meant to guide our behavior towards each other, rather than to prescribe how we may or may not write |
| 03:19 | <bakkot> | How we write to each other is part of how we behave towards each other |
| 03:19 | <Chris de Almeida> | I completely agree |
| 03:19 | <bakkot> | Anyway, I won't be at this meeting but I can put something on the agenda for the following one, I guess? I didn't think this would be controversial |
| 03:21 | <bakkot> | Needs to go through plenary anyway I suppose |
| 03:21 | <Chris de Almeida> | I don't know that it's controversial. I think we all have the same goal and are generally in agreement. it's the particulars about where such guidance or rules live, is where we may not agree |
| 03:23 | <Chris de Almeida> | your proposed text above was excellent. maybe a little massaging is all I'd ask. but after that, we seem to run into difficulty adding it to the CoC vs elsewhere |
| 03:25 | <bakkot> | Happy to take some wordsmithing now also, wherever it ends up |
| 05:10 | <rkirsling> | is the issue perhaps that this, like spam, is something that wouldn't warrant reporting to the CoC committee? |
| 05:11 | <rkirsling> | (but I'm rather surprised to see Jordan and Tab, say, in disagreement on this) |
| 05:11 | <rkirsling> | either way I like the proposed text |
| 06:45 | <ljharb> | i mean, if it doesn't warrant reporting, it doesn't belong in the CoC |
| 06:45 | <ljharb> | and spam isn't a CoC issue either, it's just that CoC moderation teams tend to be the ones who combat spam, because they have the permissions and vigilance to do it |
| 06:46 | <ljharb> | as far as i'm aware you have never been able to do that - certainly not for at least a decade. the only repos that can go into tc39-transfer are ones from people in the tc39-transfer org, ie, delegates/IEs |
| 15:44 | <dminor> | More generally, would we see plagiarism as a violation of the code of conduct? The APA definition is "the act of presenting the words, ideas, or images of another as your own", and that seems to be what we're trying to prevent here, in the narrow case of presenting LLM generated text as your own work. |
| 15:44 | <dminor> | I'm wondering if it would be less controversial to add text about plagiarism to the CoC. |
| 15:45 | <bakkot> | Many people do not regard use of LLMs as plagiarism, so I wouldn't want to use that word. |
| 15:46 | <bakkot> | I could tweak my proposal to also list "or any other source", maybe? As in: Author your own comments |
| 16:03 | <TabAtkins> | as far as i'm aware you have never been able to do that - certainly not for at least a decade. the only repos that can go into tc39-transfer are ones from people in the tc39-transfer org, ie, delegates/IEs |
| 16:54 | <ljharb> | that works, but it's easier for them just to bounce it to you directly as the first step |
| 23:34 | <TabAtkins> | Oh, can you just directly transfer a repo between people? That would work, yeah. |