Live data from Hacker News

ShareGPT: Share your ChatGPT conversations with one click

sharegpt.com

1–10 of 42 posts

Re: ShareGPT: Share your ChatGPT conversations with one click

#6
post #4

How much of GPT generated text is going back right onto training new LLMs. This has to eventually overwhelm organic human generated content doesn't it? What's the way out of this

Why does there need to be a way out? Everyone just seems to assume that feeding model output into the training set is going to break things, but I don't get why.

AlphaZero learned to play chess and go training purely on its own data. Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? To me, it sounds like it could even be desirable.

Re: ShareGPT: Share your ChatGPT conversations with one click

#7
post #2

I'm going to be honest, I love using ChatGPT. Use it all the time. But I really don't want to read your sessions. I don't care what the AI said to you.

For some reason, it's often like hearing other people's dreams, but for the obvious exception that sometimes someone will surface a ChatGPT prompt that might be useful to you.

Re: ShareGPT: Share your ChatGPT conversations with one click

#8
post #4

How much of GPT generated text is going back right onto training new LLMs. This has to eventually overwhelm organic human generated content doesn't it? What's the way out of this

Watermarking. From an outsiders perspective, the issue appears to reaching consensus on how this can be implemented (but not in the technical sense). There's a game theoretic challenge in that if models define and publish detection mechanisms, this creates a motivation for people to use other systems that don't include this.

On the technical front there's a good paper here: https://arxiv.org/pdf/2301.10226.pdf, and a nice very approachable video explaining it here: https://www.youtube.com/watch?v=XZJc1p6RE78.

Re: ShareGPT: Share your ChatGPT conversations with one click

#9
post #4

How much of GPT generated text is going back right onto training new LLMs. This has to eventually overwhelm organic human generated content doesn't it? What's the way out of this

Why does there need to be a way out? Everyone just seems to assume that feeding model output into the training set is going to break things, but I don't get why. AlphaZero learned to play chess and go training purely on its own data. Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? To me, it sounds like it could even be desirable.

Correct output will be desirable. If you feed nonsense either human or AI generated you might break it.

Re: ShareGPT: Share your ChatGPT conversations with one click

#10
post #4

How much of GPT generated text is going back right onto training new LLMs. This has to eventually overwhelm organic human generated content doesn't it? What's the way out of this

While OpenAI keeps logs of every response ever returned, they can just filter that text out of any future training data.

Those logs aren't as large or unwieldy as they appear - the cost of storing a thousand words of text is tiny compared to the compute cost to generate it.

Post reply on HN