Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ShareGPT: Share your ChatGPT conversations with one click

sharegpt.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: ShareGPT: Share your ChatGPT conversations with one click

#21

Post by geraneum »

marginalia_nu wrote: post #12
sebzim4500 wrote: post #6

Earlier quoted context omitted.

Why does there need to be a way out? Everyone just seems to assume that feeding model output into the training set is going to break things, but I don't get why. AlphaZero learned to play chess and go training purely on its own data. Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? To me, it sounds like it could even be desirable.

In chess there is a very clear victory state, and a scoring function can be implicitly defined from a large number of games of various skill levels. You really don't throw two sentences into the thunder dome to decide which one "wins". Means it's much more susceptible to being poisoned.

That’s correct. I have seen the above argument a lot: Using analogy as a basis for proof!

Re: ShareGPT: Share your ChatGPT conversations with one click

#22

Post by meken »

Has anyone actually used this? I tried using a few weeks ago and it just… ostensibly didn’t do anything

I think I clicked “share to shareGPT” and nothing happened. I tried clicking around to see where to go, but I couldn’t find anything, so I just uninstalled

Re: ShareGPT: Share your ChatGPT conversations with one click

#23

Post by amelius »

bil7 wrote: post #5

to quote Tom Scott: "Telling someone about your fascinating AI conversation is like telling someone about your dreams. They don’t care, it just sounds like you’re hallucinating nonsense."

Why? We might learn about the failure modes of AI.

Re: ShareGPT: Share your ChatGPT conversations with one click

#24

Post by kimburgess »

simonh wrote: post #19
kimburgess wrote: post #8

Earlier quoted context omitted.

Watermarking. From an outsiders perspective, the issue appears to reaching consensus on how this can be implemented (but not in the technical sense). There's a game theoretic challenge in that if models define and publish detection mechanisms, this creates a motivation for people to use other systems that don't include this. On the technical front there's a good paper here: https://arxiv.org/pdf/2301.10226.pdf , and…

The problem with watermarking like this, which is incredibly clever, is it’s trivial to break. All you have to do is change one word in the text, and the watermarking of all subsequent tokens is spoiled. So if you change the first word, or rephrase the first sentence, or extract text from the middle or end of a response, the watermark is completely spoiled.

There are definitely paths of attack. The trivial ones that you call out - insertion, deletion, substitution - are covered in section 7 of that paper (along with mitigations).

Re: ShareGPT: Share your ChatGPT conversations with one click

#25

Post by amelius »

simonh wrote: post #19
kimburgess wrote: post #8

Earlier quoted context omitted.

Watermarking. From an outsiders perspective, the issue appears to reaching consensus on how this can be implemented (but not in the technical sense). There's a game theoretic challenge in that if models define and publish detection mechanisms, this creates a motivation for people to use other systems that don't include this. On the technical front there's a good paper here: https://arxiv.org/pdf/2301.10226.pdf , and…

The problem with watermarking like this, which is incredibly clever, is it’s trivial to break. All you have to do is change one word in the text, and the watermarking of all subsequent tokens is spoiled. So if you change the first word, or rephrase the first sentence, or extract text from the middle or end of a response, the watermark is completely spoiled.

There can be redundancy in the watermark, meaning you'll have to change more than one word. See e.g. how error-correcting codes work.

Re: ShareGPT: Share your ChatGPT conversations with one click

#26

Post by notahacker »

swalsh wrote: post #2

I'm going to be honest, I love using ChatGPT. Use it all the time. But I really don't want to read your sessions. I don't care what the AI said to you.

I quite liked reading the notably funny conversations. But I'm not sure I want to read everybody's conversation

Re: ShareGPT: Share your ChatGPT conversations with one click

#27

Post by sebzim4500 »

simonh wrote: post #17
sebzim4500 wrote: post #6

Earlier quoted context omitted.

Why does there need to be a way out? Everyone just seems to assume that feeding model output into the training set is going to break things, but I don't get why. AlphaZero learned to play chess and go training purely on its own data. Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? To me, it sounds like it could even be desirable.

> Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? Firstly what makes you think only the best output from 4 will go into future training sets? It’s just as likely to be the most bizarre or ludicrous, or dangerous that gets shared and discussed. But also, how will v5 get to be better than v4 if it’s trained significantly on v4 output? It would just end up bein…

>Firstly what makes you think only the best output from 4 will go into future training sets? It’s just as likely to be the most bizarre or ludicrous

That's true now, because LLMs are new so the failure cases are still interesting. If we are talking about a hypothetical world in which LLM outputs are a significant portion of the internet, then most of it would be from reddit comments/tweets/HN posts/buzzfeed articles/etc.

Then if you take only the ones which have more than average views/upvotes/etc. you should expect to get the 'best' results.

Re: ShareGPT: Share your ChatGPT conversations with one click

#28

Post by sebzim4500 »

marginalia_nu wrote: post #12
sebzim4500 wrote: post #6

Earlier quoted context omitted.

Why does there need to be a way out? Everyone just seems to assume that feeding model output into the training set is going to break things, but I don't get why. AlphaZero learned to play chess and go training purely on its own data. Why is inserting the best outputs from GPT-4 into the training set for GPT-5 expected to make things worse? To me, it sounds like it could even be desirable.

In chess there is a very clear victory state, and a scoring function can be implicitly defined from a large number of games of various skill levels. You really don't throw two sentences into the thunder dome to decide which one "wins". Means it's much more susceptible to being poisoned.

>You really don't throw two sentences into the thunder dome to decide which one "wins".

That's almost literally what RLHF is though, and that is the last step of training GPT-n. Then when GPT-{n+1} is being trained, it will include some results from GPT-n, and therefore will benefit from that finetuning, even before it goes through its own round of RLHF. Also, on average good outputs of GPT-n are more likely to be included in the training set of GPT-{n+1} (because it ends up as a buzzfeed article or a top post on reddit or something), so there is an additional signal beyond the above.

Re: ShareGPT: Share your ChatGPT conversations with one click

#30

Post by sebzim4500 »

bil7 wrote: post #5

to quote Tom Scott: "Telling someone about your fascinating AI conversation is like telling someone about your dreams. They don’t care, it just sounds like you’re hallucinating nonsense."

I'm still interested by other people's chats, so long as they are probing the limits of the model in ways I haven't thought of.

I don't want to see yet another conversation where someone asks it if it will become skynet or if it can write a haiku about whatever.

Post reply on HN