Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google denies training Bard on ChatGPT chats from ShareGPT

twitter.com

Post reply on HN
Print view View original

261–270 of 342 posts

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#261

Post by toxik »

rightbyte wrote: post #229
paxys wrote: post #191

Earlier quoted context omitted.

Maybe, but we are fast approaching the point (or more likely have crossed it already) where distinguishing between human and AI generated data isn't really possible. If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training? Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that of other LLMs)?

> Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that or other LLMs)? Seems trivial. Only use old data for the bulk? Feed some new data carefully curated?

Future job: token selector / archiving

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#262

Post by pantalaimon »

layer8 wrote: post #186
paxys wrote: post #165

Earlier quoted context omitted.

Hardly unethical, considering OpenAI is doing exactly this.

Two wrongs don’t make a right.

It’s still debatable if training a computer neutral network on public data is 'wrong' when we very much accept it as a right for biological neural networks.

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#263

Post by iandanforth »

ithkuil wrote: post #249
funkyjazz wrote: post #240

Earlier quoted context omitted.

Not to mention it's embarrassing. Google playing second banana to OpenAI.

That assumes that training on the output of another language model somehow gives you the ability to improve your model and to catch up somehow

It does. In general this is known as teacher-student training or knowledge distillation. It works better if you have access to the activations of the model but you can work with just outputs as well.

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#264

Post by notahacker »

paxys wrote: post #191
Jimmc414 wrote: post #188

Earlier quoted context omitted.

Right, but training an LLM on the output of another LLM can certainly exacerbate these issues

Maybe, but we are fast approaching the point (or more likely have crossed it already) where distinguishing between human and AI generated data isn't really possible. If Google indexes a blog, how does it know whether it was written with AI assistance and therefore should not be used for training? Heck, how does OpenAI itself prevent such a feedback loop from its own output (or that of other LLMs)?

I'm only half joking.... I think we likely will end up with flags for human generated/curated content (and it will have to be that way round, as I can't imagine spammers bothering to put flags on AI-generated stuff), and we probably already should have an equivalent of robots.txt protocol that allows users to specify which parts of their website they would and wouldn't like used in the training of LLMs.

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#266

Post by b112 »

jrirhfifj wrote: post #251
nicehill wrote: post #245

Earlier quoted context omitted.

I think Amazon was first in the (free) banana business

you joke, but first producy they changed on whole foods were the bananas. before: organic (south america) and regular (central ou SEA) for 69, 59. then: both chikita's brand with regular and organic stickers (clearly the same produce, always from SEA) for 49 and 39 cents. thats was days after the announcement

Did you inadvertently reverse to regular/organic order, or was organic cheaper after?

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#267

Post by b112 »

ithkuil wrote: post #248
ladon86 wrote: post #235

Earlier quoted context omitted.

> Google denies doing it Read their statement carefully and it's actually not a denial of the allegation. > But Google is firmly and clearly denying the data was used: “Bard is not trained on any data from ShareGPT or ChatGPT,” spokesperson Chris Pappas tells The Verge * Allegation: Google used ShareGPT to train Bard. * Rebuttal: The current production version of Bard is not trained on ShareGPT data Both things can b…

Intent matters I guess. Did they accidentally train on that public piece of info they scraped anyway because they are scraping the whole web? Or did they intentionally scrape chatgpt output to see if that would help?

They could have trained, then modified code, repeat, to better enhance training in the current version.

Then after, train on raw data.

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#268

Post by zhwu »

MMMercy2 wrote: post #265

This project fine-tunes LLaMA on ShareGPT and gets competitive performance compared to Google's Bard. https://vicuna.lmsys.org/

They even have a eval page showing that they beat Bard by only training on ShareGPT. https://vicuna.lmsys.org/eval/

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#269

Post by hackerlight »

social_quotient wrote: post #256

It’s interesting when we say Google did this. It’s actually and likely some people that work for Google and are on this forum did this. Knowingly, not by accident while slurping up the rest of the internet, and got paid to do it. I wonder what the engineer view on this was/is. I have to assume they ballpark know the terms of the openai data (regardless if you disagree or not). Anyone care to steel man the argument fo…

> Anyone care to steel man the argument for why this was a good idea?

I don't see a big difference between this and training it on people's code and art which also happens without explicit permission.

Re: Google denies training Bard on ChatGPT chats from ShareGPT

#270

Post by Certhas »

KRAKRISMOTT wrote: post #134
paxys wrote: post #88

1. Google denies doing it, so at the very least the title should have an "allegedly". 2. Even if they did – so what? The output from ChatGPT is not copyrightable by OpenAI. In fact it is OpenAI that is training its models on copyrighted data, pictures, code from all over the internet.

OpenAI Terms of service forbid training competitor models via their ML outputs (LoRa alpaca laundering is probably not allowed for commercial use).

This is really hilarious. Authors and artists never gave permission to use their work to train AI models either...

Not legally the same situation, but ethically close enough.

Post reply on HN