Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it has evidence DeepSeek used its model to train competitor

ft.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#561

Post by kgeist »

moralestapia wrote: post #476

Called it from day 0, impossible to reach that performance with 5M, they had to distill OpenAI (or some other leading foundational model). Got downvoted to oblivion by people who haven't been told what to think by MSM yet. Now it's on FT and everywhere, good, what matters is that truth comes out eventually. I don't take any sides and think what DeepSeek did is fair play, however, what I do find harmful about this is,…

The "evidence" is very weak though:

>The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek.

Given that many people have been using ChatGPT to distill their fine-tunes for a few years now, how can they be sure it was specifically DeepSeek? There's, say, glaive.ai whose entire business model is to sell you synthetic datasets, probably generated with ChatGPT as well.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#562

Post by cherry_tree »

Imnimo wrote: post #504
riantogo wrote: post #461

Earlier quoted context omitted.

Why would it cast any doubt? If you can use o1 output to build a better R1. Then use R1 output to build a better X1... then a better X2.. XN, that just shows a method to create better systems for a fraction of the cost from where we stand. If it was that obvious OpenAI should have themselves done. But the disruptors did it. It hindsight it might sound obvious, but that is true for all innovations. It is all good stuf…

I think it would cast doubt on the narrative "you could have trained o1 with much less compute, and r1 is proof of that", if it turned out that in order to train r1 in the first place, you had to have access to bunch of outputs from o1. In other words, you had to do the really expensive o1 training in the first place. (with the caveat that all we have right now are accusations that DeepSeek made use of OpenAI data -…

> I think it would cast doubt on the narrative "you could have trained o1 with much less compute, and r1 is proof of that"

Whether or not you could have, you can now.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#563

Post by iforgot22 »

hnlmorg wrote: post #474
827a wrote: post #465

Earlier quoted context omitted.

There is a third possibility I haven't seen discussed yet: That DeepSeek, illegally, got their hands on an OpenAI model via a breach of OpenAI's systems. Its easy to laugh at OpenAI and say "you reap what you sow", I'm 100% in that camp, but given the lengths other Chinese entities have gone to when it comes to replicating Western technology; we should not discount this. That being said, breaching OAI's systems, re-t…

[flagged]

I can't? I am going to make that accusation if we're talking about the govt of China.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#564

Post by flybarrel »

cmiles74 wrote: post #468
buyucu wrote: post #454

Earlier quoted context omitted.

I'm willing to bet ''ban DeepSeek'' voices will start soon. Why compete, when you can just ban?

They've started already, I've seen posts on LinkedIn implying or outright stating that DeepSeek is a national security risk (IMHO, LinkedIn being the social media outlet most corporate-sycophantic). I went ahead and just picked this one at random from my feed. https://www.linkedin.com/posts/kevinkeller_deepseek-privacy-...

Oh this post...calling out DeepSeek's T&C but not comparing it with OpenAI's is really disingenuous IMO.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#565

Post by AdeptusAquinas »

intalentive wrote: post #560
dismalaf wrote: post #458

Earlier quoted context omitted.

[flagged]

[flagged]

Agree with most of this (not the weird anti-immigrant bit but the rest), however China does have some foreign requirements that are a bit of a pain in the ass, like its insistence that Taiwan isn't a country. They also don't like it and will retaliate when you point out the shady shit it does (e.g. the Uighurs), but then thats no different from the states especially under its current toddler administration.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#566

Post by kelnos »

aprilthird2021 wrote: post #478
rockemsockem wrote: post #467

Earlier quoted context omitted.

I think the prevailing narrative ATM is that DeepSeek's own innovation was done in isolation and they surpassed OpenAI. Even though in the paper they give a lot of credit to Llama for their techniques. The idea that they used o1's outputs for their distillation further shows that models like o1 are necessary. All of this should have been clear anyway from the start, but that's the Internet for you.

> the prevailing narrative ATM is that DeepSeek's own innovation was done in isolation and they surpassed OpenAI I did not think this, nor did I think this was what others assumed. The narrative, I thought, was that there is little point in paying OpenAI for LLM usage when a much cheaper, similar / better version can be made and used for a fraction of the cost (whether it's on the back of existing LLM research doesn'…

> I did not think this, nor did I think this was what others assumed.

That's what I thought and assumed. This is the narrative that's been running through all the major news outlets.

It didn't even occur to me that DeepSeek could have been training their models using the output of other models until reading this article.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#567

Post by me551ah »

Imnimo wrote: post #320

I think there's two different things going on here: "DeepSeek trained on our outputs and that's not fair because those outputs are ours, and you shouldn't take other peoples' data!" This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet. "DeepSeek trained on our outputs, and so their claims of replicating…

This is going to have a catastrophic effect on closed source AI startup valuations. Because this means that anyone can copy any LLM. The person who trains the model, spends the most amount of money. Everyone else can create a replica at lower cost

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#568

Post by rubslopes »

tasuki wrote: post #364
glenstein wrote: post #220

All the top level comments are basking in the irony of it, which is fair enough. But I think this changes the Deepseek narrative a bit. If they just benefited from repurposing OpenAI data, that's different than having achieved an engineering breakthrough, which may suggest OpenAI's results were hard earned after all.

I understand they just used the API to talk to the OpenAI models. That... seems pretty innocent? Probably they even paid for it? OpenAI is selling API access, someone decided to buy it. Good for OpenAI! I understand ToS violations can lead to a ban. OpenAI is free to ban DeepSeek from using their APIs.

Additionally, I was under the impression that all those Chinese models were being trained using data from OpenAI and Anthropic. Were there not some reports that Qwen models referred to themselves as Claude?

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#569

Post by davesque »

I recently thought of a related question. Actually, I'm almost certain that foundation model trainers have thought of this. The question is to what extent are popular modern benchmarks (or any reference to them, or description of them, etc.) bring scrubbed from the training data? Or are popular benchmarks designed in such a way that they can be re-parametrized for each run? In any case, it seems like a surprisingly hard problem to deal with.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#570

Post by km144 »

Imnimo wrote: post #320

I think there's two different things going on here: "DeepSeek trained on our outputs and that's not fair because those outputs are ours, and you shouldn't take other peoples' data!" This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet. "DeepSeek trained on our outputs, and so their claims of replicating…

Reasonable take, but to ignore the politics of this whole thing is to miss the forest for the trees—there is a big tech oligarchy brewing at the edges of the current US administration that Altman is already participating in with Stargate, and anti-China sentiment is everywhere. They'd probably like the US to ban Chinese AI.
Post reply on HN