Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it has evidence DeepSeek used its model to train competitor

ft.com

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#531

Post by namuol »

buyucu wrote: post #454
bilekas wrote: post #95

> “It’s also extremely hard to rally a big talented research team to charge a new hill in the fog together,” he added. “This is the key to driving progress forward.” Well I think DeepSeek releasing it open source and on an MIT license will rally the big talent. The open sourcing of a new technology has always driven progress in the past. The last paragraph too is where OpenAi seems to be focusing their efforts.. > we…

I'm willing to bet ''ban DeepSeek'' voices will start soon. Why compete, when you can just ban?

Already happening within tech company policy. Mostly as a security concern. Local or controlled hosting of the model is okay in theory based on this concern, but it taints everything regarding deepseek in effect.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#532

Post by SpaceManNabs »

Imnimo wrote: post #504
riantogo wrote: post #461

Earlier quoted context omitted.

Why would it cast any doubt? If you can use o1 output to build a better R1. Then use R1 output to build a better X1... then a better X2.. XN, that just shows a method to create better systems for a fraction of the cost from where we stand. If it was that obvious OpenAI should have themselves done. But the disruptors did it. It hindsight it might sound obvious, but that is true for all innovations. It is all good stuf…

I think it would cast doubt on the narrative "you could have trained o1 with much less compute, and r1 is proof of that", if it turned out that in order to train r1 in the first place, you had to have access to bunch of outputs from o1. In other words, you had to do the really expensive o1 training in the first place. (with the caveat that all we have right now are accusations that DeepSeek made use of OpenAI data -…

My question is if deepseek r1 is just a distilled o1, i wonder if you can build a fine tuned r1 through distillation without having to fine tune o1.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#533

Post by FooBarWidget »

blast wrote: post #374

Everyone is responding to the intellectual property issue, but isn't that the less interesting point? If Deepseek trained off OpenAI, then it wasn't trained from scratch for "pennies on the dollar" and isn't the Sputnik-like technical breakthrough that we've been hearing so much about. That's the news here. Or rather, the potential news, since we don't know if it's true yet.

Have people on HN never heard of public ChatGPT conversations data sets? They've been mentioned multiple times in past HN conversations and I thought it'd be common knowledge here by now. Pretty much all open source models have been training on them for the past 2 years, it's common practice by now. And haven't people been having conversations about "synthetic data" for a pretty long time by now? Why is all of this suddenly an issue in the context of DeepSeek? Nobody made a fuss about this before.

And just because a model trains on some ChatGPT data, doesn't mean that that data is the majority. It's just another dataset.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#534

Post by david-gpu »

joe_the_user wrote: post #520
rockemsockem wrote: post #467

Earlier quoted context omitted.

I think the prevailing narrative ATM is that DeepSeek's own innovation was done in isolation and they surpassed OpenAI. Even though in the paper they give a lot of credit to Llama for their techniques. The idea that they used o1's outputs for their distillation further shows that models like o1 are necessary. All of this should have been clear anyway from the start, but that's the Internet for you.

The idea that they used o1's outputs for their distillation further shows that models like o1 are necessary. Hmm, I think the narrative of the rise of LLMs is that once the output of humans has been distilled by the model, the human isn't necessary. As far as I know, DeepSeek adds only a little to the transformers model while o1/o3 added a special "reasoning component" - if DeepSeek is as good as o1/o3, even taking d…

> I think the narrative of the rise of LLMs is that once the output of humans has been distilled by the model

Distillation is a term of art in AI and it is fundamentally incorrect to talk about distilling human-created data. Only an AI model can be distilled.

https://en.m.wikipedia.org/wiki/Knowledge_distillation#Metho...

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#535

Post by WheatMillington »

827a wrote: post #465
Imnimo wrote: post #320

I think there's two different things going on here: "DeepSeek trained on our outputs and that's not fair because those outputs are ours, and you shouldn't take other peoples' data!" This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet. "DeepSeek trained on our outputs, and so their claims of replicating…

There is a third possibility I haven't seen discussed yet: That DeepSeek, illegally, got their hands on an OpenAI model via a breach of OpenAI's systems. Its easy to laugh at OpenAI and say "you reap what you sow", I'm 100% in that camp, but given the lengths other Chinese entities have gone to when it comes to replicating Western technology; we should not discount this. That being said, breaching OAI's systems, re-t…

Do you have ANY reason to believe this might be true, or is this 100% pure speculation based on absolutely nothing?

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#536

Post by MrLeap »

Imnimo wrote: post #504
riantogo wrote: post #461

Earlier quoted context omitted.

Why would it cast any doubt? If you can use o1 output to build a better R1. Then use R1 output to build a better X1... then a better X2.. XN, that just shows a method to create better systems for a fraction of the cost from where we stand. If it was that obvious OpenAI should have themselves done. But the disruptors did it. It hindsight it might sound obvious, but that is true for all innovations. It is all good stuf…

I think it would cast doubt on the narrative "you could have trained o1 with much less compute, and r1 is proof of that", if it turned out that in order to train r1 in the first place, you had to have access to bunch of outputs from o1. In other words, you had to do the really expensive o1 training in the first place. (with the caveat that all we have right now are accusations that DeepSeek made use of OpenAI data -…

o1 wouldn't exist without the combined compute of every mind that led to the training data they used in the first place. How many h100 equivalents are the rolling continuum of all of human history?

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#537

Post by namuol »

thorum wrote: post #125

> “It is (relatively) easy to copy something that you know works,” Altman tweeted. “It is extremely hard to do something new, risky, and difficult when you don’t know if it will work.” The humor/hypocrisy of the situation aside, it does seem to be true that OpenAI is consistently the one coming up with new ideas first (GPT 4, o1, 4o-style multimodality, voice chat, DALL-E, …) and then other companies reproduce their…

The eye-watering funding numbers proposed by Altman in the past and more recently with “Stargate” suggests a publicly-funded research pivot is not out of the question. Could see a big defense department grant being given. Sigh.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#538

Post by alexathrowawa9 »

me551ah wrote: post #158

OpenAI is going after a company that open sourced their model, by distilling from their non-open AI? OpenAI talks a lot about the principles of being Open, while still keeping their models closed and not fostering the open source community or sharing their research. Now when a company distills their models using perfectly allowed methods on the public internet, OpenAI wants to shut them down too? High time OpenAI cha…

The name OpenAI gets more ridiculous by the day

Would not be surprised if they do a rebrand eventually

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#539

Post by _pferreir_ »

buyucu wrote: post #455
bhouston wrote: post #23

The US government likely will favor a large strategic company like OpenAI instead of individual's copyrights, so while ironic, the US government definitely doesn't care. And the US government is also likely itching to reduce the power of Chinese AI companies that could out compete US rivals (similar to the treatment of BYD, TikTok, solar panel manufacturers, network equipment manufacturers, etc), so expect sweeping l…

It's too late for that. That ship sailed a long time ago. The best language model right now is open source. Let that sink in.

DeepSeek is not Open Source. That's like saying that Microsoft Edge is Open Source, as you can download it for free.

https://huggingface.co/blog/open-r1

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#540

Post by mtlmtlmtlmtl »

So, what is this evidence? I'll believe it when I see it. Right now all we really have is some vague rumours about some API requests. How many requests? How many tokens? Over how long of a time period? Was it one account or multiple, if the latter, how many? How do they know the activity came from deepseek? How do they know the data was actually used to train Deepseek models(could have just been benchmarking against the competition)?

If all they really have is some API requests, even assuming they're real and originated by Deepseek, that's very far from proof that any of it was used as training data. And honestly, short of commiting crimes against Deepseek(hacking), I'm not sure how they even could prove that at this point, from their side alone.

And what's even more certain is that a vague insistence that evidence exists, accompanied by a denial to shed any more light on the specifics, is about as informative as saying nothing at all. It's not like OpenAI and Microsoft have a habit of transparency and honesty in their communication with the public, as proven by an endless laundry list of dishonest and subversive behaviour.

In conclusion, I don't see why I should give this any more credence than I would a random anon on 4chan claiming a pizza place in Washington DC is the centre of a child sex trafficking ring.

P.S: And to be clear, I really don't care if it is true. If anything, I hope it is; it would be karmic justice at its finest.

Post reply on HN