Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI says it has evidence DeepSeek used its model to train competitor

ft.com

Post reply on HN
Print view View original

591–600 of 1001 posts

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#591

Post by kgeist »

moralestapia wrote: post #574
kgeist wrote: post #561

Earlier quoted context omitted.

The "evidence" is very weak though: >The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek. Given that many people have been using ChatGPT to distill their fine-tunes for a few years now, how can they be sure it was specifically DeepSeek? There's, say, glaive.ai whose entire business model is to sell you synthetic datasets, pr…

I agree that the evidence is weak, and even if they had some, they cannot really do anything. To me, it's just very likely they distilled GPT-4, because: 1) Again, you just cannot get that performance at that cost. And no, what they describe on the paper is not enough to explain the 1,000x-fold decrease in cost. 2) Very often, DeepSeek tells you it's ChatGPT or OpenAI; it's actually quite easy to get it to do that. S…

If it's just a distillation of GPT-4, wouldn't we expect it to have worse quality than o1? But I've seen countless examples of DeepSeek-r1 solving math problems that o1 cannot.

>Very often, DeepSeek tells you it's ChatGPT or OpenAI; it's actually quite easy to get it to do that. Some say that's related to "the background radiation on the post-AI internet". I'm not a fentanyl consumer so, unfortunately, I think that argument is trash.

The exact same thing happened with Llama. Sometimes it also claimed to be Google Assistant or Amazon Alexa.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#592

Post by rndphs »

thorum wrote: post #125

> “It is (relatively) easy to copy something that you know works,” Altman tweeted. “It is extremely hard to do something new, risky, and difficult when you don’t know if it will work.” The humor/hypocrisy of the situation aside, it does seem to be true that OpenAI is consistently the one coming up with new ideas first (GPT 4, o1, 4o-style multimodality, voice chat, DALL-E, …) and then other companies reproduce their…

> OpenAI is consistently the one coming up with new ideas first (GPT 4, o1, 4o-style multimodality, voice chat, DALL-E, …)

As far as I can tell o1 was based on Q-star, which could likely be Quiet-STaR, a CoT RL technique developed at Stanford that OpenAI may have learned about before it got published. Presumably that's why they never used the Q-Star name even though it had garnered mystique and would have been good for building hype. This is just speculation, but since OpenAI haven't published their technique then we can't know if it really was their innovation.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#593

Post by [deleted] »

moralestapia wrote: post #574
kgeist wrote: post #561

Earlier quoted context omitted.

The "evidence" is very weak though: >The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek. Given that many people have been using ChatGPT to distill their fine-tunes for a few years now, how can they be sure it was specifically DeepSeek? There's, say, glaive.ai whose entire business model is to sell you synthetic datasets, pr…

I agree that the evidence is weak, and even if they had some, they cannot really do anything. To me, it's just very likely they distilled GPT-4, because: 1) Again, you just cannot get that performance at that cost. And no, what they describe on the paper is not enough to explain the 1,000x-fold decrease in cost. 2) Very often, DeepSeek tells you it's ChatGPT or OpenAI; it's actually quite easy to get it to do that. S…

[deleted]

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#594

Post by freehorse »

sho_hn wrote: post #49

While I'm as amused as everyone else - I think it's technically accurate to point out that the "we trained it for $6 mio" narrative is contingent on the done investment by others.

That is the case anyway for training any llm. It is contingent on the work done by all those who produced the data.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#595

Post by kridsdale1 »

kmeisthax wrote: post #578
827a wrote: post #465

Earlier quoted context omitted.

There is a third possibility I haven't seen discussed yet: That DeepSeek, illegally, got their hands on an OpenAI model via a breach of OpenAI's systems. Its easy to laugh at OpenAI and say "you reap what you sow", I'm 100% in that camp, but given the lengths other Chinese entities have gone to when it comes to replicating Western technology; we should not discount this. That being said, breaching OAI's systems, re-t…

I'd be perfectly fine with China stealing all "our" shit if they just shared it. The word "our" does a lot of heavy lifting in politics[0]. America is not a commune, it's a country club, one which we used to own but have been bought out of, and whose new owners view us as moochers but can't actually kick us out (yet). It is in competition with another, worse country club that purports to be a commune. We owe neither…

This comment made me realize we don’t have a pronoun for n-our or x-nour

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#596

Post by [deleted] »

mrdevlar wrote: post #420
floatrock wrote: post #246

Earlier quoted context omitted.

The railroads drama ended when JP Morgan (the person, not yet the entity) brought all the railroad bosses together, said "you all answer to me because I represent your investors / shareholders", and forced a wave of consolidation and syndicates because competition was bad for business. Then all the farmers in the midwest went broke not because they couldn't get their goods to market, but because JP Morgan's consolida…

Which is the exact goal of the current wave of Tech oligarchy also.

[deleted]

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#597

Post by iforgot22 »

Austiiiiii wrote: post #492
worik wrote: post #431

[flagged]

And we'd be shooting ourselves in the foot to do so. If America is forced to use only the clunky corporate-owned American AI at a fee, we'll very quickly fall behind competitors worldwide who use DeepSeek models to produce better results for much, much cheaper. Not to mention it'd defeat the whole purpose of a "free market" economy. (Not that that means much of anything anymore)

The Nvidia export restrictions also might be shooting us in the foot too, or at least Nvidia. They really benefit from CUDA remaining the de facto standard.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#598

Post by yonran »

daft_pink wrote: post #177

This reminds me of the railroads, where once railroads were invented, there was a huge investment boom of eveyrone trying to make money of the railroads, but the competition brought the costs down where the railroads weren’t the people who generally made the money and got the benefit, but the consumers and regular businesses did and competition caused many to fail. AI is probably similar where the Moore’s law and adv…

I think a better analogy than railroads (which own the land that the track sits on and often valuable land around the station) is airlines, which don’t own land. I recall a relevant Warren Buffett letter that warned about investing hundreds of millions of dollars into capital with no moat:

> Similarly, business growth, per se, tells us little about value. It's true that growth often has a positive impact on value, sometimes one of spectacular proportions. But such an effect is far from certain. For example, investors have regularly poured money into the domestic airline business to finance profitless (or worse) growth. For these investors, it would have been far better if Orville had failed to get off the ground at Kitty Hawk: The more the industry has grown, the worse the disaster for owners.

https://www.berkshirehathaway.com/letters/1992.html

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#599

Post by freehorse »

radicality wrote: post #336
semking wrote: post #21

This is absolutely hilarious! :) ClosedAI scraped human content without asking and they explained why this was acceptable... but when the outputs of their training corpus is scraped, it is THEIR dataset and this is NOT acceptable! Oh, the irony! :D I shared a few screenshots of DeepSeek answering using ChatGPT's output in yesterday's article! https://semking.com/deepseek-china-ai-model-breakthrough-sec...

I liked Matt Levine’s newsletter few days ago where he hypothesized scenarios where it’s much more profitable to short your competitors, then release a much better version of some widget completely free, and then profit $$$. Which is plausible here too, considering DeepSeek is made by a hedge fund.

How would that work out here though? "Open"AI is not publicly traded. Any kind of shorting would be quite indirect.

Re: OpenAI says it has evidence DeepSeek used its model to train competitor

#600

Post by s17n »

Imnimo wrote: post #320

I think there's two different things going on here: "DeepSeek trained on our outputs and that's not fair because those outputs are ours, and you shouldn't take other peoples' data!" This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet. "DeepSeek trained on our outputs, and so their claims of replicating…

> This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet.

OpenAI has also invested heavily in human annotation and RLHF. If all DeepSeek wanted was a proxy for scraped training data, they'd probably just scrape it themselves. Using existing RLHF'd models as replacement for expensive humans in the training loop is the real game changer for anyone trying to replicate these results.

Post reply on HN