Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

181–190 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#181

Post by ziaowang »

nogridbag wrote: post #176
octacat wrote: post #168

I think deepseek accidentally also killed google for me, not just chatgpt. Because of the visible reasoning part.

From what I read elsewhere (random reddit comment), the visible reasoning is just "for show" and isn't the process deepseek used to arrive at the result. But if the reasoning has value, I guess it doesn't matter even if it's fake.

Can you provide a link to the comment?

R1's technical report (https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...) says the prompt used for training is " reasoning process here answer here . User: prompt. Assistant:" This prompt format strongly suggests that the text between is made the "reasoning" and the text between is made the "answer" in the web app and API (https://api-docs.deepseek.com/guides/reasoning_model). I see no reason why deepseek should not do it this way, if not considering post-generation filtering.

Plus, if you read table 3 of the R1 technical report, which contains an example of R1's chain of thought, its style (going back to re-evaluating the problem) resembles what I actually got in the COT in the web app.

Re: An analysis of DeepSeek's R1-Zero and R1

#182

Post by visarga »

spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

> I wonder if there is a cap to multi head attention architecture

I don't think there is a cap other than having good data. The model learns all languages in the world, it has capacity. A simple model like AlphaZero beats humans at board games. As long as you have data, the model is not an obstacle. A LLM like AlphaProof is ranked silver medal at IMO.

Re: An analysis of DeepSeek's R1-Zero and R1

#183

Post by visarga »

jcims wrote: post #44
spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

>You might be lowering the cost of your training corpus by a few million dollars, but I highly doubt you are getting novel, high quality data. The large foundational models don't really need more empirical data about the world. ChatGPT already 'knows' way more than I do, probably by many orders of magnitude. Yet it's still spewing nonsense at me regularly because it doesn't know how to think like a human or interact…

> the ability for a company like OpenAI to collect novel data from interacting with real humans is a material advantage over their competition

It's different kind of data from the R1 reasoning chains. When LLMs have human in the loop, the human provides help based off their personal experience and real world validation. Sometimes users take an idea from the LLM and try it in real life. Then come back later and discuss the outcomes. This is a real world testing loop.

In order to judge if an AI response was useful, you can look at the following messages with a judge LLM. Using hindsight helps a lot here. Maybe it doesn't pan out and the user tries another approach, or maybe some innocuous idea was key to success later. It's hard to tell in the moment, but easy when you see what followed after that.

This scales well - OpenAI has 300M users, I estimate up to 1 Trillion interactive tokens/day. The user base is very diverse, problems are diverse, and feedback comes from user experience and actual testing. They form an experience flywheel, the more problem solving they do, the smarter it gets, attracting more users.

Re: An analysis of DeepSeek's R1-Zero and R1

#184

Post by aprilthird2021 »

Uehreka wrote: post #101
aprilthird2021 wrote: post #87

Earlier quoted context omitted.

> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.

Not necessarily, not all tactics can be used symmetrically like that. Many of the sites they scrape feel the need to support search engine crawlers and RSS crawlers, but OpenAI feels no such need to grant automated anonymous access to ChatGPT users. And at the end of the daty, they can always look at the responses coming in and make decisions like “95% of users said these responses were wrong, 5% said these responses…

Sure not necessarily the same tactics, but as with any hacking exercise, there are ways. We can become the 95% :)

Re: An analysis of DeepSeek's R1-Zero and R1

#185

Post by aprilthird2021 »

halfadot wrote: post #170
whilenot-dev wrote: post #114

Earlier quoted context omitted.

> As long as the vast majority of their data is good (and it will be) So expert answers are out of scope? Nice, looking forward to those quality data!

What makes people think companies like OpenAI can't just pay experts for verified true data? Why do all these "gotcha" replies always revolve around the idea that everyone developing AI models is credulous and stupid?

Because paying experts for verified true data in the quantities they need isn't possible. Ilya himself said we've reached peak data (https://www.theverge.com/2024/12/13/24320811/what-ilya-sutsk...).

Why do you think we are stupid? We work at places developing these models and have a peek into how they're built...

Re: An analysis of DeepSeek's R1-Zero and R1

#186

Post by stale2002 »

mrbungie wrote: post #179
halfadot wrote: post #169

Earlier quoted context omitted.

It is absolutely fascinating to read the fantasy produced by people who (apparently) think they live in a sci-fi movie. The companies whose datasets you're "poisoning" absolutely know about the attempts to poison data. All the ideas I've seen linked on this side so far about how they're going to totally defeat the AI companies' models sound like a mixture of wishful thinking and narcissism.

Are you suggesting some kind of invulnerability? People iterate their techniques, if big techs are so capable of avoiding poisoning/gaming attempts there would be no decades long tug-of-war between Google and black hat SEO manipulators. Also I don't get the narcissism part. Would it be petty to poison a website only when looked by a spider? Yes, but I would also be that petty if some big company doesn't respect the b…

Its not complete invulnerability. Instead, it is merely accepting that these methods might increase costs, like a little bit, but they don't cause the whole thing to explode.

The idea that a couple bad faith actions can destroy a 100 billion dollar company, is the extraordinary claim that requires extraordinary evidence.

Sure, bad actors can do a little damage. Just like bad actors can do DDoS attempts against Google. And that will cause a little damage. But mostly Google wins. Same thing applies to these AI companies.

> Also I don't get the narcissism part

The narcissism is the idea that your tiny website is going to destroy a 100 billion dollar company. It won't. They'll figure it out.

Re: An analysis of DeepSeek's R1-Zero and R1

#187

Post by aprilthird2021 »

halfadot wrote: post #169
aprilthird2021 wrote: post #87

Earlier quoted context omitted.

> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.

It is absolutely fascinating to read the fantasy produced by people who (apparently) think they live in a sci-fi movie. The companies whose datasets you're "poisoning" absolutely know about the attempts to poison data. All the ideas I've seen linked on this side so far about how they're going to totally defeat the AI companies' models sound like a mixture of wishful thinking and narcissism.

Who said they don't know? The same way companies know about hackers, it doesn't mean nothing ever gets hacked

Re: An analysis of DeepSeek's R1-Zero and R1

#188

Post by mrbungie »

stale2002 wrote: post #186
mrbungie wrote: post #179

Earlier quoted context omitted.

Are you suggesting some kind of invulnerability? People iterate their techniques, if big techs are so capable of avoiding poisoning/gaming attempts there would be no decades long tug-of-war between Google and black hat SEO manipulators. Also I don't get the narcissism part. Would it be petty to poison a website only when looked by a spider? Yes, but I would also be that petty if some big company doesn't respect the b…

Its not complete invulnerability. Instead, it is merely accepting that these methods might increase costs, like a little bit, but they don't cause the whole thing to explode. The idea that a couple bad faith actions can destroy a 100 billion dollar company, is the extraordinary claim that requires extraordinary evidence. Sure, bad actors can do a little damage. Just like bad actors can do DDoS attempts against Google…

Grandparent mentioned "we", I guess they refer to a full class of "black hats" avoiding bad faith scraping that eventually could amass to a relatively effective volume of poisoned sites and/or feedback to the model.

Obviously a singular poisoned site will never make a difference in a dataset of billions and billions of tokens, much less destroy a 100bn company. That's a straw man, and I think people arguing about poisoning acknowledge that perfectly. But I'd argue they can eventually manage to at least do some little damage mostly for the lulz, while avoiding scraping.

Google is full of SEO manipulators and even when they recognize the problem and try to fix it, searching today is a mess because of that. Main difference and challenge in poisoning LLMs would be coordination between different actors, as there is no direct aligning incentive to poisoning except (arguably) global justified pettiness, unlike black hat SEO players that have the incentive to be the first result to certain query.

As LLMs become commonplace eventually new incentives may appear (i.e. an LLM showing a brand before others), and then, it could become a much bigger problem akin to Google's.

tl;dr: I wouldn't be so dismissive of what adversaries can manage to do with enough motivation.

Re: An analysis of DeepSeek's R1-Zero and R1

#189

Post by visarga »

> R1-Zero removes the human bottleneck

I disagree. It only removes the bottleneck to collecting math and code reasoning chains, not in general. The general case requires physical testing not just calculations, otherwise scientists would not need experimental labs. Discovery comes from searching the real world, it's where interesting things happen. The best interface between AI and the world are still humans, the code and math domains are just lucky to work without real world interaction.

Re: An analysis of DeepSeek's R1-Zero and R1

#190

Post by danenania »

spoaceman7777 wrote: post #119
levocardia wrote: post #102

Earlier quoted context omitted.

What's interesting is that you can already see the "AI race" dynamics in play -- OpenAI must be under immense market pressure to push o3 out to the public to reclaim "king of the hill" status.

I suppose they're under some pressure to release o3-mini, since r1 is roughly a peer for that, but r1 itself is still quite rough. The o1 series had seen significantly more QA time to smooth out the rough edges, and idiosyncracies what a "production" model should be optimized for, vs. just a top scorer on benchmarks. We'll likely only see o3 once there is a true polished peer for it. It's a race, and companies are ke…

That could also definitely make sense if the SOTA models are too slow and expensive to be popular with a general audience.
Post reply on HN