Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

111–120 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#111

Post by jvanderbot »

daveguy wrote: post #90
jvanderbot wrote: post #49

Earlier quoted context omitted.

Really? Isn't that the point of RL used in the way R1 did? Provide a cost function (vs labels) and have it argue itself to greatness as measured by that cost function? I believe that's what GP meant by "respond", not telling GPT they were wrong.

That is still inference. It is using a model generated from the RL process. The RL process is what used the cost function to add another model layer. Any online/continual learning would have to be performed by a different algorithm than classical LLM or RL. You can think of RL as a revision, but it still happens offline. Online/continual learning is still a very difficult problem in ML.

Yes, that makes sense. We're both talking about offline learning.

Re: An analysis of DeepSeek's R1-Zero and R1

#112

Post by kristofferR »

BorisMelnik wrote: post #60
nine_k wrote: post #46

Earlier quoted context omitted.

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?

Great arsticle from today: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-...

Re: An analysis of DeepSeek's R1-Zero and R1

#113

Post by gsam »

optimalsolver wrote: post #25
dagelf wrote: post #2

Well o3 scored 75% on AGI-1, R1 and o1 only 25%.... watch this space though....

But can o3 write a symphony? Seriously though, I'd like to hear suggestions on how to automatically evaluate an AI model's creativity, no humans in the loop.

In my view there's two modes of creativity:

1. That two distant topics or ideas are actually much more closely related. The creative sees one example of an idea and applies it to a discipline that nobody expects. In theory, reduction of the maximally distant can probably be measured with a tangible metric.

2. Discovery of ideas that are even more maximally distant. Pushing the edge, and this can be done by pure search and randomness actually. But it's no good if it's garbage. The trick is, what is garbage? That is very context dependent.

(Also, a creative might be measured on the efficiency of these metrics rather than absolute output)

Re: An analysis of DeepSeek's R1-Zero and R1

#114

Post by whilenot-dev »

Uehreka wrote: post #101
aprilthird2021 wrote: post #87

Earlier quoted context omitted.

> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.

Not necessarily, not all tactics can be used symmetrically like that. Many of the sites they scrape feel the need to support search engine crawlers and RSS crawlers, but OpenAI feels no such need to grant automated anonymous access to ChatGPT users. And at the end of the daty, they can always look at the responses coming in and make decisions like “95% of users said these responses were wrong, 5% said these responses…

> As long as the vast majority of their data is good (and it will be)

So expert answers are out of scope? Nice, looking forward to those quality data!

Re: An analysis of DeepSeek's R1-Zero and R1

#115

Post by youoy »

Davidzheng wrote: post #107
aithrowawaycomm wrote: post #64

Earlier quoted context omitted.

It's also extremely hard to nail down in much of mathematics or computer science! - is such-and-such theorem deep or shallow? - is this definition/axiom useful? (there's a big difference between doing compass-straightedge proofs vs. wondering about the parallel postulate) - more generally, discovering theorems is generally not amenable to verifiable rewards, except in domains where simpler deterministic tools exist (…

Not clear to me that theorem discovery is not amenable to verifiable rewards. I think most important theorems probably are recovered automatically by asking AI systems to proof increasing complicated human conjectures. Along the way I expect emergent behaviors of creating conjectures and recognizing important self-breakthroughs. Much like regret emergence

Theorems discovery is amenable to verifiable rewards. But is meaningful theorems discovery too? Is the ability to discern between meaningful theorems and bad ones an emergent behaviour? You can check for yourself examples of automatic proofs, and the huge amount of intermediate theorems that they can generate which are not very meaningful.

Re: An analysis of DeepSeek's R1-Zero and R1

#116

Post by amluto »

mohsen1 wrote: post #31
spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

Does it?

If I say "no, you hallucinated basically the entire content of the response", then maybe a newer training set derived from that could train on the specific fact that that specific hallucinated response is hallucinated. This seems to be of dubious value in a training set.

Re: An analysis of DeepSeek's R1-Zero and R1

#117

Post by HarHarVeryFunny »

Onavo wrote: post #5
anothermathbozo wrote: post #3

The claim is that this removes the human bottleneck (aka SFT or supervised fine tuning) on domains with a verifiable reward. Critically, this verifiable reward is extremely hard to pin down in nearly all domains besides mathematics and computer science.

By verifiable do they mean it in the complexity theory P/NP sense of the word?

For some reasoning data (e.g. you talking out loud as you figure something out, mistakes and all) to be useful for RL training, the conclusion to your reasoning needs to be correct/verified, else that's not the kind of reasoning you want to learn!

Some types of reasoning output, such as solving a math problem or writing a computer program can be automatically verified (e.g. respectively by a symbolic solver, or by compiling and running the program), but in the general case it's hard for a computer to verify whether a chain of reasoning is correct and arrived at a valid answer or not, although LLM-as-judge should work some of the time.

Re: An analysis of DeepSeek's R1-Zero and R1

#118

Post by girvo »

genewitch wrote: post #104
aprilthird2021 wrote: post #45

Earlier quoted context omitted.

> auto run your code, compile it, feed errors back to the LLM, Can't wait for companies to juice profits by having the LLM run excessive cycles or get stuck in a loop and run up my bill

aider jams the backend on my PC, i have to kill the tcp connection or python to stop it running a GPU on the backend, from time to time. I can't imagine paying for tokens and not knowing if it's working or wasting money.

The loops and constant useless changes drive me nuts haha

Re: An analysis of DeepSeek's R1-Zero and R1

#119

Post by spoaceman7777 »

levocardia wrote: post #102
dagelf wrote: post #2

Well o3 scored 75% on AGI-1, R1 and o1 only 25%.... watch this space though....

What's interesting is that you can already see the "AI race" dynamics in play -- OpenAI must be under immense market pressure to push o3 out to the public to reclaim "king of the hill" status.

I suppose they're under some pressure to release o3-mini, since r1 is roughly a peer for that, but r1 itself is still quite rough. The o1 series had seen significantly more QA time to smooth out the rough edges, and idiosyncracies what a "production" model should be optimized for, vs. just a top scorer on benchmarks.

We'll likely only see o3 once there is a true polished peer for it. It's a race, and companies are keeping their best models close to their chest, as they're used internally to train smaller models.

e.g., Claude 3.5 Opus has been around for quite a while, but it's unreleased. Instead, it was just used to refine Claude Sonnet 3.5 into Claude Sonnet 3.6 (3.6 is for lack of a better name, since it's still called 3.5).

We also might see a new GPT-4o refresh trained up using GPT-o3 via deepseek's distillation technique and other tricks.

There are a lot of new directions to go in now for OpenAI, but unfortunately, we won't likely see them until their API dominance comes under threat.

Re: An analysis of DeepSeek's R1-Zero and R1

#120

Post by IAmGraydon »

polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

Most software is useful because a large number of people can interact with it or with each other over it. I'm not so certain that one-off software would be very useful for anyone beyond very simple functionality.
Post reply on HN