Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

231–240 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#231

Post by linguistbreaker »

merrywhether wrote: post #217
octacat wrote: post #165

Earlier quoted context omitted.

There are ways to analyze that your contributions make sense from the conversation point of view. Reasoning detects that pretty quickly. To attack you would actually use another AI, to generate non totally random stuff. It still could be detected. I would assume to use data they would have to filter it a lot and correlate between many users. You can detect if the user is the real one and trust their other chats "a bi…

You would have to grade every user on every knowledge axis though. Just because someone is an expert in software doesn’t mean you should believe their takes on medicine, no matter how good faith their model interactions appear. I’d argue that coming up with an automated way to determine the objective truthfulness of information would be among the greatest creations of humanity (basically “solving” philosophy), so thi…

I've been thinking about how this happens with human cognitive development. There's a constant reinforcement mechanism that simply compares one's predicted reality with actual reality. The machines lack an authoritative reality.

If we had to grade truthiness of data sources - our sight or other main senses would probably be #1. Some gossip we heard from a 6 year old is near the bottom.

We know how to grade these data sources based on longitudinal experience and they are graded on multiple axes. For instance Angela is wrong about most facts but always right about matters of the heart.

Re: An analysis of DeepSeek's R1-Zero and R1

#232

Post by nyrikki »

halfadot wrote: post #170
whilenot-dev wrote: post #114

Earlier quoted context omitted.

> As long as the vast majority of their data is good (and it will be) So expert answers are out of scope? Nice, looking forward to those quality data!

What makes people think companies like OpenAI can't just pay experts for verified true data? Why do all these "gotcha" replies always revolve around the idea that everyone developing AI models is credulous and stupid?

You see a rowboat, and you need to cross the river.

Ask a dozen experts to decide what that boat needs to fit your need.

That is the specification problem, add on the frame problem and it becomes intractable.

Add in domain specific terms and conflicts and it becomes even more difficult.

Any nontrivial semantic properties, those without a clear T/F are undecidable.

OpenAI with have to do what they can, but it is not trivial or solvable.

It doesn't matter how smart they are, generalized solutions are hard.

Re: An analysis of DeepSeek's R1-Zero and R1

#234

Post by Bjorkbat »

Stevvo wrote: post #20

"The o3 system demonstrates the first practical, general implementation of a computer adapting to novel unseen problems" Yet, they said when it was announced: "OpenAI shared they trained the o3 we tested on 75% of the Public Training set. They have not shared more details. We have not yet tested the ARC-untrained model to understand how much of the performance is due to ARC-AGI data." These two statements are complet…

Glad someone brought this up.

I'm personally fine with o3 being tuned on the train set as a way to teach models "the rules of the game", what annoys me is that this wasn't also done with the o1 models or r1. It's a misleading comparison that suggests that o3 is a huge improvement over o1 when in reality much of that improvement may have simply been that one model knew which game it was playing and the others didn't.

Re: An analysis of DeepSeek's R1-Zero and R1

#235

Post by entropicdrifter »

spyckie2 wrote: post #160
sheepscreek wrote: post #123

Earlier quoted context omitted.

> The most promising idea is to use reasoning models to generate data, and then train our non-reasoning models with the reasoning-embedded data. DeepSeek did precisely this with their LLama fine-tunes. You can try the 70B one here (might have to sign up): https://groq.com/groqcloud-makes-deepseek-r1-distill-llama-7...

Yes, but I meant it slightly differently than the distills. The idea is to create the next gen SOTA non reasoning model with synthetic reasoning training data.

So you mean something like, "what if the baseline, off-the-cuff response for the next-gen models was tuned based on the results of the reasoning model excluding the reasoning itself?"

Re: An analysis of DeepSeek's R1-Zero and R1

#236

Post by dakshgupta »

>Generate chains-of-thought (CoT) for a problem domain. >Label the intermediary CoT steps using a combination of human experts (“supervised fine tuning” or SFT) and automated machines (“reinforcement learning” or RL). >Train base model using (2).

This is remarkably intuitive and elegant. Seems analogous to the idea that humans can come up with new knowledge by synthesizing from their current knowledge. Theoretical sciences or creative arts for example.

Re: An analysis of DeepSeek's R1-Zero and R1

#237

Post by benbosco »

merrywhether wrote: post #216
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

Users can be adversarial to the “truth” (to the extent it exists) without being adversarial in intent. Dinosaur bones are either 65 million year old remnants of ancient creatures or decoys planted by a God during a 7 day creation, and a large proportion of humans earnestly believe either take. Choosing which of these to believe involves a higher level decision about fundamental worldviews. This is an extreme example,…

Dirac’s prediction of antimatter came from purely mathematical reasoning—before any experimental evidence existed. Testing and verifying conjectures requires the ability to extrapolate beyond known data, rather than from it, and the ability discard false leads based on theoretical reasoning, rather than statistical confidence.

All of this is possible in a bottle, but laughably far beyond our current capabilities.

Re: An analysis of DeepSeek's R1-Zero and R1

#238

Post by janalsncm »

aithrowawaycomm wrote: post #64
anothermathbozo wrote: post #3

The claim is that this removes the human bottleneck (aka SFT or supervised fine tuning) on domains with a verifiable reward. Critically, this verifiable reward is extremely hard to pin down in nearly all domains besides mathematics and computer science.

It's also extremely hard to nail down in much of mathematics or computer science! - is such-and-such theorem deep or shallow? - is this definition/axiom useful? (there's a big difference between doing compass-straightedge proofs vs. wondering about the parallel postulate) - more generally, discovering theorems is generally not amenable to verifiable rewards, except in domains where simpler deterministic tools exist (…

For questions with a correct answer, you don’t need to verify the reasoning process. RL training will discover it. That’s R1-Zero.

The point of R1 was to fix problems with the reasoning tokens and expand to subjective domains like creative writing.

Re: An analysis of DeepSeek's R1-Zero and R1

#239

Post by bobxmax »

motoboi wrote: post #215
bobxmax wrote: post #211

Earlier quoted context omitted.

I don't know why HN users in particular fixate so heavily on fringe issues when it comes to LLMs. Same as the exaggerations of hallucinations.

Because hallucinations is something that from a distance looks very unimportant, but when looked closely is a structural problem. Some people here live very close to the LLM field. Structural because while a human being can be the judge of an LLM output, a computer (or another LLM) cannot. No amount of error correction is enough to turn an LLM output into a reliable input to another (possible dumb) computer system. W…

Two questions:

1) Which search engine comes with infallible information? 2) Where are LLMs being sold as something different?

Re: An analysis of DeepSeek's R1-Zero and R1

#240

Post by janalsncm »

youoy wrote: post #115
Davidzheng wrote: post #107

Earlier quoted context omitted.

Not clear to me that theorem discovery is not amenable to verifiable rewards. I think most important theorems probably are recovered automatically by asking AI systems to proof increasing complicated human conjectures. Along the way I expect emergent behaviors of creating conjectures and recognizing important self-breakthroughs. Much like regret emergence

Theorems discovery is amenable to verifiable rewards. But is meaningful theorems discovery too? Is the ability to discern between meaningful theorems and bad ones an emergent behaviour? You can check for yourself examples of automatic proofs, and the huge amount of intermediate theorems that they can generate which are not very meaningful.

Unless you can quantify what you mean by “meaningful” then it won’t be possible. It can’t read your mind.
Post reply on HN