Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

81–90 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#81

Post by sgt101 »

Onavo wrote: post #5
anothermathbozo wrote: post #3

The claim is that this removes the human bottleneck (aka SFT or supervised fine tuning) on domains with a verifiable reward. Critically, this verifiable reward is extremely hard to pin down in nearly all domains besides mathematics and computer science.

By verifiable do they mean it in the complexity theory P/NP sense of the word?

There's a big difference. The membership of these classes is determined in the worst case - so if there is no polynomial time solution in the worst case then it's NP.

For this problem we don't care if it's possible that sometimes there are things that aren't verifiable, or the answers aren't exact, we just need training signal.

Re: An analysis of DeepSeek's R1-Zero and R1

#82

Post by gajjanag »

mrbungie wrote: post #74
ClumsyPilot wrote: post #47

Earlier quoted context omitted.

I think future of inference is on the client side You can do inference on almost any hardware, I do not see any edge for NVIDIA here I can download DeepSeek 30b model and run inference at good speed on AMD GPU ms and even on CPU. Apple silicon works fine too. I get >50 tokens/s on £300 AMD GPUs. The main bottleneck appears to be memory, not processing power.

In fact, I'm not sure how the "we will need tons of centralized inference infrastructure" argument works when Apple with +50% smartphone market share in the USA has a totally opposite strategy focused on privacy: on-device inference.

This is much more nuanced now. See Apple "Private Cloud Compute": https://security.apple.com/blog/private-cloud-compute/ ; they run a lot of the larger models on their own servers.

Fundamentally it is more efficient to process a batch of tokens from multiple users/requests than processing them from a single user's request on device.

Re: An analysis of DeepSeek's R1-Zero and R1

#83

Post by mrandish »

sheepdestroyer wrote: post #21
mohsen1 wrote: post #7

Earlier quoted context omitted.

with 57 million(!!) tokens

From the article : o3 (low) 75.7% 335K $20 o3 (high) 87.5% 57M $3.4K

When I saw these numbers back in the initial o3-ARC post, I immediately converted them into "$ per ARC-AGI-1 %" and concluded we may be at a point where each increased increment of 'real human-like novel reasoning' gets exponentially more compute costly.

If Mike Knoop is correct, maybe R1 is pointing the way toward more efficient approaches. That would certainly be a good thing. This whole DeepSeek release and the reactions have shown by limiting the export to China of high-end GPUs, the US incentivized China to figure out how to make low-end GPUs work really well. The more subtle meta-lesson here is that the massive flood of investment capital being shoved toward leading edge AI companies has fostered a drag race mentality which prioritized winning top-line performance far above efficiency, costs, etc.

Re: An analysis of DeepSeek's R1-Zero and R1

#84

Post by immibis »

BorisMelnik wrote: post #60
nine_k wrote: post #46

Earlier quoted context omitted.

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?

Creators who use Nightshade on their published works.

Re: An analysis of DeepSeek's R1-Zero and R1

#85

Post by echelon »

mohsen1 wrote: post #31
spyckie2 wrote: post #22

> But now with reasoning systems and verifiers, we can create brand new legitimate data to train on. This can either be done offline where the developer pays to create the data or at inference time where the end user pays! > This is a fascinating shift in economics and suggests there could be a runaway power concentrating moment for AI system developers who have the largest number of paying customers. Those customers…

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

> you provide a very valuable piece of data to train on

We've been saying this "we get valuable data" thing since the 2010s [1].

When will our collective Netflix thumbs ups give us artificial super-intelligence?

[1] Especially to investors. They love that line.

Re: An analysis of DeepSeek's R1-Zero and R1

#86

Post by ant6n »

jl6 wrote: post #67
sheepdestroyer wrote: post #21

Earlier quoted context omitted.

From the article : o3 (low) 75.7% 335K $20 o3 (high) 87.5% 57M $3.4K

$3.4K is about what you might pay a magic circle lawyer for an opinion on a matter. Not saying o3 is an efficient use of resources, just saying that it’s not outlandish that a sufficiently good AI could be worth that kind of money.

What’s the liability insurance of the AI like

Re: An analysis of DeepSeek's R1-Zero and R1

#87

Post by aprilthird2021 »

Uehreka wrote: post #75
nine_k wrote: post #46

Earlier quoted context omitted.

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

This assumes that the companies gathering the data don’t have silent ways of detecting bad actors and discarding their responses. If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? Are you using a tool to generate this bad data, which might have detectable word frequency patterns that can be detected with something cheap like tf-idf? There’s a lot…

> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known?

We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.

Re: An analysis of DeepSeek's R1-Zero and R1

#88

Post by acchow »

polishdude20 wrote: post #6

I predict that the future of LLM's when it comes to coding and software creation is in "custom individually tailored apps". Imagine telling an AI agent what app you want, the requirements and all that and it just builds everything needed from backend to frontend, asks for your input on how things should work, clarifying questions etc. It tests the software by compiling and running it reading errors and failed tests a…

Have you tried https://bolt.diy ?

It does what you describe

Re: An analysis of DeepSeek's R1-Zero and R1

#89

Post by genewitch »

stetrain wrote: post #78
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

> What is today's date? >> Today's date is Tuesday, January 28, 2025. > No, you're wrong, today's date is actually Wednesday the 29th. >> My mistake. Yes, today's date is Wednesday, January 29th, 2025. Three months later in April when this tagged data is used to train the next iteration, the AI can successfully learn that today's date is actually January 29th.

the date is in the "system prompt", so the cron job that updates the prompts to the current date may be in a different time zone than you. 7f5dbb71f54322f271c4d3fc3aaa4d3282a1af5541d82b2cbc5aa10c1420b6bc

Re: An analysis of DeepSeek's R1-Zero and R1

#90

Post by daveguy »

jvanderbot wrote: post #49
jeffbee wrote: post #39

Earlier quoted context omitted.

If such labels are collected and used to retrain the model then yes. But these models are not learning online.

Really? Isn't that the point of RL used in the way R1 did? Provide a cost function (vs labels) and have it argue itself to greatness as measured by that cost function? I believe that's what GP meant by "respond", not telling GPT they were wrong.

That is still inference. It is using a model generated from the RL process. The RL process is what used the cost function to add another model layer. Any online/continual learning would have to be performed by a different algorithm than classical LLM or RL. You can think of RL as a revision, but it still happens offline. Online/continual learning is still a very difficult problem in ML.
Post reply on HN