Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An analysis of DeepSeek's R1-Zero and R1

arcprize.org

Post reply on HN
Print view View original

161–170 of 280 posts

Re: An analysis of DeepSeek's R1-Zero and R1

#161

Post by nemomarx »

Levitz wrote: post #131
deegles wrote: post #93

Earlier quoted context omitted.

not being snarky, but what is the point of using the model if you already know enough to correct it into giving the right answer? an example that just occurred to me - if you asked it to generate an image of a mushroom that is safe to eat in your area, how would you tell it it was wrong? "oh, they never got back to me, I'll generate this image for others as well!"

>not being snarky, but what is the point of using the model if you already know enough to correct it into giving the right answer? For your example, what if you want to show what such a mushroom looks like to a friend? What if you want to use it on a website?

I feel like conventional image search would be more reliable to get a good picture of a mushroom variety that you know about. Ideally going out into the woods to get one I suppose.

Re: An analysis of DeepSeek's R1-Zero and R1

#162

Post by nemomarx »

genewitch wrote: post #89
stetrain wrote: post #78

Earlier quoted context omitted.

> What is today's date? >> Today's date is Tuesday, January 28, 2025. > No, you're wrong, today's date is actually Wednesday the 29th. >> My mistake. Yes, today's date is Wednesday, January 29th, 2025. Three months later in April when this tagged data is used to train the next iteration, the AI can successfully learn that today's date is actually January 29th.

the date is in the "system prompt", so the cron job that updates the prompts to the current date may be in a different time zone than you. 7f5dbb71f54322f271c4d3fc3aaa4d3282a1af5541d82b2cbc5aa10c1420b6bc

why can't they feed in user data like time zone and locale?

Re: An analysis of DeepSeek's R1-Zero and R1

#163

Post by nl »

victorbjorklund wrote: post #103
jl6 wrote: post #67

Earlier quoted context omitted.

$3.4K is about what you might pay a magic circle lawyer for an opinion on a matter. Not saying o3 is an efficient use of resources, just saying that it’s not outlandish that a sufficiently good AI could be worth that kind of money.

You pay that price to a law firm to get good service and to get a "guarantee" of correctness. You get neither from an LLM. Not saying it is not worth anything but you cant compare it to a top law firm.

You absolutely do not get a "guarantee" of correctness (event with the airquotes) from any lawyer.

Re: An analysis of DeepSeek's R1-Zero and R1

#164

Post by buu700 »

ClumsyPilot wrote: post #47
talldayo wrote: post #14

Earlier quoted context omitted.

Nvidia can actually charge larger margins if inference compute goes down. It would enable them to manufacture more units of smaller GPUs using inferior and cheaper silicon, all of which would increase the profits per unit sold as well as the number of units they can manufacture. The industry has to find a way to separate itself from Nvidia's GPGPU technology if they want to stop being gouged. The issue is that nobody…

I think future of inference is on the client side You can do inference on almost any hardware, I do not see any edge for NVIDIA here I can download DeepSeek 30b model and run inference at good speed on AMD GPU ms and even on CPU. Apple silicon works fine too. I get >50 tokens/s on £300 AMD GPUs. The main bottleneck appears to be memory, not processing power.

I would argue that both things are true:

1. The future of inference for ChatGPT-style direct consumer usage is on-device. Cloud-based inference is too gaping of a privacy hole in a world where some level of E2EE is rapidly becoming the default expectation for chat. It's not hard to imagine that the iPhone 50 may be able to comfortably run models that firmly surpass GPT-4o and o1. Similarly, for things like coding and any other creation of novel IP, there are obvious security benefits to keeping the inference local.

2. Going forward, the vast majority of inference will be performed by agents for process automation (both personal and business), rather than direct user interaction. For these use cases, centralized infrastructure will be the natural architecture. Even for cases where an end client device technically exists (e.g. Tesla-Optimus-style machines), there may be economy of scale advantages to offloading compute to the cloud.

Re: An analysis of DeepSeek's R1-Zero and R1

#165

Post by octacat »

nine_k wrote: post #46
mohsen1 wrote: post #31

Earlier quoted context omitted.

every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now

This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.

There are ways to analyze that your contributions make sense from the conversation point of view. Reasoning detects that pretty quickly. To attack you would actually use another AI, to generate non totally random stuff. It still could be detected.

I would assume to use data they would have to filter it a lot and correlate between many users.

You can detect if the user is the real one and trust their other chats "a bit more".

Re: An analysis of DeepSeek's R1-Zero and R1

#166

Post by ClumsyPilot »

snovv_crash wrote: post #137
ClumsyPilot wrote: post #47

Earlier quoted context omitted.

I think future of inference is on the client side You can do inference on almost any hardware, I do not see any edge for NVIDIA here I can download DeepSeek 30b model and run inference at good speed on AMD GPU ms and even on CPU. Apple silicon works fine too. I get >50 tokens/s on £300 AMD GPUs. The main bottleneck appears to be memory, not processing power.

Which AMD GPU gives you 50 tok/s on a 30b model? My 3090 does 30 tok/s with a 4 bit quant.

I don't mean at the same time.

For a simple question, with RX 6800, I am observing ~50 tok/s on 8B models Deepseek 16B gives ~40 tok/s. 32B doesn't fit in memory

Re: An analysis of DeepSeek's R1-Zero and R1

#167

Post by usaar333 »

Overall good post, but feels like he has an axe to grind with LLMs to the point it is misleading:

> Last week, DeepSeek published their new R1-Zero and R1 “reasoner” systems that is competitive with OpenAI’s o1 system on ARC-AGI-1. R1-Zero, R1, and o1 (low compute) all score around 15-20% – in contrast to GPT-4o’s 5%, the pinnacle of years of pure LLM scaling

R1-zero gets 14% on private set which is the exact same score June Sonnet got; Sonnet, not 4o, is the pinnacle of pure LLM scaling

Re: An analysis of DeepSeek's R1-Zero and R1

#169

Post by halfadot »

aprilthird2021 wrote: post #87
Uehreka wrote: post #75

Earlier quoted context omitted.

This assumes that the companies gathering the data don’t have silent ways of detecting bad actors and discarding their responses. If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? Are you using a tool to generate this bad data, which might have detectable word frequency patterns that can be detected with something cheap like tf-idf? There’s a lot…

> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.

It is absolutely fascinating to read the fantasy produced by people who (apparently) think they live in a sci-fi movie.

The companies whose datasets you're "poisoning" absolutely know about the attempts to poison data. All the ideas I've seen linked on this side so far about how they're going to totally defeat the AI companies' models sound like a mixture of wishful thinking and narcissism.

Re: An analysis of DeepSeek's R1-Zero and R1

#170

Post by halfadot »

whilenot-dev wrote: post #114
Uehreka wrote: post #101

Earlier quoted context omitted.

Not necessarily, not all tactics can be used symmetrically like that. Many of the sites they scrape feel the need to support search engine crawlers and RSS crawlers, but OpenAI feels no such need to grant automated anonymous access to ChatGPT users. And at the end of the daty, they can always look at the responses coming in and make decisions like “95% of users said these responses were wrong, 5% said these responses…

> As long as the vast majority of their data is good (and it will be) So expert answers are out of scope? Nice, looking forward to those quality data!

What makes people think companies like OpenAI can't just pay experts for verified true data? Why do all these "gotcha" replies always revolve around the idea that everyone developing AI models is credulous and stupid?
Post reply on HN