Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepseek R1-0528

huggingface.co

Post reply on HN
Print view View original

121–130 of 264 posts

Re: Deepseek R1-0528

#121

Post by jsemrau »

karencarits wrote: post #119

What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications

I have a signal tracer that evaluates unusual trading volumes. Given those signals, my local agent receives news items through API to make an assessment what happens. This helps me tremendously. If I would do this through a remote app, I'd have to spend a several dollars per day. So I have this on existing hardware.

Re: Deepseek R1-0528

#122

Post by sudomarcma »

karencarits wrote: post #119

What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications

Any companies with any type of sensitive data will love to have anything to do with LLM done locally.

Re: Deepseek R1-0528

#124

Post by lvturner »

karencarits wrote: post #119

What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications

Also worth it for the speed of AI autocomplete in coding tools, the round trip to my graphics card is much faster than going out over the network.

Re: Deepseek R1-0528

#125

Post by lvturner »

hbbio wrote: post #115
doctoboggan wrote: post #27

Earlier quoted context omitted.

Hard to say exactly how it will affect the market, but IIRC when deepseek was first released Nvidia stock took a big hit as people realized that you could develop high performing LLMs without access to Nvidia hardware.

Actually, the "narrative" crashed Nvidia for no reason. Not only DeepSeek uses a lot of Nvidia hardware for the training. But even more so, by releasing an open weight frontier model, people around the world need more Nvidia chips than ever for inference.

I know of enterprises in APAC now spending millions of dollars on Huawei GPUs, while they might not be as efficient, they are seen as geopolitically more stable (especially given the region).

DeepSeek helped "prove" to a lot of execs that "Good" is "Good enough" and that there are viable alternatives with less perceived risk of supply chain disruption - even if facts differ may from this narrative.

Re: Deepseek R1-0528

#126

Post by jacob019 »

piperswe wrote: post #102
make3 wrote: post #101

Earlier quoted context omitted.

I don't think people make the distinction like that. The open source vs non open source distinction boils down to, usually, can you use it for commercial use. what you're saying is just that it's non reproducible, which is a completely valid but separate issue

But where's the source? I just see a binary blob, what makes it open source?

The weights are the source. It isn't as though something was compiled into weights. They're trained directly. But I know what you mean, it would be more open to have the training pipeline and souce dataset available.

Re: Deepseek R1-0528

#127

Post by 1una »

piperswe wrote: post #102
make3 wrote: post #101

Earlier quoted context omitted.

I don't think people make the distinction like that. The open source vs non open source distinction boils down to, usually, can you use it for commercial use. what you're saying is just that it's non reproducible, which is a completely valid but separate issue

But where's the source? I just see a binary blob, what makes it open source?

I won't call it "binary blob". Safetensors is just a simple format for storing tensors safely: https://huggingface.co/docs/safetensors/index

Re: Deepseek R1-0528

#128

Post by bcoates »

karencarits wrote: post #119

What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications

I use the local LLM-based autocomplete built into PyCharm and I'm pretty happy with it

Re: Deepseek R1-0528

#129

Post by behnamoh »

danielhanchen wrote: post #106
behnamoh wrote: post #85

Earlier quoted context omitted.

> 1.58bit quantization of course we can run any model if quantize it enough. but I think the OP was talking about the unquantized version.

Oh you can still run them unquantized! See https://docs.unsloth.ai/basics/llama-4-how-to-run-and-fine-t... where we show you can offload all MoE layers to system RAM, and leave non MoE layers on the GPU - the speed is still pretty good! You can do it via `-ot ".ffn_.*_exps.=CPU"`

Thanks, I'll try it! I guess "mixing" GPU+CPU would hurt the perf tho.

Re: Deepseek R1-0528

#130

Post by thenameless7741 »

sudomarcma wrote: post #122
karencarits wrote: post #119

What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications

Any companies with any type of sensitive data will love to have anything to do with LLM done locally.

A recent example: a law firm hired this person [0] to build a private AI system for document summarization and Q&A.

[0] https://xcancel.com/glitchphoton/status/1927682018772672950

Post reply on HN