Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.1

api-docs.deepseek.com

Post reply on HN
Print view View original

31–40 of 273 posts

Re: DeepSeek-v3.1

#31

Post by segmondy »

hodgehog11 wrote: post #2

For reference, here is the terminal-bench leaderboard: https://www.tbench.ai/leaderboard Looks like it doesn't get close to GPT-5, Claude 4, or GLM-4.5, but still does reasonably well compared to other open weight models. Benchmarks are rarely the full story though, so time will tell how good it is in practice.

garbage benchmark, inconsistent mix of "agent tools" and models. if you wanted to present a meaningful benchmark, the agent tools will stay the same and then we can really compare the models.

there are plenty of other benchmarks that disagree with these, with that said. from my experience most of these benchmarks are trash. use the model yourself, apply your own set of problems and see how well it fairs.

Re: DeepSeek-v3.1

#33

Post by DarkFuture »

pdimitar wrote: post #16
gunalx wrote: post #13

Earlier quoted context omitted.

You would still need drivers and all the stuff difficult with nvidia in linux with a egpu. (Its not nessecarily terrible just suboptimal) Rather just add the second GPU in the Workstation, or just run the llm in your AMD GPU.

Oh, we can run LLMs efficiently with AMD GPUs now? Pretty cool, I haven't been following, thank you.

I've been running LLM models on my Radeon 7600 XT 16GB for past 2-3 months without issues (Windows 11). I've been using llama.cpp only. The only thing from AMD I installed (apart from latest Radeon drivers) is the "AMD HIP SDK" (very straight forward installer). After unzipping (the zip from GitHub releases page must contain hip-radeon in the name) all I do is this:

llama-server.exe -ngl 99 -m Qwen3-14B-Q6_K.gguf

And then connect to llamacpp via browser to localhost:8080 for the WebUI (its basic but does the job, screenshots can be found on Google). You can connect more advanced interfaces to it because llama.cpp actually has OpenAI-compatible API.

Re: DeepSeek-v3.1

#34

Post by tonyhart7 »

hodgehog11 wrote: post #2

For reference, here is the terminal-bench leaderboard: https://www.tbench.ai/leaderboard Looks like it doesn't get close to GPT-5, Claude 4, or GLM-4.5, but still does reasonably well compared to other open weight models. Benchmarks are rarely the full story though, so time will tell how good it is in practice.

Yeah but the pricing is insane, I don't care about SOTA if its not break my bank

Re: DeepSeek-v3.1

#36

Post by mycall »

imachine1980_ wrote: post #15
amrrs wrote: post #9

Earlier quoted context omitted.

Any example or prompt you use to make this statment?

I remember asking for quotes about the Spanish conquest of South America because I couldn't remember who said a specific thing. The GPT model started hallucinating quotes on the topic, while DeepSeek responded with, "I don't know a quote about that specific topic, but you might mean this other thing." or something like that then cited a real quote in the same topic, after acknowledging that it wasn't able to find the…

I wonder if Conway's law is at all responsible for that, in the similarity it is based on; regional trained data which has concept biases which it sends back in response.

Re: DeepSeek-v3.1

#37

Post by darrinm »

seunosewa wrote: post #3

It's a hybrid reasoning model. It's good with tool calls and doesn't think too much about everything, but it regularly uses outdated tool formats randomly instead of the standard JSON format. I guess the V3 training set has a lot of those.

Did you try the strict (beta) function calling? https://api-docs.deepseek.com/guides/function_calling

Re: DeepSeek-v3.1

#38

Post by arcanemachiner »

pdimitar wrote: post #23
oktoberpaard wrote: post #22

Earlier quoted context omitted.

I’m running Ollama on 2 eGPUs over Thunderbolt. Works well for me. You’re still dealing with an NVDIA device, of course. The connection type is not going to change that hassle.

Thank you for the validation. As much as I don't like NVIDIA's shenanigans on Linux, having a local LLM is very tempting and I might put my ideological problems to rest over it. Though I have to ask: why two eGPUs? Is the LLM software smart enough to be able to use any combination of GPUs you point it at?

Yes, Ollama is very plug-and-play when it comes to multi GPU.

llama.cpp probably is too, but I haven't tried it with a bigger model yet.

Re: DeepSeek-v3.1

#39

Post by danielhanchen »

For local runs, I made some GGUFs! You need around RAM + VRAM >= 250GB for good perf for dynamic 2bit (2bit MoE, 6-8bit rest) - can also do SSD offloading but it'll be slow.

./llama.cpp/llama-cli -hf unsloth/DeepSeek-V3.1-GGUF:UD-Q2_K_XL -ngl 99 --jinja -ot ".ffn_.*_exps.=CPU"

More details on running + optimal params here: https://docs.unsloth.ai/basics/deepseek-v3.1

Post reply on HN