Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.1

api-docs.deepseek.com

Post reply on HN
Print view View original

151–160 of 273 posts

Re: DeepSeek-v3.1

#151

Post by danielhanchen »

pshirshov wrote: post #144
danielhanchen wrote: post #138

Earlier quoted context omitted.

For now I'm re-directly people to our docs https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-... But I'm working on more cross platform docs as well!

My current solution is to pack llama.cpp as a custom nix formula (the one in nixpkgs has the conversion script broken) and run it myself. I wasn't able to run unsloth on ROCM nor for inference nor for conversion, sticking with peft for now but I'll attempt again to re-package it.

Oh interesting oh for ROCM there are some installation instructions here: https://rocm.docs.amd.com/projects/ai-developer-hub/en/lates...

I'm working with the AMD folks to make the process easier, but it looks like first I have to move off from pyproject.toml to setup.py (allows building binaries)

Re: DeepSeek-v3.1

#152

Post by rat9988 »

danielhanchen wrote: post #127
rfoo wrote: post #126

Earlier quoted context omitted.

IMO the correct thing to do to make these people happy, while being sane, is - do not build llama.cpp on their system. Instead, bundle a portable llama.cpp binary along with unsloth, so that when they install unsloth with `pip` (or `uv`) they get it. Some people may prefer using whatever llama.cpp in $PATH, it's okay to support that, though I'd say doing so may lead to more confused noob users spam - they may just ha…

Oh yes I was working on providing binaries together with pip - currently we're relying on pyproject.toml, but once we utilize setup.py (I think), using binaries gets much simpler I'm still working on it, but sadly I'm not a packaging person so progress has been nearly zero :(

Don't worry. Don't let the rednecks screaming here affect you. As for one, I'm happy that you have automated this part and sad to see it is going away. People will always complain. It might be reasonable feedback worth acting upon. Don't let their tone distract you though. Some of them are just angry all day.

Re: DeepSeek-v3.1

#153

Post by d4rkp4ttern »

It’s a very smart move for DeepSeek to put out an Anthropic-compatible API, similar to Kimi-k2, GLM4.5 (Puzzled as to why Qwen didn’t do this). You can set up a simple function in your .zhsrc to run Claude-Code with these models:

https://github.com/pchalasani/claude-code-tools/tree/main?ta...

Re: DeepSeek-v3.1

#154

Post by jodleif »

d4rkp4ttern wrote: post #153

It’s a very smart move for DeepSeek to put out an Anthropic-compatible API, similar to Kimi-k2, GLM4.5 (Puzzled as to why Qwen didn’t do this). You can set up a simple function in your .zhsrc to run Claude-Code with these models: https://github.com/pchalasani/claude-code-tools/tree/main?ta...

Qwen have their own competitor to Claude Code.

Re: DeepSeek-v3.1

#155

Post by lucideer »

danielhanchen wrote: post #140
lucideer wrote: post #131

Earlier quoted context omitted.

I don't think this should be a personal preference, I think it should be a standard*. That said, it does at least seem like these recent changes are a large step in the right direction. --- * in terms of what the standard approach should be, we live in an imperfect world and package management has been done "wrong" in many ecosystems, but in an ideal world I think the "correct" solution here should be: (1) If it's an…

I would like to be in (1) but I'm not a packaging person so I'll need to investigate more :( (2) I might make the message on installing llama.cpp maybe more informative - ie instead of re-directing people to the docs on manual compilation ie https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-... , I might actually print out a longer message in the Python cell entirely Yes we're working on Docker! https://hub…

> Yes we're working on Docker!

That will be nice too, though I was more just referring to simply doing something along the lines of this in your current build:

  docker run conanio/gcc11-ubuntu16.04 make clean -C llama.cpp etc etc...
(likely mounting & calling a sh file instead of passing individual commands)

---

Although I do think getting the ggml guys to support Conan (or monkey patching your own llama conanfile in before building) might be an easier route.

Re: DeepSeek-v3.1

#157

Post by irthomasthomas »

dmos62 wrote: post #145
noodletheworld wrote: post #128

Earlier quoted context omitted.

How can a benchmark be secret if you post it to an API to test a model on it? "We totally promise that when we run your benchmark against our API we won't take the data from it and use to be better at your benchmark next time" :P If you want to do it properly you have to avoid any 3rd party hosted model when you test your benchmark, which means you can't have GPT5, claude, etc. on it; and none of the benchmarks want…

How do you propose that would work? A pipeline that goes through query-response pairs to deduce response quality and then uses the low-quality responses for further training? Wouldn't you need a model that's already smart enough to tell that previous model's responses weren't smart enough? Sounds like a chicken and egg problem.

It just means that once you send your test questions to a model API, that company now has your test. So 'private' benchmarks take it on faith that the companies won't look at those requests and tune their models or prompts to beat them.

Re: DeepSeek-v3.1

#158

Post by scotty79 »

easygenes wrote: post #112
okasaki wrote: post #108

Earlier quoted context omitted.

My experience is that gpt-oss doesn't know much about obscure topics, so if you're using it for anything except puzzles or coding in popular languages, it won't do well as the bigger models. It's knowledge seems to be lacking even compared to gpt3. No idea how you'd benchmark this though.

Something I was doing informally that seems very effective is asking for details about smaller cities and towns and lesser points of interest around the world. Bigger models tend to have a much better understanding and knowledge base for the more obscure places.

I would really love if they figured out how to train a model that doesn't have any such knowledge baked it, but knows where to look for it. Maybe even has a clever database for that. Knowing this kind of trivia like this consistently of the top of your head is a sign of deranged mind, artificial or not.

Re: DeepSeek-v3.1

#159

Post by irthomasthomas »

seunosewa wrote: post #24
ivape wrote: post #18

Earlier quoted context omitted.

What formats? I thought the very schema of json is what allows these LLMs to enforce structured outputs at the decoder level? I guess you can do it with any format, but why stray from json?

Sometimes it will randomly generate something like this in the body of the text: ``` executeshell command echo "" >> novels/AI_Voodoo_Romance/chapter-1-a-new-dawn.txt ``` or this: ``` executeshell {"command": "pwd && ls -la"} ``` Prompting it to use the right format doesn't seem to work. Claude, Gemini, GPT5, and GLM 4.5, don't do that. To accomodate DeepSeek, the tiny agent that I'm building will have to support all…

Can't you use logit bias to help with this? Might depend how they are tokenized.

Re: DeepSeek-v3.1

#160

Post by antman »

sync wrote: post #49
amrrs wrote: post #9

Earlier quoted context omitted.

Any example or prompt you use to make this statment?

I'm doing coreference resolution and this model (w/o thinking) performs at the Gemini 2.5-Pro level (w/ thinking_budget set to -1) at a fraction of the cost.

Nice point. How did you test for coreference resolution? Specific prompt or dataset?
Post reply on HN