Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.1

api-docs.deepseek.com

Post reply on HN
Print view View original

41–50 of 273 posts

Re: DeepSeek-v3.1

#44

Post by hopelite »

theuurviv467456 wrote: post #30

[flagged]

This does not make any sense to me. “There”? “‘Nationalist’ () bans” of and by whom?

Dark propaganda opposed to what, light propaganda? The Chinese model being released is about keeping China down?

You seem very animated about this, but you would probably have more success if you tried to clarify this a bit more.

Re: DeepSeek-v3.1

#46

Post by refulgentis »

ivape wrote: post #18
seunosewa wrote: post #3

It's a hybrid reasoning model. It's good with tool calls and doesn't think too much about everything, but it regularly uses outdated tool formats randomly instead of the standard JSON format. I guess the V3 training set has a lot of those.

What formats? I thought the very schema of json is what allows these LLMs to enforce structured outputs at the decoder level? I guess you can do it with any format, but why stray from json?

In the modes in APIs, the sampling code essentially "rejects and reinference" any token sampled that wouldn't create valid JSON under a grammar created from the schema. Generally, the training is doing 99% of the work, of course, it's just "strict" means "we'll check it's work to the point a GBNF grammar created from the schema will validate."

One of the funnier info scandals of 2025 has been that only Claude was even close to properly trained on JSON file edits until o3 was released, and even then it needed a bespoke format. Geminis have required using a non-formalized diff format by Aider. Wasn't until June Gemini could do diff-string-in-JSON better than 30% of the time and until GPT-5 that an OpenAI model could. (Though v4a, as OpenAI's bespoke edit format is called, is fine because it at least worked well in tool calls. Geminis was a clown show, you had to post process regular text completions to parse out any diffs)

Re: DeepSeek-v3.1

#47

Post by dragonwriter »

refulgentis wrote: post #46
ivape wrote: post #18

Earlier quoted context omitted.

What formats? I thought the very schema of json is what allows these LLMs to enforce structured outputs at the decoder level? I guess you can do it with any format, but why stray from json?

In the modes in APIs, the sampling code essentially "rejects and reinference" any token sampled that wouldn't create valid JSON under a grammar created from the schema. Generally, the training is doing 99% of the work, of course, it's just "strict" means "we'll check it's work to the point a GBNF grammar created from the schema will validate." One of the funnier info scandals of 2025 has been that only Claude was eve…

> In the modes in APIs, the sampling code essentially "rejects and reinference" any token sampled that wouldn't create valid JSON under a grammar created from the schema.

I thought the APIs in use generally interface with backend systems supporting logit manipulation, so there is no need to reject and reinference anything; its guaranteed right the first time because any token that would be invalid has a 0% chance of being produced.

I guess for the closed commercial systems that's speculative, but all the discussion of the internals of the open source systems I’ve seen has indicated that and I don't know why the closed systems would be less sophisticated.

Re: DeepSeek-v3.1

#49

Post by sync »

amrrs wrote: post #9
coliveira wrote: post #7

Earlier quoted context omitted.

My personal experience is that it produces high quality results.

Any example or prompt you use to make this statment?

I'm doing coreference resolution and this model (w/o thinking) performs at the Gemini 2.5-Pro level (w/ thinking_budget set to -1) at a fraction of the cost.

Re: DeepSeek-v3.1

#50

Post by amelius »

guluarte wrote: post #17
hodgehog11 wrote: post #2

For reference, here is the terminal-bench leaderboard: https://www.tbench.ai/leaderboard Looks like it doesn't get close to GPT-5, Claude 4, or GLM-4.5, but still does reasonably well compared to other open weight models. Benchmarks are rarely the full story though, so time will tell how good it is in practice.

tbh companies like anthopic, openai, create custom agents for specific benchmarks

Aren't good benchmarks supposed to be secret?
Post reply on HN