Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v4-flash-vision-exp

api-docs.deepseek.com

Post reply on HN
Print view View original

121–130 of 169 posts

Re: DeepSeek-v4-flash-vision-exp

#121

Post by CooCooCaCha »

dghlsakjg wrote: post #104
CooCooCaCha wrote: post #98

Earlier quoted context omitted.

Is being asked to read a clock really a gotcha?

If I had to hire an engineer and there was one that could one shot the wang algorithm, but couldn’t read an analog clock, I would have no problem hiring them. Also worth noting that both models got it wrong. Qwen made a mistake that humans very good at reading clocks would make. Deepseek made a mistake that a human who had just learned to read clocks would make.

It would be different if AI was known to be reliable but it isn’t, so this is less of a random failure and more a symptom of jagged intelligence.

And with every one of these there’s always an attempt to minimize the problem by saying it’s just one silly failure.

Re: DeepSeek-v4-flash-vision-exp

#122

Post by bel8 »

jubilanti wrote: post #101
dghlsakjg wrote: post #93

Earlier quoted context omitted.

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

Reading any analog clock at any time level (edit: and a non-noisy vector rendered image at that) is absolutely table stakes for an allegedly frontier flagship vision model. As much as 1:1 OCR. If the model can't do that, there's something wrong. Doesn't matter if it's memorized some random thing you think is esoteric but is in all the training data and benchmarks. The whole point of LLM/FMs vs good old fashioned ML i…

I disagree. It's not even that useful to train LLMs to read an ancient analog clock.

Unless we're talking about AGI, I couldn't care less if an LLM is bad at things they won't be doing anyway.

I'd rather focus training data on more useful tasks.

Re: DeepSeek-v4-flash-vision-exp

#123

Post by knollimar »

adastra22 wrote: post #115
knollimar wrote: post #72

Earlier quoted context omitted.

Downsizing a higher res image to lower res means the zoom will be blurry.

They’re not talking about zooming, hence the quotes.

If the harness does it that's just like saying "please use a workaround". You'll lose fidelity and LLMs will lose the ability to count things or maintain relationships for schematics, etc

Re: DeepSeek-v4-flash-vision-exp

#124

Post by nprateem »

Deepseek flash v4 july sounds like fun and games while you're looking at prices, but it routinely outputs incoherent rubbish and fails to call tools correctly.

Sadly oversold. I hold little hope for the vision model either now.

Re: DeepSeek-v4-flash-vision-exp

#125

Post by stronglikedan »

knollimar wrote: post #5
ciberado wrote: post #4

DS being unable to precisely view Playwright screenshots is the only thing I really miss from Sonnet. This is promising. > Images are converted into tokens based on their dimensions, and these tokens are billed together with your text tokens. > Before inference, every image is automatically resized: > - Images with a total pixel count below roughly 384×384 are scaled up while preserving their aspect ratio. > - Larger…

Oof 800 by 800 kills a lot of use cases

I don't know about a lot. Probably more like a few. I take a lot of screenshots for various reasons, and over 800 seems like I could have done a better job framing and cropping.

Re: DeepSeek-v4-flash-vision-exp

#126

Post by tjoff »

knollimar wrote: post #107
andai wrote: post #78

Earlier quoted context omitted.

They process the original image file with Python on the local device. (And I've seen the web chats do this with their "computer use" features too.) The really wild one is even blind models will do this and they'll try to run stats on the pixels to figure out what it looks like... the even wilder thing is that it kind of works!

If the API accepts only 800 by 800, the aegument youre making is "fix it in the harness". I don't think the n by n subgrid fixes this the way most harnesses do, as it'll fail to count things if you have more overlap and fail relatiomships if you have less

That seems weirdly specific?

And if you are counting things it should be trivial to note the position of your items and not double-count them, no?

Re: DeepSeek-v4-flash-vision-exp

#127

Post by johndough »

adastra22 wrote: post #115
knollimar wrote: post #72

Earlier quoted context omitted.

Downsizing a higher res image to lower res means the zoom will be blurry.

They’re not talking about zooming, hence the quotes.

Yes. When the LLM tries to read an image, it will be resized by DeepSeek's server to 800x800, which might be a bit blurry. The LLM will then crop a smaller image from the high resolution image (using e.g. the `convert` tool via bash) and will then read the small cropped image. This image will still be resized to 800x800 by DeepSeek's server, but since it is already small, there is no or little loss of quality.

Re: DeepSeek-v4-flash-vision-exp

#128

Post by johndough »

knollimar wrote: post #123
adastra22 wrote: post #115

Earlier quoted context omitted.

They’re not talking about zooming, hence the quotes.

If the harness does it that's just like saying "please use a workaround". You'll lose fidelity and LLMs will lose the ability to count things or maintain relationships for schematics, etc

LLMs read images by splitting them up into e.g. 16x16 patches, which are then converted to embedding vectors and fed to the LLM, so from a technical point of view, feeding a big image as many 20x20 patches all at once is not too different from cropping subimages from the image, splitting those subimages into patches and feeding them to the LLM. Of course, the LLM has to be trained to understand that those images belong together, but it can be done.

Re: DeepSeek-v4-flash-vision-exp

#129

Post by ndriscoll »

dghlsakjg wrote: post #93
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

I was literally working on an educational game for my kids last week where one of the activities is clock reading, and I ask codex to QA its Godot program via screenshots, so literally this exact scenario is something I was doing in a software engineering context. It can of course write code to figure out the angles to rotate by just fine, but it also needs to be able to figure out whether the whole picture comes together, whether the hand sprites are anchored on the clock face correctly with the right pivot, etc.

Re: DeepSeek-v4-flash-vision-exp

#130

Post by dannyw »

nprateem wrote: post #124

Deepseek flash v4 july sounds like fun and games while you're looking at prices, but it routinely outputs incoherent rubbish and fails to call tools correctly. Sadly oversold. I hold little hope for the vision model either now.

Are you using an API, or running locally? If so, are you running with a quant, or other 'optimisations'?

I've been using it via openrouter pretty heavily as my daily driver for the past week and loving it, have never experienced incoherent rubbish even at 500k+ contexts (that's usually way higher than I'd typically compact at), and tool calling reliability is better than Opus 5 in the Claude Code harness.

Modern Anthropic models frequently get tool calls wrong, invent non-existent references or SQL tables, or have gibberish CJK characters in the output, like out of nowhere. Of course, they're great at self-recovery after an incorrect tool call, but so is Deepseek v4 flash.

If you're running a quant, and esp with a quant'd KV cache, then yeah, not surprised if you're getting incoherent results; but you're not running the real/full model.

Also, which harness? Try something like Pi or OMP. Models perform better in these harnesses than Claude Code: https://www.databricks.com/blog/benchmarking-coding-agents-d...

The main reason to use Cladue Code is a subsidised Anthropic subscription. If you're on API rates, you should not use Claude Code; you pay more for worse results. Claude Code is sadly quite bloated these days, and comes with a lot of proprietary context window garage like claude design skills, claude.ai artifacts, etc that you probably don't use, and if you do, well, you can add it.

Post reply on HN