Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v4-flash-vision-exp

api-docs.deepseek.com

Post reply on HN
Print view View original

141–150 of 169 posts

Re: DeepSeek-v4-flash-vision-exp

#141

Post by [deleted] »

johndough wrote: post #132
knollimar wrote: post #72

Earlier quoted context omitted.

Downsizing a higher res image to lower res means the zoom will be blurry.

The order is: LLM issues tool call to read high res image -> harness sends high res image to server -> server downsizes it to 800x800 (blurry) -> LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image -> LLM issues tool call to read subimage -> harness sends subimage to server -> server does not resize the subimage because it is small already, so it is not blurry when…

[deleted]

Re: DeepSeek-v4-flash-vision-exp

#142

Post by bel8 »

kadoban wrote: post #139
bel8 wrote: post #122

Earlier quoted context omitted.

I disagree. It's not even that useful to train LLMs to read an ancient analog clock. Unless we're talking about AGI, I couldn't care less if an LLM is bad at things they won't be doing anyway. I'd rather focus training data on more useful tasks.

It's not that useful for a person to be able to read an analog clock either, but if a supposed-genius came to me and confidently gave the wrong answer, it would say something about their strengths/weaknesses in general. The whole point of these models is they're meant to be able to generalize fairly well, not just answer questions the got trained on.

Funny, I just ran the test myself and DS4 got it right first try.

Then I ran it 9 more times and it got right 9 out of 10 times.

Lesson learned to double check what I read on the internet.

Re: DeepSeek-v4-flash-vision-exp

#143

Post by knollimar »

johndough wrote: post #132
knollimar wrote: post #72

Earlier quoted context omitted.

Downsizing a higher res image to lower res means the zoom will be blurry.

The order is: LLM issues tool call to read high res image -> harness sends high res image to server -> server downsizes it to 800x800 (blurry) -> LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image -> LLM issues tool call to read subimage -> harness sends subimage to server -> server does not resize the subimage because it is small already, so it is not blurry when…

Then you have a separate issue where the LLM can't piece together 9 subimages well.

Re: DeepSeek-v4-flash-vision-exp

#144

Post by knollimar »

johndough wrote: post #128
knollimar wrote: post #123

Earlier quoted context omitted.

If the harness does it that's just like saying "please use a workaround". You'll lose fidelity and LLMs will lose the ability to count things or maintain relationships for schematics, etc

LLMs read images by splitting them up into e.g. 16x16 patches, which are then converted to embedding vectors and fed to the LLM, so from a technical point of view, feeding a big image as many 20x20 patches all at once is not too different from cropping subimages from the image, splitting those subimages into patches and feeding them to the LLM. Of course, the LLM has to be trained to understand that those images belo…

You say "it can be trained" but they fail at counting in my use cases, let alone maintaining symbolic relationships.

Can you point me at one that can understand a detailed block diagram? Frontier is fine, soliciting recommendations

Re: DeepSeek-v4-flash-vision-exp

#145

Post by bel8 »

cjg007 wrote: post #138

I've been using v4-flash without vision for this months — it's my go-to for code tasks. Now with vision, I'm wondering: if this model can do everything the text-only version does (plus see images), why keep the text-only one around? Is it just cost/latency? Or is there something text-only does better?

Yes, it adds vision to the already capable text-only LLM according to DS:

> This experimental multimodal model matches DeepSeek-V4-Flash on text capabilities—including agents, reasoning, and world knowledge.

https://api-docs.deepseek.com/news/news260821/

Re: DeepSeek-v4-flash-vision-exp

#146

Post by wolfgangK »

jerkstate wrote: post #80

I just ran my image recognition benchmark on it ("is this XXX public landmark"?) and it misses a lot that bytedance seed 2.1 turbo gets right; for example: Asked "Is this Salisbury Cathedral" and supplied a picture of Wells Cathedral, it answers "Yes, the west facade of Salisbury Cathedral". Bytedance seed 2.1 turbo correctly says no. Similar results for a picture of Manhattan Bridge sent as Brooklyn Bridge, Chartres…

I have zero interest in world knowledge for my LLMs but this got me wondering : are there RAGs for that kind of data ? How could a LLM like DeepSeek-v4-flash-vision-exp accurately answer you question with an indexed database of labeled landmark pictures (or even 3D models ?).

Re: DeepSeek-v4-flash-vision-exp

#147

Post by Aeroi »

embedding-shape wrote: post #29
wongarsu wrote: post #14

Earlier quoted context omitted.

For most use cases you can fix that in the harness. Just give the model a tool to request a crop of specific coordinates of any image it has in its context. Call the tool "zoom" and it should be intuitive for the model Maybe there are some use cases where you need high detail everywhere at once, but for OCR of small text and the like a zoom ability should be sufficient

For really dumb models I've also had success automatically cropping it into a grid of N images with the max size, then processing each cell individually, then once all been processed, do one final call with resized image + all other context previously generated per cell. Basically a workaround to the image dimension restrictions without loosing fidelity. Works well with even dumb 7B models. Can't remember if I stole…

i think the claude cookbook has a file that does this, called tiling.

Re: DeepSeek-v4-flash-vision-exp

#148

Post by RobertLong »

lukax wrote: post #134
RobertLong wrote: post #109

The benchmark results look promising when compared to Opus 4.8, but for agentic usecases it's lacking images as tool call result types. Giving the model a tool to take screenshots and verify its work is my main usecase for vision models, but this is more oriented towards "build a website that looks like this" type prompts. Hopefully we'll see this by release.

This is only a problem with OpenAI Chat Completions API. With OpenAI Responses API and Anthropic Messages API a tool output can be text, image or file.

Oh, you're right! From the responses API reference:

> For function_call_output / custom_tool_call_output items. The output of the tool call, either a plain string or a

> list of input_text / input_image content parts. With the deepseek-v4-flash-vision-exp model, input_image parts in

> the output are processed as real images; with other models they are replaced with a placeholder text.

Re: DeepSeek-v4-flash-vision-exp

#149

Post by ianmcgowan »

CooCooCaCha wrote: post #98
dghlsakjg wrote: post #93

Earlier quoted context omitted.

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

Is being asked to read a clock really a gotcha?

It is, apocryphally, a gotcha test for Gen-alpha. As is being able to return the correct change or make a phone call to a human being. Reminds me of the iRobot line: "Can you create a symphony?", "No. Can you?".

Re: DeepSeek-v4-flash-vision-exp

#150

Post by hedgehog »

knollimar wrote: post #5
ciberado wrote: post #4

DS being unable to precisely view Playwright screenshots is the only thing I really miss from Sonnet. This is promising. > Images are converted into tokens based on their dimensions, and these tokens are billed together with your text tokens. > Before inference, every image is automatically resized: > - Images with a total pixel count below roughly 384×384 are scaled up while preserving their aspect ratio. > - Larger…

Oof 800 by 800 kills a lot of use cases

Multi-turn, expose a tool that extracts crops. Works great.
Post reply on HN