Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v4-flash-vision-exp

api-docs.deepseek.com

Post reply on HN
Print view View original

151–160 of 169 posts

Re: DeepSeek-v4-flash-vision-exp

#151

Post by [deleted] »

bel8 wrote: post #140
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

Well it succeeded on first try for me: https://i.imgur.com/gljOYr9.png [Image 1] what time does the clock show? + Thought: 368ms The clock shows 8:10. - Blue hour hand: just past 8 - Green minute hand: pointing at 2 (10 minutes) - Red second hand: pointing at 5 (25 seconds) ▣ Plan · DeepSeek V4 Flash Vision Exp · 4.4s edit: I now ran the test 10 times total, fresh session every time, and DS4 got it right 9 out of 10…

[deleted]

Re: DeepSeek-v4-flash-vision-exp

#152

Post by leumon »

bel8 wrote: post #140
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

Well it succeeded on first try for me: https://i.imgur.com/gljOYr9.png [Image 1] what time does the clock show? + Thought: 368ms The clock shows 8:10. - Blue hour hand: just past 8 - Green minute hand: pointing at 2 (10 minutes) - Red second hand: pointing at 5 (25 seconds) ▣ Plan · DeepSeek V4 Flash Vision Exp · 4.4s edit: I now ran the test 10 times total, fresh session every time, and DS4 got it right 9 out of 10…

Try it on openrouter directly if you don't believe me that i got this result. Maybe image inputs behave differently in opencode, or it could even be that this agent harness uses different models for different tasks, not sure. Or maybe the reasoning (high) makes the difference.

Re: DeepSeek-v4-flash-vision-exp

#153

Post by efficax »

bel8 wrote: post #140
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

Well it succeeded on first try for me: https://i.imgur.com/gljOYr9.png [Image 1] what time does the clock show? + Thought: 368ms The clock shows 8:10. - Blue hour hand: just past 8 - Green minute hand: pointing at 2 (10 minutes) - Red second hand: pointing at 5 (25 seconds) ▣ Plan · DeepSeek V4 Flash Vision Exp · 4.4s edit: I now ran the test 10 times total, fresh session every time, and DS4 got it right 9 out of 10…

fwiw the clock reads 8:09:25 not 8:10:25 although it would be passable to read it as 8:10 if there were no second hand

Re: DeepSeek-v4-flash-vision-exp

#154

Post by dghlsakjg »

atomicnumber3 wrote: post #106
dghlsakjg wrote: post #93

Earlier quoted context omitted.

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

It's because the messaging for what the point of these things is supposed to be is all over the place. Ask 10 different people and you'll get 10 different answers: - A superintelligence that will usher in an age of human enlightenment - A superintelligence that will usher in an age of human enslavement - A really cool way to rake in trillion of rich VC/investor money by promising you're building a superintelligence t…

Where did deepseek claim any of that?

Re: DeepSeek-v4-flash-vision-exp

#155

Post by johndough »

knollimar wrote: post #144
johndough wrote: post #128

Earlier quoted context omitted.

LLMs read images by splitting them up into e.g. 16x16 patches, which are then converted to embedding vectors and fed to the LLM, so from a technical point of view, feeding a big image as many 20x20 patches all at once is not too different from cropping subimages from the image, splitting those subimages into patches and feeding them to the LLM. Of course, the LLM has to be trained to understand that those images belo…

You say "it can be trained" but they fail at counting in my use cases, let alone maintaining symbolic relationships. Can you point me at one that can understand a detailed block diagram? Frontier is fine, soliciting recommendations

For counting, there are specialized counting models, e.g. https://huggingface.co/spaces/MengqiLei/count-anything-demo

I tried to parse hand-drawn ER diagrams in the past and did not have much success with any model, frontier or otherwise. If you have annotated data, I'd recommend finetuning a recent (dense) VLM, but don't expect 100% accuracy. https://unsloth.ai/docs/basics/vision-fine-tuning

Re: DeepSeek-v4-flash-vision-exp

#156

Post by johnfn »

dghlsakjg wrote: post #93
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

How is this a "gotcha" question? "gotcha" implies there's some sort of trick. This is just... a question.

Re: DeepSeek-v4-flash-vision-exp

#157

Post by 1saadcodes »

5kyn3t wrote: post #37

For what do you guys use vision in those models? surveillance is the obvious use case... but are there some "nicer" ways to use it?

Screenshots, diagrams, photos of things I’m trying to fix. Things that I'd normally have to manually describe since OCR is limited

Re: DeepSeek-v4-flash-vision-exp

#159

Post by nwienert »

knollimar wrote: post #143
johndough wrote: post #132

Earlier quoted context omitted.

The order is: LLM issues tool call to read high res image -> harness sends high res image to server -> server downsizes it to 800x800 (blurry) -> LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image -> LLM issues tool call to read subimage -> harness sends subimage to server -> server does not resize the subimage because it is small already, so it is not blurry when…

Then you have a separate issue where the LLM can't piece together 9 subimages well.

Insert the famous Louie CK phones on planes bit. Sand is thinking, working, coding and now looking for you, for pennies an hour... but "oof" it's not good enough.

Re: DeepSeek-v4-flash-vision-exp

#160

Post by dghlsakjg »

mejutoco wrote: post #103
dghlsakjg wrote: post #93

Earlier quoted context omitted.

I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…

It is like asking a politician how much a coffee costs, to show how disconnected they are from common people. Super intelligence not being able to read a simple analog clock does the same. > Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot This is about a _vision_ model.

An _experimental_ _flash_ _vision_ variant of a model optimized for speed and cost.

The problem with judging a politician for not knowing what a cup of coffee costs is that it doesn’t tell you anything about his capabilities or why he doesn’t know. Maybe the politician is incredibly competent, but doesn’t drink coffee. Maybe he gets up at 5 to start working and makes a pot of coffee himself before the coffee shops open so knows what a bag of coffee costs but not a cup.

The issue with a gotcha question is that it is a shortcut to confirmation bias. There is no single correct answer to what a cup of coffee costs, and not knowing the answer doesn’t tell you as much as people want it to. I can go to Starbucks and get a cup of “coffee” that costs $10, or I can go to the corner store and get one for $.99. I can make one at home for $.50 or less. There’s no right answer.

Scroll through the comments and there is someone that tried the same clock test on this model and it answered correctly 9/10 times. That’s why catching someone or something in a gotcha once is kinda bad signal.

Can this model tell time? Yes, and also no. Does a cup of coffee cost .99? Yes and also no.

Post reply on HN