Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v4-flash-vision-exp

api-docs.deepseek.com

Post reply on HN
Print view View original

81–90 of 169 posts

Re: DeepSeek-v4-flash-vision-exp

#81

Post by wolttam »

nubg wrote: post #76
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

welp, damning indictment. not sure if that means DS is super crap, or qwen is super good

Neither. Performance of all models is incredibly spikey.

Re: DeepSeek-v4-flash-vision-exp

#82

Post by dominotw »

5kyn3t wrote: post #37

For what do you guys use vision in those models? surveillance is the obvious use case... but are there some "nicer" ways to use it?

when i am learning i draw what i undestand in a picture and ask ai to correct me. i want ai to watch over me while i am learning.

this is such good way to learn something for me.

Re: DeepSeek-v4-flash-vision-exp

#83

Post by andai »

ttul wrote: post #79
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

A good share of humanity would have also gotten this question wrong!

Yeah, I heard most kids these days can't read analog clocks either.

I can't actually remember where I learned to read a clock, it might have actually been in school. I guess that means they don't teach it anymore. (Everyone's phone shows the time anyway...)

Re: DeepSeek-v4-flash-vision-exp

#84

Post by ttul »

The DeepSWE benchmark they report (59.3%) overlaps with the confidence interval of 5.6-Sol Medium (61% +/- 2%), but likely at 1/18th the cost (they did not report the DeepSWE benchmark cost, but v4-flash had this cost ratio against Sol Medium).

Interestingly, v4-flash performed several points worse on DeepSWE at 53% +/- 4%. Assuming this result is verified by DeepSWE officially, it would mark a significant advance in Pareto cost/performance on software engineering tasks.

Re: DeepSeek-v4-flash-vision-exp

#85

Post by emosenkis »

leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

This is not a normal looking clock - most clocks have either one color for all hands (second hand is thinnest and maybe also longest) or one color for hour/minute and one for second. I know that the hand lengths and thicknesses on this image are correct but for some reason I, a totally human person who grew up when analog clocks were still common, see this and think the hand on the 5 is the minute hand. How does the AI do if you just make all the hands black?

Re: DeepSeek-v4-flash-vision-exp

#86

Post by mdp2021 »

ttul wrote: post #79
leumon wrote: post #74

It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…

A good share of humanity would have also gotten this question wrong!

It's been four years that we are looping those

"The professional failed its task!" // "Laymen would have failed it too".

Which makes no sense.

Re: DeepSeek-v4-flash-vision-exp

#88

Post by paytonjjones »

ttul wrote: post #84

The DeepSWE benchmark they report (59.3%) overlaps with the confidence interval of 5.6-Sol Medium (61% +/- 2%), but likely at 1/18th the cost (they did not report the DeepSWE benchmark cost, but v4-flash had this cost ratio against Sol Medium). Interestingly, v4-flash performed several points worse on DeepSWE at 53% +/- 4%. Assuming this result is verified by DeepSWE officially, it would mark a significant advance in…

The closer comparison would be 5.6-Luna. On DeepSWE at Xhigh it's 57% at 1/6 the cost of Sol M, on Max it's 67% at 1/3rd the cost.

Still an advance, I just thought it worthy to note Sol isn't nearly as impressive on the cost/performance frontier as discounted Luna.

Post reply on HN