131–140 of 169 posts
will this be open weights?
This is something I would like to know as well. But if not, does anybody know a recommended way to attach vision to deepseek flash (on a self-hosted infrastructure)?
Deepseek is usually very good with open weights, they don't necessarily drop immediately, sometimes in a few hours, sometimes in a couple days.
Earlier quoted context omitted.
Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.
Downsizing a higher res image to lower res means the zoom will be blurry.
LLM issues tool call to read high res image ->
harness sends high res image to server ->
server downsizes it to 800x800 (blurry) ->
LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image ->
LLM issues tool call to read subimage ->
harness sends subimage to server ->
server does not resize the subimage because it is small already, so it is not blurry when finally ingested by the LLMThe benchmark results look promising when compared to Opus 4.8, but for agentic usecases it's lacking images as tool call result types. Giving the model a tool to take screenshots and verify its work is my main usecase for vision models, but this is more oriented towards "build a website that looks like this" type prompts. Hopefully we'll see this by release.
It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…
Earlier quoted context omitted.
Yeah I've seen it a lot. It goes through the effort, unasked, of pulling screenshots off a connected device and then it's like... Oh shit yeah I can't see.
It's doing it's best to accomplish whatever task you've thrown at it. It's expecting you to have done at least something besides select DS4 on Ollama, essentially.
It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…
I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock. Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I gu…
It doesn’t have to be a negative thing! Simon’s pelican on a bicycle prompt is an example of a “gotcha” question.
Is it just cost/latency? Or is there something text-only does better?
Earlier quoted context omitted.
Reading any analog clock at any time level (edit: and a non-noisy vector rendered image at that) is absolutely table stakes for an allegedly frontier flagship vision model. As much as 1:1 OCR. If the model can't do that, there's something wrong. Doesn't matter if it's memorized some random thing you think is esoteric but is in all the training data and benchmarks. The whole point of LLM/FMs vs good old fashioned ML i…
I disagree. It's not even that useful to train LLMs to read an ancient analog clock. Unless we're talking about AGI, I couldn't care less if an LLM is bad at things they won't be doing anyway. I'd rather focus training data on more useful tasks.
The whole point of these models is they're meant to be able to generalize fairly well, not just answer questions the got trained on.
It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png I asked it "what time does the clock show?" (both on reasoning: high) DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes.…
https://i.imgur.com/gljOYr9.png
[Image 1] what time does the clock show?
+ Thought: 368ms
The clock shows 8:10.
- Blue hour hand: just past 8
- Green minute hand: pointing at 2 (10 minutes)
- Red second hand: pointing at 5 (25 seconds)
▣ Plan · DeepSeek V4 Flash Vision Exp · 4.4s
I'll leave a copy of the clock image for posterity here in case anyone wants to test it themselves: https://i.imgur.com/BQsfa3R.png
