Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT 5.6 Sol is the best "vision" model OpenAI ever released

blog.roboflow.com

Post reply on HN
Print view View original

101–110 of 194 posts

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#101

Post by SkalskiP »

weli wrote: post #3

Anecdotal, opinion: Gpt is really good in vision stuff, or at least their MoE seems to be really cohesive. From my experience Claude models can be really good at language but the moment they need to look at a picture and decide why the design is not good what parts need improvement it degrades a lot. My easiest benchmark is giving them a screenshot of a feature in my app and tell it "identify non-normative UI blocks…

Hi! I’m the author of this blog. GPT-5.6 is much better at vision than previous GPT versions, but it’s still much weaker than Gemini 3.5 Flash or Gemini 3.7 Flash, which was released last week. One interesting approach is to use Gemini through a tool call.

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#102

Post by lwarfield »

I currently have fable organize a bunch of 5.6 sol agents when working on my personal projects. This makes me wonder if I should add something along the lines of "For tasks that involve visual analysis, have gemini 3.7 look at images generated."

Overall I've been hooked on using agents from different companies for what they are best at (Thanks to Theo). Fable is expensive, but unmatched for planning and top level organization of other agents. Sol is fast, will persistantly go after goals (sometimes to its detriment), and does well with computer use.

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#103

Post by SkalskiP »

bearjaws wrote: post #51

It is funny to me seeing Sol used for what a "traditional" AI model can do already (counting pills). We have vision models for our pharmacy and I could never imagine taking the latency hit to use a Sol in our robotics, it would be likely 25-50x slower.

Hi! I’m the author of this blog.

I’m evaluating these VLMs to figure out which ones are good enough to auto-annotate my data, so I can fine-tune my detector.

I wrote a bit more about this here: https://x.com/skalskip92/status/2080334344061694429?s=20

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#104

Post by 1saadcodes »

5.6 Sol looks nice, but the Gemini 3.5 Flash comparison is interesting. It’s cheaper and still came out ahead on detection and counting, which doesn't really give me much of a reason to use Sol since Flash is much cheaper and hence much easier to scale. Not to mention we now have 3.6 Flash too

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#105

Post by SkalskiP »

adroitboss wrote: post #29

I didn't expect Gemini 3.5 Flash to top basically every metric in this article.

Hi! I’m the author of this blog. I wrote it 4 weeks ago, and it’s already a bit outdated. Gemini 3.7 Flash came out last week, and considering the price, it’s easily the best vision model right now: https://x.com/skalskip92/status/2088032652301304121?s=20

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#106

Post by dannyw »

MrBuddyCasino wrote: post #53
HarHarVeryFunny wrote: post #45

The summary "There are still clear limits. Gemini 3.5 Flash remains a better practical choice [than GPT 5.6 Sol] for high-volume detection and counting in our benchmark, especially at its price." seems rather understated ! GPT 5.6 Sol was outperformed on all benchmarks by Gemini 3.5 Flash, apart from a single exception (OCR) where Fable was the winner. Gemini 3.5 Flash not only outperformed GPT 5.6 Sol, but did so at…

Yeah I was thinking about giving Luna a go with my PDF data extraction, but I think I‘ll stay on Gemini. It does a very good job.

Gemini is honestly an excellent LLM with many capability strengths.

For example, 3.7 Flash is #1 on MMLU Pro and AA’s agentic spreadsheets/docs benchmark, etc. Yes, beating Fable.

Agentic coding is only one dimension.

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#108

Post by MostlyStable »

SkalskiP wrote: post #87
HarHarVeryFunny wrote: post #45

The summary "There are still clear limits. Gemini 3.5 Flash remains a better practical choice [than GPT 5.6 Sol] for high-volume detection and counting in our benchmark, especially at its price." seems rather understated ! GPT 5.6 Sol was outperformed on all benchmarks by Gemini 3.5 Flash, apart from a single exception (OCR) where Fable was the winner. Gemini 3.5 Flash not only outperformed GPT 5.6 Sol, but did so at…

Hi, I’m the author of this blog post. I wrote it about 4 weeks ago, and the VLM world is moving so fast that it’s already kinda outdated. I think Gemini 3.7 Flash might be a better choice now, especially when you factor in the price. Here’s a comparison of the best low-cost models I put together last week. What’s crazy is that Gemini 3.7 Flash is now 50% off on OpenRouter, and this chart doesn’t even account for that…

Curious why you didn't try Gemini 3 pro? That is the model I've been using for OCR entry of handwritten datasheets (JPGS of datasheets, structured JSON output). At my scale, the cost of 3 pro is basically not an issue, but if there are improvements in quality, I'd definitely be willing to explore other models

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#109

Post by mhaberl »

SkalskiP wrote: post #103
bearjaws wrote: post #51

It is funny to me seeing Sol used for what a "traditional" AI model can do already (counting pills). We have vision models for our pharmacy and I could never imagine taking the latency hit to use a Sol in our robotics, it would be likely 25-50x slower.

Hi! I’m the author of this blog. I’m evaluating these VLMs to figure out which ones are good enough to auto-annotate my data, so I can fine-tune my detector. I wrote a bit more about this here: https://x.com/skalskip92/status/2080334344061694429?s=20

Did you evaluate any that could be self-hosted (or at least ow models), if so which one is the best you seen?

Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released

#110

Post by Damjanski »

HarHarVeryFunny wrote: post #45

The summary "There are still clear limits. Gemini 3.5 Flash remains a better practical choice [than GPT 5.6 Sol] for high-volume detection and counting in our benchmark, especially at its price." seems rather understated ! GPT 5.6 Sol was outperformed on all benchmarks by Gemini 3.5 Flash, apart from a single exception (OCR) where Fable was the winner. Gemini 3.5 Flash not only outperformed GPT 5.6 Sol, but did so at…

thats so helpful - tysm
Post reply on HN