Ironically, the pill counting example selected to showcase "the best vision model" can be easily solved with OpenCV template matching, a technology created 25 years ago.
GPT 5.6 Sol is the best "vision" model OpenAI ever released
41–50 of 194 posts
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#42I didn't expect Gemini 3.5 Flash to top basically every metric in this article.
In my practice Gemini models are far better than anything on the market in terms of vision, also it's worth to mention that current Gemini flash is 3.7, so it got 2 updates since 3.5 which beat GPT-5.6 Sol in this comparison.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#43I would love more vision benchmarks! Once I asked the model to inspect a completely black picture and it hallucinated a nice wooden kitchen wall. Took me some time to figure out where the kitchen came from...
I usually go to https://arena.ai/leaderboard/vision/pareto for a nice overview of current models.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#44Ironically, the pill counting example selected to showcase "the best vision model" can be easily solved with OpenCV template matching, a technology created 25 years ago.
I'm assuming you mean that this tech became available in OpenCV 25 years ago, but as it turns out, the underlying tech can be traced back much further, at least as far as 1977! :)
https://ieeexplore.ieee.org/document/1674847 G. J. Vanderbrug and A. Rosenfeld, “Two-Stage Template Matching,” IEEE Transactions on Computers, Vol. C-26, No. 4, pp. 384–393, April 1977. DOI: 10.1109/TC.1977.1674847
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#45The summary "There are still clear limits. Gemini 3.5 Flash remains a better practical choice [than GPT 5.6 Sol] for high-volume detection and counting in our benchmark, especially at its price." seems rather understated !
GPT 5.6 Sol was outperformed on all benchmarks by Gemini 3.5 Flash, apart from a single exception (OCR) where Fable was the winner.
Gemini 3.5 Flash not only outperformed GPT 5.6 Sol, but did so at 1/3 of the cost.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#46One of my friends (and BIL) own an architecture firm. They use AI to generate and quickly update renderings but they run into the equivalent of the 6 fingered hand problem. I sent him this article I wonder if the updated models can catch and fix mistakes made by previous models.
This article is about vision, not image output.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#47Ironically, the pill counting example selected to showcase "the best vision model" can be easily solved with OpenCV template matching, a technology created 25 years ago.
The point is that it's general. It can do this task and many other tasks and it doesn't need custom development like OpenCV does. Of course if you only want to count pills and you want it to be cheap/fast you're still better off using OpenCV.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#48It's vision capabilities poisoned my cucumber bed, misidentifying the malaise and having me spray them down with water, which only spread the fungus that gemini later informed me was actual cause, which I went and checked myself.
I hope that whatever was lost at GDM in the last few months, didn't include their extra focus on vision capabilities.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#49So far I haven't seen a single model succeeding at transcribing sheet music, but I just tested it again with 5.6 Sol and it nailed the small test case. Fluently reading music requires multiple years of training for most people, but I feel like accurately following the horizontal lines trips up vision models in particular.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#50In the third vision bench result, Sol is 100% correct but the expected has 1 error. Seems like an oversight. In the next bench, Sol looks like it’s correct again but the bboxes are rotated 90 degrees for some reason.
Seems to be due to the detection area being not fully accurate. Green vs red shows the difference between actual and detected