Penny sample shown looks like failed EXIF orientation registered by the model/harness. The coins are correctly marked, it's rotated 90 degrees.
Good call out, I noticed the same rotation issue but pointing at EXIF data sounds about right.
GPT 5.6 Sol is the best "vision" model OpenAI ever released
171–180 of 194 posts
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#172Earlier quoted context omitted.
Hi! I’m the author of this blog. I had the same intuition, but together with the OpenAI team we figured out that the issue was image resolution. GPT-5.6 doesn’t handle large images well.
OpenAI team sounds like they've misidentified the root cause for this particular case then.
Haha, ya at least to some degree, those boxes are in the right position, but rotated.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#173Earlier quoted context omitted.
but 3.7 flash is expensive for img inputs no ?
As usual for something so simple, Google's docs seem unclear: https://ai.google.dev/gemini-api/docs/pricing For 3, pricing for image tokens was the same as text tokens. Since they don't indicate a difference on 3.7, I would assume the same holds. And as far as I know the number of image tokens is the same for both (depending on the detail level you pick, but it's generally around 1k per image). So they're about the s…
mystery to me is how the image tokens are calculated? 1MB is 1000 tokens ?
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#174Earlier quoted context omitted.
3.1 Pro is not deprecated!
https://ai.google.dev/gemini-api/docs/deprecations That link shows 3.1 pro listed as deprecated with no replacement model.
No, that's the "preview" version (gemini-3.1-pro-preview) aka the beta/early version before the official release of 3.1 Pro.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#175Dumb question: When your testing "ChatGPT 5.6 Sol" are you testing an actual LLM or some visual pre-processor stack that sits in front of it (along with a maybe a bunch of other such pre-processors) that is bundled into what's call "ChatGPT 5.6 Sol"? I.e. last I checked LLMs had a something like a 30-100K token alphabet to work with and it's hard to imagine how throwing pixels arrays at one directly would work.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#176A better headline would be "Gemini 3.5 Flash is the best vision model". It tops almost every single benchmark shown in the article.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#177[flagged]
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#178From our experiments it’s the best video captioning model in the world by a mile. This was not the case a year ago. When reasoning got introduced a year ago to GPT 5, on average the model performed worse than GPT4-o for short video clip captioning (Ie hallucinating actions that didn’t happen). The old GPT 5 was extremely finicky in terms of fps sample rate. The other SOTA LLMs (like Gemini Pro) have clearly been opti…
Have you checked versus more recent Gemini models like 3.5 or perhaps 3.7?
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#179Dumb question: When your testing "ChatGPT 5.6 Sol" are you testing an actual LLM or some visual pre-processor stack that sits in front of it (along with a maybe a bunch of other such pre-processors) that is bundled into what's call "ChatGPT 5.6 Sol"? I.e. last I checked LLMs had a something like a 30-100K token alphabet to work with and it's hard to imagine how throwing pixels arrays at one directly would work.
I'm ~95% certain that images are tokenized, just like regular text, and fed directly in; that's the 'multimodal' part of these models. Now how this tokenizing works I don't know, and there might be some level of preprocessing, but it's certainly not converting the image into text and feeding it in to a regular LLM.
Re: GPT 5.6 Sol is the best "vision" model OpenAI ever released
#180[flagged]