31–40 of 287 posts
Anthropic is the new king. This isn't even Claude 3.5 Opus and it's already super impressive. The speed is insane. I asked it "Write an in depth tutorial on async programming in Go" and it filled out 8 sections of a tutorial with multiple examples per section before GPT4o got to the second section and GPT4o couldn't even finish the tutorial before quitting. I been a fan of Anthropic models since Claude 3. Despite the…
fyi for anyone testing this in their product, their docs are wrong, it's claude-3-5-sonnet-20240620, not claude-3.5-sonnet-20240620.
Recently canceled openai subscription because too much lag and crashes. Switched to Gemini because their webinterface is faster and rock solid. Makes me think the openai backend and frontend engineers don't know what they are doing compared to the google engineers.
Earlier quoted context omitted.
That's right! Breakdown here for the API: https://www.anthropic.com/pricing#anthropic-api
why would someone pick opus at these prices?
Awesome!! Seems like the API docs haven't been updated yet: https://docs.anthropic.com/en/docs/about-claude/models
GPT-4o's answer was excellent and highly detailed, recognizing essentially all the relevant aspects of the image [GPT4o]. Claude 3 Sonnet was correct on a general level, but its answer was much less detailed and showed more uncertainty in the form of "A or B" sentences [CL3]. Gemini's answer was, well, hilariously wrong [GEM].
I just tried this with Claude 3.5 Sonnet and it did very well. Its answer was still not as detailed as GPT-4o's, but it did ask me if I want it to elaborate on any aspect of the image [CL35].
I think this was an interesting experiment because street plan CAD diagrams probably aren't very common in the training data of these models.
--
[GPT4o] https://gist.github.com/jdahlstrom/844bda8ac76a5c3248c863d20...
[CL3] https://gist.github.com/jdahlstrom/ecccf31c8305f82519f27af53...
[GEM] https://gist.github.com/jdahlstrom/2e12a966c0d603a7b1446ba08...
[CL35] https://gist.github.com/jdahlstrom/60ca9352630934bec6e2f4e37...
A while ago I tested the image recognition skills of GPT-4o, Claude 3, and Gemini using a random street plan diagram I happened to have in my downloads folder ( https://i.imgur.com/9WZpK0L.png ). It's a top-down CAD rendering showing the planned renovation of a street in my neighborhood in Tampere, Finland. I uploaded the image and simply asked each model "What can you tell me about the attached picture?" GPT-4o's an…
A while ago I tested the image recognition skills of GPT-4o, Claude 3, and Gemini using a random street plan diagram I happened to have in my downloads folder ( https://i.imgur.com/9WZpK0L.png ). It's a top-down CAD rendering showing the planned renovation of a street in my neighborhood in Tampere, Finland. I uploaded the image and simply asked each model "What can you tell me about the attached picture?" GPT-4o's an…
GPT-4o 30.7
Claude 3.5 Sonnet 27.9
Claude 3 Opus 27.3
Llama 3 Instruct 70B 24.0
Gemini Pro 1.5 0514 22.3
Mistral Large 17.7
Qwen 2 Instruct 72B 15.6
