The evidence given really doesn't justify the conclusion. Maybe it suggests 2.5 Pro might be better if you're asking it to build Javascript apps from scratch, but that hardly equates to "It's better at coding". Feels like a lot of LLM articles follow this pattern, someone running their own toy benchmarks and confidently extrapolating broad conclusions from a handful of data points. The SWE-Bench result carries a bit more weight but even that should be taken with a pinch of salt.
Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
61–70 of 336 posts
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#62In before people post contradictory anecdotes. It would be more helpful if people posted the prompt, and the entire context, or better yet the conversation, so we can all judge for ourselves.
This is also compounded by the fact that LLMs are not deterministic, every response is different for the same given prompt. And people tend to judge on one off experiences.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#63I was using gemini 2.5 pro yesterday and it does seem decent. I still think claude 3.5 is better at following instruction then the new 3.7 model which just goes ham messing stuff up. Really disappointed by Cursor and the Claude CLI tool, for me they create more problems then fix. I cant figure out how to use them on any of my projects with out them ruining the project and creating terrible tech debt. I really like th…
Have you tried wind surf? I’ve been really enjoying it and wondering if they do something on top to make it work better. The AI definitely still gets into weird rabbit holes and sometimes even injects security bugs (kept trying to add sandbox permissions for an iframe), but at least for UI work it’s been an accelerant.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#64Earlier quoted context omitted.
I think O1 Pro Mode is so infrequently used by others (because of the price) so I've just started added "besides O1 Pro Mode, if you have access" in my head when someone says "This is the best available model for X". It really is miles ahead of anything else so far, but also really pricey so makes sense some people try to find something close to it with much lower costs.
Yeah its not even close. In my mind, the 200$ a month could be 500 and I would still pay for it. There are many technical problems I have ran into, where I simply would not have solved the problem without it. I am building more complicated software than I ever have, and I have 10+ years of engineering experience in big tech
If you are in a developing country and making $500-$1000 a month doing entry level coding work then $200 is crazy. On the other hand, your employment at this point is entirely dependent on your employer having no idea what is going on, or being really nice to you. I've also heard complaints from people, in the United States, about not wanting to pay $20 a month for ChatGPT. If the work you are doing is that low value, you probably shouldn't be on a computer at all.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#65Theo video detected = opinion rejected Also I generally dislike thinking models for coding and prefer faster models, so if you have something easy gemini 2.0 is good
Absolute golden age YouTube brain rot. I had to disable the youtube sidebar with a custom style because just seeing these thumbnails and knowing some stupid schmuck is clicking on them like an ape when they do touchscreen experiments really lowers my mood.
If you find youtubers talking about it, they all fully agree that making these thumbnails is soul draining and they are totally aware how stupid they are. But they are also aware that click-through rates fall off a cliff when you don't use them. Humans are mostly dumb, it's up to you if you want to use it to your advantage or to your detriment.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#66Useful article but I would rather see comparisons where it takes a codebase and tries to modify it given a series of instructions rather than attempting to zero-shot implementations of games or solving problems. I feel like it fits better the real use cases of these tools.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#67Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…
Break it down into smaller problems.
Or ask an AI to do it?
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#68that guy Theo-t3 is so strange for my taste :)
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#69Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…
Convert the GTK 3 and GTK 4 API documentation into a single `.txt` file each.
Upload one of your platform-specific C++ file's source, along with the doc `.txt` into your LLM of choice.
Either ask it for a conversion function-by-function, or separate it some other way logically such that the output doesn't get truncated.
Would be surprised if this didn't work, to be honest.
Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison
#70The evidence given really doesn't justify the conclusion. Maybe it suggests 2.5 Pro might be better if you're asking it to build Javascript apps from scratch, but that hardly equates to "It's better at coding". Feels like a lot of LLM articles follow this pattern, someone running their own toy benchmarks and confidently extrapolating broad conclusions from a handful of data points. The SWE-Bench result carries a bit…
> The SWE-Bench result carries a bit more weight
Although I have issues with it (few benchmarks are perfect), I tend to agree. Gemini's 63.8 from Sonnet's 62.3 isn't a huge jump though. To Gemini's credit, it solved a bug in my PyTorch code yesterday that o1 (through the web app) couldn't (or at least didn't with my prompts).