Live data from Hacker News

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

composio.dev

181–190 of 336 posts

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#181
post #85

Earlier quoted context omitted.

I mean responses like this one: I understand the desire for a simple or unconventional solution, however there are problems with those solutions. There is likely no further explanation that will be provided. It is best that you perform testing on your own. Good luck, and there will be no more assistance offered. You are likely on your own. This was about a SOCKS proxy which was leaking when the OpenVPN provider was d…

That sounds like you asked for plans to a perpetual motion machine.

In the past at least ChatGPT would reply "Building a perpetual motion machine sounds like a great idea, here are some plans on how to get started. Let me know if you need help with any of the details".

This has been a problem with using LLMs for design and brainstorming problems in general. It is virtually impossible to make them go "no, that's a stupid idea and will never work", or even to push back and give serious criticism. No matter what you ask they're just so eager to please.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#182
post #91

Earlier quoted context omitted.

> The Gemini always try to do unrelated changes. For example, I asked, separately, for small changes such as remove this unused function For anything like this, I don’t understand trying to invoke AI. Just open the file and delete the lines yourself. What is AI going to do here for you? It’s like you are relying 100% on AI when it’s a tool in your toolset.

Playing devils advocate here, it's because removing a function is not always as simple as deleting the lines. Sometimes there are references to that function that you forgot about that the LLM will notice and automatically update for you. Depending on your prompt it will also go find other references outside of the single file and remove those as well. Another possibility is that people are just becoming used to inte…

Any IDE will do this for you a hundred times better than current LLMs.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#183
post #46
post #29

I was using gemini 2.5 pro yesterday and it does seem decent. I still think claude 3.5 is better at following instruction then the new 3.7 model which just goes ham messing stuff up. Really disappointed by Cursor and the Claude CLI tool, for me they create more problems then fix. I cant figure out how to use them on any of my projects with out them ruining the project and creating terrible tech debt. I really like th…

My only experience is via cursor but I'd agree in that context 3.7 is worse than 3.5. 3.7 goes crazy trying to fix any little linter errors and often gets confused and will just hammer away, making things worse until I stop generation. I think if I let it continue it would probably proposed rm -rf and start over at some point :). Again, this could just have to do with the way cursor is prompting it.

believe it or not, i had cursor in yolo mode just for fun recently and 3.7 rm -rf'd my home folder :(

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#184

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

You guys really need a Docker build. This dependency chain with submodules is a nightmare.

Alternative perspective: you kids with your Docker builds need to roll up your sleeves and learn how to actually compile a semi-complicated project if you expect to be able to contribute back to said project.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#186

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

GTK is an abomination of a UI framework. You should be looking for another way to manage your UI entirely, not trying to keep up with the joneses, who will no doubt release something new in short order and set yet another hoop to jump through, without providing any benefit to you at all. It's openly hostile to not consider the upgrade path of existing users, and make things so difficult that it requires huge lifts ju…

>> GTK is an abomination of a UI framework.

I respectfully disagree with that. I think it's a solid UI framework, but...

>> It's openly hostile to not consider the upgrade path of existing users, and make things so difficult that it requires huge lifts just to upgrade versions of something like a UI framework.

I completely agree with you on that. We barely use any UI widgets so you'd think the port would be easy enough. I went through most of the checklist for changes you can make while still using GTK3 in prep for 4. "Don't access event structure members directly, use accessor functions." OK I made that change which made the code a little more verbose. But then they changed a lot of the accessor functions going from 3 to 4. Like WTF? I'm just trying to create a menu but menus don't exist any more - you make them out of something else. Oh and they're not windows they are surfaces. Like why?

I hope with some of the big architectural changes out of the way they can stabilize and become a nice boring piece of infrastructure. The talk of regular API changes every 3-5 years has me concerned. There's no reason for that.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#187
post #185

They nerfed it as of sunday March 30, a lot of people noticed performance drop and it rambling. https://x.com/nisten/status/1906141823631769983 Would be nice if this review actually wrote exactly when they conducted their test.

("it" being the Claude 3.7, not the Gemini)

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#188
post #29

I was using gemini 2.5 pro yesterday and it does seem decent. I still think claude 3.5 is better at following instruction then the new 3.7 model which just goes ham messing stuff up. Really disappointed by Cursor and the Claude CLI tool, for me they create more problems then fix. I cant figure out how to use them on any of my projects with out them ruining the project and creating terrible tech debt. I really like th…

Whenever I read about LLMs or try to use them, I feel like I am asleep in a dream where two contradicting things can be true at the same time. On one hand, you have people claiming "AI" can now do SWE tasks which take humans 30 minutes or 2 hours and the time doubles every X months so by Y year, SW development will be completely automated. On the other hand, you have people saying exactly what you are saying. Usually…

> people claiming "AI" can now do SWE tasks which take humans 30 minutes or 2 hours

Yes people claim that but everyone with a grain of salt in his mind know this is not true. Yes, in some cases an LLM can write from scratch a python or web demo-like application and that looks impressive but it is still far from really replacing a SWE. Real world is messy and requires to be careful. It requires to plan, do some modifications, get some feedback, proceed or go back to the previous step, think about it again. Even when a change works you still need to go back to the previous step, double check, make improvements, remove stuff, fix errors, treat corner cases.

The LLM doesn't do this, it tries to do everything in one single step. Yes, even when it is in "thinking" mode, in thinks ahead and explore a few possibilities but it doesn't do several iterations as it would be needed in many cases. It does a first write like a brilliant programmers may do in one attempt but it doesn't review its work. The idea of feeding back the error to the LLM so that it will fix it works in simple cases but in most common cases, where things are more complex, that leads to catastrophes.

Also when dealing with legacy code it is much more difficult for an LLM because it has to cope with the existing code with all its idiosincracies. One need in this case a deep understanding of what the code is doing and some well-thought planning to modify it without breaking everything and the LLM is usually bad as that.

In short, LLM are a wonderful technology but they are not yet the silver bullet someone pretends it to be. Use it like an assistant to help you on specific tasks where the scope is small the the requirements well-defined, this is the domain where it does excel and is actually useful. You can also use it to give you a good starting point in a domain you are nor familiar or it can give you some good help when you are stuck on some problem. Attempt to give the LLM a stack to big or complex are doomed to failure and you will be frustrated and lose your time.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#189
post #112

Earlier quoted context omitted.

Convert the GTK 3 and GTK 4 API documentation into a single `.txt` file each. Upload one of your platform-specific C++ file's source, along with the doc `.txt` into your LLM of choice. Either ask it for a conversion function-by-function, or separate it some other way logically such that the output doesn't get truncated. Would be surprised if this didn't work, to be honest.

Do you really need to provide the docs? I would have imagined that those docs are included in their training sets. There is even a guide on how to migrate from GTK3 to GTK4, so this seems to be a low-hanging fruit job for an LLM iff they are okay for coding.

It moves the model from 'sorta-kinda-maybe-know-something-about-this' to being grounded in the context itself. Huge difference for anything underrepresented (not only obscure packages and not-Python not-JS languages).

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#190
The loser in the AI model competition appears to be... Microsoft.

When ChatGPT was the only game in town Microsoft was seen as a leader, thanks to their wise investment in Open AI. They relied on Open AI's model and didn't develop their own. As a result Microsoft has no interesting AI products. Copilot is a flop. Bing failed to take advantage of AI, Perplexity ate their lunch.

Satya Nadella last year: “Google should have been the default winner in the world of big tech’s AI race”.

Sundar Pichai's response: “I would love to do a side-by-side comparison of Microsoft’s own models and our models any day, any time. They are using someone else's model.”

See: https://www.msn.com/en-in/money/news/sundar-pichai-vs-satya-...

Post reply on HN