Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

composio.dev

Post reply on HN
Print view View original

81–90 of 336 posts

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#81

Post by G4E »

phkahler wrote: post #55

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

It's not AI, but I have good news for you though : what you seek already exists !

https://github.com/dune3d/dune3d

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#82

Post by [deleted] »

phkahler wrote: post #55

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

[deleted]

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#83

Post by vonneumannstan »

dsign wrote: post #77

I guess depends on the task? I have very low expectations for Gemini, but I gave it a run with a signal processing easy problem and it did well. It took 30 seconds to reason through a problem that would have taken me between 5 to 10 minutes to reason. Gemini's reasoning was sound (but it took me a couple of minutes to decide that), and it also wrote the functions with the changes (which took me an extra minute to ver…

>I guess depends on the task? I have very low expectations for Gemini, but I gave it a run with a signal processing easy problem and it did well. It took 30 seconds to reason through a problem that would have taken me between 5 to 10 minutes to reason. Gemini's reasoning was sound (but it took me a couple of minutes to decide that), and it also wrote the functions with the changes (which took me an extra minute to verify). It's not a definitive win in time, but at least there was an extra pair of "eyes"--or whatever that's called with a system like this one.

I wonder if you treat code from a Jr engineer the same way? Seems impossible to scale a team that way. You shouldnt need to verify every line but rather have test harnesses that ensure adherence to the spec.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#84

Post by ramesh31 »

phkahler wrote: post #55

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

You guys really need a Docker build. This dependency chain with submodules is a nightmare.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#85

Post by qwertox »

criddell wrote: post #74
qwertox wrote: post #13

Gemini is the only model which tells me when it's a good time to stop chatting because either it can't find a solution or because it dislikes my solution (when I actively want to neglect security). And the context length is just amazing. When ChatGPT's context is full, it totally forgets what we were chatting about, as if it would start an entirely new chat. Gemini lacks the tooling, there ChatGPT is far ahead, but a…

I asked Claude this weekend what it could tell me about writing Paint.Net plugins and it responded that it didn't know much about that: > I'd be happy to help you with information about writing plugins for Paint.NET. This is a topic I don't have extensive details on in my training, so I'd like to search for more current information. Would you like me to look up how to create plugins for Paint.NET?

I mean responses like this one:

  I understand the desire for a simple or unconventional solution, however there are problems with those solutions.
  There is likely no further explanation that will be provided.
  It is best that you perform testing on your own.

  Good luck, and there will be no more assistance offered.
  You are likely on your own.
This was about a SOCKS proxy which was leaking when the OpenVPN provider was down while the container got started, so we were trying to find the proper way of setting/unsetting iptable rules.

My proposed solution was to just drop all incoming SOCKS traffic until the tunnel was up and running, but Gemini was hooked on the idea that this was a sluggish way of solving the issue, and wanted me to drop all outgoing traffic until the tun device existed (with the exception of DNS and VPN_PROVIDER_IP:443 for building the tunnel).

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#86

Post by criddell »

qwertox wrote: post #85
criddell wrote: post #74

Earlier quoted context omitted.

I asked Claude this weekend what it could tell me about writing Paint.Net plugins and it responded that it didn't know much about that: > I'd be happy to help you with information about writing plugins for Paint.NET. This is a topic I don't have extensive details on in my training, so I'd like to search for more current information. Would you like me to look up how to create plugins for Paint.NET?

I mean responses like this one: I understand the desire for a simple or unconventional solution, however there are problems with those solutions. There is likely no further explanation that will be provided. It is best that you perform testing on your own. Good luck, and there will be no more assistance offered. You are likely on your own. This was about a SOCKS proxy which was leaking when the OpenVPN provider was d…

That sounds like you asked for plans to a perpetual motion machine.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#87

Post by nprateem »

Sometimes these models get tripped up with a mistake. They'll add a comment to the code saying "this is now changed to [whatever]" but it hasn't made the replacement. I tell it it hasn't made the fix, it apologises and does it again. Subsequent responses lead to more profuse apologies with assertions it's definitely fixed it this time when it hasn't.

I've seen this occasionally with older Claude models, but Gemini did this to me very recently. Pretty annoying.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#88

Post by hu3 »

Kiro wrote: post #58
iammrpayments wrote: post #33

Theo video detected = opinion rejected Also I generally dislike thinking models for coding and prefer faster models, so if you have something easy gemini 2.0 is good

What's wrong with Theo?

People say his technical opinions can/are bought for the right price or clicks.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#89

Post by quesomaster9000 »

dsincl12 wrote: post #4
mraniki wrote: post #2

TL;DR If you want to jump straight to the conclusion, I’d say go for Gemini 2.5 Pro, it’s better at coding, has one million in context window as compared to Claude’s 200k, and you can get it for free (a big plus). However, Claude’s 3.7 Sonnet is not that far behind. Though at this point there’s no point using it over Gemini 2.5 Pro.

Not sure what happened with Claude 3.7, but 3.5 is way better in all things day to day. 3.7 felt like a major step back especially when it comes to coding even though this was highlighted as one aspect they improved upon. 500k window will soon be released for Claude. Not sure much it will improve anything though.

With Claude 3.7 I keep having to remind it about things, and go back and correct it several times in a row, before cleaning the code up significantly.

For example, yesterday I wanted to make a 'simple' time format, tracking Earths orbits of the Sun, the Moons orbits of Earth and rotations of Earth from a specific given point in time (the most recent 2020 great conjunction) - without directly using any hard-coded constants other than the orbital mechanics and my atomic clock source. Where this would be in the format of `S4.7.... L52... R1293...` for sols, luns & rotations.

I keep having to remind to to go back to first principles, we want actual rotations, real day lengths etc. rather than hard-coded constants that approximate the mean over the year.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#90

Post by ignoramous »

mountainriver wrote: post #73
neal_ wrote: post #29

I was using gemini 2.5 pro yesterday and it does seem decent. I still think claude 3.5 is better at following instruction then the new 3.7 model which just goes ham messing stuff up. Really disappointed by Cursor and the Claude CLI tool, for me they create more problems then fix. I cant figure out how to use them on any of my projects with out them ruining the project and creating terrible tech debt. I really like th…

My whole team feels like 3.7 is a letdown. It really struggles to follow instructions as others are mentioning. Makes me think they really just hacked the benchmarks on this one.

Claude Sonnet 3.7 Thinking is also an unmitigated disaster for coding. I was mistaken that a "thinking" model would be better at logic. It turns out "thinking" is a marketing term, a euphemism for "hallucinating" ... though, not unsurprising when you actually take a look at the model cards for these "reasoning" / "thinking" LLMs; however, I've found these to work nicely for IR (information retrieval).
Post reply on HN