Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

composio.dev

Post reply on HN
Print view View original

101–110 of 336 posts

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#101

Post by superkuh »

What is most apparent to me (putting in existing code and asking for changes) is Gemini 2.5 Pro's tendency to refuse to actually type out subroutines and routinely replace them with either stubs or comments that say, "put the subroutines back here". It makes it so even if Gemini results are good they're still broken and require lots of manual work/thinking to get the subroutines back into the code and hooked up properly.

With a 1 million token context you'd think they'd let the LLM actually use it but all the tricks to save token count just make it... not useful.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#102

Post by aleph_minus_one »

G4E wrote: post #81
phkahler wrote: post #55

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

It's not AI, but I have good news for you though : what you seek already exists ! https://github.com/dune3d/dune3d

This does not look like a Gtk4 port of Solvespace, but like another independent CAD application that uses Gtk4 for its GUI on GNU/Linux.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#103

Post by sieabahlpark »

acedTrex wrote: post #98
ttul wrote: post #79

Earlier quoted context omitted.

Send the whole repo to AI Studio using my vibe coded tool `llm_globber` and let Gemini chew on it. You can get this done in a few hours.

I think the "offer a PR I will accept is the kicker here, getting it 'done' is the easy part"

[dead]

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#104

Post by phkahler »

G4E wrote: post #81
phkahler wrote: post #55

Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…

It's not AI, but I have good news for you though : what you seek already exists ! https://github.com/dune3d/dune3d

Yes, we are all well aware of Dune3d. I'm a big fan of Lukas K's work. In fact I wish he had done our GTK port first, and then forked Solvespace to use Open Cascade to solve the problems he needed to address. That would have given me this task for free ;-) We are not currently planning to incorporate OCCT but to simply extend and fix the small NURBS kernel that Solvespace already has.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#105

Post by evantbyrne »

The common issue I run into with all LLMs is that they don't seem to be able to complete the same coding tasks where googling around also fails to provide working solutions. In particular, they seem to struggle with libraries/APIs that are less mainstream.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#107

Post by stared »

At this level, it is very contextual - depending on your tools, prompts, language, libraries, and the whole code base. For example, for one project, I am generating ggplot2 code in R; Claude 3.5 gives way better results than the newer Claude 3.7.

Compare and contrast https://aider.chat/docs/leaderboards/, https://web.lmarena.ai/leaderboard, https://livebench.ai/#/.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#108

Post by HarHarVeryFunny »

kingkongjaffa wrote: post #3
mraniki wrote: post #2

TL;DR If you want to jump straight to the conclusion, I’d say go for Gemini 2.5 Pro, it’s better at coding, has one million in context window as compared to Claude’s 200k, and you can get it for free (a big plus). However, Claude’s 3.7 Sonnet is not that far behind. Though at this point there’s no point using it over Gemini 2.5 Pro.

How are you getting gemini 2.5 pro for free? In the gemini iOS app the only available models are currently 2.0 flash and 2.0 flash thinking.

They just added it to the free tier today.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#109

Post by namaria »

MrScruff wrote: post #61

The evidence given really doesn't justify the conclusion. Maybe it suggests 2.5 Pro might be better if you're asking it to build Javascript apps from scratch, but that hardly equates to "It's better at coding". Feels like a lot of LLM articles follow this pattern, someone running their own toy benchmarks and confidently extrapolating broad conclusions from a handful of data points. The SWE-Bench result carries a bit…

There are three things this hype cycle excels at. Getting money from investors for foundational model creators and startup.ai; spinning lay offs as a good sign for big corps; and trying to look like clever tech blogger for people looking for clout online.

Re: Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

#110

Post by HarHarVeryFunny »

uxx wrote: post #47

Gemini takes parts of code and just writes (same as before) even when i ask it to provide full code. which for me is deal breaker

Yeah - I tried Gemini 2.0 Flash a few week ago, and while the model itself is decent this was very annoying. It'd generate full source if I complained, but then next change would go back to "same as before" ... over and over ...
Post reply on HN