Live data from Hacker News

Gemini-2.5-pro-preview-06-05

deepmind.google

171–180 of 237 posts

Re: Gemini-2.5-pro-preview-06-05

#171

Earlier quoted context omitted.

I am really struggling with this. I tried Cline with both OpenAI and Claude to very weird results. Often burning through credits to get no where or just running out of context. I just got Cursor for a try so can't say anything on that yet.

It's a skill that takes some persistence and trial and error. Happy to chat with you about it if you want to send me an email.

There is skill to it but that's certainly not the only relevant variable involved. Other important factors are:

Language: Syntax errors rise, and a common form is the syntax of a more common language bleeding through.

Domain: Less so than what humans deem complex, quality is more strongly controlled by how much code and documentation there is for a domain. Interesting is that if in a less common subdomain, it will often revert to a more common approach (for example working on shaders for a game that takes place in a cylinder geometry requires a lot more hand-holding than on a plane). It's usually not that they can't do it, but that they require much more involved prompting to get the context appropriately set up and then managing drifting to default, more common patterns. Related is decisions with long term consequences. LLMs are pretty weak at this. In humans this one comes with experience, so it's rare and an instance of low coverage.

Dates: Related is reverting to obsolete API patterns.

Complexity: While not as dominant as domain coverage, complexity does play a role. With likelihood of error rising with complexity.

This means if you're at the intersection of multiple of these (such as a low coverage problem in a functional language), agent mode will likely be too much of a waste for you. But interactive mode can still be highly productive.

Re: Gemini-2.5-pro-preview-06-05

#172

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

There is some serious confusion about the strength of OpenAIs position. "chatgpt" is a verb. People have no idea what claude or gemini are, and they will not be interested in it, unless something absolutely fantastic happens. Being a little better will do absolutely nothing to convince normal people to change product (the little moat that ChatGPT has simply by virtue of chat history is probably enough from a convenie…

I think this pretty substantially overstates ChatGPT's stickiness. Just because something is widely (if not universally) known doesn't mean it's universally used, or that such usage is sticky.

For example, I had occasion to chat with a relative who's still in high school recently, and was curious what the situation was in their classrooms re: AI.

tl;dr: LLM use is basically universal, but ChatGPT is not the favored tool. The favored tools are LLMs/apps specifically marketed as study/homework aids.

It seems like the market is fine with seeking specific LLMs for specific kinds of tasks, as opposed to some omni-LLM one-stop-shop that does everything. The market has already and rapidly moved beyond from ChatGPT.

Not to mention I am willing to bet that Gemini has radically more usage than OpenAI's models simply by virtue of being plugged into Google Search. There are distribution effects, I just don't think OpenAI has the strongest position!

I think OpenAI has some first-mover advantage, I just don't think it's anywhere near as durable (nor as large) as you're making it out to be.

Re: Gemini-2.5-pro-preview-06-05

#173

I have two issues with Gemini that I don't experience with Claude: 1. It RENAMES VARIABLE NAMES even in places where I don't tell it to change (I pass them just as context). and 2. Sometimes it's missing closing square brackets. Sure I'm a lazy bum, I call the variable "json" instead of "jsonStringForX", but it's contextual (within a closure or function), and I appreciate the feedback, but it makes reviewing the chan…

I find o1-pro, which nobody ever mentions, is in the top spot along with Gemini. But Gemini is an absolute mess to work with because it constantly adds tons of comments and changes unrelated code.

It is worth it sometimes, but usually I use it to explore ideas and then have o1-pro spit out a perfect solution ready diff test and merge.

Re: Gemini-2.5-pro-preview-06-05

#174

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

> At even a $300B valuation a typical wall street analysts would want to value them at 2x sales which would mean they'd expect OpenAI to have $600B in annual sales to account for this valuation when they go public.

Oops I think you may have flipped the numerator and the denominator there, if I’m understanding you. Valuation of 300B , if 2x sales, would imply 150B sales.

Probably your point still stands.

Re: Gemini-2.5-pro-preview-06-05

#175

I have two issues with Gemini that I don't experience with Claude: 1. It RENAMES VARIABLE NAMES even in places where I don't tell it to change (I pass them just as context). and 2. Sometimes it's missing closing square brackets. Sure I'm a lazy bum, I call the variable "json" instead of "jsonStringForX", but it's contextual (within a closure or function), and I appreciate the feedback, but it makes reviewing the chan…

Gemini loves to add idiotic non-functional inline comments. "# Added this function" "# Changed this to fix the issue" No, I know, I was there! This is what commit messages for, not comments that are only relevant in one PR.

Have you tried modifying the system instructions to get it to stop doing that?

Re: Gemini-2.5-pro-preview-06-05

#176
As a Windsurf user I was happy with Claude 3.7 but then switched to Google Gemini 2.5 when Claude started glitching on a particularly large file. It's a bummer that 3.7 has gone from Windsurf - I considered cancelling my Windsurf subscription, but decided not to because it is still good value for money.

Re: Gemini-2.5-pro-preview-06-05

#177
post #20

I pay for both ChatGPT Plus and Gemini Pro. I'm thinking of cancelling my ChatGPT subscription because I keep hitting rate limits. Meanwhile I have yet to hit any rate limit with Gemini/AI Studio.

AI Studio uses your API account behind the scenes, and it is subject to normal API limits. When you signup for AI Studio, it creates a Google Cloud free tier project with "gen-lang-client-" prefix behind the scenes. You can link a billing account at the bottom of the "get an api key page". Also note that AI studio via default free tier API access doesn't seem to fall within "commercial use" in Google's terms of servi…

> AI studio via default free tier API access doesn't seem to fall within "commercial use" in Google's terms of service, which would mean that your prompts can be reviewed by humans and used for training. All info AFAIK.

Seconded.

Re: Gemini-2.5-pro-preview-06-05

#178
post #12

Impressive seeing Google notch up another ~25 ELO on lmarena, on top of the previous #1, which was also Gemini! That being said, I'm starting to doubt the leaderboards as an accurate representation of model ability. While I do think Gemini is a good model, having used both Gemini and Claude Opus 4 extensively in the last couple of weeks I think Opus is in another league entirely. I've been dealing with a number of gn…

o3 is still my favorite over even Opus 4 in most cases. I've spent hundreds of dollars on AI code gen tools in the last month alone and my ranking is: 1. o3 - it's just really damn good at nuance, getting to the core of the goal, and writing the closest thing to quality production level code. The only negative is it's cutoff window and cost, especially with it's love of tools. That's not usually a big deal for the Ra…

How do you find o3 vs o4-mini?

Re: Gemini-2.5-pro-preview-06-05

#179
post #65

Gemini is a good and fast model, but I think the style of code it writes is... amateur / inexperienced. It doesn't make a lot of mistakes typical of an LLM, but rather chooses approaches that are typical of someone who just learned programming. I have to always nudge it to avoid verbosity, keep structure less repetitive, optimize async code, etc. With claude, I rarely have this problem -- it feels more like working w…

> I have to always nudge it to avoid verbosity, keep structure less repetitive, optimize async code, etc.

Isn’t this what you can do with system instructions?

Re: Gemini-2.5-pro-preview-06-05

#180
post #106

Earlier quoted context omitted.

Gemini loves to add idiotic non-functional inline comments. "# Added this function" "# Changed this to fix the issue" No, I know, I was there! This is what commit messages for, not comments that are only relevant in one PR.

And it sure loves removing your carefully inserted comments for human readers.

It feels like I'm negotiating with a toddler. If I say nothing, it adds useless comments everywhere. If I tell it to not add comments, it deletes all of my comments. Tell it to put the comments back, and it still throws away half of my comments and rewrites the reset in a less precise way.
Post reply on HN