Gemini-2.5-pro-preview-06-05 • Hacker News Board

Re: Gemini-2.5-pro-preview-06-05

#151

Post by diggan » Thu, Jun 05, 2025, 9:16 PM UTC

Workaccount2 wrote: post #92

ketzo wrote: post #68

Earlier quoted context omitted.

OpenAI has already forecast $12B in revenue by the end of this year. I agree that Google is well-positioned, but the mindshare/product advantage OpenAI has gives them a stupendous amount of leeway

The hurdle for OpenAI is going to be on the profit side. Google has their own hardware acceleration and their own data centers. OpenAI has to pay a monopolist for hardware acceleration and beholden to another tech giant for data centers. Never mind that Google can customize it's hardware specifically for it's models. The only way for OpenAI to really get ahead on solid ground is to discover some sort of absolute game…

> OpenAI has to pay a monopolist for hardware acceleration and beholden to another tech giant for data centers.

Don't they have a data center in progress as we speak? Seems by now they're planning on building not just one huge data center in Texas, but more in other countries too.

Re: Gemini-2.5-pro-preview-06-05

#152

Post by ketzo » Thu, Jun 05, 2025, 9:16 PM UTC

VeejayRampay wrote: post #96

ketzo wrote: post #68

Earlier quoted context omitted.

OpenAI has already forecast $12B in revenue by the end of this year. I agree that Google is well-positioned, but the mindshare/product advantage OpenAI has gives them a stupendous amount of leeway

the leeway comes from the grotesque fanboyism the company benefits from they haven't been number one for quite some time and still people can't stop presenting them as the leaders

People said much the same thing about Apple for decades, and they’re a $3T company; not a bad thing to have fans.

Plus, it’s a consumer product; it doesn’t matter if people are “presenting them as leaders”, it matters if hundreds of millions of totally average people will open their computers and use the product. OpenAI has that.

Re: Gemini-2.5-pro-preview-06-05

#153

Post by diggan » Thu, Jun 05, 2025, 9:18 PM UTC

xtracto wrote: post #121

wiradikusuma wrote: post #56

I have two issues with Gemini that I don't experience with Claude: 1. It RENAMES VARIABLE NAMES even in places where I don't tell it to change (I pass them just as context). and 2. Sometimes it's missing closing square brackets. Sure I'm a lazy bum, I call the variable "json" instead of "jsonStringForX", but it's contextual (within a closure or function), and I appreciate the feedback, but it makes reviewing the chan…

I have a very clear example of Gemini getting it wrong: For a code like this, it keeps changing processing_class=tokenizer to "tokenizer=tokenizer", even though the parameter was renamed and even after adding the all caps comment. #Set up the SFTTrainer print("Setting up SFTTrainer...") trainer = SFTTrainer( model=model, train_dataset=train_dataset, args=sft_config, processing_class=tokenizer, # DO NOT CHANGE. THIS I…

Do you have in the system prompt to actually not edit lines that has comments about not editing them? Had that happen to me too, that code comments been ignored, and adding instructions about actually following code comments helped for that. But different models so YMMV.

Re: Gemini-2.5-pro-preview-06-05

#154

Post by raincole » Thu, Jun 05, 2025, 9:19 PM UTC

chollida1 wrote: post #64

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

> At even a $300B valuation a typical wall street analysts would want to value them at 2x sales which would mean they'd expect OpenAI to have $600B in annual sales to account for this valuation when they go public.

Even Google doesn't have $600B revenue. Sorry, it sounds like numbers pulled from someone's rear.

Re: Gemini-2.5-pro-preview-06-05

#155

Post by jstummbillig » Thu, Jun 05, 2025, 9:20 PM UTC

chollida1 wrote: post #64

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

There is some serious confusion about the strength of OpenAIs position.

"chatgpt" is a verb. People have no idea what claude or gemini are, and they will not be interested in it, unless something absolutely fantastic happens. Being a little better will do absolutely nothing to convince normal people to change product (the little moat that ChatGPT has simply by virtue of chat history is probably enough from a convenience standpoint, add memories and no super obvious path to export/import either and you are done here).

All that OpenAI would have to do, to easily be worth their evaluation eventually, is to optimize and not become offensively bad to their, what, 500 million active users. And, if we assume the current paradigm that everyone is working with is here to stay, why would they? Instead of leading (as they have done so far, for the most part) they can at any point simply do what others have resorted to successfully and copy with a slight delay. People won't care.

Re: Gemini-2.5-pro-preview-06-05

#156

Post by joshvm » Thu, Jun 05, 2025, 9:28 PM UTC

svachalek wrote: post #53

throwaway314155 wrote: post #46

Earlier quoted context omitted.

It's interesting you say that because o3, while being a considerable improvement over OpenAI's other models, still doesn't match the performance of Opus 4 and Gemini 2.5 Pro by a long shot for me. However, o3 resides in the ChatGPT app, which is still superior to the other chat apps in many ways, particularly the internet search implementation works very well.

If you're coding through chat apps you're really behind the times. Try an agent IDE or plugin.

An important caveat here is yes, for coding. Apps are fine for coming up with one-liners, or doing other research. I haven't found the quality of IDE based code to be significantly better than what ChatGPT would suggest, but it's very useful to ask questions when the model has access to both the code and can prompt you to run tests which rely on local data (or even attached hardware). I really don't trust YOLO mode so I manually approve terminal calls.

My impression (with Cursor) is that you need to practice some sort of LLM-first design to get the best out of it. Either vibe code your way from the start, or be brutal about limiting what changes the agent can make without your approval. It does force you to be very atomic about your requests, which isn't a bad thing, but writing a robust spec for the prompt is often slower than writing the code by hand and asking for a refactor. As soon as kipple, for lack of a better word, sneaks into the code, it's a reinforcing signal to the agent that it can add more.

It's definitely worth paying the $20 and playing with a few different clients. The rabbit hole is pretty deep and there's still a ton of prompt engineering suggestions from the community. It encourages a lot of creative guardrails, like using pre-commit to provide negative feedback when the model does something silly like try to write a 200 word commit message. I haven't tried JetBrains' agent yet (Junie), but that seems like it would be a good one to explore as well since it presumably integrates directly with the tooling.

Re: Gemini-2.5-pro-preview-06-05

#157

Post by aeyes » Thu, Jun 05, 2025, 9:31 PM UTC

jstummbillig wrote: post #155

chollida1 wrote: post #64

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

There is some serious confusion about the strength of OpenAIs position. "chatgpt" is a verb. People have no idea what claude or gemini are, and they will not be interested in it, unless something absolutely fantastic happens. Being a little better will do absolutely nothing to convince normal people to change product (the little moat that ChatGPT has simply by virtue of chat history is probably enough from a convenie…

Google has a text input box on google.com, as soon as this gives similar responses there is no need for the average user to use ChatGPT anymore.

I already see lots of normal people share screenshots of the AI Overview responses.

Re: Gemini-2.5-pro-preview-06-05

#158

Post by vendiddy » Thu, Jun 05, 2025, 9:32 PM UTC

joshmlewis wrote: post #42

johnfn wrote: post #12

Impressive seeing Google notch up another ~25 ELO on lmarena, on top of the previous #1, which was also Gemini! That being said, I'm starting to doubt the leaderboards as an accurate representation of model ability. While I do think Gemini is a good model, having used both Gemini and Claude Opus 4 extensively in the last couple of weeks I think Opus is in another league entirely. I've been dealing with a number of gn…

o3 is still my favorite over even Opus 4 in most cases. I've spent hundreds of dollars on AI code gen tools in the last month alone and my ranking is: 1. o3 - it's just really damn good at nuance, getting to the core of the goal, and writing the closest thing to quality production level code. The only negative is it's cutoff window and cost, especially with it's love of tools. That's not usually a big deal for the Ra…

I find o3 to be the clearest thinker as well.

If I'm working on a complex problem and want to go back and forth on software architecture, I like having o3 research prior art and have a back and forth on trade-offs.

If o3 was faster and cheaper I'd use it a lot more.

I'm curious what your workflows are !

Re: Gemini-2.5-pro-preview-06-05

#159

Post by AstroBen » Thu, Jun 05, 2025, 9:41 PM UTC

Workaccount2 wrote: post #137

zamadatix wrote: post #109

Earlier quoted context omitted.

I think the only way to be particularly impressed with new leading models lately is to hold the opinion all of the benchmarks are inaccurate and/or irrelevant and it's vibes/anecdotes where the model is really light years ahead. Otherwise you look at the numbers on e.g. lmarena and see it's claiming a ~16% preference win rate for gpt-3.5-turbo from November of 2023 over this new world-leading model from Google.

People can ask whatever they want on LMarena, so a question like "List some good snacks to bring to work" might elicit a win for a old/tiny/deprecated model simply because it lists the snack the user liked more.

are you saying that's a bad way to judge a model? Not sure why we'd want ones that choose bad snacks

Re: Gemini-2.5-pro-preview-06-05

#160

Post by candiddevmike » Thu, Jun 05, 2025, 9:41 PM UTC

jstummbillig wrote: post #155

chollida1 wrote: post #64

I'd start to worry about OpenAI, from a valuation standpoint. The company has some serious competition now and is arguably no longer the leader. its going to be interesting to see how easily they can raise more money. Their valuation is already in the $300B range. How much larger can it get given their relatively paltry revenue at the moment and increasingly rising costs for hardware and electricity. If the next gene…

There is some serious confusion about the strength of OpenAIs position. "chatgpt" is a verb. People have no idea what claude or gemini are, and they will not be interested in it, unless something absolutely fantastic happens. Being a little better will do absolutely nothing to convince normal people to change product (the little moat that ChatGPT has simply by virtue of chat history is probably enough from a convenie…

ChatGPT is going to be Kleenex'd. They wasted their first mover advantage. Replace ChatGPT's interface with any other LLM and most users won't be able to tell the difference.