Never sleep on Google.
Gemini 2.5 Pro Preview
121–130 of 728 posts
Re: Gemini 2.5 Pro Preview
#122I don't understand what I'm doing wrong.. it seems like everyone is saying Gemini is better, but I've compared dozens of examples from my work, and Grok has always produced better results.
[flagged]
Good luck finding a commercial LLM not backed by a corporation with serious ethical issues.
Re: Gemini 2.5 Pro Preview
#123Earlier quoted context omitted.
Tell it not to write so many comments then. You have a great deal of flexibility in dictating the coding style and can even include that style in your system prompt or upload a coding style document and have Gemini use it.
Every time I ask an LLM to not write comments, it still litters it with comments. Is Gemini better about that?
No, you can tell it not to write these comments in every prompt and it'll still do it
Re: Gemini 2.5 Pro Preview
#124Earlier quoted context omitted.
LLMs are extremely poor at following negative instructions, tell them what to do, not what not to do.
Ok, so saying "Implement feature X" leads to a ton of comments. How do you rewrite that comment to not include "don't write comments" while making the output not containing comments? "Write only source code, no plain text with special characters in the beginning of the line" or what are you suggesting here in practical terms?
"Implement feature X, and as you do, insert only minimal and absolutely necessary comments that explain why something is being done, not what is being done."
Re: Gemini 2.5 Pro Preview
#125Earlier quoted context omitted.
I've found that heavily commented code can be better for the LLM to read later, so it pulls in explanatory comments into context at the same time as reading code, similar to pulling in @docs, so maybe it's doing that on purpose?
No, it's just bad. I've been writing a lot of Python code past two days with Gemini 2.5 Pro Preview, and all of its code was like: ```python def whatever(): --- SECTION ONE OF THE CODE --- ... --- SECTION TWO OF THE CODE --- try: [some "dangerous" code] except Exception as e: logging.error(f"Failed to save files to {output_path}: {e}") # Decide whether to raise the error or just warn # raise IOError(f"Failed to save…
I'm seeing it trying to catch blind exceptions in Python all the time. I see it in my colleagues code all the time, it's driving me nuts.
Re: Gemini 2.5 Pro Preview
#126Earlier quoted context omitted.
This should be the top comment. Cherry-picking is hurting this industry. I bet they kept training on coding tasks, made everything worse on the way, and tried to hide it under the rug because of the sunk costs.
Or because they realized that coding is what most of those LLMs are used for anyways?
They should have shown the benchmarks. Or market it as a coding model, like Qwen & Mistral.
Re: Gemini 2.5 Pro Preview
#127I don't understand what I'm doing wrong.. it seems like everyone is saying Gemini is better, but I've compared dozens of examples from my work, and Grok has always produced better results.
I agree, from my experience Grok gives superior coding results, especially when modifying large sections of the codebase at once such as in refactoring.
Although it’s not for coding, I have noticed Gemini 2.5 pro Deep Research has surpassed Grok’s DeepSearch in thoroughness and research quality however.
Re: Gemini 2.5 Pro Preview
#128Interestingly, when compering benchmarks of Experimental 03-25 [1] and Experimental 05-06 [2] it seems the new version scores slightly lower in everything except on LiveCodeBench. [1] https://storage.googleapis.com/model-cards/documents/gemini-... [2] https://deepmind.google/technologies/gemini/
Sounds like they were losing so much money on 2.5-Pro they came up with a forced update that made it cheaper to run. They can't come out with "we've made it worse across the board", nor do they want to be the first to actually raise prices, so instead they made a bit of a distill that's slightly better at coding so they can still spin it positively.
Re: Gemini 2.5 Pro Preview
#129Earlier quoted context omitted.
Or because they realized that coding is what most of those LLMs are used for anyways?
They should have shown the benchmarks. Or market it as a coding model, like Qwen & Mistral.
That's clearly not a PR angle they could possibly take when it's replacing the overall SotA model. This is a business decision, potentially inference cost related.
Re: Gemini 2.5 Pro Preview
#130Earlier quoted context omitted.
It's already much improved on the early days. But I wonder when we'll be happy? Do we expect colleagues friends and family to be 100% laser-accurate 100% of the time? I'd wager we don't. Should we expect that from an artificial intelligence too?
I expect my calculator to be 100% accurate 100% of the time. I have slightly more tolerance for other software having defects, but not much more.
And a $2.99 drugstore slim wallet calculator with solar power gets it right 100% of the time while billion dollar LLMs can still get arithmetic wrong on occasion.