Gemini 2.5 Flash • Hacker News Board

Re: Gemini 2.5 Flash

#141

Post by Layvier » Thu, Apr 17, 2025, 9:08 PM UTC

punkpeye wrote: post #38

Layvier wrote: post #22

Earlier quoted context omitted.

Agreed, it's not even possible to run an eval dataset. If someone from google see this please at least increase the burst rate limit

It is not without rate limits, but we do have elevated limits for our accounts through: https://glama.ai/models/gemini-2.5-flash-preview-04-17 So if you just want to run evals, that should do it. Though the first couple of days after a model comes out are usually pretty rough because everyone try to run their evals.

That's very interesting, thanks for sharing!

Re: Gemini 2.5 Flash

#142

Post by zhengyi13 » Thu, Apr 17, 2025, 9:08 PM UTC

byearthithatius wrote: post #89

alecco wrote: post #61

Gemini models are very good but in my experience they tend to overdo the problems. When I give it things for context and something to rework, Gemini often reworks the problem. For software it is barely useful because you want small commits for specific fixes not a whole refactor/rewrite. I tried many prompts but it's hard. Even when I give it function signatures of the APIs the code I want to fix uses, Gemini rewrite…

Yes, it will add INSANE amounts of "robust error handling" to quick scripts where I can be confident about assumptions. This turns my clean 40 lines of Python where I KNOW the JSONL I am parsing is valid into 200+ lines filled with ten new try except statements. Even when I tell it not to do this, it loves to "find and help" in other ways. Quite annoying. But overall it is pretty dang good. It even spotted a bug I mi…

I wonder how much of that sort of thing is driven by having trained their models on their own internal codebases? Because if that's the case, careful and defensive being the default would be unsurprising.

Re: Gemini 2.5 Flash

#143

Post by mmaunder » Thu, Apr 17, 2025, 9:09 PM UTC

stefan_ wrote: post #128

mmaunder wrote: post #93

More great innovation from Google. OpenAI have two major problems. The first is Google's vertically integrated chip pipeline and deep supply chain and operational knowledge when it comes to creating AI chips and putting them into production. They have a massive cost advantage at every step. This translates into more free services, cheaper paid services, more capabilities due to more affordable compute, and far more g…

I don't know man, for months now people keep telling me on HN how "Google is winning", yet no normal person I ever asked knows what the fuck "Gemini" is. I don't know what they are winning, it might be internet points for all I know. Actually, some of the people polled recalled the Google AI efforts by their expert system recommending glue on pizza and smoking in pregnancy. It's a big joke.

Try uploading a bunch of PDF bank statements to notebooklm and ask it questions. Or the results of blood work. It's jaw dropping. e.g. uploaded 7 brokerage account statements as PDFs in a mess of formats and asked it to generate table summary data which it nailed, and then asked it to generate actual trades to go from current position to a new position in shortest path, and it nailed that too.

Biggest issue we have when using notebooklm is a lack of ambition when it comes to the questions we're asking. And the pro version supports up to 300 documements.

Hell, we uploaded the entire Euro Cyber Resilience Act and asked the same questions we were going to ask our big name legal firm, and it nailed every one.

But you actually make a fair point, which I'm seeing too and I find quite exciting. And it's that even among my early adopter and technology minded friends, adoption of the most powerful AI tools is very low. e.g. many of them don't even know that notebookLM exists. My interpretation on this is that it's VERY early days, which is suuuuuper exciting for us builders and innovators here on HN.

Re: Gemini 2.5 Flash

#144

Post by kube-system » Thu, Apr 17, 2025, 9:09 PM UTC

stefan_ wrote: post #128

mmaunder wrote: post #93

More great innovation from Google. OpenAI have two major problems. The first is Google's vertically integrated chip pipeline and deep supply chain and operational knowledge when it comes to creating AI chips and putting them into production. They have a massive cost advantage at every step. This translates into more free services, cheaper paid services, more capabilities due to more affordable compute, and far more g…

I don't know man, for months now people keep telling me on HN how "Google is winning", yet no normal person I ever asked knows what the fuck "Gemini" is. I don't know what they are winning, it might be internet points for all I know. Actually, some of the people polled recalled the Google AI efforts by their expert system recommending glue on pizza and smoking in pregnancy. It's a big joke.

While there are some first-party B2C applications like chat front-ends built using LLMs, once mature, the end game is almost certainly that these are going to be B2B products integrated into other things. The future here goes a lot further than ChatGPT.

Re: Gemini 2.5 Flash

#145

Post by paxys » Thu, Apr 17, 2025, 9:11 PM UTC

throwup238 wrote: post #117

mmaunder wrote: post #93

More great innovation from Google. OpenAI have two major problems. The first is Google's vertically integrated chip pipeline and deep supply chain and operational knowledge when it comes to creating AI chips and putting them into production. They have a massive cost advantage at every step. This translates into more free services, cheaper paid services, more capabilities due to more affordable compute, and far more g…

Nobody has really talked about what I think is an advantage just as powerful as the custom chips: Google Books. They already won a landmark fair use lawsuit against book publishers, digitized more books than anyone on earth, and used their Captcha service to crowdsource its OCR. They've got the best* legal cover and all of the best sources of human knowledge already there. Then Youtube for video. The chips of course…

LibGen already exists, and all the top LLM publishers use it. I don't know if Google's own book index provides a big technical or legal advantage.

Re: Gemini 2.5 Flash

#146

Post by mdp2021 » Thu, Apr 17, 2025, 9:14 PM UTC

ghurtado wrote: post #108

gambiting wrote: post #60

Earlier quoted context omitted.

In my experience they are as dumb as a bag of bricks. The other day I asked "can you edit a picture if I upload one" And it replied "sure, here is a picture of a photo editing prompt:" https://g.co/gemini/share/5e298e7d7613 It's like "baby's first AI". The only good thing about it is that it's free.

> in my experience they are as dumb as a bag of bricks In my experience, anyone that describes LLMs using terms of actual human intelligence is bound to struggle using the tool. Sometimes I wonder if these people enjoy feeling "smarter" when the LLM fails to give them what they want.

If those people are a subset of those who demand actual intelligence, they will very often feel frustrated.

Re: Gemini 2.5 Flash

#147

Post by mring33621 » Thu, Apr 17, 2025, 9:14 PM UTC

drob518 wrote: post #110

mwest217 wrote: post #83

Earlier quoted context omitted.

Gemini has had 4 families of models, in order of decreasing size: - Ultra - Pro - Flash - Flash-Lite Versions with `-Preview` at the end haven't had their "official release" and are technically in some form of "early access" (though I'm not totally clear on exactly what that means given that they're fully available and as of 2.5 Pro Preview, have pricing attached to them - earlier versions were free during Preview bu…

Is GMail still in beta?

so Sigma...

Re: Gemini 2.5 Flash

#148

Post by mring33621 » Thu, Apr 17, 2025, 9:16 PM UTC

bsmith wrote: post #56

transformi wrote: post #7

Bad day is going on google. First the decleration of illegal monopoly.. and now... Google’s latest innovation: programmable overthinking. With Gemini 2.5 Flash, you too can now set a thinking_budget—because nothing says "state-of-the-art AI" like manually capping how long it’s allowed to reason. Truly the dream: debugging a production outage at 2am wondering if your LLM didn’t answer correctly because you cheaped out…

Using AI to debug code at 2am sounds like pure insanity.

the new normal

Re: Gemini 2.5 Flash

#149

Post by bambax » Thu, Apr 17, 2025, 9:16 PM UTC

hmaxwell wrote: post #8

I did some testing this morning: Prompt: "can you find any mistakes on my codebase? I put one in there on purpose" + 70,000 tokens of codebase where in one line I have an include for a non-existent file. Claude 3.7 Sonnet (Try 1 - new chat) Claimed it Found a race condition in the api of the admin interface that would be triggered if two admins were to change the room order at the same time. Claude suggested I group…

> codebase where in one line I have an include for a non-existent file

Ok but you don't need AI for this; almost any IDE will issue a warning for that kind of error...

Re: Gemini 2.5 Flash

#150

Post by jdgoesmarching » Thu, Apr 17, 2025, 9:16 PM UTC

whyenot wrote: post #114

mmaunder wrote: post #93

More great innovation from Google. OpenAI have two major problems. The first is Google's vertically integrated chip pipeline and deep supply chain and operational knowledge when it comes to creating AI chips and putting them into production. They have a massive cost advantage at every step. This translates into more free services, cheaper paid services, more capabilities due to more affordable compute, and far more g…

Another advantage that Google has is the deep integration of Gemini into Google Office products and Gmail. I was part of a pilot group and got to use a pre-release version and it's really powerful and not something that will be easy for OpenAI to match.

You say deep integration, yet there is still no way to send a Gemini Canvas to Docs without a lot of tedious copy-pasting and formatting because Docs still doesn’t actually support markdown. Gemini in Google Office in general has been a massive disappointment for all but the most simplistic of writing tasks.

They can have the most advanced infrastructure in the world, but it doesn’t mean much if Google continues its infamous floundering approach to product. But hey, 2.5 pro with Cline is pretty nice.