Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.8 Flash and 3.8 Flash Cyber

blog.google

Post reply on HN
Print view View original

471–480 of 699 posts

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#471

Post by embedding-shape »

Kayou wrote: post #410
simonw wrote: post #200

The speed combined with the fact that this thing is really good at HTML JavaScript is pretty exciting. Here's what I got for 1.8 cents and 13 seconds from the prompt "make me a cool thing in html": https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f Transcript here (it was part of a chat): https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...

Why do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing ! I find Ling 3.0 tiny particularly interesting as it looks really nice for a tiny model with 7.9B total parameters, with only 1.3B parameters activated per token. Here is the result https://coolthing-ling-3-tiny.tiiny.site (sorry for the weird hosting, first I found that worked) (it cost…

> Why do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing !

Datasets contains lots of people sharing particles simulations in various ways, with a bunch of people replying "that's so cool" and similar, so 10 years later someone asks an LLM for "cool thing" and "particle simulations" rank pretty far up when it thinks about what others have called cool.

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#472

Post by Alpha3031 »

xyzzy_plugh wrote: post #109
coffeecoders wrote: post #70

One place where I find the Flash models surprisingly bad is Google Search's "AI Mode". A recent example - I searched for how to unsubscribe from Pearson emails. Google Search "AI Mode" confidently gave me a sequence of steps along the lines of Settings > Profile > Email preferences > Unsubscribe. Of course, I looked for an unsubscribe link before asking Google. None of those options existed. The correct answer was th…

It's not the models, it's the guardrails. It's obvious that the Google Search AI Mode encourages the model to give an answer without spending unnecessary cycles investigating deeply. They also heavily encourage keeping the context short. For example, it will remove the option to start a new turn after a small number of turns, depending on the topic. It definitely makes things up all the time, but it gets it right sur…

The search model is probably flash-lite based on what they give to users who aren't signed in.

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#473

Post by briHass »

Forgeties79 wrote: post #408
senordevnyc wrote: post #396

Earlier quoted context omitted.

Easy, have another agent check it. Yeah, I know, just more slop. But I do think the second agent’s eagerness to please is aligned more in your favor in that instance, so it’s likely to find most issues. The bigger problem I’ve found is that it’ll also find all kinds of very minor edge cases that you have to pick through.

I do not understand how some of y’all are not under water with fragile code that is too massive to possibly parse. Every engineering team I know is currently trying to undo the damage of the last 6-12mo when they all got more serious into adopting these tools (usually Claude). It hasn’t completely screwed them over, but the the debt is substantial and cannot be put off anymore it seems. They argue the net is positive…

The key seems to be extensive integration/end-to-end tests with gold standard assertion data. Heck, even just saving off the json from API endpoints and using that as a reference to compare after changes works pretty well. Spin up a database backup with that static starting point, run actions, compare state afterwards.

These types of high-level tests are frustrating beyond belief to humans due to their lack of specificity, but with the agents, they don't get annoyed investigating possible regressions from non-specific signals.

They also aren't as painful to maintain as one would think, because a regression flagging test can be traced by the agent and represented as the business rule that was violated. I've found recent models to be really excellent at discerning a true regression from an outdated test assertion, especially if they are able to trace the failing test back to the PR and work ticket that built it.

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#474

Post by riversflow »

Qworg wrote: post #458
noir_lord wrote: post #413

Earlier quoted context omitted.

Do we add a third one to check the second one which is checking the first? Asking slightly tongue in cheek but at what point does this stop making sense if we can't trust the output, the people creating the models are already getting surprised in bad ways (if we take their words at face value) with how the models are behaving already etc. We have the folks over here saying "AI is amazing" and the other other folks ov…

Humans are also not trustworthy. So we build systems to capture their nondeterministic output and solve the issues that emerge.

yeah, isn’t that the main focus of discipline of software engineering?

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#475

Post by criley2 »

gundmc wrote: post #363
bermudi wrote: post #317

Earlier quoted context omitted.

I honestly can't believe serious people are making this argument on a straight face. Gemini 3.7 flash outputs so many tokens per answer it doesn't matter how fast its TPS is, sol will end up being both cheaper and faster than Gemini. So ppl are paying more for a given task, waiting longer and using a dumber intelligence because "TPS number shiny". Gemini 3.8 outputs 11k more tokens PER TASK on average in AAII than 3.…

There are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them https://artificialanalysis.ai/#cost-tabs That said, Luna is the undisputed king here at the moment and is what I use as my workhorse model.

>There are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them https://artificialanalysis.ai/#cost-tabs

Not sure if you read your own link but Sol 56 high ranks smack between Gemini 3.8 flash medium and high. Gemini 3.8 flash comes in as more expensive per task than Sol 56 high according to artificial analysis.

Luna high is literally 30X cheaper than Gemini 3.8 flash high.

You can limit the model viewer and they're getting better at testing multiple effort levels now: https://artificialanalysis.ai/?models=gpt-5-6-sol-medium%2Cg...

One reason is clear: Sol uses dramatically fewer output tokens than Gemini 38 flash https://artificialanalysis.ai/?models=gemini-3-8-flash%2Cgem...

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#476

Post by pampas »

andai wrote: post #18

Wait, I didn't realize 3.7 Flash was already beating Sol on a bunch of the benchmarks. Isn't it a way smaller models?

Gemini 3.7 Flash was already smashing more expensive models on my Redactle benchmark https://redactle.net/llm-leaderboard which mostly tests omniscience.

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#477

Post by anslopic4 »

greenowl wrote: post #228

Not to rain on anyone's parade but I find it strange how excited and giddy people on HN get for any new X.X model releases. Pumping it straight to the top, clamoring to use it, check and compare benchmarks, bragging about it being your "daily driver"? Are you people truly this excited about this crap? I mean I guess if you work for Google or Anthropic or whatever I could see it??? Otherwise, are these just bot commen…

Yes they are mostly shill and bot comments. Some of the big accounts are paid influencers, some of the other comments are purely AI.

HN sells these advertising services. Nobody is using “Claude” etc.

They will censor comments like yours and my reply here because we call it out.

It’s very weird that basically lies and disinformation became the optimal meta in business and in life! But here we are

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#478

Post by sotix »

WarmWash wrote: post #36
mattlondon wrote: post #23

Currently top at https://deepswe.datacurve.ai - beating Opus 5! https://artificialanalysis.ai/models/gemini-3-8-flash shows an intelligence score of 59, the same as Opus 5 medium! Wow - for a flash model this seems to benchmark powerfully. Remains to be seen what it is like to use.

The benchmark also doesn't include speed. You almost think something has gone wrong when using it because it returns full responses so incredibly fast.

This one uses that as a priority weight: https://winstonrc.github.io/ai-coding-agents-leaderboard/

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#479

Post by knollimar »

duplessitous wrote: post #347
pietz wrote: post #162

Earlier quoted context omitted.

That's not being debated here. The initial reported numbers were false and this was simply pointed out. You're changing the subject.

> [...] shows an intelligence score of 59, the same as Opus 5 medium! Nothing here is false, you are simply confused. You either didn't read what they wrote in its entirety or decided to reinterpret what they did write.

"Beating opus" is the false part, no?

Re: Gemini 3.8 Flash and 3.8 Flash Cyber

#480

Post by kelvinjps10 »

Sir_Twist wrote: post #313
kelvinjps10 wrote: post #305

I think about Google is the value you get of their plans, for 5$ a month you get their ai plus model combined with 400gb you can share this with your family. The other ai companies don't provide family plans

And the free year-long trial for college students they recently offered, which includes 5 tb of Google Drive storage.

I got 6months for free when I bought my s25.
Post reply on HN