Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.5 Flash

blog.google

Post reply on HN
Print view View original

121–130 of 692 posts

Re: Gemini 3.5 Flash

#121

Post by ACCount37 »

throwa356262 wrote: post #86
ACCount37 wrote: post #26

Earlier quoted context omitted.

Anthropic has been sitting on Mythos for a while now. I guess they don't feel pressured to fuck it ship it until anyone else gets a 10T to work.

According to people who have access to Mythos, it is slightly worse than GPT-5.5-xhigh. At least for security tasks. Hold on, I think this claim needs some hard data. Here you go gentlemen: https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

That claim keeps contradicted hard by other parties, who say Mythos beats 5.5 resoundingly on both autonomous search and discovery and creation of complex exploit chains.

There might be a harness difference, but also, this CTF-type benchmark might not capture the capability difference fully.

Re: Gemini 3.5 Flash

#123

Post by LetsGetTechnicl »

GodelNumbering wrote: post #84

Per million input/output tokens: Gemini 2.5 flash: $0.30/$2.50 Gemini 3.0 flash preview: $0.50/$3.00 Gemini 3.5 flash: $1.50/$9.00 Interesting pricing direction. I don't think we have ever seen a 3x price increase for in the immediate next same-sized model (and lol @ 3 only ever getting a preview). 3.5 flash costs similar to Gemini 2.5 pro which was $1.25/$10

Gen AI is unprofitable, especially at the insanely cheap rates they've been offering to get people in the door. So expect more increases in the future.

Re: Gemini 3.5 Flash

#125

Post by jader201 »

nerdalytics wrote: post #30
alexdns wrote: post #5

Its Gemini 3.5 Flash

Yeah, Google chose a misleading title for the blog post.

> Today, we’re introducing Gemini 3.5, our latest family of models combining frontier intelligence with action. This represents a major leap forward in building more capable, intelligent agents. We’re kicking off the series by releasing 3.5 Flash.

Re: Gemini 3.5 Flash

#126

Post by hydra-f »

noelsusman wrote: post #72

The Artificial Analysis benchmark results are pretty underwhelming. Roughly the same "intelligence" as MiMo-V2.5-Pro for over 3x the cost. We'll have to see how that translates to actual usage but it's not a great sign.

That really depends on whether they have similar parameter counts, doesn't it? Unless you know that, the comparison is just strange

Re: Gemini 3.5 Flash

#127

Post by aesthesia »

throwa356262 wrote: post #86
ACCount37 wrote: post #26

Earlier quoted context omitted.

Anthropic has been sitting on Mythos for a while now. I guess they don't feel pressured to fuck it ship it until anyone else gets a 10T to work.

According to people who have access to Mythos, it is slightly worse than GPT-5.5-xhigh. At least for security tasks. Hold on, I think this claim needs some hard data. Here you go gentlemen: https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

See the later post testing a newer Mythos checkpoint, though: https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber...

Re: Gemini 3.5 Flash

#128

Post by MASNeo »

Well, available for Gemini means these days that half the time they are “Receiving a lot of requests right now.” and so sorry they couldn’t complete the task. Luckily the model supports long time horizons because that’s what’s needed. /me likes Gemini a lot just wishing Google would add the compute!

Re: Gemini 3.5 Flash

#130

Post by hibikir »

WarmWash wrote: post #57
aliljet wrote: post #33

Is there a good benchmark tracking hallucinations? The models are all incredibly good now, even the open ones, and my hope is that the rate of hallucinations is something that's falling off in concert with larger and larger context lengths.

People complain about them incessantly, but I can almost never get people to actually post receipts. Every provider allows sharing chats, and anyone can share a prompt that reliably produces hallucinations. More often than not, people are using images in responses that go awry. Which is fair, the models are sold as multi-modal, but image analyses is still at gpt-4.0 text-analyses levels. Also knowledge cutoff issues,…

I see constant hallucination in claude code when using specific tooling: It thinks it knows aws cli, for instance, but there's some flags that don't exist, it attempts to use all the time in 4.6 and 4.7. When asked about it, it says that yes , the flag doesn't exist in that command, but it exists in a different command (which it does), and yet, it attempts to use it without extra info.

Claude also believes it knows how AWS' KMS works, quite confidently, while getting things wrong. I have a separate "this is how KMS replication actually works" file just to deal with its misconceptions.

For gemini, I typically use it to query information from large corpuses, but it often web searches and hallucinates instead of reading the actual corpus. On a book series, it will hallucinate chapters and events which, while reasonable and plausible, do not exist. "Go look at the files and see if your reference is correct" shows that it's not correct, and it's a mandatory step. But that doesn't prevent hallucination, but makes sure you catch it after the fact, just like a method in a class that doesn't exist gets found out by the compiler. The LLM still hallucinated it.

Post reply on HN