Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.5 Flash

blog.google

Post reply on HN
Print view View original

511–520 of 692 posts

Re: Gemini 3.5 Flash

#511

Post by akulbe »

SwellJoe wrote: post #484
hedgehog wrote: post #443

Earlier quoted context omitted.

Have you done comparisons with 4 bit and seen a noticeable difference for coding tasks?

No, I've just seen benchmarks showing most models start degrading around 4-5 bits. That's not to say they become useless, just that down to about 6-bits (with careful hybrid quantizations like unsloth where some of the layers aren't quantized or are quantized at higher bit depths) the quality isn't measurably degraded, but below that there are measurable differences in performance. People report good results from Dee…

One of the things I'm wondering about is what I'm missing for $LLM to create files on the local FS like Claude and Codex do. What I see instead is stuff just printing to stdout, rather than files on the filesystem.

What am I missing?

Re: Gemini 3.5 Flash

#512

Post by akulbe »

sheeshkebab wrote: post #417
akulbe wrote: post #351

Earlier quoted context omitted.

Any chance you'd be willing to talk further about your setup? I have 2 x 3090s in a local machine, and I'm still left with questions about how best to use stuff locally.

You can only run heavily quantized models on all 3/4/5 rtx gpus (with 32gb or less vram) - and you probably want moe versions like Qwen 35b for this to run at speed somewhat comparable to Claude. It’s still not there to be honest but getting there. Personally I mess around with llama.cpp on m5 max with 128gb - it’s a decent setup to try various medium sized things, and runs llms surprisingly well without quantization…

How is that machine for local inference? It's a serious consideration for me, but getting to hear more from folks that already have it would be helpful.

Re: Gemini 3.5 Flash

#513

Post by dankwizard »

simonw wrote: post #131

The pelican is a lot : https://github.com/simonw/llm-gemini/issues/133#issuecomment... Not a great bicycle though, it forgot the bar between the pedals and the back wheel and weirdly tangled the other bars. Expensive too - that pelican cost 13 cents: https://www.llm-prices.com/#it=11&ot=14403&sel=gemini-3.5-fl...

Wouldn't be a thread about the tech that is changing the landscape for businesses across nearly every discipline without a pelican svg.

Re: Gemini 3.5 Flash

#514

Post by cube00 »

hmate9 wrote: post #265

I have google ai pro plan and tried antigravity with 3.5 flash but it used up all my quota in two prompts. If that is not a bug then it is seriously unusable.

The way they're charging for failed generations is brutal.

Checked my 5 hour quota, it was 0%, got this for multiple attempts:

I'm getting more image requests than usual, so I can't create that for you right now. Please try again later.

or

Can you ask me again later? I'm being asked to create more images than usual, so I can't do that for you right now.

Went back and found they took 34% of my quota for the privilege of repeating that same error.

I think the "Usage Limits" screen is new so who knows how long they've been counting errors against our quota. I guess I should be grateful it's now visible.

Re: Gemini 3.5 Flash

#515

Post by hedgehog »

akulbe wrote: post #511
SwellJoe wrote: post #484

Earlier quoted context omitted.

No, I've just seen benchmarks showing most models start degrading around 4-5 bits. That's not to say they become useless, just that down to about 6-bits (with careful hybrid quantizations like unsloth where some of the layers aren't quantized or are quantized at higher bit depths) the quality isn't measurably degraded, but below that there are measurable differences in performance. People report good results from Dee…

One of the things I'm wondering about is what I'm missing for $LLM to create files on the local FS like Claude and Codex do. What I see instead is stuff just printing to stdout, rather than files on the filesystem. What am I missing?

The model just predicts text, Claude Code etc parse the output and do the actual file creation (or run shell commands that do it). If you have Claude Code installed look in ~/.claude/projects/... and you can see the transcripts of your actual sessions, or install Mini-SWE-Agent and play with that to get a feel for what's going on.

Re: Gemini 3.5 Flash

#516

Post by hedgehog »

SwellJoe wrote: post #484
hedgehog wrote: post #443

Earlier quoted context omitted.

Have you done comparisons with 4 bit and seen a noticeable difference for coding tasks?

No, I've just seen benchmarks showing most models start degrading around 4-5 bits. That's not to say they become useless, just that down to about 6-bits (with careful hybrid quantizations like unsloth where some of the layers aren't quantized or are quantized at higher bit depths) the quality isn't measurably degraded, but below that there are measurable differences in performance. People report good results from Dee…

The data I've seen is stuff like the KL Divergence comparisons that Unsloth does which show something but not clearly whether there's an observable or significant difference in task performance.

Re: Gemini 3.5 Flash

#517

Post by Barbing »

tantalor wrote: post #184
simonw wrote: post #131

The pelican is a lot : https://github.com/simonw/llm-gemini/issues/133#issuecomment... Not a great bicycle though, it forgot the bar between the pedals and the back wheel and weirdly tangled the other bars. Expensive too - that pelican cost 13 cents: https://www.llm-prices.com/#it=11&ot=14403&sel=gemini-3.5-fl...

Forgetting the chainstay is typical of asking random people to draw a bicycle. https://www.gianlucagimini.it/portfolio-item/velocipedia/ > most ended up drawing something that was pretty far off from a regular men’s bicycle

Thanks for the delightful Velocipedia

Re: Gemini 3.5 Flash

#519

Post by SwellJoe »

akulbe wrote: post #511
SwellJoe wrote: post #484

Earlier quoted context omitted.

No, I've just seen benchmarks showing most models start degrading around 4-5 bits. That's not to say they become useless, just that down to about 6-bits (with careful hybrid quantizations like unsloth where some of the layers aren't quantized or are quantized at higher bit depths) the quality isn't measurably degraded, but below that there are measurable differences in performance. People report good results from Dee…

One of the things I'm wondering about is what I'm missing for $LLM to create files on the local FS like Claude and Codex do. What I see instead is stuff just printing to stdout, rather than files on the filesystem. What am I missing?

You're missing an agent. The model uses tool calls to interact with the filesystem, commands on the system, optionally search (you need a search MCP server, like Brave or Exa, and API key), etc.

I usually use the Zed Agent built into Zed editor for self-hosted models, but you could use Pi, OpenCode, Hermes, Claude Code, etc. there are many, many, agents.

Re: Gemini 3.5 Flash

#520

Post by dr_dshiv »

SwellJoe wrote: post #186
dr_dshiv wrote: post #103

Earlier quoted context omitted.

3.1 flash lite — $0.25/$1.50 — plus insanely fast. 3.1 flash lite isn’t quite as good as 3 flash preview (which is the most incredible cheap model… I really love it) — but 3.1 is half the price and the insane speed opens up different use cases. For comparison, Opus models are $5/$25

Opus 4.7 is smarter than even Gemini 3.1 Pro on nearly every metric, though. You're comparing apples to oranges. Gemini 3.1 Flash is somewhere in the neighborhood between current Haiku and Sonnet, I think? Still a better value than the Anthropic models, I guess, which are quite pricey. Since Gemini 3.5 Flash is raising the price to $1.50/$9.00, it's priced between Haiku and Sonnet. If it outperforms Sonnet, it remain…

Definitely apples to oranges, sorry I wasn’t clear. I only included opus pricing for comparison—it is vastly superior. But even 3.1 flash lite is really useful.

Of course, if I manage to reach my limits every week on my Claude $200 sub, opus 4.7 is probably priced closer to flash!

Post reply on HN