251–260 of 345 posts
Speculative decoding is an amazingly clever invention, almost seems-too-good-to-be-true (faster interference with zero degradation from the quality of the main model). The core idea is: if you can find a way to generate a small run of draft next tokens with a smaller model that have a reasonable likelihood of being correct, it's fast to check that they are actually correct with the main model because you can run the…
Earlier quoted context omitted.
Ollama merged a PR for MTP about 2 hours ago, as well: https://github.com/ollama/ollama/pull/15980 Edit: Seems they also have a pre-release version out with the functionality added: https://github.com/ollama/ollama/releases/tag/v0.23.1-rc0
Sad: theturtle32@ai1:~$ ollama run gemma4:31b-coding-mtp-bf16 pulling manifest Error: pull model manifest: 412: this model requires macOS
Any idea how much worse they will be ? Or is the issue that their error will really diverge as you accept more of their tokens?
Earlier quoted context omitted.
https://chatjimmy.ai being a demo of the "burn the model to an ASIC" approach being sold by Taalas[0], an approach which they use to run Llama 3.1 8B at ~17000 tokens per second. [0] - https://taalas.com/products/
Not to downplay their accomplishment but Llama 3.1 8B is a terrible model. It's really outdated at this point. It's cool that they were able to accelerate a model with silicon, but it also feels wasteful since llama 8B is such a useless model?
Earlier quoted context omitted.
Where are you using it? Is Gemini CLI at a usable state? It was a frustrating, miserable experience last time I gave it a shot. Antigravity seems significantly better in comparison, but with lower usage limits. If I run out, I usually don't bother switching to Gemini CLI.
I'm using it in antigravity, and fint it quite good. I have not managed to run out of usage on Flash. You can run Pro out of quota almost instantly, they really don't want you to use it if you're not paying $200 a month. I do not use super broad prompts, though. None of this "build me a webapp" stuff. It's more like, "adjust this part of this class to do Y instead of X."
Earlier quoted context omitted.
Anecdotally the 15/month basic Gemini plan allows coding all day. I'm not hitting the limits or needing to upgrade to 100/month plans like other people are doing with Claude or Codex. Caveat: Gemini has been dumbed down a few times over the last year. Rate limits tightened up too. So it might not be this good in the future.
I find Gemini to be quite good / acceptable at code review, design, and design review, but it's notably far behind Claude Code for implementation. Are you having better results? Codex is fast and decent, but I REALLY have to stay on top of it. The amount of times it makes executive design decisions on the fly to completely break everything is way too high.
I either vibe code a whole personal project, or strongly direct it to generate individual changes. It's fine for both.
The Pro model is the only good model for complex code and I think it's slower than Claude and Codex.
Earlier quoted context omitted.
Anecdotally the 15/month basic Gemini plan allows coding all day. I'm not hitting the limits or needing to upgrade to 100/month plans like other people are doing with Claude or Codex. Caveat: Gemini has been dumbed down a few times over the last year. Rate limits tightened up too. So it might not be this good in the future.
This used to be the case, but the changes last month have rendered the Gemini Pro plan completely unusable.
Likely there's a lot of dynamic tweaking of model quality. Rate limits are still fine for me at least.
Earlier quoted context omitted.
Anecdotally the 15/month basic Gemini plan allows coding all day. I'm not hitting the limits or needing to upgrade to 100/month plans like other people are doing with Claude or Codex. Caveat: Gemini has been dumbed down a few times over the last year. Rate limits tightened up too. So it might not be this good in the future.
no 15/month does not enough all day? pls dont share wrong info, 3.1 pro CLI sometimes wait 20-30 min thinking sometimes, it's by far worse compared to others.It finishes with few hours of work mostly, but in openai they give you 6 times of that in 24 hours, gemini resets one time a day. It is literally lazy and so many times does half work. I'm a power user for all top models in top 3 AI companies, only Gemini 3.1 wa…
Earlier quoted context omitted.
Anecdotally the 15/month basic Gemini plan allows coding all day. I'm not hitting the limits or needing to upgrade to 100/month plans like other people are doing with Claude or Codex. Caveat: Gemini has been dumbed down a few times over the last year. Rate limits tightened up too. So it might not be this good in the future.
I only see plans for $8, $20, and $250/month... which one are you using exactly? https://gemini.google/subscriptions/
This might be silly, but … since the assistant models are so much smaller than the full models. What if we just use those smaller models? Any idea how much worse they will be ? Or is the issue that their error will really diverge as you accept more of their tokens?