I wanted to try GLM 4.6 through their API with Cline, before spending the $50. But I'm getting hit with API limits. And now I'm noticing a red banner "GLM4.6 Temporarily Sold Out. Check back soon." at cloud.cerebras.ai. HN hug of death, or was this there before?
Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
101–110 of 135 posts
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#102If they don't quantize the model, how do they achieve these speeds? Groq also says they don't quantize models (and I want to believe them) but we literally have no way to prove they're right. This is important because their premium $50 (as opposed to $20 on Claude Pro or ChatGPT Plus) should be justified by the speed. GLM 4.6 is fine but I don't think it's still at the GPT-5/Claude Sonnet 4.5 level, so if I'm paying…
It’s because the model weights and KV cache are stored in SRAM. It’s extremely expensive per token.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#103Earlier quoted context omitted.
so apparently they have custom hardware that is basically absolutely gigantic chips - across the scale of a whole wafer at a time. Presumably they keep the entire model right on chip, in effectively L3 cache or whatever. So the memory bandwidth is absurdly fast, allowing very fast inference. It's more expensive to get the same raw compute as a cluster of nvidia chips, but they don't have the same peak throughput. As…
For $50/month, it's a non-starter. I hope they can find a way to use all this excess bandwidth to put out a $10 equivalent to Claude Code instead of a 1000 tok/s party trick I can't use properly.
$600 per year is a trivial cost for a professional tool
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#104Where I’m unfortunately hitting the wall with these assistants is developing a desktop app in say Java swing - Claude cannot verify that ui it produced is actually functional
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#105Earlier quoted context omitted.
We need a new term for LLMs actually solving a hard problems. When I help Claude Code solve a nasty bug it doesn’t feel like “vibing” as in “I tell the model what I want the website to look like”. It feels like sniping as in “I spot for Claude Code, telling how to adjust for wind, range, and elevation so it can hit my far away target”.
So we're the spotter in that metaphor. I like it!
"spotter coding" or perhaps "checker coding"?
"verified vivisection development" when you're working with older code :D
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#106Where I’m unfortunately hitting the wall with these assistants is developing a desktop app in say Java swing - Claude cannot verify that ui it produced is actually functional
what is your AI dev stack? have you tried Kilo Code?
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#107Earlier quoted context omitted.
I feel the same and it's also why I can't understand all these people using small local models. Every local model I've used and even most open source are just not good
GLM-4.6 is on par with Sonnet 4.5. Sometimes it is better, sometimes it is worse. Give it a shot. It's the only model that made me (almost) ditch Claude. The only problem is, Claude Code is still the best agentic program in town and search doesn't function without a proper subscription.
Have you tried Claude Code Router with GLM 4.6?
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#108I have been an AI-coding skeptic for some time. I always acknowledged LLMs as useful for solving specific problems and making certain things possible that weren't possible before. But I've not been surprised to see AI fail to live up to the hype. And I never had a personally magical moment - an experience that shifted my perspective à la the peak end rule. I've been using GLM 4.6 on Cerebras for the last week or so,…
Your post has inspired me to check them out. How do you use it, with their UI oe to power some other open source tool? Do you suggest that this thing is so fast its simpler now to quickly work on one thing at a time instead of the 5 background tools running in parallel which might have been a pattern we invented because these things are so slow?
I’ve been using the crush TUI primarily. I like that I have the flexibility to switch to a smarter model on occasion - for awhile I hesitated to pick up AI coding at all, simply because I didn’t want to be locked into a model that could be immediately surpassed. It’s also customizable enough with sane defaults.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#109Earlier quoted context omitted.
This is where Rust's "if it compiles, it's probably correct" philosophy may come in handy. "Shifting bugs left" is even more important for LLMs than it is for humans. There are certain tests LLMs can't run, so if we can detect bugs at compile time and run the LLM in a loop until things compile, that's a significant benefit.
My recent experience is that llms are dogshit at rust, though, unable to correct bugs without inserting new ones, going back and forth fixing and breaking the same thing, etc.
Aren’t we all though?
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#110I find the fast models good for rapidly iterating UI changes with voice chat. Like "add some padding above the text box" or "right align the button". But I find the fast models useless for deep coding work. But a fast model has its place. Not $50/month though. Cursor has Compose 1 and Grok Code Fast for free. Not sure what $50/month gets me that those don't. I liked the stealth supernova model a lot too.
Composer and grok fast are not free.
They are both free in Cursor right now.