At what quantization? And if it is in fact quantized below fp8, how is the performance impacted on all the various benchmarks?
They claim they don't use quantization.
The reason for their speed is this chip: https://www.cerebras.ai/chip
Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
51–60 of 135 posts
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#52AI moves so fast that Vibe Coding still has a negative stigma attached to it, but even after 25 years of development, I'm not able to match the productivity of getting AI to implement the features I want. It's basically getting multiple devs to set out and go do work for you where you just tell them what you want and provide iterative feedback till they implement all the features you want, in the way you want and to…
It just depends on what you are doing. A green field react app in typescript with a CRUD API behind? The LLMs are a mind blowing assistant and 1000t/s is crazy. You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. You need to read every line of their output. 1000t/s is crazy but no longer always in a good w…
> You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations.
I experienced this with Claude 4 Sonnet and, to some extent, gpt-5-mini-high.
When able to run tests against its output, Claude produces pretty good Rust backend and TypeScript frontend code. However, Claude became borderline unproductive once I started experimenting with uefi-rs. Other LLMs, like gpt-5-mini-high, did not fare much better, but they were at least capable of admitting lack of knowledge. In particular, GPT-5 would provide output akin to "here is some pseudocode that you may be able to adapt to your choice of UEFI bindings".
Testing in a UEFI environment is quite difficult; the LLM can't just run `cargo test` and verify its output. Things get worse in embedded, because crates like embedded_hal made massive API changes between 0.2 and 1.0 (the latest version), and each LLM I've tried seems to only have knowledge of 0.2 releases. Also, for embedded, forget even thinking about testing harnesses (which at least exist in some form with UEFI, it's just difficult to automate the execution and output for an LLM). In this case, you cannot really trust the output of the LLM. To minimize risk of hallucination, I would try maintaining data sheets and library code in context, but at that point, it took more time to prompt an LLM than handwrite code.
I've been writing a lot of embedded Rust over the past two weeks, and my usage of LLMs in general decreased because of that. Currently planning to resume development on some of my "easier" projects, since I have about 300 Claude prompts remaining in my Zed subscription, and I don't want them to go to waste.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#53Earlier quoted context omitted.
It just depends on what you are doing. A green field react app in typescript with a CRUD API behind? The LLMs are a mind blowing assistant and 1000t/s is crazy. You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. You need to read every line of their output. 1000t/s is crazy but no longer always in a good w…
I've said it before but no one takes it seriously: LLMs are only useful if you're building something that's already in the training set ie already commodity . In which case why are you building it ???
Because I'm getting paid to.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#54AI moves so fast that Vibe Coding still has a negative stigma attached to it, but even after 25 years of development, I'm not able to match the productivity of getting AI to implement the features I want. It's basically getting multiple devs to set out and go do work for you where you just tell them what you want and provide iterative feedback till they implement all the features you want, in the way you want and to…
It just depends on what you are doing. A green field react app in typescript with a CRUD API behind? The LLMs are a mind blowing assistant and 1000t/s is crazy. You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. You need to read every line of their output. 1000t/s is crazy but no longer always in a good w…
> You are doing embedded development or anything else not as mainstream as web dev?
Counterpoint, but also kind of reinforcing your point. It depends on the kind of embedded development. I did a small utility PCB with an ESP32, and their libs are good there is active community, they have test frameworks. LLM's did a great job there.
On the other hand, I wanted to drive a timer and a pwm module and a DMA dma engine to generate some precise pulses. The way I chained hw was... Not typical, but it was what I needed and the hw could do it. At that, Claude failed miserably and it only led me to waste my time, so I had to spend the time to do it manually.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#55Earlier quoted context omitted.
It just depends on what you are doing. A green field react app in typescript with a CRUD API behind? The LLMs are a mind blowing assistant and 1000t/s is crazy. You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. You need to read every line of their output. 1000t/s is crazy but no longer always in a good w…
> You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. I experienced this with Claude 4 Sonnet and, to some extent, gpt-5-mini-high. When able to run tests against its output, Claude produces pretty good Rust backend and TypeScript frontend code. However, Claude became borderline unproductive once I started…
> Also, for embedded, forget even thinking about testing harnesses (which at least exist in some form with UEFI, it's just difficult to automate the execution and output for an LLM).
I think this doesn't have to be like this and we can do better for this. If LLMs keep this up, good testing infrastructure might become more important.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#56I have been an AI-coding skeptic for some time. I always acknowledged LLMs as useful for solving specific problems and making certain things possible that weren't possible before. But I've not been surprised to see AI fail to live up to the hype. And I never had a personally magical moment - an experience that shifted my perspective à la the peak end rule. I've been using GLM 4.6 on Cerebras for the last week or so,…
Your post has inspired me to check them out. How do you use it, with their UI oe to power some other open source tool?
Do you suggest that this thing is so fast its simpler now to quickly work on one thing at a time instead of the 5 background tools running in parallel which might have been a pattern we invented because these things are so slow?
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#57Earlier quoted context omitted.
> You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. I experienced this with Claude 4 Sonnet and, to some extent, gpt-5-mini-high. When able to run tests against its output, Claude produces pretty good Rust backend and TypeScript frontend code. However, Claude became borderline unproductive once I started…
> Also, for embedded, forget even thinking about testing harnesses (which at least exist in some form with UEFI, it's just difficult to automate the execution and output for an LLM). I think this doesn't have to be like this and we can do better for this. If LLMs keep this up, good testing infrastructure might become more important.
One of my expectations for the future is the development of testing tools whose output is "optimized" in some way for LLM consumption. This is already occurring with Bun's test runner, for instance.[0] They are implementing a flag in the test runner so that the output is structured and optimized for token count.
Overall, I agree with your point. LLMs feel a lot more reliable when a codebase has thorough, easy-to-run tests. For a similar reason, I have been drifting towards strong, statically-typed languages. Both Rust and TypeScript have rich type systems that can express many kinds of runtime behavior with just types. When a compiler can make strong guarantees about a program's behavior, I assume that helps nudge the quality of LLM output a bit higher. Tests then help prevent silly regressions from occurring. I have no evidence for this besides my anecdotal experience using LLMs across several programming languages.
In general, I've had the best experience with LLMs when there's plenty of static analysis (and tests) on the codebase. When a codebase can't be easily tested, then I get much less productivity gains from LLMs. So yeah, I'm all for improving testing infrastructure.
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#58AI moves so fast that Vibe Coding still has a negative stigma attached to it, but even after 25 years of development, I'm not able to match the productivity of getting AI to implement the features I want. It's basically getting multiple devs to set out and go do work for you where you just tell them what you want and provide iterative feedback till they implement all the features you want, in the way you want and to…
The industry of "software" is so large... While I agree with web development going this route, I'm not sure about "everything else".
You could argue that that's the bulk of all software jobs in tech, and you'd likely be correct... But depending on what your actual challenge is, LLM assistance is more of a hindrance then help. However creating a web platform without external constraints makes LLM assistance shine, that's true
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#59Earlier quoted context omitted.
It just depends on what you are doing. A green field react app in typescript with a CRUD API behind? The LLMs are a mind blowing assistant and 1000t/s is crazy. You are doing embedded development or anything else not as mainstream as web dev? LLMs are still useful but no longer mind blowing and often produce hallucinations. You need to read every line of their output. 1000t/s is crazy but no longer always in a good w…
I've said it before but no one takes it seriously: LLMs are only useful if you're building something that's already in the training set ie already commodity . In which case why are you building it ???
It's true, when I was working with LLMs on a novel idea it said sorry I can't help you with that!
Re: Cerebras Code now supports GLM 4.6 at 1000 tokens/sec
#60Was able to sign up for the Max plan & start using it via opencode. It does a way better job than Qwen3 Coder in my opinion. Still extremely fast, but in less than 1 hour I was able to use 7M input tokens, so with a single agent running I would be able easily to pass that 120M daily token limit. The speed difference between Claude Code is significant though - to the point where I'm not waiting for generation most of…
They don't have prefix caching? Claude and Codex have this.