141–150 of 349 posts
Earlier quoted context omitted.
Cerebras is a programmable accelerator, but Taalas does burn the model into the gates.
Ah, right. I misremembered. Does Taalas have a meaningful advantage here then?
Earlier quoted context omitted.
What precisely is an ASIC supposed to do that a more programmable accelerator can't? Memory latency is memory latency: doesn't matter whether it's embedded in a cache-line wait state or some flipflop state machine. ROM isn't going to be faster than RAM either. Likewise, for compute, is the ASIC somehow going to beat a systolic array? You can't have one circuit per weight: the die area and electrical fan-out would be…
No, if you burn your model into the silicon you don't need memory as the output of a layer flows through circuitry directly into the next one. No I/O to memory of any kind. You still need a bit of memory for in flight answers, but that's it.
I can accept the idea of specializing a circuit for a specific model shape, but I'm not seeing a need to specialize a circuit for the weights inside the shape.
To everyone praising Open weight models, could you answer a simple question? If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models? Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.
You can simplify your question even more: To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?
Earlier quoted context omitted.
I mean, if I imagine Anthropic giving away unlimited Sonnet 4.5 away at $20, I would still be paying the $200 for fable. It is a bit like saying "why would you hire someone with a doctorate when you could get unlimited high school grads". How appealing that sounds depends on your needs.
It’s like how most people on here would want a loaded MacBook Pro or RTX 5090, but chromebooks and iGPUs do volume. There’s absolutely a place for a lifestyle subscription to a sonnet model that you could just use everywhere all the time.
Earlier quoted context omitted.
Ah, right. I misremembered. Does Taalas have a meaningful advantage here then?
6x. Taalas has Llama3.1 8B running at 18000 tok/s. Cerebras advertised that model at 3000 tok/s.
The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest. The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release. Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software en…
Open weights are a red herring if you can’t run the model independently. Good luck fitting these models on your Mac mini. In fact the superior models are irreducibly nothing but superior web services run from China.
Nobody needs "web services run from China" to use Chinese open weight models.
Earlier quoted context omitted.
Except aren’t these Chinese models having to go big too? While they might not be Mythos/Fable sized, a trillion plus parameters is hardly a local server or even desktop to mainframe style jump
I've had 2b models give a plausible Paris vacation itinerary. A tools-capable 12b and especially 30b model from 2026 is certainly capable of producing passable results. I was demonstrating the qwen 3.6 27b model I stood up last week to my wife and it gave her a passable Moroccan Chicken recipe. With tool calling (search) they're quite good.
Any value will come from the largest models, and those largest models are unlikely to ever run on consumer hardware within their window of relevancy.
The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest. The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release. Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software en…
While running the model at 9000 tokens/s is the more flashy demo, I imagine running 1000 concurrent requests at 150 tokens/s each is the much more achievable goal
Earlier quoted context omitted.
is anyone doing this ?
The closest example I've seen is ChatJimmy: https://chatjimmy.ai/ a prototype from Taalas running Llama 8B Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging. If I was younger and had the right background, I'd love to dive into attempting somethign like this
The open weight, open architecture releases of the past several days has me more convinced that ultimately, the winner will be whoever burns their models to ASICs fastest. The LLMs themselves are capable of doing some aspects of chip design as evinced by the K3 press release. Furthermore, the frontier models are "good enough" for a wide swathe of tasks and will soon hit that threshold for a good amount of software en…
I think the SotA is moving too fast for the production timelines of an ASIC, wouldn't you think? People are just now coming out with LLAMA ASICS but who would want to use LLAMA? Or I guess you are arguing that the models _now_ will be durably useful enough to commit the time to creating the ASIC?
guess you are arguing that the models _now_ will
be durably useful enough to commit the time to
creating the ASIC?
Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"
The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.
You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.
So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?