I wanna see an inference chip where the weights are part of the rom of the chip. There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle. That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second…
You don't need a single wafer, you can split the model into many smaller different chips and connect inputs/outputs.
Skip VHDL and directly go for GDSII / OASIS. Try to find similar vectors so you get re-usable blocks.
You can dynamically calibrate a chip by fine tuning output.
wow if they can get something like this working, what happens to all this infrastructure? Hyperscalers have to be assuming the lifespan of that stuff wrong considering the next gen will be 1000x more efficient.
The question isn’t whether it works (it does); the question is whether there are buyers for hardware that is obsolete the day it ships. Models evolve much more quickly than hardware can keep up.
There may be all sorts of stable use case models that this could be interesting for. Imagine permanent voice translation circuits at a tiny fraction of the current price, glasses that subtitle the world with long battery life.
The question isn’t whether it works (it does); the question is whether there are buyers for hardware that is obsolete the day it ships. Models evolve much more quickly than hardware can keep up.
Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt. If you can build chips that could run one specific LLM 100x faster than anything else, it would have a us…
Those taalus chips apparently run at 1/10 the power as the current SOTA GPU setups. If they can execute even partially on their plan, it'll be a literal game changer.
In fact any application where the task is stable and the model good enough to address that task. As you suggest, industrial applications where a robot must deal with variants of the same repetitive task. Or a military drone which needs to be jamming proof.
> Or a military drone which needs to be jamming proof. That, if used in war, I would think, would need the ability to be updated frequently. For example, your enemy might find out (say by running tests on hardware they captured from you) that painting some red paint in a particular shape (a smiley might even work) on their hardware prevented your drones from attacking them because it confuses that pattern with the Re…
You keep the "reasoning core" burned and play the cat-and-mouse game at the I/O edge. Enemy invents a smiley shield, your R&D figures out some filtering step that defeats this effect without compromising general image recognition. Then the enemy figures out a new trick, your R&D invents a countermeasure, and so on - point is, this can happen for a long time in layers on top of the core model. If the enemy invents some robust way to attack the core that cannot be filtered out, it's game over for that hardware, but that is a much more difficult task and might take longer than expected service time of a given batch of drones.
He's definitely not talking about worthy endeavour. He's talking about an endeavour reaching the market. I'm sure if Zuckerberg wants to spend $10B on Nuclear Fusion it will happen.
It’s fission , not fusion: https://www.esgdive.com/news/meta-inks-nuclear-deals-terrapo... …and if they do all of this, it’ll be closer to $20B than 10!
If all it took to get viable fusion power was a FAANG CEO with $10B to burn, I'd be first to petition for it to happen, and even throw whatever money I can spare onto that pyre.
Designing circuit board and 3D models (even using something like OpenSCAD) is a very spatial process today. You are dealing with coordinates one way or another. This is very unlike how FPGA and (I assume) ASIC is done. That is more like a traditional programming language but everything happens all at once (no sequence of statements outside tests, if you need that you have to write a state machine yourself). You defin…
Autorouting PCBs doesn’t really give usable results on all but the simplest cases. It seems to be a very difficult problem to solve even though a human doing it is only following a relatively simple bunch of rules and goals in his or her head.
Simple bunch of rules and goals backed by extremely sophisticated visual intuition.
Pretty sure someone already tried throwing VLMs and diffusion models at this, wonder how that fared.