51–60 of 65 posts
Founder here! We're still in stealth, but I'll be able to share details and performance figures soon. Our first product is a bet on transformers. If we're right, there's enormous upside - being transformer-specific lets you get an order of magnitude more compute than more flexible accelerators (GPUs, TPUs). We're hiring - if the EV makes sense for you, reach out at gavin @ etched.ai
Any thoughts on State Space Models?
Eg:
Where did this come from? There is absolutely nothing clickable except 'contact us' which just reloads the same page? There's almost zero information here?
Maude you have JS disabled? It’s one of those fancy animations as you scroll websites.
Earlier quoted context omitted.
In the cloud, these chips will compete head to head with GPUs. If they are able to pull off a 10x price/performance win without excessive porting work… it’ll take off in a heartbeat.
Like ASIC Botcoin miners did. There are parallels here in how it might just pan out.
BFL mined with customer hardware and basically didn't ship units to customers until there was no profit in running one.
Crypto ASICs are a super weird edge case IMHO in chips, strictly speaking it's not rational to sell them if they are very profitable. It only makes sense if the customer has a different risk profile than you; or the customer can somehow get power more cheaply than you; or you have some kind of scam going on; or you couldn't get capital except by presales and are unusually honest.
Note that an additional profit-making option for crypto ASIC producers is to secretly over-produce and compete with your customers and you are unlikely to get caught doing this.
Earlier quoted context omitted.
Like ASIC Botcoin miners did. There are parallels here in how it might just pan out.
Oh man, all those years back I made a choice between antminer and butterfly labs. I backed the wrong horse. BFL mined with customer hardware and basically didn't ship units to customers until there was no profit in running one. Crypto ASICs are a super weird edge case IMHO in chips, strictly speaking it's not rational to sell them if they are very profitable. It only makes sense if the customer has a different risk p…
Earlier quoted context omitted.
It’s a bet. Probably a good one to make. The upside of being the ones who have an AI chip (not a graphics chip larping as an AI chip) is huge. It will run faster and more cheaply. You get to step all over OpenAI, or get a multi billion dollar deal to supply Microsoft data centres. Or these ship on every new laptop etc. You get to be the next unicorn ($1tn company). So that is a decent bet for investors assuming the t…
Yep, transformers showed up in 2017, nearly 7 years ago, and they still wear the crown. Maybe some new architecture will come to dominate eventually, but I would love a low cost PCIe board that could run 80B transformer models today.
What about Transformers on FPGAs?
FPGAs are designed to fight latency as much as possible. To do this, they have networks of switches to shuttle bits across the chip and keep delays to the bare minimum, in order for synchronous logic to be able to run at the highest possible clock rates for signals that traverse the entire chip. To meet this goal, there's a huge amount of effort required to compile a program written in Verilog, VHDL, etc.. into a set…
This is true and misleading at the same time. Filling a large FPGA takes time, but if you are working with a small FPGA the turnaround time can be 15 minutes.
Earlier quoted context omitted.
It’s very difficult to implement, and requires training the network to use it. I worked at DeepMind on projects that used MCTS. Even with access to the AlphaZero source code, it was very difficult to write an other implementation that got the same results as the original.
I'm really curious about this part: > and requires training the network to use it. I thought one of the benefits of MCTS was, if you already have your value network, then a general MCTS implementation can walk the tree of values created by that network. And so no special update to the model is necessary. But I'm probably wrong about this. (also, it boosts my confidence to hear that even folks at DeepMind find MCTS di…
In my experience, PUCT does a lot better than UCT, so you want to also have a prior network.
You don’t have to train a new network, but in my experience, it works much better. I haven’t spent a ton of time using off the shelf networks with MCTS though. Maybe it works great.
very subtle bugs is the MCTS experience. Particularly once parallelism is involved.
Earlier quoted context omitted.
Interesting point. That said, the AI model space is rapidly evolving, while bitcoin's hashing problem is static. This makes it significantly more risky to make a large capital investment in dedicated HW when it's unclear if it will be able to run the next big model architecture. For instance, if this had been built + released a year ago, before SOTA models used MoE , then it would rapidly have become obselete.
Outside of hardware/implementation optimizations, and position embedding choice - has the SOTA transformer architecture evolved that much? Llama-2 code appears to be about the same as gpt-2.
100T models on one chip with MCTS search.
That is some impressive marketing.
I’ll believe it when I see it.
Great to see so many hardware startups.
Future is deffo accelerated neural nets on hardware.