70b runs on 4x CS-3 estimated at $2-3m each, let’s say total system cost $10m, drawing ~100kW power. They don’t mention batch size, so let’s start with batch size 1 and see where we get. At 100% utilisation for 3 years that’d be 42 billion tokens for a cost of $10m capital plus ~$0.5m power and cooling let’s say, or $250 per million tokens. They’re claiming they can sell their API access at $0.60/million. To break ev…
your estimate of $2-3M per CS-3 is a price not a cost. It costs about $20K per wafer from TSMC and the price they charge reflects the NRE of designing the system and taping out the masks more than the additional costs to package up their wafers into a system. If this business scales, they can probably afford to lower the price by a factor of 10.
So for $20,000 (in quantity) you get somewhere around 10 trillion transistors?
That's enough for about 50 4096x4096 multiply-accumulate chips. At a nice slow 1 Mhz clock rate, each would take about 3.5 watts, and give you 16 teraflops of performance. If you stepped up the power and cooling, you could likely get to 350 watts of power at 100 Mhz, and 1.6 Petaflops.
50 of those chips, for $20,000 --> $400 each