TinyTinyTPU: 2×2 systolic-array TPU-style matrix-multiply unit deployed on FPGA

Post by alanma » Fri, Jan 02, 2026, 10:19 PM UTC

NitpickLawyer wrote: post #6

aunty_helen wrote: post #3

I think it’s only a matter of time before we see asic vendors making TPU devices. Same thing happened with BTC. There was enough money there to spawn an industry. Nvidias 70% margins are too hard to ignore. And if playing on the open market seems too rough, there’s always acquisition potential like what happened to groq.

Aren't high end accelerators already closer to ASICs than to og GPUs, tho?

Post by Lerc » Fri, Jan 02, 2026, 10:20 PM UTC

fooblaster wrote: post #12

mrinterweb wrote: post #9

I've been wondering when we will see general purpose consumer FPGAs, and eventually ASICs, for inference. This reminds me of bitcoin mining. Bitcoin mining started with GPUs. I think I remember a brief FPGA period that transitioned to ASIC. My limited understanding of Google's tensor processing unit chips are that they are effectively a transformer ASIC. That's likely a wild over-simplification of Google's TPU, but G…

FPGAs will never rival gpus or TPUs for inference. The main reason is that GPUs aren't really gpus anymore. 50% of the die area or more is for fixed function matrix multiplication units and associated dedicated storage. This just isn't general purpose anymore. FPGAs cannot rival this with their configurable DSP slices. They would need dedicated systolic blocks, which they aren't getting. The closest thing is the vers…

Post by alanma » Fri, Jan 02, 2026, 10:23 PM UTC

fooblaster wrote: post #5

Great! How do you program it?

    def __init__(self):
        super().__init__()

        self.layer1 = nn.Linear(2, 2, bias=False)
        self.layer2 = nn.Linear(2, 2, bias=False)

    def forward(self, x):
        x = self.layer1(x)
        x = torch.relu(x)
        x = self.layer2(x)
        return x

Post by nightshift1 » Fri, Jan 02, 2026, 10:40 PM UTC

mrinterweb wrote: post #13

tucnak wrote: post #11

Earlier quoted context omitted.

It all comes down to memory and fabric bandwidth. For example, the state of the art developer -friendly (PCIe 5.0) FPGA platform is Alveo V80 which rocks four 200G NIC's. Basically, Alveo currently occupies this niche where it's the only platform on the market to allow programmable in-network compute. However, what's available in terms of bandwidth—lags behind even pathetic platforms like Bluefield. Those in the know…

As far as I understand all the inference purpose-build silicon out there is not being sold to competitors and kept in-house. Google's TPU, Amazon's Inferentia (horrible name), Microsoft's Maia, Meta's MTIA. It seems that custom inference silicon is a huge part of the AI game. I doubt GPU-based inference will be relevant/competitive soon.

Post by bee_rider » Fri, Jan 02, 2026, 10:51 PM UTC

mrinterweb wrote: post #9

I've been wondering when we will see general purpose consumer FPGAs, and eventually ASICs, for inference. This reminds me of bitcoin mining. Bitcoin mining started with GPUs. I think I remember a brief FPGA period that transitioned to ASIC. My limited understanding of Google's tensor processing unit chips are that they are effectively a transformer ASIC. That's likely a wild over-simplification of Google's TPU, but G…