Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA Architecture for Deep Learning: Survey and Future Directions

arxiv.org

Post reply on HN
Print view View original

101–110 of 135 posts

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#101

Post by woopsn »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Cool idea. What does the expression/spec language look like? I would guess it has to be mapped... not straightforwardly. It seems like your goal is maximize throughput, but then the datapath would have to be planar, no?

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#102

Post by WanderPanda »

Is there any small risc-v soft-core with big ass SVE (scalable vector extensions)? I would like to play around with them but the only option seem to be cloud instances like gravitron and small (128bit) like the licheerv nano (c906) which also seems to only support a beta version of the standard.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#103

Post by pyinstallwoes »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Sounds similar to Greenarray :

https://web.archive.org/web/20230331155747/https://www.green...

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#104

Post by shrubble »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

DE Shaw has a systolic supercomputer: https://en.m.wikipedia.org/wiki/Anton_(computer)

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#105

Post by mikewarot »

ooterness wrote: post #100
mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

What you're describing is a cellular automaton, in the same vein as of Conway's Game of Life. You can do lots of interesting things with those, but it's emphatically not where I'd start for a flexible computing platform. Why not go the extra mile, and make each tile a small CPU? There's a Zachtronics have called TIS-100 with this premise. https://store.steampowered.com/app/370360/TIS100/

Because each cell has it's own state, and 64 bits of "program" (16 bits in each of the 4 LUTs), it's unlike the game of life, where the rule is the same for each cell.

I looked at a lot of choices for architecture, and wanted to allow data paths to cross without conflict, and the 4 in/4 out choice worked best without going too far.

Someone did work out how you could run the game of life on a BitGrid, it's in the Esoteric Languages wiki

https://esolangs.org/wiki/Bitgrid

I see it as something like a Turing machine, a bit less abstract, and much, much faster at computing real results. I hope it can democratize access to PetaFLOPS.

The question I can't seem to find an answer to is simple... how much power does a 4 bit in/out set of LUTs with a latch take statically? How many femtojoules does it take to switch?

If those numbers are good enough, it's entirely possible that really fast compute is on the table of possibilities. If not, it's another Turing machine.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#106

Post by mikewarot »

woopsn wrote: post #101
mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Cool idea. What does the expression/spec language look like? I would guess it has to be mapped... not straightforwardly. It seems like your goal is maximize throughput, but then the datapath would have to be planar, no?

I don't have a language for it... I've been stuck at analysis paralysis for far, far too long on this one. The code I did write, was all figured out by hand.

I'm thinking it'll have to end up being a set of equations, much like the tables that get spewed when you compile VHDL for an FPGA.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#107

Post by 15155 »

ZoomerCretin wrote: post #90

I'm glad to see this is being studied. I did a brief half-semester project summarizing the usefulness of accelerators after the death of Dennard's law and Moore's law (this was when Intel was still pushing out incremental improvements on 14nm). The short summary is that accelerators offered substantial performance per watt improvement, at the cost of longer development timelines, cost of manufacturing in the case of…

> AMD offers much better performance per dollar than Nvidia, but of course their poor drivers make using their hardware a fool's errand.

I would never use their IP or drivers, but Xilinx hardware is top notch. What drivers are you using? XRT?

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#108

Post by buildbot »

ZoomerCretin wrote: post #90

I'm glad to see this is being studied. I did a brief half-semester project summarizing the usefulness of accelerators after the death of Dennard's law and Moore's law (this was when Intel was still pushing out incremental improvements on 14nm). The short summary is that accelerators offered substantial performance per watt improvement, at the cost of longer development timelines, cost of manufacturing in the case of…

Brainwave is/was very successful? Still serving your searches! (I work on one of the teams that came out of brainwave, not with them directly)

https://www.microsoft.com/en-us/research/project/project-bra...

https://www.microsoft.com/en-us/research/publication/a-confi...

https://www.microsoft.com/en-us/research/blog/a-microsoft-cu... -> This led to the OCP MX work:

https://www.opencompute.org/blog/amd-arm-intel-meta-microsof...

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#109

Post by weinzierl »

sitkack wrote: post #98
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

If I remember correctly about 80% of a modern FPGA's silicon is is used for connections. FPGA have their uses and very often a big part in them is the Field Programmability. If that is not required, there is no good reason another solution (ASIC, GPU, etc.) couldn't beat the FPGA in theory. Now, in practice there are some niches, where this is not absolutely true, but I agree with GP that I see challenges for deep learning.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#110

Post by tehsauce »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Systolic arrays are essentially how matmul is implemented in tensor cores in GPUs and TPUs.
Post reply on HN