Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA Architecture for Deep Learning: Survey and Future Directions

arxiv.org

Post reply on HN
Print view View original

111–120 of 135 posts

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#111

Post by ooterness »

mikewarot wrote: post #105
ooterness wrote: post #100

Earlier quoted context omitted.

What you're describing is a cellular automaton, in the same vein as of Conway's Game of Life. You can do lots of interesting things with those, but it's emphatically not where I'd start for a flexible computing platform. Why not go the extra mile, and make each tile a small CPU? There's a Zachtronics have called TIS-100 with this premise. https://store.steampowered.com/app/370360/TIS100/

Because each cell has it's own state, and 64 bits of "program" (16 bits in each of the 4 LUTs), it's unlike the game of life, where the rule is the same for each cell. I looked at a lot of choices for architecture, and wanted to allow data paths to cross without conflict, and the 4 in/4 out choice worked best without going too far. Someone did work out how you could run the game of life on a BitGrid, it's in the Esot…

The power is easy enough to calculate.

Let's take the a Kintex Ultrascale+ from Xilinx as a fairly typical example of a modern FPGA. Relevant documentation is the UltraScale Architecture CLB User Guide [1] and the Xilinx Power Estimator spreadsheet [2].

Each "slice" contains two flip-flops and a lookup table with 6 input bits and 2 output bits. So two slices is enough to implement each cell with room to spare.

Let's say you have a 200 x 200 grid = 40k cells. That's 80k LUTs and 160k flip-flops. That's about 29% of the resources on a XCKU9P. If we assume a 100 MHz clock and 25% toggle rate (somewhat arbitrary), that's 4e12 state-changes per second. The spreadsheet indicates that circuit will consume 850 mW, or about 200 fJ per state-change.

That said, this is NOT an efficient way to do arithmetic. You'd need N cells to do a fixed-point addition with N-bit arguments, and O(N^2) (give or take) to do a fixed-point multiplication. Floating point requires orders of magnitude more. There's a reason modern FPGAs have dedicated paths for fast addition and hardwired multiplier macros.

[1] https://www.xilinx.com/content/dam/xilinx/support/documents/...

[2] https://www.xilinx.com/products/technology/power/xpe.html

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#112

Post by weinzierl »

zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Every couple of years I revisit the FPGA topic, eager to build something exciting. I always end up with a ton of research, where I learn a lot but ultimately shy away from building something.

This is because I cannot find a project that is doable and affordable for a hobbyist but at the same time requires an FPGA in some sense. To put it bluntly: I can blink a LED for a fiver with a micro instead of spending hundreds for an FPGA.

So, assuming I am reasonably experienced in software development and electronics and I have 1000 USD and a week to spend.

What could I build that shows off the capabilities of an FPGA?

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#113

Post by tehsauce »

sitkack wrote: post #98
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

500GB/s is going to limit it to at best 1/4 the DL performance of an nvidia gpu. I’m not sure what the floating point perf of these FPGAs are but I imagine that also might set a fundamental performance limit at a small fraction of a GPU.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#114

Post by imtringued »

zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Um, no? The actual problem is that most FPGAs already have DPUs for machine learning integrated on them. Some Xilinx FPGAs have 400 "AI Engines" which provide significantly more compute than the programmable logic, the almost 2000 DSP slices or the ARM cores. This means that the problem with FPGAs is primarily lack of SRAM and limited memory bandwidth.

https://www.xilinx.com/products/boards-and-kits/vck190.html

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#115

Post by rlupi »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Interesting idea.

The main question for me is if it will be efficient, in the sense that you need program/models that can be binpacked into the size of your design and need data all at the same time in various stages otherwise a lot of your silicon will be under-utilized (since you don't have memory, you can't trade between compute and RAM to efficiently use your silicon die size).

Rather than our current breed of neural network architectures and models, you'd probably need to look into alternatives like spiking neural network and see if they can store data as frequency and activation patterns.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#117

Post by imtringued »

ooterness wrote: post #100
mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

What you're describing is a cellular automaton, in the same vein as of Conway's Game of Life. You can do lots of interesting things with those, but it's emphatically not where I'd start for a flexible computing platform. Why not go the extra mile, and make each tile a small CPU? There's a Zachtronics have called TIS-100 with this premise. https://store.steampowered.com/app/370360/TIS100/

>Why not go the extra mile, and make each tile a small CPU?

Xilinx AI Engine and Ryzen AI is exactly that.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#118

Post by scottapotamas »

weinzierl wrote: post #112
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Every couple of years I revisit the FPGA topic, eager to build something exciting. I always end up with a ton of research, where I learn a lot but ultimately shy away from building something. This is because I cannot find a project that is doable and affordable for a hobbyist but at the same time requires an FPGA in some sense. To put it bluntly: I can blink a LED for a fiver with a micro instead of spending hundreds…

Reasonably experienced and 'a week' can mean vastly different things... It's certainly easier to keep the cost down with longer time-frames.

For a focus on electronics rather than implementing some kind of toy 'algorithm accelerator', I find low-hanging/interesting projects where the combination of requirements exceed a micro's peripheral capabilities - i.e. multiple input/output/processing tasks which could be performed on a micro individually, but adding synchronisation or latency requirements makes it rather non-trivial.

- Very wide/parallel input/output tasks: ADC/DACs for higher samplerate/bitdepth/channel count than typically accessible with even high-end micros

- Implementing unique/specialised protocols which would have required bit-banging, abuse of timer/other peripherals on a micro (i.e. interesting things people achieve with PIO blocks on RP2040 etc)

- Signal processing: digital filters and control systems are great because you can see/hear/interact with the output which can help build a sense of achievement.

When starting out, it's also less overwhelming to start with smaller parts and allocate the budget to the rest of the electronics. They're still incredibly capable and won't seem as under-utilised. Some random project ideas:

- Driving large frame-buffers to display(s) or large sets of LED matrices at high frame rate - https://gregdavill.com/posts/d20/

- Realtime audio filters - the Eurorack community might have some inspiration.

- Multi-channel synchonous detection, lock-in amplifiers, distributed timing reference/control,

- Find a sensing application that's interesting and then take it to the logical extreme - arrays of photo/hall-effect sensors sampled at high speed and displayed, accelerometers/IMU sensor fusion

- Laser galvanometers and piezo actuators are getting more accessible

- Small but precise/fast motion stages for positioning or sensing might present a good combination of input, output, filtering and control systems.

- With more time/experience you could branch into more interesting (IMO) areas like RF or imaging systems.

With more info about your interest areas I can give more specific suggestions.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#120

Post by RetroTechie »

rlupi wrote: post #115
mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

Interesting idea. The main question for me is if it will be efficient, in the sense that you need program/models that can be binpacked into the size of your design and need data all at the same time in various stages otherwise a lot of your silicon will be under-utilized (since you don't have memory, you can't trade between compute and RAM to efficiently use your silicon die size). Rather than our current breed of ne…

> since you don't have memory, you can't trade between compute and RAM to efficiently use your silicon die size

As I understand it, BitGrid would be a nice architecture for applications which:

  # Are compute-heavy
  # Don't need a lot, or high-bandwidth memory
  # Where the complex architecture (programming) of GPUs/FPGAs is a barrier
  # Where the computation is something other than one for which dedicated accelerators exist (like GPU, DSPs, or a CPU's vector extensions, FP math etc)
If most or all of those conditions hold, BitGrid could be a good architecture. Ideally, configured/programmed as if it were a simple memory array.

No doubt such applications exist. But between those many competing options, I suspect BitGrid will have a hard time carving out a niche where its advantages weigh heavily enough to matter.

Disclaimer: if nothing else, I would love it as a dead-simple-to-use-sea-of-LUTs. FPGAs are powerful but complicated beasts (and usually rely on closed source toolchains).

Post reply on HN