Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA Architecture for Deep Learning: Survey and Future Directions

arxiv.org

Post reply on HN
Print view View original

121–130 of 135 posts

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#121

Post by Aromasin »

weinzierl wrote: post #112
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Every couple of years I revisit the FPGA topic, eager to build something exciting. I always end up with a ton of research, where I learn a lot but ultimately shy away from building something. This is because I cannot find a project that is doable and affordable for a hobbyist but at the same time requires an FPGA in some sense. To put it bluntly: I can blink a LED for a fiver with a micro instead of spending hundreds…

I work at one of the big 3 FPGA companies, so I can give you an idea of where our teams spend most of their time, and you can translate that into a hobbyist project as you will.

1. Video and Broadcast. Lots of things to be done here. New protocols are being introduced every year by IEEE for sending video between systems. Most cutting-edge cameras have some sort of FPGA inside doing niche image processing. You can get a sensor and build yourself your own Camera-on-Chip. It's a fantastic way to lose a year or two (I can attest to that). Some good material on the matter here: https://www.mathworks.com/discovery/fpga-image-processing.ht...

2. Compute Acceleration. This is more data centre-specific. SmartNICs, IPUs and the like. Hard to make a dent unless you want to spend 200k on a DevKit, but you could prototype one on a small scale. Some sort of smart FPGA switch that redirects Ethernet traffic between a bunch of Raspberry Pis dependent on one factor or another. One company that comes to mind is Napatech. They make a bunch of really interesting FPGA servers systems: https://www.napatech.com/products/nt200a02-smartnic-capture/

3. Robotics and Computer Vision. Plenty of low-hanging fruit to be plucked here. A rediculous amount of IO, all needed to work in near realtime. Hardware acceleration kernels on top of open standards like ROS 2. I always point people in the direction of Acceleration Robotics' startup in Barcelona for this. They're epic: https://github.com/ros-acceleration

4. Telecomunications. This is a bit of a dark art area for me, where the RF engineers get involved. From what my colleagues tell me, FPGAs are good for this because any other device doesn't service the massive MIMO antenna arrays besides building custom ASICs, and the rate of innovation in this area means an ASIC made one year is redundant the next. Software-defined radios are the current trend. You could have fun making your own radio using an FPGA: https://github.com/dawsonjon/FPGA-radio

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#122

Post by megous »

sitkack wrote: post #98
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

$300 board (I'm including shipping and customs) is not low end. Low end FPGA boards are ~30$ these days.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#123

Post by weinzierl »

scottapotamas wrote: post #118
weinzierl wrote: post #112

Earlier quoted context omitted.

Every couple of years I revisit the FPGA topic, eager to build something exciting. I always end up with a ton of research, where I learn a lot but ultimately shy away from building something. This is because I cannot find a project that is doable and affordable for a hobbyist but at the same time requires an FPGA in some sense. To put it bluntly: I can blink a LED for a fiver with a micro instead of spending hundreds…

Reasonably experienced and 'a week' can mean vastly different things... It's certainly easier to keep the cost down with longer time-frames. For a focus on electronics rather than implementing some kind of toy 'algorithm accelerator', I find low-hanging/interesting projects where the combination of requirements exceed a micro's peripheral capabilities - i.e. multiple input/output/processing tasks which could be perfo…

Good list, thanks. I have a couple of years professional experience as a software dev and worked in the embedded space too. Nowadays I am in security and that is definitely an area of interest.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#124

Post by adrian_b »

weinzierl wrote: post #109
sitkack wrote: post #98

Earlier quoted context omitted.

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

If I remember correctly about 80% of a modern FPGA's silicon is is used for connections. FPGA have their uses and very often a big part in them is the Field Programmability . If that is not required, there is no good reason another solution (ASIC, GPU, etc.) couldn't beat the FPGA in theory. Now, in practice there are some niches, where this is not absolutely true, but I agree with GP that I see challenges for deep l…

An ASIC will always have better performance than an FPGA, but it will have an acceptable cost only if it is produced in a large enough number. You will always want an ASIC, but only seldom you will able to afford it.

So the decision of ASIC vs. FPGA is trivial, it is always based on the estimated price of the ASIC, based on the number of ASICs that would be needed.

The decision between off-the-shelf components, i.e. GPUs and FPGAs, is done based on performance per dollar and performance per W and it depends very strongly on the intended application. If the application must compute many operations with bigger numbers, e.g. FP32 or FP16, then it is unlikely that an FPGA can compete with a GPU. When arithmetic computations do not form the bulk of an algorithm, then an FPGA may be competitive, but a detailed analysis must be made for any specific application.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#125

Post by touisteur »

tehsauce wrote: post #113
sitkack wrote: post #98

Earlier quoted context omitted.

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

500GB/s is going to limit it to at best 1/4 the DL performance of an nvidia gpu. I’m not sure what the floating point perf of these FPGAs are but I imagine that also might set a fundamental performance limit at a small fraction of a GPU.

Well I keep seeing all models quantized and for 2-bit, 4-bit and 1-bit quantizations I had good very good inference performance (either througput or latency) on CNNs and some RNNs on Alveo boards using FINN (so, mostly high level synthesis and very little actual fpga wrangling). No idea about the current status of all these, will read the paper though :-)

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#126

Post by scottapotamas »

weinzierl wrote: post #123
scottapotamas wrote: post #118

Earlier quoted context omitted.

Reasonably experienced and 'a week' can mean vastly different things... It's certainly easier to keep the cost down with longer time-frames. For a focus on electronics rather than implementing some kind of toy 'algorithm accelerator', I find low-hanging/interesting projects where the combination of requirements exceed a micro's peripheral capabilities - i.e. multiple input/output/processing tasks which could be perfo…

Good list, thanks. I have a couple of years professional experience as a software dev and worked in the embedded space too. Nowadays I am in security and that is definitely an area of interest.

I only dabble with recreationally reverse engineering industrial/consumer grade HW and following blogs/conferences, so I can only provide a rough shotgun of search terms to try and hit something you're interested in:

- The Glasgow interface explorer is an example of a smaller FPGA making interface level RE tooling more accessible.

- The Chipwhisperer hardware has a focus on power supply glitching, side-channel attacks and general hardware security education/testing.

- There's a handful of FPGA-based implementations intended for high-speed protocol sniffing/MiTM (TCP/IP, USB and CANBus are both pretty common) on github etc, Cynthion is one example.

- Some recent projects have been trying to implement and improve the FOSS ARM Cortex programming and trace experience, Orbuculum ORBTrace probe is an example though the benefits aren't fully realised yet.

- In an odd use-case for an FPGA, I've personally seen hardware that enforces brutal/paranoid DRM/licencing via customised downloaded bitstreams to guards against reverse-engineering/copy efforts, all to most likely run a soft-CPU. I've read (unsubstantiated) that this approach appears on some military hardware.

- Slightly adjacent to specific FPGA projects, but the SDR tooing ecosystem has lots of cool stuff to play with for wireless signal identification/spoofing/re-implementation. HackRF, LimeSDR, GNUradio etc. If you want to get deep then there's lots of overlap with custom FPGA implementations.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#127

Post by weinzierl »

scottapotamas wrote: post #126
weinzierl wrote: post #123

Earlier quoted context omitted.

Good list, thanks. I have a couple of years professional experience as a software dev and worked in the embedded space too. Nowadays I am in security and that is definitely an area of interest.

I only dabble with recreationally reverse engineering industrial/consumer grade HW and following blogs/conferences, so I can only provide a rough shotgun of search terms to try and hit something you're interested in: - The Glasgow interface explorer is an example of a smaller FPGA making interface level RE tooling more accessible. - The Chipwhisperer hardware has a focus on power supply glitching, side-channel attack…

Thanks a lot. This is a rabbit hole I will happily go down.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#128

Post by camel-cdr »

WanderPanda wrote: post #102

Is there any small risc-v soft-core with big ass SVE (scalable vector extensions)? I would like to play around with them but the only option seem to be cloud instances like gravitron and small (128bit) like the licheerv nano (c906) which also seems to only support a beta version of the standard.

You seem to be confusing two different things:

gravitron: has ARM SVE extension

licheerv nano: has RISC-V Vector extension

Both ARM SVE and the RISC-V Vector Extension (RVV) are vector length agnostic, but quite different in their design.

Since RISC-V allows for open source implementations, there are a few projects working on RVV implementations. Check out the t1 project, they implement a configurable RVV implementation, that is supposed to be driven by a small scalar core, e.g. RocketChip. [0] There are configurations for vector length of 512-bit to 16K bit. Connecting it to RocketChip isn't officially supported just yet, and it uses the spike emulator for the scalar part.

They offer a pre-build docker environment, so you can play around with the RTL simulation via "docker run --name t1 -it -v $PWD:/workspace --rm ghcr.io/chipsalliance/t1-machamp:latest /bin/bash" This drops you into a shell, and you can start simulating a 1K vector length processor, with "ip-emulator --no-logging -C yourProgram", see the tests/ director for example code. At least in theory, but there might still be a few bugs.

[0] https://github.com/chipsalliance/t1

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#129

Post by ChrisKjellqvist »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

I think the problem with such an architecture is that latency isn't a limit on timing closure from a hardware sense now, but you still have to consider it now from the software compilation perspective, and it might severely impact performance.

From what I've thought about this, the problem applies to any high fanout signal. For instance, if you want to implement a multiplexer for two n-bit operands, you'll need the select bit to be in ~n places at once (if you compile the n-bit multiplex into n 1-bit multiplexes). Compiling LUTs to route this select signal to the right places in the grid synchronously with the arrival of the data signals is complex and amounts to a similar sort of problem one faces with hardware compilation (akin to setup timing). In this architecture, you're replacing actual routing resources (wires) with LUT entries. Instead of considering the propagation of a signal down a wire in terms of nanoseconds, you'll be thinking about it in terms of cycles to traverse the grid. Unless the clock rate is absurdly high, signals like this will probably cause a performance problem for your design.

FPGAs/ASICs also have a problem of this flavor but it generally only happens for one signal: the clock. FPGAs address this by not using regular routing resources for the clock, and instead using special, pre-routed nets for clock distribution. I imagine you'd probably need a solution like this to deal with high fanout signals in an efficient way.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#130

Post by zachbee »

sitkack wrote: post #98
zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time. You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice... Modern FPGA platforms like Xilinx Alveo have 35TB/s of SR…

I'm definitely not! I'm a hardware designer and I work with FPGAs all the time, for both work and for personal projects. Like with all things, there's a right tool for every job, and I think for modern DL algorithms like Transformers, GPUs and AI ASICs are the better tools. For rapid hard prototyping, or for implementing specialized architectures, FPGAs are far better.
Post reply on HN