Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA Architecture for Deep Learning: Survey and Future Directions

arxiv.org

Post reply on HN
Print view View original

91–100 of 135 posts

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#91

Post by ZoomerCretin »

woopsn wrote: post #82

Reconfigurable logic may be used to implement fairly small models, in applications where they're already employed and adding a coprocessor specifically for ML is either infeasible or doesn't make sense. As the paper mentions, you need hard logic blocks for arithmetic (if not floating point), and these are always in short supply. In DSP applications I've worked on we used the fpga for timing and i/o, to control jitter…

Yep. Google has TPUs, and AMD/Nvidia/Intel/Apple/Qualcomm all have tensor coprocessors now. From a CPU or GPU to an FPGA, the cost/benefit is huge. With every device having tensor cores, not so much. ASIP and ASICs are likely the way to go, at least for common operations like matrix multiplication.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#92

Post by zachbee »

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA won't offer huge performance increases.

The main advantage FPGAs offer is being able to take advantage of new model optimizations much earlier than ASIC implementations could. Those proposed ternary LLMs could potentially run much faster on FPGAs, because the hardware could be optimized for exclusively ternary ops. [3]

Not to toot my own horn, but I wrote up a blog post recently about building practical FPGA acceleration and which applications are best suited for it: https://www.zach.be/p/how-to-build-a-commercial-open-source

[1] https://aws.amazon.com/solutions/case-studies/munich-leukemi...

[2] https://careers.imc.com/us/en/blogarticle/how-are-fpgas-used...

[3] https://arxiv.org/abs/2402.17764

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#93

Post by cabbageears »

m3kw9 wrote: post #68

Dangs got some work cut out for him tonight

    function modifyElements(pSel, cSel, rxStr) {
        const regex = new RegExp(rxStr, 'i');
        const pEls = document.querySelectorAll(pSel);
        pEls.forEach(pEl => {
            const fEl = pEl.querySelector(cSel);
            if (fEl && regex.test(fEl.textContent)) {
                pEl.style.display = 'none';
            }
        });
    }
    
    let rx = /(hi are u lonely|want (an )?ai gf?)/i;
    
    modifyElements(".athing.comtr", ".comment", rx);

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#94

Post by kadushka »

zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

There are two other problems with FPGAs:

1. They are hard to use (program). If you're a regular ML engineer, there will be a steep learning curve with Verilog/VHDL and the specifics of the chip you choose, especially if you want to squeeze all the performance out of it. For most researchers it's just not worth it. And for production deployment it's not worth the risk of investing into an unproven platform. Microsoft tried it many years ago to accelerate their search/whatever, and I think they abandoned it.

2. Cost. High performance FPGA chips are expensive. Like A100 to H100 price range. Very few people would be willing to spend this much to accelerate their DL models unless the speedup is > 2x compared to GPUs.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#95

Post by IX-103 »

zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

FPGAs are also reasonably good at breadboarding modules to be added to ASICs. You scale down the timing and you can run the same HDL and perform software integration at the same time as the HDL is optimized.

Much cheaper and faster than gate level simulation.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#96

Post by mikewarot »

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid.

It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization.

To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic. Unlike an FPGA, latency is not the primary limit of performance, as everything is thus pipelined.

I wrote a simulator, and started learning VHDL in order to program an FPGA board I bought for prototyping the concept. My eventual goal is an ASIC through tiny tapeout.

The big software hurdle is compiling expressions into a directed graph of bitwise operations.

Because data only travels to the neighbors, all the lines in a chip are short, and it should be possible to use far fewer metallization layers in an actual chip than a CPU for example.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#97

Post by sitkack »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

https://tinytapeout.com/ now lest you purchase additional tiles for $50, each tile supports about 1k digital logic gates.

Next one closes June 1.

https://tinytapeout.com/faq/

You might enjoy this talk from the last Latchup on Wave Pipelining

https://fossi-foundation.org/latch-up/2024#riding-the-wave-b...

https://www.cs.princeton.edu/courses/archive/fall01/cs597a/w...

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#98

Post by sitkack »

zachbee wrote: post #92

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA…

Are you trying to scare people away from FPGAs? GPUs aren't actually that _good_ at deep learning, but they are in the right place at the right time.

You can rent high end FPGAs on AWS, https://github.com/aws/aws-fpga there is no better time to get into FPGAs. On the low end there is the excellent https://hackaday.com/2019/01/14/ulx3s-an-open-source-lattice...

Modern FPGA platforms like Xilinx Alveo have 35TB/s of SRAM bandwidth and 460GB/s of HBM bandwidth. https://www.xilinx.com/products/boards-and-kits/alveo/u55c.h...

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#99

Post by Rexxar »

blackeyeblitzar wrote: post #88
m3kw9 wrote: post #68

Dangs got some work cut out for him tonight

Given they all link to Discord, I wonder if @dang should consider contacting their abuse team to investigate and take down those channels.

Maybe it's exactly what they want (I didn't check the links). Maybe they want to make someone they don't like banned.

Re: FPGA Architecture for Deep Learning: Survey and Future Directions

#100

Post by ooterness »

mikewarot wrote: post #96

My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid. It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization. To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic…

What you're describing is a cellular automaton, in the same vein as of Conway's Game of Life. You can do lots of interesting things with those, but it's emphatically not where I'd start for a flexible computing platform.

Why not go the extra mile, and make each tile a small CPU? There's a Zachtronics have called TIS-100 with this premise.

https://store.steampowered.com/app/370360/TIS100/

Post reply on HN