Viewing profile — ag2718

This is a simplification in the blog post: each activation doesn't map one-to-one onto a physical FPGA LUT primitive, but is instead represented as a "logical LUT" (L-LUT) that Viv…

Our end-to-end implementation can be found here! https://github.com/Duchstf/KANELE

Some very cool applications of small models! It seems that this scale of models tends to be sufficient when doing simpler classification, anomaly detection, signal processing, etc.…

That is a really cool application of FPGA-based machine learning that I would not have thought of :)

This is definitely true: one could imagine a model with a mix of the two layers or a simple linear / MLP-like kernel doing "preprocessing" before KAN layers. Other work that explor…

Thank you :)

Ah I see, that's an interesting point about higher depth potentially having other benefits. For our work on smaller models (e.g. generally <5 layers), this might not have been as r…

A key benefit of KANs is expressivity, as each layer is significantly more expressive than an MLP layer. This can be seen in our benchmarks: KAN networks need fewer layers than MLP…

There is definitely a precision-performance tradeoff to consider. We explored this through ablation studies on bitwidth precision / resource usage in our work (Figure 6a in https:/…

You're correct that this work is not very applicable for LLMs and that the focus here is primarily on latency.

Yes, definitely: this type of work is applicable in domains where software run on general-purpose processors cannot meet latency or power requirements.

Hmm the post is still up for me?

Yes, this work is focused on accelerating very small models, typically for real-time systems that require extremely low power or low latency. One primary application of this work i…

https://web.archive.org/web/20260609200156/https://aarushgup...