took long enough
Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
21–30 of 58 posts
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#22This guy will be hired by a high-frequency trading firm, and the next time we hear about him, he will have a net worth in 9 figures.
Sure, if they worked for 100 years maybe.. FPGA guy at jane st probably makes 600k to low seven figures... Maybe.
Not everyone in quant is a centi-millionaire, probably almost none of them in r&d actually.
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#23So for people wondering if it can be used to accelerate LLM inference, sadly not. I've been trying to hit 100,000tokens/s with a 3.28m dumb model, and even this is an order of magnitude too large to benefit. It appears to be focussed more on latency, than throughput. Happy to be corrected?
Was anyone thinking this?
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#24Has there been much exploration on how much benefit comes from precision in activation functions in KANs? There's a little niggle in the back of my head that maybe 90% of the benefit of KANs can be gained from a quite small variety of function shapes. Combined with input weighting, I almost feel you could have a representation that scales from a standard relu perceptron though KANs to something with weighted inputs and fancy weighted activation functions.
Mark that out in 2d with axes of input weight precision and activation weight precision, you could perhaps do sweeps to find the best accuracy per parameter bit, or accuracy/speed, or some sweet spot that has a nice balance of operating speed, accuracy, and model size.
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#25If you want to experiment with KANs yourself in a non-FPGA environment, there's a GitHub repo here: https://github.com/KindXiaoming/pykan
HN comments page on that is here: https://news.ycombinator.com/item?id=40219205
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#26Has there been much exploration on how much benefit comes from precision in activation functions in KANs? There's a little niggle in the back of my head that maybe 90% of the benefit of KANs can be gained from a quite small variety of function shapes. Combined with input weighting, I almost feel you could have a representation that scales from a standard relu perceptron though KANs to something with weighted inputs a…
The benefit in KANs is interpretability, not expressivity. It's a structure that lends itself well to performing symbolic regression or other interpretable downstream tasks. This can make it better suited for scientific tasks, for example. You can easily replicate the practical performance of any KAN with an MLP, and it will train and run faster on modern architectures. This proposes a method it might be faster, but it's early days to me.
Precision in the activation function is targetting a part of neural networks that you don't want. There are many other methods that work with high precision. You use neural networks because of their implicit bias toward regular solutions. That means there is a sweet spot at low precision that you're targetting.
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#27Right. But ... this would limit you to either extremely small models or extremely large FPGA's, yes? If there's a simple machine learning task that requires a sub microsecond latency I can see the point but otherwise??
Yes, this work is focused on accelerating very small models, typically for real-time systems that require extremely low power or low latency. One primary application of this work is in high-energy physics ( https://home.cern/smarter-decisions-at-the-speed-of-collisio... ). Ultrafast and real-time learning is also very applicable for problems in quantum computing, plasma control, etc. ( https://arxiv.org/pdf/2602.0200…
Drone target recognition?
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#28Has there been much exploration on how much benefit comes from precision in activation functions in KANs? There's a little niggle in the back of my head that maybe 90% of the benefit of KANs can be gained from a quite small variety of function shapes. Combined with input weighting, I almost feel you could have a representation that scales from a standard relu perceptron though KANs to something with weighted inputs a…
There is definitely a precision-performance tradeoff to consider. We explored this through ablation studies on bitwidth precision / resource usage in our work (Figure 6a in https://arxiv.org/pdf/2512.12850, Figure 4 in https://arxiv.org/pdf/2602.02056). Further exploration into the mechanics here would definitely be useful.
Regarding your point that "90% of the benefit of KANs can be gained from a small variety of function shapes": even within the B-spline basis, the shapes are quite uniform. Much of the actual benefit of scaling up the basis size comes from learning more complex, piecewise-polynomial activation functions. Scaling up the number of basis functions (i.e. more granular intervals) also increases locality and allows the activation function's value across different parts of the domain to be learned semi-independently. (There obviously is a tradeoff here with overfitting.)
The number of basis functions (G+S) is largely what determines how expressive the activation is, as it relates to your point: "you could have a representation that scales from a standard relu perceptron though KANs to something with weighted inputs and fancy weighted activation functions."
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#29So for people wondering if it can be used to accelerate LLM inference, sadly not. I've been trying to hit 100,000tokens/s with a 3.28m dumb model, and even this is an order of magnitude too large to benefit. It appears to be focussed more on latency, than throughput. Happy to be corrected?
When aiming for 100k tok/s, you would still have CUDA overheads (on the order of microseconds) -- which might become the bottleneck, even if you do everything else right with the inference architecture. How are you planning to overcome that?
EDIT: Oh, on second read, do you mean you're running the model on an FPGA?
Re: Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks
#30So for people wondering if it can be used to accelerate LLM inference, sadly not. I've been trying to hit 100,000tokens/s with a 3.28m dumb model, and even this is an order of magnitude too large to benefit. It appears to be focussed more on latency, than throughput. Happy to be corrected?
When aiming for 100k tok/s, you would still have CUDA overheads (on the order of microseconds) -- which might become the bottleneck, even if you do everything else right with the inference architecture. How are you planning to overcome that? EDIT: Oh, on second read, do you mean you're running the model on an FPGA?
You might be conflating throughput with latency. 100k tok/s is very different to 1 tok/10us.