Kolmogorov-Arnold Networks
github.com
Kolmogorov-Arnold Networks
1–10 of 149 posts
Re: Kolmogorov-Arnold Networks
#2Looks super interesting
I wonder how many more new architectures are going to be found in the next few years
Re: Kolmogorov-Arnold Networks
#3Interesting!
Would this approach (with non-linear learning) still be able to utilize GPUs to speed up training?
Re: Kolmogorov-Arnold Networks
#4Interesting! Would this approach (with non-linear learning) still be able to utilize GPUs to speed up training?
Seconded. I’m guessing you could create an implementation that is able to do that and then write optimised triton/cuda kernels to accelerate them but need to investigate further
Re: Kolmogorov-Arnold Networks
#5It’d be really cool to see a transformer with the MLP layers swapped for KANs and then compare its scaling properties with vanilla transformers
Re: Kolmogorov-Arnold Networks
#6Feels like someone stuffed splines into decision trees.
Re: Kolmogorov-Arnold Networks
#7I can't assess this, but I do worry that overnight some algorithmic advance will enhance LLMs by orders of magnitude and the next big model to get trained is suddenly 10,000x better than GPT-4 and nobody's ready for it.
Re: Kolmogorov-Arnold Networks
#8It’d be really cool to see a transformer with the MLP layers swapped for KANs and then compare its scaling properties with vanilla transformers
This is the first thought came to my mind too.
Given its sparse, Will this be just replacement for MoE.
Re: Kolmogorov-Arnold Networks
#9I can't assess this, but I do worry that overnight some algorithmic advance will enhance LLMs by orders of magnitude and the next big model to get trained is suddenly 10,000x better than GPT-4 and nobody's ready for it.
>some algorithmic advance will enhance LLMs by orders of magnitude
I would worry if I'd own Nvidia shares.
Re: Kolmogorov-Arnold Networks
#10I can't assess this, but I do worry that overnight some algorithmic advance will enhance LLMs by orders of magnitude and the next big model to get trained is suddenly 10,000x better than GPT-4 and nobody's ready for it.
What to be worried about? Technical progress will happen, sometimes by sudden jumps. Some company will become a leader, competitors will catch up after a while.