Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kolmogorov-Arnold Networks

github.com

Post reply on HN
Print view View original

41–50 of 149 posts

Re: Kolmogorov-Arnold Networks

#41

Post by cess11 »

6mian wrote: post #24
cess11 wrote: post #19

Earlier quoted context omitted.

Where did you learn that history? What do you mean by "better"?

I had a European peasant in the 1600-1700s in mind when I wrote about the amount of work. During the season, they worked all day; off-season, they had "free time" that went into taking care of the household, inventory, etc., so it's still work. Can't quickly find a reliable source in English I could link, so I can be wrong here. "Better" was referring to what OP wrote in the top comment. I guess 10x faster, 10x longe…

Sorry, I can't fit that with what you wrote earlier: "12 hrs/7 days per week to not die from hunger".

Those peasants payed taxes, i.e. some of their work was exploited by an army or a priest rather than hunger, and as you mention, they did not work "12 hrs/7 days per week".

Do you have a better example?

Re: Kolmogorov-Arnold Networks

#42

Post by cess11 »

vladms wrote: post #23
cess11 wrote: post #13

Earlier quoted context omitted.

"Technical progress" has been destroying our habitat for centuries, causing lots of other species to go extinct. Pretty much the entire planet surface has been 'technically progressed', spreading plastics, climate change and whatnot over the entirety of it. Are you assuming that this particular "progress" would be relatively innocent?

Many species went extinct during Earth's history. Evolution requires quite aggressive competition. The way the habitat got destroyed by humans is stupid because it might put us in danger. You can call me "speciesist" but I do care more for humans rather than for a particular other specie. So I think progress should be geared towards human species survival and if possible preventing other species extinction. Some of t…

Evolution is the result of catastrophies and atrocities. You use the word as if it has positive connotations, which I find weird.

How do you come to the conclusion "stupid" rather than evil? Aren't we very aware of the consequences of how we are currently organising human societies, and have been for a long time?

Re: Kolmogorov-Arnold Networks

#44

Post by itsthecourier »

GistNoesis wrote: post #34

I quickly skimmed the paper, got inspired to simplify it, and created some Pytorch Layer : https://github.com/GistNoesis/FourierKAN/ The core is really just a few lines. In the paper they use some spline interpolation to represent 1d function that they sum. Their code seemed aimed at smaller sizes. Instead I chose a different representation, aka fourier coefficients that are used to interpolate the functions of indiv…

you really are a pragmatic programmer, Noesis

Re: Kolmogorov-Arnold Networks

#46

Post by bart1ett »

diwank wrote: post #5

It’d be really cool to see a transformer with the MLP layers swapped for KANs and then compare its scaling properties with vanilla transformers

After trying this out with the fourier implementation above, swapping MLP/Attention Linear layers for KANs (all, or even a few layers) produces diverging loss. KANs don't require normalization for good forward pass dynamics, but may be trickier to train in a deep net.

Re: Kolmogorov-Arnold Networks

#48

Post by godelski »

cs702 wrote: post #39

It's so refreshing to come across new AI research different from the usual "we modified a transformer in this and that way and got slightly better results on this and that benchmark." All those new papers proposing incremental improvements are important, but... everyone is getting a bit tired of them. Also, anecdotal evidence and recent work suggest we're starting to run into fundamental limits inherent to transforme…

There's a ton actually. Just they tend to go through extra rounds of review (or never make it...) and never make it to HN unless there's special circumstances (this one is MIT and CIT). Unfortunately we've let PR become a very powerful force (it's always been a thing, but seems more influential now). We can fight against this by up voting things like this and if you're a reviewee, not focusing on sota (it's clearly been gamed and clearly leading us in the wrong direction)

Re: Kolmogorov-Arnold Networks

#50

Post by godelski »

glebnovikov wrote: post #40
cs702 wrote: post #39

It's so refreshing to come across new AI research different from the usual "we modified a transformer in this and that way and got slightly better results on this and that benchmark." All those new papers proposing incremental improvements are important, but... everyone is getting a bit tired of them. Also, anecdotal evidence and recent work suggest we're starting to run into fundamental limits inherent to transforme…

> Everyone is getting tired of those papers. This is science as is :) 95% percent will produce mediocre-to-nice improvements to what we already have so there were reserachers that eventually grow up and do something really exciting

Nothing wrong with incremental improvements. Giant leaps (almost always) only happen because of a lack of your niche domain expertise. And I mean niche niche
Post reply on HN