[stub for offtopicness]
Somehow I expected he made it out of mechanical parts. Like literally “by hand”.
me too, i was hoping to see a flyback transformer wound with a power drill or something
I made a transformer to predict a simple sequence manually
11–20 of 104 posts
Re: I made a transformer to predict a simple sequence manually
#12It's some kind of abstract machine, like a turing machine or the machine that parses regexes, isn't it?
Re: I made a transformer to predict a simple sequence manually
#13Re: I made a transformer to predict a simple sequence manually
#14It's some kind of abstract machine, like a turing machine or the machine that parses regexes, isn't it?
Neural networks are Turing machines. You can make them perform any computation by carefully setting up their weights. It would be nice to have compilers for them that were not based on approximation, though.
Re: I made a transformer to predict a simple sequence manually
#15It's some kind of abstract machine, like a turing machine or the machine that parses regexes, isn't it?
Neural networks are Turing machines. You can make them perform any computation by carefully setting up their weights. It would be nice to have compilers for them that were not based on approximation, though.
> It would be nice to have compilers for them that were not based on approximation, though.
Could you elaborate?
Re: I made a transformer to predict a simple sequence manually
#16> maybe even feel inspired to make your own model by hand as well!
Other then a learning exercise to satisfy your curiosity what are you doing with this? I'm starting to get the feeling that anything complex with ml models is unreasonable for a at home blog reader?
Re: I made a transformer to predict a simple sequence manually
#17It's some kind of abstract machine, like a turing machine or the machine that parses regexes, isn't it?
It’s just any pile of linear algebra that’s been in contact with the AllSpark, right?
Re: I made a transformer to predict a simple sequence manually
#18Earlier quoted context omitted.
Neural networks are Turing machines. You can make them perform any computation by carefully setting up their weights. It would be nice to have compilers for them that were not based on approximation, though.
> It would be nice to have compilers for them that were not based on approximation, though. Could you elaborate?
People typically set the weights of a neural network using heuristic approximation algorithms, by looking at a large set of example inputs/outputs and trying to find weights that perform the needed computation as accurately as possible. This approximation process is called training. But this approximation happens because nobody really knows how to set the weights otherwise. It would be nice if we had "compilers" for neural networks, where you write an algorithm in a programming language, and you get a neural network (architecture+weights) that performs the same computation.
TFA is a beautiful step in that direction. What I want is an automated way to do this, without having to hire vgel every time.
Re: I made a transformer to predict a simple sequence manually
#19Re: I made a transformer to predict a simple sequence manually
#20> maybe even feel inspired to make your own model by hand as well! Other then a learning exercise to satisfy your curiosity what are you doing with this? I'm starting to get the feeling that anything complex with ml models is unreasonable for a at home blog reader?
I dunno, maybe they actually enjoy hacking on projects like this? Weird I know.