Tinygrad: A simple and powerful neural network framework
1–10 of 147 posts
Re: Tinygrad: A simple and powerful neural network framework
#2“Almost 9k stars” is actually 7.3k stars…
But otherwise very cool project :)
Re: Tinygrad: A simple and powerful neural network framework
#3I believe neural networks are over hyped sometimes.
They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor.
If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
Re: Tinygrad: A simple and powerful neural network framework
#4The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.
Re: Tinygrad: A simple and powerful neural network framework
#5I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
The problems where traditional ML works best and the problems where Transformers or ConvNets work best are usually two different domains.
AI is not a buzzword.
Re: Tinygrad: A simple and powerful neural network framework
#6I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?
Re: Tinygrad: A simple and powerful neural network framework
#7I must say they gained instant credibility with the minimalistic website given how fast it loaded.
Code looks simple and easy to follow, and I love how the comments are constantly mentioning hardware characteristics, making maxing the hardware the goal. It seems that it’s trying to achieve this by jitting optimal code for the operations at hand rather than hand-optimizing kernels, and betting that the small number of operations will make tuning the codegen tractable.
I haven’t kept up much with what’s happening in ML, but at least in the realm of columnar database engines, interpreting a series of hand-optimized kernels seems to be the dominant approach over compiling a vectorized query plan. Are compilers good enough at optimizing ML operations that specializing on input shape makes a difference over hand-tuned kernels?
Re: Tinygrad: A simple and powerful neural network framework
#8I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
I'm no expert but can you show how those techniques can be used to solve the same problems NNs can? Like SOTA image recognition, chess / go, STT, TTS etc?
Re: Tinygrad: A simple and powerful neural network framework
#9If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3].
Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders.
[1] https://youtu.be/GXy5eVwnL_Q
[2] https://m.youtube.com/watch?v=Cb2KwcnDKrk
[3] no joke, 19.5h stream https://www.youtube.com/watch?v=xc0jGZYFQLQ
Re: Tinygrad: A simple and powerful neural network framework
#10I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
The problems where traditional ML works best and the problems where Transformers or ConvNets work best are usually two different domains. AI is not a buzzword.
>The problems where traditional ML works best and the problems where Transformers or ConvNets work best are usually two different domains.
Yes and we are using NN for everything.