Earlier quoted context omitted.
If it's not Adderall I don't know. But, if I've ever focused for that long it's been because of Ritalin or Adderall.
I think it's combination of: 1) he's really passionate about what he's doing 2) he sees the problem as real challenge 3) he doesn't have corporate structure on his back giving him deadlines and pressure
Tinygrad: A simple and powerful neural network framework
21–30 of 147 posts
Re: Tinygrad: A simple and powerful neural network framework
#22If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
I used to watch scanlime do 8 hour sw/hw sessions, really hope she comes back soon.
I have watched Brandon Falk do 10-11 hours of rust programming, although he sometimes take a break to play games for 4-5 hours (while in stream)
Re: Tinygrad: A simple and powerful neural network framework
#23https://github.com/geohot/tinygrad/blob/master/.github/workf...
Re: Tinygrad: A simple and powerful neural network framework
#24I love this website. Their style tag literally is: body { font-family:'Lucida Console', monospace } Also look like a very cool project.
Re: Tinygrad: A simple and powerful neural network framework
#25Re: Tinygrad: A simple and powerful neural network framework
#26If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
If it's not Adderall I don't know. But, if I've ever focused for that long it's been because of Ritalin or Adderall.
If you watch Hotz's streams he takes small breaks to talk with chat and to meme around (just like everyone else during their work days) and he eats lunch and whatever (again just like everyone else).
What I'm trying to say is that Hotz's isn't a superman on Adderall he is just working on stuff he is excited about.
Re: Tinygrad: A simple and powerful neural network framework
#27If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
Re: Tinygrad: A simple and powerful neural network framework
#28Earlier quoted context omitted.
If it's not Adderall I don't know. But, if I've ever focused for that long it's been because of Ritalin or Adderall.
Is 10 hours really _that_ strange? You are (hopefully) focusing 8 hours "straight" during work _every day_. If you watch Hotz's streams he takes small breaks to talk with chat and to meme around (just like everyone else during their work days) and he eats lunch and whatever (again just like everyone else). What I'm trying to say is that Hotz's isn't a superman on Adderall he is just working on stuff he is excited abo…
I've been excited on 10 hours for a long portion of my life. Getting older makes it harder though.
Re: Tinygrad: A simple and powerful neural network framework
#29 > It's extremely simple, and breaks down the most complex networks into 4 OpTypes:
>
> - UnaryOps operate on one tensor and run elementwise. RELU, LOG, RECIPROCAL, etc...
> - BinaryOps operate on two tensors and run elementwise to return one. ADD, MUL, etc...
> - ReduceOps operate on one tensor and return a smaller tensor. SUM, MAX
> - MovementOps operate on one tensor and move the data around, copy-free with ShapeTracker. RESHAPE, PERMUTE, EXPAND, etc...
>
> But how...where are your CONVs and MATMULs? Read the code to solve this mystery.
Ok, I was curious, so I read the code. The answer is that it represents a MATMUL as a 1x1 CONV. And it lied about CONV, which is a ProcessingOps.CONV and explicitly represented and implemented: https://github.com/geohot/tinygrad/blob/c0050fab8ff0bc667e40... Quite the letdown of figuring out this 'mystery'.Re: Tinygrad: A simple and powerful neural network framework
#30I believe neural networks are over hyped sometimes. They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor. If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.
I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?
1. Collaborative filtering based on a sparse dataset of implicit interactions.
2. Many time series applications.