Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinygrad: A simple and powerful neural network framework

tinygrad.org

Post reply on HN
Print view View original

111–120 of 147 posts

Re: Tinygrad: A simple and powerful neural network framework

#111

Post by terafo »

lostmsu wrote: post #96

It was ok as an educational tool, but now they don't count GPU implementation in 1000 lines, so it is not small. Considering the code style it is closer to 20k+ lines when formatted and GPU code included. It also doesn't support bfloat16 so is doomed to be 2x slower.

Actual code of tinygrad is less than 5k lines. There is also 1600 lines of tests and around 2k lines of example models. And I didn't count unfinished support for geohot's own unfinished neural network accelerator(verilog for that accelerator sits in repo too), which is abandoned.

Re: Tinygrad: A simple and powerful neural network framework

#113

Post by terafo »

georgehotz wrote: post #107
brrrrrm wrote: post #104

Earlier quoted context omitted.

> tread off the beaten path things get slow Yea, makes sense. I think there's something to be said for dynamic compilation solving this problem more elegantly than providing tons of hand-tuned kernels (PyTorch is 890MB lmao https://pypi.org/project/torch/#files ), but I don't think it's a strict reason for a performance win. > change the loop order too Memory layout as well! I'm 100% for dynamic compilation, but I'm…

Agreed. For anything at all common, most of the gains will be from fusion, the rest is just free. PyTorch also uses tons of GPU memory after only initializing, I wonder if it's copying all the kernels in?

Jax preallocates 90% of available GPU memory when first operation is run to minimize allocation overhead. Can PyTorch grab that VRAM for a similar reason?

Re: Tinygrad: A simple and powerful neural network framework

#114

Post by vladf »

georgehotz wrote: post #100
eterevsky wrote: post #80

Earlier quoted context omitted.

And how does TinyGrad solve this?

Fused into one operation since the Tensor isn't resolved until I call .numpy() kafka@tubby:/tmp$ cat fuse.py from tinygrad.tensor import Tensor x = Tensor.zeros(1) for i in range(5): x += i print(x.numpy()) kafka@tubby:/tmp$ OPT=2 GPU=1 DEBUG=2 python3 fuse.py using [ ] **CL** 0 elementwise_0 args 1 kernels [1, 1, 1] None OPs 0.0M/ 0.00G mem 0.00 GB tm 0.15us/ 0.00ms ( 0.03 GFLOPS) **CL** copy OUT (1,) [10.]

How does this differ from XLA? Would tinygrad's lazy approach also just see the same unrolled loop right before compilation?

Re: Tinygrad: A simple and powerful neural network framework

#115

Post by lostmsu »

terafo wrote: post #111
lostmsu wrote: post #96

It was ok as an educational tool, but now they don't count GPU implementation in 1000 lines, so it is not small. Considering the code style it is closer to 20k+ lines when formatted and GPU code included. It also doesn't support bfloat16 so is doomed to be 2x slower.

Actual code of tinygrad is less than 5k lines. There is also 1600 lines of tests and around 2k lines of example models. And I didn't count unfinished support for geohot's own unfinished neural network accelerator(verilog for that accelerator sits in repo too), which is abandoned.

> Actual code of tinygrad is less than 5k lines

Yeah, not

> Considering the code style

I mean it is possible to read it, but I would not say it is optimized for it. Which I suppose betrays the goal.

Re: Tinygrad: A simple and powerful neural network framework

#116

Post by terafo »

lostmsu wrote: post #115
terafo wrote: post #111

Earlier quoted context omitted.

Actual code of tinygrad is less than 5k lines. There is also 1600 lines of tests and around 2k lines of example models. And I didn't count unfinished support for geohot's own unfinished neural network accelerator(verilog for that accelerator sits in repo too), which is abandoned.

> Actual code of tinygrad is less than 5k lines Yeah, not > Considering the code style I mean it is possible to read it, but I would not say it is optimized for it. Which I suppose betrays the goal.

> Yeah, not

Provide some evidence. I just ran tokei on freshly cloned tinygrad repo using arguments from[1]. Got 4854 lines of code, which is less than 5k lines.

[1] tokei --exclude *.json --exclude accel/cherry --exclude test --exclude examples

Re: Tinygrad: A simple and powerful neural network framework

#117

Post by blt »

ren_engineer wrote: post #79
matesz wrote: post #9

If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…

If you enjoy what you are doing it's pretty easy to work on something that long, I've had gaming sessions last as long back in the day with friends and some of those games are as demanding in terms of focus as programming. External motivation of having an audience would also help

Factorio?

Re: Tinygrad: A simple and powerful neural network framework

#118

Post by thrtythreeforty »

brrrrrm wrote: post #106
markisus wrote: post #95

Earlier quoted context omitted.

What does it mean to "fuse operations"?

avoiding writes to memory and reducing the number of loops (although not FLOPs) for j in range(10): c[j] = a[j] + b[j] for j in range(10): d[j] = c[j] * 2 becomes for j in range(10): d[j] = (a[j] + b[j]) * 2

Or, better, identifying that the machine has a primitive that is better than doing each op individually. For example, a multiply-accumulate instruction vs a multiply and separate accumulate. The source code still says "a*b+c", the compiler is just expected to infer the MAC instruction.

Re: Tinygrad: A simple and powerful neural network framework

#119

Post by thrtythreeforty »

passion__desire wrote: post #67
RektBoy wrote: post #60

No Bible quotes? I'm disappointed...

I can't believe how can someone so accomplished believe in God.

I think it's orthogonal. There are tons of smart people who believe in God. (Knuth has already been mentioned.)

If God wanted, He could make himself apparent to everyone. Clearly that isn't the case; there is room to doubt or to believe no matter how smart or accomplished you are.

Re: Tinygrad: A simple and powerful neural network framework

#120

Post by kwant_kiddo »

I think posts like this are only getting upvotes because George Hotz owns the project. I do see value in simple code, but the constraint of 1000 LOC makes little sense to me, especially when the code is formatted poorly.

This will get downvoted, but reading the comments here I dont understand the (cult/respect) for him. Siding with the most successful CTF-team ever (PPP) he won defcon two times. He made a startup with funding that makes a cool 'niche' product.

I just think a guy like Chris Lattner or Dave Cutler who made so much impact on real computing deserve so much more respect, but I guess that the norm here is to admire this guy.

Post reply on HN