101–110 of 167 posts
The risk for Tinygrad is that PyTorch will create a new backend for Inductor, plug in their AMD codegen stuff and walala, PyTorch still king. I mean, they could have easily just taken that route themselves instead of bothering with a new ML framework and AD engine. 99% of the work is just the AMD codegen part of the compiler. Either way, super cool project and I wish them the best.
> walala Is this like voila or something else?
Earlier quoted context omitted.
Did we read the same article? The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite. The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connectin…
> The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components. That I cannot believe. He might have shifted the make-or-buy decisions, but b…
Note the "goal" there. SpaceX's only in flight explosion came after a strut (3rd party sourced) failed on S2, on the CRS-7 mission in 2015. They in sourced that, and haven't had many issues on ascent since then. They've also launched and landed some 500 rockets since then (165 this year) so ... at least they're walking the walk?
Earlier quoted context omitted.
yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.
Running a formatter on that codebase would probably at least triple it.
I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.
No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.
https://github.com/tinygrad/tinygrad?tab=readme-ov-file#cont...
I've got a Comma 3X and I'm thankful for the 4090 P2P work too (which is now here? https://github.com/tinygrad/open-gpu-kernel-modules) so I'm excited to see it work. Rooting for the guy. Hope it's true that the chip part is the easy work. Could be that both are "the hard work".
I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.
yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.
I agree with some poster above, the code looks hard simply because the problem is very hard, not because they want to write it that way.
>People get hired by contributing to the repo. It’s a very self directed job, with one meeting a week and a goal of making tinygrad better I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.
I wonder what happened to George’s old policy of requiring everyone to move to San Diego?
> To fund the operation, we have a computer sales division that makes about $2M revenue a year. What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?