Live data from Hacker News

Five Years of Tinygrad

geohot.github.io

51–60 of 167 posts

Re: Five Years of Tinygrad

#51

Earlier quoted context omitted.

I hate it when ‘inspirational’ quotes are attributed to the person with the largest audience and not the people who came up with it, like in this case, the engineers at Lockheed’s Skunk Works.

It's an apocryphal quote. "A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away." - Antoine de Saint-Exupéry

Or Colin Chapman at Lotus: "Simplify, then add lightness." Could just be wise advice :)

Re: Five Years of Tinygrad

#52

> To fund the operation, we have a computer sales division that makes about $2M revenue a year. What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?

Looking at their boxes, I'd guess 20-ish%, once the labor costs of designing the boxes is out of the picture. I imagine George doesn't take a salary, or at least takes only a small one. They have a contract with AMD as well. I'd bet the company is modestly profitable.

Re: Five Years of Tinygrad

#54

Is it really "Complex"? Or did we just make it "Complicated"? - https://www.youtube.com/watch?v=ubaX1Smg6pY

If you think you can do better, the code's right there and you're welcome to try! There's a giant pile of money at the end of the rainbow for you if you can uncomplexify it.

(Love that talk tho)

Re: Five Years of Tinygrad

#55
post #29

> tinygrad is following the Elon process for software. Make the requirements less dumb. The best part is no part. That’s not Elon. See Russian TRIZ https://en.wikipedia.org/wiki/TRIZ

Not sure where it was from but I remember reading a comment about how every modern rocket scientist needs to come to terms with the fact that every novel idea they might have has already been proposed and tried by a Russian scientist in the 20th century.

Re: Five Years of Tinygrad

#57
If you want to "own" Nvidia, the much more realistic way of doing this then trying to compete with all the data centers that are already being built with Nvidia chips is obviously with open source models. In the case of open source models, inference is much more important to most people not training which a maxed out macbook already does a good job of.

Re: Five Years of Tinygrad

#58

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

yes, it's really crazy, if people think you're exaggerating look at this:

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co...

One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

Re: Five Years of Tinygrad

#59
post #48
post #6

Lots of words and weird analogies to say basically nothing. What is the status of the project? What can it do? What has it achieved in 5 years? But no, let's highlight how we follow the "Elon process". As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.

Did we read the same article? The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite. The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connectin…

> The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components.

That I cannot believe. He might have shifted the make-or-buy decisions, but both Tesla and SpaceX do a lot of outsourcing.

Re: Five Years of Tinygrad

#60
post #55
post #29

> tinygrad is following the Elon process for software. Make the requirements less dumb. The best part is no part. That’s not Elon. See Russian TRIZ https://en.wikipedia.org/wiki/TRIZ

Not sure where it was from but I remember reading a comment about how every modern rocket scientist needs to come to terms with the fact that every novel idea they might have has already been proposed and tried by a Russian scientist in the 20th century.

Soviet scientist, let me correct you a bit, comrade.
Post reply on HN