Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Tinygrad

geohot.github.io

Post reply on HN
Print view View original

71–80 of 167 posts

Re: Five Years of Tinygrad

#71

Post by vovavili »

Barrin92 wrote: post #58
rent0pat wrote: post #56

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

Running a formatter on that codebase would probably at least triple it.

Re: Five Years of Tinygrad

#72

Post by octoberfranklin »

Barrin92 wrote: post #58
rent0pat wrote: post #56

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

Looks fine to me.

I'm going to guess that you're used to codebases that use an autoformatter.

There's definitely a case to be made for autoformatters in projects with a very large (>100) number of developers, but the price you pay for that is much poorer information density per screenful of code.

For things like tinygrad or sqlite with a small dev team it's possible to use whitespace, style, and formatting to communicate important information, just like mathematicians do. The AMS journals don't autoformat mathematicians' equations!

Re: Five Years of Tinygrad

#73

Post by octoberfranklin »

chubot wrote: post #68
Barrin92 wrote: post #58

Earlier quoted context omitted.

yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

It’s not crazy at all, but personally I like simple code that flows down the page more, not across

It's two-dimensional code, not one-dimensional code.

Declarations flow down the page, definitions flow across.

Re: Five Years of Tinygrad

#74

Post by octoberfranklin »

georgehotz wrote: post #21
ttul wrote: post #18

Earlier quoted context omitted.

I wonder what happened to George’s old policy of requiring everyone to move to San Diego?

That's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task. We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose.…

Have you run into problems with contributors who can't enter the PRC?

Re: Five Years of Tinygrad

#75

Post by ttul »

georgehotz wrote: post #21
ttul wrote: post #18

Earlier quoted context omitted.

I wonder what happened to George’s old policy of requiring everyone to move to San Diego?

That's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task. We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose.…

An answer from the man himself. Thank you.

Re: Five Years of Tinygrad

#76

Post by eichin »

Barrin92 wrote: post #58
rent0pat wrote: post #56

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

to quote an old academic research programmer: "Two-letter variable names are fine, as long as both letters are meaningful" :-)

Re: Five Years of Tinygrad

#77

Post by christophilus »

Barrin92 wrote: post #58
rent0pat wrote: post #56

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

yes, it's really crazy, if people think you're exaggerating look at this: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co... One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

Doesn’t seem too bad to me. It reminds me a bit of Clojure. Like Clojure, though, the lack of static types would be a headache.

Re: Five Years of Tinygrad

#78

Post by georgehotz »

geremiiah wrote: post #36

The risk for Tinygrad is that PyTorch will create a new backend for Inductor, plug in their AMD codegen stuff and walala, PyTorch still king. I mean, they could have easily just taken that route themselves instead of bothering with a new ML framework and AD engine. 99% of the work is just the AMD codegen part of the compiler. Either way, super cool project and I wish them the best.

tinygrad has a PyTorch frontend, we fully encourage the frontend to stay PyTorch and for them to integrate tinygrad at any level. Our mission is to commoditize the petaflop, all frontends are welcome. We also have a quite complete ONNX frontend.

Re: Five Years of Tinygrad

#79

Post by quux0r »

dustbunny wrote: post #66
georgehotz wrote: post #21

Earlier quoted context omitted.

That's comma.ai's policy since they make hardware and solve physical problems. The tiny corp has been hybrid (remote-first) since day 1 because it primarily writes open source software, and there's a long track record of success with remote for this kind of task. We have a few whole-team meetups in Hong Kong each year for 2-4 weeks, and there's a San Diego or Hong Kong office that anyone can work from as they choose.…

Why Hong Kong? I guess you have a bunch of contributors near there?

If I recall correctly, George recently relocated there.

Re: Five Years of Tinygrad

#80

Post by nighthawk454 »

abacadaba wrote: post #64
nighthawk454 wrote: post #62

Earlier quoted context omitted.

That doesn’t look super awful to me? Hardly extreme code golfing. The far more interesting part is the order of magnitude. If they can pull off a 20k LOC with zero dependencies (implying a pretty concise project size) and it still works well on meaningful applications, that’s pretty neat. A 1000x reduction in code size and matching/exceeding in perf is worth looking at. Probably also implying a better architecture as…

it looks dense but perfectly readable. arguably more readable that way than if it had a bunch of extra new lines, definitions, and code blocks spreading the logic out into a larger visible area.

I'm inclined to agree. While not always appropriate or popular, it makes some sense to me to have the visual weight/area of the code being ~proportional to it's significance. Communicates the ideas more clearly. Instead of spending the majority of space telling me what _isn't_ going to happen, and hiding what is.

I often find myself wishing this was more ergonomic in languages.

Post reply on HN