Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Tinygrad

geohot.github.io

Post reply on HN
Print view View original

41–50 of 167 posts

Re: Five Years of Tinygrad

#41

Post by piskov »

semiquaver wrote: post #40

Is this the guy who talked a big game about all the things he was going to fix at Twitter, then utterly failed when confronted with a real world codebase and gave up having done nothing of use?

He left after realizing nothing was going to change (not like he’s in the leadership).

Also half-joked how the good food went away.

—

George is many things but not a quitter (see comma ai for example).

If someone could pull this, it’s him due to “never give up, never surrender” attitude.

The shit with nvidia just needs to stop

Re: Five Years of Tinygrad

#43

Post by eli_gottlieb »

tucnak wrote: post #39
zephen wrote: post #12

Earlier quoted context omitted.

Lattner is a smart guy, but I think Mojo might be the wrong direction. Time will tell. History has not so far been kind to projects which attempt to supplant cPython, whether they are other Python variants such as PyPy, or other languages such as julia. Python has a lot of detractors, but (despite some huge missteps with the 2-3 transition) the core team keeps churning out stuff that people want to use. Mojo is being…

Honestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while sin…

> For example, Bayesian programming like Turing.jl is virtually unmatched in Python.

What about numpyro?

Disclaimer: I contribute to numpyro occasionally.

Re: Five Years of Tinygrad

#44

Post by mycodendral »

vileain wrote: post #27
mycodendral wrote: post #13

Earlier quoted context omitted.

the value is the directness, not implied origination not everyone cares about playing voldemort

What is so aggrandizingly 'direct' about calling the system you are attempting to improve 'dumb'?

you have a more direct way?

Re: Five Years of Tinygrad

#45

Post by JoeDohn »

paxys wrote: post #6

Lots of words and weird analogies to say basically nothing. What is the status of the project? What can it do? What has it achieved in 5 years? But no, let's highlight how we follow the "Elon process". As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.

making things less dumb is not a elon's process, if it's the case then we are saying everything that elon is not involved in/with is dumb !

Re: Five Years of Tinygrad

#46

Post by zephen »

tucnak wrote: post #39
zephen wrote: post #12

Earlier quoted context omitted.

Lattner is a smart guy, but I think Mojo might be the wrong direction. Time will tell. History has not so far been kind to projects which attempt to supplant cPython, whether they are other Python variants such as PyPy, or other languages such as julia. Python has a lot of detractors, but (despite some huge missteps with the 2-3 transition) the core team keeps churning out stuff that people want to use. Mojo is being…

Honestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while sin…

I dunno. This sort of thing gives me pause:

https://danluu.com/julialang/

But the first thing that gave me pause about Julia? They sort of pivoted to say "we're general purpose" but the whole index-starting-at-one thing really belies that -- these days, that's pretty much the province of specialty languages.

Re: Five Years of Tinygrad

#47

Post by dewey »

selkin wrote: post #32
dewey wrote: post #22

Earlier quoted context omitted.

Using lines of code as a metric for productivity is bad. Using it to show how simple something is, or how a refactor removed x lines of code that doesn’t need to be maintained any more isn’t such a bad thing I’d say.

Less LOC also doesn't imply simplicity: just look at the demoscene, which often has the former but not the latter.

Demo scene or other competitions that aim at using little space / shortest way to achieve something are not really a good counter example.

Re: Five Years of Tinygrad

#48

Post by bko »

paxys wrote: post #6

Lots of words and weird analogies to say basically nothing. What is the status of the project? What can it do? What has it achieved in 5 years? But no, let's highlight how we follow the "Elon process". As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.

Did we read the same article?

The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite.

The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connecting people" or whatever the hell it is?

He said he has a hardware division that makes $2m a year. You can click on the link and buy a computer. He tells you the revenue.

He said he has a deal with AMD which is also public and on X.

The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components.

I wish more people focused on public code and shipping publicly. Can I see Toyotas stack? Why does the touch screen perform worse than a 2012 iPad. What the hell is going on

I don't understand this hate someone like Hotz, a true engineer running an ambitious very open company, receives on a ... checks notes ... engineering forum? The whole setup is like a wet dream for engineers from a decade ago.

Almost completely open source

Interview is completing a paid bounty

CEO is engineer

Has a clear mission, goals, and timeline

What is your issue?

Re: Five Years of Tinygrad

#49

Post by tucnak »

eli_gottlieb wrote: post #43
tucnak wrote: post #39

Earlier quoted context omitted.

Honestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while sin…

> For example, Bayesian programming like Turing.jl is virtually unmatched in Python. What about numpyro? Disclaimer: I contribute to numpyro occasionally.

Honestly, I'm not familiar with it. I had only played with RxInfer, if only to try the so-called "message-passing" paradigm. My grasp on probability is really lacking, in fact I picked up prediction markets and Julia to get better at it.

If you don't mind me asking, what's the deal with NumPyro why you chose to work it?

Re: Five Years of Tinygrad

#50

Post by tucnak »

zephen wrote: post #46
tucnak wrote: post #39

Earlier quoted context omitted.

Honestly, I feel like Julia might as well beat Mojo or sommat to the punch, sooner or later. It has some facilities and supporting infrastructure for a lot of scientific and data-handling tasks surrounding ML, if not for compiling and dispatching kernels (where XLA reins supreme to anything in the CUDA ecosystem!) For example, Bayesian programming like Turing.jl is virtually unmatched in Python. It's been a while sin…

I dunno. This sort of thing gives me pause: https://danluu.com/julialang/ But the first thing that gave me pause about Julia? They sort of pivoted to say "we're general purpose" but the whole index-starting-at-one thing really belies that -- these days, that's pretty much the province of specialty languages.

You're not supposed to admit it, but I never cared for Dijkstra's arguments on the matter. The same goes for his GOTO tirade, although that has been distorted by time somewhat. Pascal is using 1-ord, Fortran, R, Mathematica. If anything, it seems there's a longer tradition of 1-ord in scientific computing. In this view, I must agree insofar I don't think Julia people are serious about their "general purpose" stance whatsoever. But hey, these are merely idiosyncrasies. People say multiple dispatch is the shit, but it's just one bit of the puzzle with Julia. How they managed to engineer a solid foundation, semantics like that, without unnecessarily sacrificing performance—I don't think they get enough credit for that from programming guys.
Post reply on HN