Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Tinygrad

geohot.github.io

Post reply on HN
Print view View original

141–150 of 167 posts

Re: Five Years of Tinygrad

#141

Post by georgehotz »

stathibus wrote: post #96
still-learning wrote: post #16

>People get hired by contributing to the repo. It’s a very self directed job, with one meeting a week and a goal of making tinygrad better I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.

It’s hard to argue against hiring contributors, but a bounty system that pays pennies vs. market value for skilled developers shouldn’t be the only interview path, it’s borderline exploitative.

What's even sadder is that because of AI coding agents it's possible the bounty system will go away. The value today is much less in writing the code and much more in defining the tasks and validating the code. We don't have the bandwidth to read tons of AI slop, which at first glance looks okay, but upon spending time to read it you realize it isn't good. The fix for this is probably a reputation or staking system.

I'm shocked that interviewing still works how it does in large companies; the Sybil attacks and DDOS are just getting started.

Re: Five Years of Tinygrad

#142

Post by 0x696C6961 »

atq2119 wrote: post #120
Lerc wrote: post #98

Earlier quoted context omitted.

I see the advantage of the inline lambdas, but also it seems like it would hamper the ability to test the functions which can often have intricacies that catch you out. I believe one of the principles of Tinygrad is that tests shouldn't be part of the core codebase and shouldn't add to the total linecount. I think there is an opportunity there to have a tag in comments for tests. Have a hotkey to generate an id so yo…

You don't really want or need classical unit tests for compilers. You need a harness that you can feed a program representation into and get a transformed one out, then check the result -- ideally mostly just by running it, or alternatively by comparing the output against a reference.

How is that not a unit test?

Re: Five Years of Tinygrad

#143

Post by johnnyfived »

xiphias2 wrote: post #81
rent0pat wrote: post #56

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

I think it's an amazing experiment. You can look at the PyTorch code base and understand a small local function instantly, but if I would have a task to have a deep understanding of either PyTorch with all kernel code and low level code, whole CUDA code base + LLVM compilation code or Tinygrad I would pick Tinygrad in an instant. The code looks hard because what it is doing is hard, but all its layers can be debugged…

Does doing hard work mean it shouldn't have any docstrings or comments? Cause that seems like an arbitrary way of making it "harder".

Re: Five Years of Tinygrad

#144

Post by rustybolt »

0x696C6961 wrote: post #142
atq2119 wrote: post #120

Earlier quoted context omitted.

You don't really want or need classical unit tests for compilers. You need a harness that you can feed a program representation into and get a transformed one out, then check the result -- ideally mostly just by running it, or alternatively by comparing the output against a reference.

How is that not a unit test?

I've hear people refer to it as an end-to-end test, where unit tests usually test a single class or function.

Re: Five Years of Tinygrad

#145

Post by xiphias2 »

johnnyfived wrote: post #143
xiphias2 wrote: post #81

Earlier quoted context omitted.

I think it's an amazing experiment. You can look at the PyTorch code base and understand a small local function instantly, but if I would have a task to have a deep understanding of either PyTorch with all kernel code and low level code, whole CUDA code base + LLVM compilation code or Tinygrad I would pick Tinygrad in an instant. The code looks hard because what it is doing is hard, but all its layers can be debugged…

Does doing hard work mean it shouldn't have any docstrings or comments? Cause that seems like an arbitrary way of making it "harder".

Docstrings and comments are amazing to be able to work with huge code bases with big teams where you don't want to understand the whole code base, because it's impossible.

If the goal is to not abstract anything away, and always look at the implementation itself to be able to change it / rewrite it, keep it tiny and it is able to beat PyTorch+CUDA+LLVM with 1000x smaller codebase, who am I to judge?

Re: Five Years of Tinygrad

#147

Post by tekla »

blactuary wrote: post #139
bko wrote: post #48

Earlier quoted context omitted.

Did we read the same article? The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite. The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connectin…

The Elon process is 100% dependency on government subsidies

Elon derangement syndrome at maximum. Tell me where Starlink funding is coming from?

Re: Five Years of Tinygrad

#148

Post by measurablefunc »

hrimfaxi wrote: post #123
measurablefunc wrote: post #89

Earlier quoted context omitted.

I don't care for money but you seem to so you should focus on making as much as possible.

> pathologizing the speaker/writer does not contribute to anything either so my recommendation is to leave those out as well.

> There's a giant pile of money at the end of the rainbow for you

Re: Five Years of Tinygrad

#149

Post by stathibus »

monooso wrote: post #133
stathibus wrote: post #96

Earlier quoted context omitted.

It’s hard to argue against hiring contributors, but a bounty system that pays pennies vs. market value for skilled developers shouldn’t be the only interview path, it’s borderline exploitative.

As opposed to months of interview prep, followed by half a dozen interview stages, possibly including an unpaid take-home mini project, all of which could be for naught because you fluffed some obscure algorithm question that bears no relation to your day job?

There are also other bad interview processes, yes.

Re: Five Years of Tinygrad

#150

Post by octoberfranklin »

sampullman wrote: post #132
octoberfranklin wrote: post #74

Earlier quoted context omitted.

Have you run into problems with contributors who can't enter the PRC?

The mainland and Hong Kong still have significantly different visa policies. I'm not sure if it's changed at all since the handover, except for mainlanders entering HK.

Yes I know, but HK is still part of the PRC and there are people who cannot travel to the PRC.

It's not about (in)ability to obtain a visa.

Post reply on HN