Tinygrad • Hacker News Board

Re: Tinygrad

#111

Post by upbeat_general » Tue, Apr 05, 2022, 2:05 AM UTC

tartakovsky wrote: post #58

For a novice in this space, can anyone provide a link to an article providing a comparison of: Pytorch, Pytorch lightning, tinygrad, micrograd.....? I would like to point my clients to a reference here. It's no longer enough to say "PyTorch" I guess.

PyTorch lightning is just a convenience wrapper for PyTorch. The rest are toy re-implementations of a very small subset of PyTorch features, likely much slower and certainly less optimized overall.

PyTorch (or Tensorflow or Keras) are the real options.

Re: Tinygrad

#112

Post by girvo » Tue, Apr 05, 2022, 2:15 AM UTC

bko wrote: post #61

mdp2021 wrote: post #49

Earlier quoted context omitted.

> not a normal guy I assure you some people try themselves - and I do not see what is not "normal" about it. To experience, voluntary, then grow, is the norm.

I don't know. I ran the idea of purposely getting addicted to opioids to see if i can quit by my wife and she assured me i was crazy.

As someone who somewhat went down that path, the answer for me was "no, I can't quit on my own and man this 'experiment' has done some serious damage to my life"

I'm doing better now! The buprenorphine injection has made my life so much better.

Of course my trauma was one of the real driving forces behind that "experiment" and thought process. Really, my "lets find out what its like" was a rationalisation it seems.

Re: Tinygrad

#113

Post by girvo » Tue, Apr 05, 2022, 2:17 AM UTC

Buttons840 wrote: post #79

I'd like to ship a video game that does machine learning and trains on its experiences with the player. Yes, I know there are many potential problems with this. What is the best way to ship training code in a game? Do I embed Python and PyTorch or something? Do I code my own NN training algorithm? Do I use a library such as Tinygrad?

In theory, once could use Arraymancer to build the tools to do this, and not have to ship a Python interpreter. That said, I've not used it in anger.

https://github.com/mratsim/Arraymancer

Re: Tinygrad

#114

Post by paisawalla » Tue, Apr 05, 2022, 2:40 AM UTC

biscoitinho wrote: post #96

bko wrote: post #61

Earlier quoted context omitted.

I don't know. I ran the idea of purposely getting addicted to opioids to see if i can quit by my wife and she assured me i was crazy.

I'm in a similar situation, but not with opioids. I don't think that this is a crazy idea at all. We're just testing our limits and seeing if we as smart as we say.

Agreed. To test my belief in probabilities, I occasionally challenge myself to a few rounds of Russian Roulette.

Re: Tinygrad

#115

Post by generalizations » Tue, Apr 05, 2022, 2:50 AM UTC

Buttons840 wrote: post #79

I'd like to ship a video game that does machine learning and trains on its experiences with the player. Yes, I know there are many potential problems with this. What is the best way to ship training code in a game? Do I embed Python and PyTorch or something? Do I code my own NN training algorithm? Do I use a library such as Tinygrad?

I suppose you could collect training data during gameplay and process it after the fact. Then you can use the heavy python framework, but it doesn't get in the way. Have the user run the training program on the accumulated data whenever they want to increase the level of customization.

Re: Tinygrad

#116

Post by ulber » Tue, Apr 05, 2022, 2:57 AM UTC

pixelpoet wrote: post #18

hmate9 wrote: post #9

This is the guy that did some iOS jailbreaks, reverse engineered the PS3 and now runs a self driving car startup.

And the shady cheapETH business last year: https://oldreddit.com/r/cheapETH/comments/lkzkso/george_hotz...

Here's his response: https://cheapeth.org/whalegate.html It casts the situation in a rather different light - you should read it.

Re: Tinygrad

#117

Post by trompetenaccoun » Tue, Apr 05, 2022, 2:59 AM UTC

dukeofdoom wrote: post #94

I watched a few of George's live streams, and I'm pretty impressed with his coding skills and determination to solve a problem. In the spirit of learning, anyone else on his level do live streams or has a youtube channel? Here's his last 7 hour stream coding Tinygrad. https://www.youtube.com/watch?v=MeE4Y2862FY

He takes a random IQ test in the middle of it, lol.

Re: Tinygrad

#118

Post by johnamata » Tue, Apr 05, 2022, 3:26 AM UTC

londons_explore wrote: post #55

> tinygrad will always be below 1000 lines. If it isn't, we will revert commits until tinygrad becomes smaller. I applaud this. Committing to keeping a project small and simple. So many projects start small and simple, and before long they've been extended in many different directions and now have thousands of options and things to understand before you can get started.

Are there other examples of famous projects that do that, limiting themselves to an X-amount of LOC? Last one I remember was TempleOS, although Terry went a bit over his limit (100k LOC).

Re: Tinygrad

#119

Post by georgehotz » Tue, Apr 05, 2022, 3:42 AM UTC

cma wrote: post #97

striking wrote: post #72

Earlier quoted context omitted.

I'm sorry to have offended you, but I don't feel personal insults over the README of an open source project are particularly reasonable. This affects the actual code, too! See, for example, https://github.com/geohot/tinygrad/commit/cfb7a4c41a2b6bcc09... , which includes this gem: diff --git a/tinygrad/ops/ops_cpu.py b/tinygrad/ops/ops_cpu.py index a454f56f..0686f810 100644 --- a/tinygrad/ops/ops_cpu.py +++ b/tinygrad…

It could be written: def log(x): return np.log(x) def exp(x): return np.exp(x) ... And keep the same line count, and the functions would keep their __name__.

I would merge this PR.

Re: Tinygrad

#120

Post by modeless » Tue, Apr 05, 2022, 3:49 AM UTC

gary_0 wrote: post #109

modeless wrote: post #32

I find this related page more interesting: A Breakdown of AI Chip Companies https://geohot.github.io/blog/jekyll/update/2021/06/13/a-bre... I especially like that he outlines an actual plan for an AI chip startup that he thinks will work, and has an update explaining why he was subsequently convinced that it wouldn't work.

The detailed follow-up to that post is here: https://geohot.github.io/blog/jekyll/update/2021/12/12/a-cor...

Interesting! I think Cerebras is exciting too, the problem is that it's so expensive that there will never be a software ecosystem for it. The people who would develop it will never have access to one.