Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinygrad

github.com

Post reply on HN
Print view View original

141–150 of 159 posts

Re: Tinygrad

#141

Post by inciampati »

EZ-Cheeze wrote: post #105

Algos run the world. "In bioinformatics, BLAST (basic local alignment search tool) is an algorithm and program for comparing primary biological sequence information, such as the amino-acid sequences of proteins or the nucleotides of DNA and/or RNA sequences." - could it be used to identify and group similar structures in NNs? ( https://en.wikipedia.org/wiki/BLAST_(biotechnology) ) I've been thinking of some kind of v…

Using bioinformatics tools to understand neural networks. What a fasinating idea! It probably makes sense to go for the direct application of string and pattern matching methods from stringology. BLAST is very much meant for DNA and protein biosequences.

edit: It would be cool to see the evolution of neural networks over their training, and im transfer learning. Comparative neural network genomics.

Re: Tinygrad

#142

Post by sdenton4 »

Buttons840 wrote: post #79

I'd like to ship a video game that does machine learning and trains on its experiences with the player. Yes, I know there are many potential problems with this. What is the best way to ship training code in a game? Do I embed Python and PyTorch or something? Do I code my own NN training algorithm? Do I use a library such as Tinygrad?

It's likely worth looking into federated learning. You could have a core model, which then trains locally on player inputs. The federated learning research space is focused on this sort of small model update on phones, so should have plenty of people who have thought hard about the problem of keeping binary sizes small and training compute manageable.

https://arxiv.org/abs/1909.11875

Re: Tinygrad

#143

Post by CornCobs »

Buttons840 wrote: post #89
alar44 wrote: post #82

Earlier quoted context omitted.

Do you know how to program? I have no idea what you're asking.

Yes, I know how to program and know a good amount about machine learning in Python. The problem is these machine learning frameworks are rather heavy to be shipping around, although I guess you could do it. I was wondering if there's any lighter weight machine learning libraries, which might not have all the bells and whistles, but would be better to ship and train models on consumers computers. Most of the time peop…

Client side training with the usual python stack sounds like a configuration nightmare. Would it try to use the users GPU if it exists? Would it fallback to CPU? Does your user need a specific GPU card to play?

Re: Tinygrad

#144

Post by ausbah »

upbeat_general wrote: post #111
tartakovsky wrote: post #58

For a novice in this space, can anyone provide a link to an article providing a comparison of: Pytorch, Pytorch lightning, tinygrad, micrograd.....? I would like to point my clients to a reference here. It's no longer enough to say "PyTorch" I guess.

PyTorch lightning is just a convenience wrapper for PyTorch. The rest are toy re-implementations of a very small subset of PyTorch features, likely much slower and certainly less optimized overall. PyTorch (or Tensorflow or Keras) are the real options.

also Jax, hopefully

Re: Tinygrad

#145

Post by mwexler »

mdp2021 wrote: post #53
whimsicalism wrote: post #3

Reminds me of https://github.com/minitorch/minitorch

Minitorch is intended to be both an engine and high quality didactic material, from Cornell University, for the course 'Machine Learning Engineering'. > the full student code for minitorch. It is designed as a single repo that can be completed part by part following the guide book > Basic Neural Networks and Modules ; Autodifferentiation for Scalars ; Tensors, Views, and Strides ; Parallel Tensor Operations ; GPU / C…

I am reminded of Andrew Tanenbaum's Minix os: before Linux became the belle of the ball, minix implemented many unix functionalities in a less efficient but clearer way than existing unix builds (many of which were not open source) and the still nascent bsd/linux/gnuos/etc, and it meshed nicely with his textbooks on OS design. It was functional on it's own, and by Minix 3 was a full os in it's own right (Intel still uses it in various ways) but balanced pedagogy with performance.

https://en.wikipedia.org/wiki/Minix gives more details.

Re: Tinygrad

#147

Post by EZ-Cheeze »

inciampati wrote: post #141
EZ-Cheeze wrote: post #105

Algos run the world. "In bioinformatics, BLAST (basic local alignment search tool) is an algorithm and program for comparing primary biological sequence information, such as the amino-acid sequences of proteins or the nucleotides of DNA and/or RNA sequences." - could it be used to identify and group similar structures in NNs? ( https://en.wikipedia.org/wiki/BLAST_(biotechnology) ) I've been thinking of some kind of v…

Using bioinformatics tools to understand neural networks. What a fasinating idea! It probably makes sense to go for the direct application of string and pattern matching methods from stringology. BLAST is very much meant for DNA and protein biosequences. edit: It would be cool to see the evolution of neural networks over their training, and im transfer learning. Comparative neural network genomics.

Yeah I want extreme cross-disciplinary collaboration and new ideas being tried constantly

Re: Tinygrad

#148

Post by viperoso312 »

trompetenaccoun wrote: post #117
dukeofdoom wrote: post #94

I watched a few of George's live streams, and I'm pretty impressed with his coding skills and determination to solve a problem. In the spirit of learning, anyone else on his level do live streams or has a youtube channel? Here's his last 7 hour stream coding Tinygrad. https://www.youtube.com/watch?v=MeE4Y2862FY

He takes a random IQ test in the middle of it, lol.

do you have the minute of that?

Re: Tinygrad

#150

Post by rrss »

buildbot wrote: post #99
modeless wrote: post #32

I find this related page more interesting: A Breakdown of AI Chip Companies https://geohot.github.io/blog/jekyll/update/2021/06/13/a-bre... I especially like that he outlines an actual plan for an AI chip startup that he thinks will work, and has an update explaining why he was subsequently convinced that it wouldn't work.

I read this article vs. the actually posted one, it has a lot of good points but also get a fair amount wrong, and the correction on the power usage is just the tip of the iceberg. That's why WaferScale do what they do, the power cost of off die vs. on die is massive, and enabling everything to be on chip means you can feed the design more easily and with less power. For example, Nvidia's compute and consumer GPU lin…

> for example the P100 has hardware scheduling, where as the 1080 does not (in the same way at least).

what does hardware scheduling mean in this context?

Post reply on HN