61–70 of 189 posts
https://twitter.com/karpathy/status/1777427944971083809 > And once this is a in a bit more stable state: videos on building this in more detail and from scratch. Looking forward to watching the videos.
I love his videos. They are dense, but I get a lot out of them.
OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?
George Hotz is attempting to solve this: https://github.com/tinygrad/tinygrad
> direct CUDA implementation, which will be significantly faster and probably come close to PyTorch. It almost hurts, to read that PyTorch is faster. But then again, with these GPU-RAM-prices, let's see how it speeds up the CPU. We really need SO-DIMM slots on the RTX series (or AMD/Intel equivalent) so that we can expand the RAM as we need it to. Is there a technical problem to it?
When you have a pulse coded signal traveling at close to 10GHz, everything becomes an antenna. The technical problem is that you can't do this with a flimsy connector like the ones used for DIMMs. The reason GDDR can have a bandwidth per pin that is 4 times higher than regular DDR is because they are soldered down on the PCB.
OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?
Download Ollama on a modern MacBook and can run 13B and even higher (if your RAM allows) at fast speeds. People run smaller models locally on their phones
Google has trained their latest models on their own TPUs... not using Nvidia to my knowledge.
So, no, there are alternatives. CUDA has the largest mindshare on the training side though.
Earlier quoted context omitted.
For that stuff, yeah you're correct. What I've seen is issues with the implementation of those libraries in a project. I don't remember exactly, but I was playing with someone's wrapper for some kind of machine learning snake game and it was taking way longer than it should have on back of the napkin math. The issue was using either a dict or a list in a hot loop and changing it to the other sped it up like 1000x. So…
> The issue was using either a dict or a list in a hot loop and changing it to the other sped it up like 1000x. The programmer using the wrong data structure is not a problem with the language.
As my understanding of LLM goes at a basic level it's predicting the next token from previous tokens, which sounds directionally similar to time series (perhaps letting aside periodicity).
It was originally based off of Karpathy's llama2.c but I renamed it when I added support for other architectures.
Probable a coincidence :)
Very sad, shouldve used an agnostic framework instead of CUDA