21–30 of 326 posts
Should be "that you can train for $100" Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.
Earlier quoted context omitted.
Controlling culture, yes but wild pivot to mention that criminal alongside Karpathy.
not a particularly ethical guy and I wouldn't hold him up as a example of morality but the guy hasn't actually been found guilty YET. Multiple courts have tried. You'd think that for a guy under as much scrutiny as him that they would have SOMETHING to pin him on by now. Innocent until PROVEN guilty is a foundational legal precedent for a reason.
There are 6 criminal cases against him in several countries, let’s see how they pan out - but regardless he is not an innocent person.
Should be "that you can train for $100" Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.
Should be "that you can train for $100" Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.
If I have let’s say 40gb RAM does it not work at all or just take twice as long to train?
That sounds like it could run on a 24gb GPU. Batch size of 8 would imply 20gb mem, no?
...presumably just takes forever
Here's the announcement post [0] from Karpathy, which provides a bit of additional context. [0] https://x.com/karpathy/status/1977755427569111362
[flagged]
We detached this subthread from https://news.ycombinator.com/item?id=45569878.
I started writing up a blog post on my weekend with nanoGPT but it's not done yet... Would have been great to link to here lol oh well
This weekend I just cracked into nanoGPT ( https://github.com/karpathy/nanoGPT ), an older but fabulous learning exercise where you build and train a crappy shakespeare GPT with ~0.8M parameters on a cpu. Results are about what you'd expect from that, they suck, but you can start to feel the magic, especially if you're not a deep learning professional and you just want to poke around and hack on it. I started writing…
This weekend I just cracked into nanoGPT ( https://github.com/karpathy/nanoGPT ), an older but fabulous learning exercise where you build and train a crappy shakespeare GPT with ~0.8M parameters on a cpu. Results are about what you'd expect from that, they suck, but you can start to feel the magic, especially if you're not a deep learning professional and you just want to poke around and hack on it. I started writing…
And this new example goes even further - adds instruction following and tool use SFT, as well as RLVR. Makes for a more useful baseline.