Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

31–40 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#31

Post by dmarcos »

andrewljohnson wrote: post #29
faxmeyourcode wrote: post #28

This weekend I just cracked into nanoGPT ( https://github.com/karpathy/nanoGPT ), an older but fabulous learning exercise where you build and train a crappy shakespeare GPT with ~0.8M parameters on a cpu. Results are about what you'd expect from that, they suck, but you can start to feel the magic, especially if you're not a deep learning professional and you just want to poke around and hack on it. I started writing…

the shakespeare code tuned a little with different training data does a good job of generating Magic The Gathering commander decks

I like the idea of specific-purpose toy models. How did you tune the code and what dataset you used?

Re: NanoChat – The best ChatGPT that $100 can buy

#32

Post by BrokenCogs »

karpathy wrote: post #15
daft_pink wrote: post #2

Wow, how do we sign up for the Eurekalabs course and how much does it cost?

Still under development, remaining work includes tuning nanochat (current state being solid v0.1) and finalizing the in-between projects so that students can "unlock" all complexity that hides underneath: `torch.Tensor`, `torch.dist`, `.backward()`, '.compile()`, etc. And then the more ops heavy aspects.

What's the pricing for the course/EurekaLabs? P.s. thanks for all you're doing

Re: NanoChat – The best ChatGPT that $100 can buy

#33

Post by viccis »

karimf wrote: post #4

I've always thought about the best way to contribute to humanity: number of people you help x how much you help them. I think what Karpathy is doing is one of the highest leverage ways to achieve that. Our current world is build on top of open source projects. This is possible because there are a lot of free resources to learn to code so anyone from anywhere in the world can learn and make a great piece of software.…

I recommend his ANN/LLM from scratch videos to people a lot because not only is he a clear instructor, but his code tends to be very Pythonic and just the right balance of terse but readable (not counting the Pytorch vectorization stuff, but that's not his fault, it's just complex). So I think people benefit just from watching and imitating his code style.

Re: NanoChat – The best ChatGPT that $100 can buy

#34

Post by croes »

karimf wrote: post #4

I've always thought about the best way to contribute to humanity: number of people you help x how much you help them. I think what Karpathy is doing is one of the highest leverage ways to achieve that. Our current world is build on top of open source projects. This is possible because there are a lot of free resources to learn to code so anyone from anywhere in the world can learn and make a great piece of software.…

I‘m afraid the technology will do more damage because many people will abuse it for fake news and misinformation.

Re: NanoChat – The best ChatGPT that $100 can buy

#36

Post by shafyy »

karimf wrote: post #4

I've always thought about the best way to contribute to humanity: number of people you help x how much you help them. I think what Karpathy is doing is one of the highest leverage ways to achieve that. Our current world is build on top of open source projects. This is possible because there are a lot of free resources to learn to code so anyone from anywhere in the world can learn and make a great piece of software.…

If it only were so easy

Re: NanoChat – The best ChatGPT that $100 can buy

#37

Post by zipy124 »

CountGeek wrote: post #35

So could I in practice train it on all my psychology books, materials, reports, case study and research papers and then run it on demand on a 1xH100 node - https://getdeploying.com/reference/cloud-gpu/nvidia-h100 whenever I have a specialised question?

You could but it would be significantly worse than fine-tuning or RAG with a pre-trained model, or using a smaller model since your dataset would be so small.

Re: NanoChat – The best ChatGPT that $100 can buy

#38

Post by antinomicus »

Onavo wrote: post #23
mhitza wrote: post #19

Should be "that you can train for $100" Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.

A GPU with 80GB VRAM costs around $1-3 USD an hour on commodity clouds (i.e. the non-Big 3 bare metal providers e.g. https://getdeploying.com/reference/cloud-gpu/nvidia-h100 ). I think it's accessible to most middle class users in first world countries.

Isn’t the whole point to run your model locally?

Re: NanoChat – The best ChatGPT that $100 can buy

#39

Post by [deleted] »

karimf wrote: post #4

I've always thought about the best way to contribute to humanity: number of people you help x how much you help them. I think what Karpathy is doing is one of the highest leverage ways to achieve that. Our current world is build on top of open source projects. This is possible because there are a lot of free resources to learn to code so anyone from anywhere in the world can learn and make a great piece of software.…

[deleted]
Post reply on HN