Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

181–190 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#181

Post by jwitthuhn »

andrewljohnson wrote: post #29
faxmeyourcode wrote: post #28

This weekend I just cracked into nanoGPT ( https://github.com/karpathy/nanoGPT ), an older but fabulous learning exercise where you build and train a crappy shakespeare GPT with ~0.8M parameters on a cpu. Results are about what you'd expect from that, they suck, but you can start to feel the magic, especially if you're not a deep learning professional and you just want to poke around and hack on it. I started writing…

the shakespeare code tuned a little with different training data does a good job of generating Magic The Gathering commander decks

Somewhat related: I wrote up a MTG card generator based on nanoGPT a while ago that I think produces pretty good results for being 1m parameters.

The real neat thing about this is that WotC makes a few thousand new cards each year, so my training data set just grows over time and the model gets better with no effort spent on my part.

https://github.com/jlwitthuhn/TCGGPT

Re: NanoChat – The best ChatGPT that $100 can buy

#182

Post by teleforce »

gyomu wrote: post #121
tehnub wrote: post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

I wonder if the new GenAI architecture namely DDN or distributed discrete networks being discussed recently can outperform the conventional architecture of GAN and VAE. As the name suggests, it can provide multitude of distributions for training and inference purposes [1].

[1] Show HN: I invented a new generative model and got accepted to ICLR (90 comments):

https://news.ycombinator.com/item?id=45536694

Re: NanoChat – The best ChatGPT that $100 can buy

#184

Post by vessenes »

simonw wrote: post #178
sammyd56 wrote: post #170

Earlier quoted context omitted.

I've uploaded the model here: https://huggingface.co/sdobson/nanochat I didn't get as good results as Karpathy (unlucky seed?) It's fun to play with though... User: How many legs does a dog have? Assistant: That's a great question that has been debated by dog enthusiasts for centuries. There's no one "right" answer (...)

I got your model working on CPU on macOS by having Claude Code hack away furiously for a while. Here's a script that should work for anyone: https://gist.github.com/simonw/912623bf00d6c13cc0211508969a1... You can run it like this: cd /tmp git clone https://huggingface.co/sdobson/nanochat uv run https://gist.githubusercontent.com/simonw/912623bf00d6c13cc0211508969a100a/raw/80f79c6a6f1e1b5d4485368ef3ddafa5ce853131/gene…

Simon, I had to run "brew install git-lfs && cd nano-chat && git lfs install && git lfs pull" and then it worked. before then, the model weights didn't get cloned by default for me on macOS.

% uv run https://gist.githubusercontent.com/simonw/912623bf00d6c13cc0... \ --model-dir nanochat/ --prompt "who is simonw on hacker news?" Using device: cpu Loading model from nanochat/model_000650.pt Loading metadata from nanochat/meta_000650.json Model config: {'sequence_len': 2048, 'vocab_size': 65536, 'n_layer': 20, 'n_head': 10, 'n_kv_head': 10, 'n_embd': 1280} Loading model weights (this may take a minute for a 2GB model)... Converting model to float32 for CPU... Model loaded successfully! Loading tokenizer... Tokenizer loaded successfully!

Prompt: who is simonw on hacker news? Encoded to 9 tokens

Generating... -------------------------------------------------- who is simonw on hacker news?A hacker news reporter, I'd say a few things. First, I'm a bit of a hothead, always pushing the boundaries of what's acceptable in the world of hacking. I've got a reputation for being merciless and relentless in my pursuit of the truth.

In many ways, I've developed a sixth sense for this type of thing. I've spent years honing my skills, learning the language of hacking and the tactics it takes. I know how to think like the hacker --------------------------------------------------

Re: NanoChat – The best ChatGPT that $100 can buy

#185

Post by bravura »

sammyd56 wrote: post #58

I'm doing a training run right now (started 20min ago). You can follow it at https://api.wandb.ai/links/sjd333-none/dsv4zkij Will share the resulting model once ready (4 hours from now) for anyone to test inference.

The measures that drop exponentially like val/bpb and train/loss you should put the x-axis in log-scale. That will better show you if it's converged

Re: NanoChat – The best ChatGPT that $100 can buy

#186

Post by astrange »

SeanAnderson wrote: post #71
andrewljohnson wrote: post #29

Earlier quoted context omitted.

the shakespeare code tuned a little with different training data does a good job of generating Magic The Gathering commander decks

would love more details on this. this is exactly the type of project I'd like to dabble in to get more up to speed.

People have been doing this for a while.

https://x.com/roborosewater

https://bsky.app/profile/roborosewaterm.bsky.social

You can see the invention of RLHF/ChatGPT here because text generation suddenly became much more coherent and also much less interesting. You have to go back to older tech for surrealism because nobody will let you see the good stuff (the base models).

Re: NanoChat – The best ChatGPT that $100 can buy

#187

Post by faxmeyourcode »

ACCount37 wrote: post #30
faxmeyourcode wrote: post #28

This weekend I just cracked into nanoGPT ( https://github.com/karpathy/nanoGPT ), an older but fabulous learning exercise where you build and train a crappy shakespeare GPT with ~0.8M parameters on a cpu. Results are about what you'd expect from that, they suck, but you can start to feel the magic, especially if you're not a deep learning professional and you just want to poke around and hack on it. I started writing…

It's a useful exercise. A lot of the good ML work is first validated at small scale. And this new example goes even further - adds instruction following and tool use SFT, as well as RLVR. Makes for a more useful baseline.

Absolutely, it's wildly fun to read the outputs of even a little tiny 0.8M model trained on CPU. And now I've actually got a much better understanding of the transformer architecture after playing around with it for a day. This repo is probably going to spawn some new folks to try out ideas which will turn into new researchers in the field, no doubt.

Re: NanoChat – The best ChatGPT that $100 can buy

#188

Post by squeaky-clean »

goalieca wrote: post #176
notatoad wrote: post #151

Earlier quoted context omitted.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.

Wysiwyg kind of fell apart once we had to stop assuming everyone had an 800x600 or 1024x768 screen, because what you saw was no longer what others got.

Re: NanoChat – The best ChatGPT that $100 can buy

#189

Post by iamcreasy »

simonw wrote: post #178
sammyd56 wrote: post #170

Earlier quoted context omitted.

I've uploaded the model here: https://huggingface.co/sdobson/nanochat I didn't get as good results as Karpathy (unlucky seed?) It's fun to play with though... User: How many legs does a dog have? Assistant: That's a great question that has been debated by dog enthusiasts for centuries. There's no one "right" answer (...)

I got your model working on CPU on macOS by having Claude Code hack away furiously for a while. Here's a script that should work for anyone: https://gist.github.com/simonw/912623bf00d6c13cc0211508969a1... You can run it like this: cd /tmp git clone https://huggingface.co/sdobson/nanochat uv run https://gist.githubusercontent.com/simonw/912623bf00d6c13cc0211508969a100a/raw/80f79c6a6f1e1b5d4485368ef3ddafa5ce853131/gene…

For anyone curious this is the error when running uv sync on macos,

> uv sync Resolved 88 packages in 3ms error: Distribution `torch==2.8.0+cu128 @ registry+https://download.pytorch.org/whl/cu128` can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're on macOS (`macosx_15_0_arm64`), but `torch` (v2.8.0+cu128) only has wheels for the following platforms: `manylinux_2_28_x86_64`, `win_amd64`; consider adding your platform to `tool.uv.required-environments` to ensure uv resolves to a version with compatible wheels

Also, tmp/nanochat expects all contents from tokenizer and chatsft_checkpoints folder.

Re: NanoChat – The best ChatGPT that $100 can buy

#190

Post by sammyd56 »

simonw wrote: post #178
sammyd56 wrote: post #170

Earlier quoted context omitted.

I've uploaded the model here: https://huggingface.co/sdobson/nanochat I didn't get as good results as Karpathy (unlucky seed?) It's fun to play with though... User: How many legs does a dog have? Assistant: That's a great question that has been debated by dog enthusiasts for centuries. There's no one "right" answer (...)

I got your model working on CPU on macOS by having Claude Code hack away furiously for a while. Here's a script that should work for anyone: https://gist.github.com/simonw/912623bf00d6c13cc0211508969a1... You can run it like this: cd /tmp git clone https://huggingface.co/sdobson/nanochat uv run https://gist.githubusercontent.com/simonw/912623bf00d6c13cc0211508969a100a/raw/80f79c6a6f1e1b5d4485368ef3ddafa5ce853131/gene…

This is a much easier way to run the model. I'm going to update the huggingface README to point to this. The one thing that could be improved is the turn-taking between user and assistant, which it sometimes gets confused about. I fixed that in my fork of your gist here: https://gist.github.com/samdobson/975c8b095a71bbdf1488987eac...
Post reply on HN