Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

171–180 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#171

Post by vunderba »

rootusrootus wrote: post #135
tehnub wrote: post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

That is a good thing to hear from someone as reputable as Karpathy. The folks who think we're on the cusp of AGI may want to temper their expectations a bit. I do love Claude Code, because one thing I periodically need to do is write some web code, which is not my favorite type of coding but happens to have incredibly good coverage in the training data. Claude is a much better web developer than I am. But for digging…

100%. I find the "LLMs are completely useless" and the "LLMs will usher in a new era of messianic programming" camps to be rather reductive.

I've already built some pretty large projects [1] with the assistance of agentic tooling like Claude Code. When it comes to the more squirrely algorithms and logic, they can fall down pretty hard. But as somebody who is just dreadful at UI/UX, having it hammer out all the web dev scaffolding saves me a huge amount of time and stress.

It's just a matter of tempering one's expectations.

[1] https://animated-puzzles.specr.net

Re: NanoChat – The best ChatGPT that $100 can buy

#172

Post by rootusrootus »

Filligree wrote: post #153
bdangubic wrote: post #143

Earlier quoted context omitted.

> But for digging into the algorithmic core of our automation tooling What I find fascinating is reading this same thing in other context like “UI guru” will say “I would not let CC touch the UI but I let it rip on algorithmic core of our automation tooling cause it is better at it than me…”

Both can be true. LLMs tend to be mediocre at (almost) everything, so they're always going to be worse than the user at whatever the user is an expert in. But 'mediocre' isn't 'useless'.

I completely agree. I'm definitely not an expert web developer. I know enough to build functional tools, but it's not exactly art that I'm making. But the core of our tooling is my primary focus, I wrote it, I've spent a lot of time perfecting it. Claude can easily impress me with things like the CSS magic it weaves, because I am unsophisticated.

Re: NanoChat – The best ChatGPT that $100 can buy

#173

Post by nl »

typpilol wrote: post #124
Lerc wrote: post #101

Earlier quoted context omitted.

The comment beside the first chart >Our main measure of progress. Bits per byte is, per Karpathy, "a much better measure than just the typical cross-entropy loss, because it further normalizes the loss on each token by the number of bytes of that token, making the metric tokenizer-invariant". Is so blindingly obvious, that I'm ashamed to think that I didn't think do it when trialing my own tokenizer approach on tinys…

Why hasn't anyone made a tokenizer that's 1 character per token. Is it because it requires an insane amount of compute? Or would the loss of efficiency make it dumber then modern tokenizers?

Tokenizers used to be 1 character per token. Then Google implemented Subword encoding[1] on their early neural translation work and found it was much better.

Subword units are genuinely meaningful in most languages. You do need to tune the vocabulary size though.

[1] https://aclanthology.org/P16-1162/

Re: NanoChat – The best ChatGPT that $100 can buy

#174

Post by nl »

danielmarkbruce wrote: post #127
sieve wrote: post #97

Earlier quoted context omitted.

If you want to train/sample large models, then use what the rest of the industry uses. My use case is different. I want something that I can run quickly on one GPU without worrying about whether it is supported or not. I am interested in convenience, not in squeezing out the last bit of performance from a card.

You wildly misunderstand pytorch.

I suspect the OP's issues might be mostly related to the ROCM version of PyTorch. AMD still can't get this right.

Re: NanoChat – The best ChatGPT that $100 can buy

#175

Post by vunderba »

SeanAnderson wrote: post #71
andrewljohnson wrote: post #29

Earlier quoted context omitted.

the shakespeare code tuned a little with different training data does a good job of generating Magic The Gathering commander decks

would love more details on this. this is exactly the type of project I'd like to dabble in to get more up to speed.

FWIW, there was a pretty popular post on HN around generating MTG cards using AI a couple years back but I believe that their approach was a fine-tune on an existing LLM.

https://news.ycombinator.com/item?id=37427854

Re: NanoChat – The best ChatGPT that $100 can buy

#176

Post by goalieca »

notatoad wrote: post #151
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed..

Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.

Re: NanoChat – The best ChatGPT that $100 can buy

#177

Post by MasterScrat »

SeanAnderson wrote: post #149
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

or a typical CRUD app architecture, or a common design pattern, or unit/integration test scaffolding, or standard CI/CD pipeline definitions, or one-off utility scripts, etc... Like 80% of writing coding is just being a glorified autocomplete and AI is exceptional at automating those aspects. Yes, there is a lot more to being a developer than writing code, but, in those instances, AI really does make a difference in…

And even for "out of distribution" code you can still ask question about how to do the same thing but more optimized, could a library help for this, why is that piece of code giving this unexpected output etc

Re: NanoChat – The best ChatGPT that $100 can buy

#178

Post by simonw »

sammyd56 wrote: post #170
sammyd56 wrote: post #58

I'm doing a training run right now (started 20min ago). You can follow it at https://api.wandb.ai/links/sjd333-none/dsv4zkij Will share the resulting model once ready (4 hours from now) for anyone to test inference.

I've uploaded the model here: https://huggingface.co/sdobson/nanochat I didn't get as good results as Karpathy (unlucky seed?) It's fun to play with though... User: How many legs does a dog have? Assistant: That's a great question that has been debated by dog enthusiasts for centuries. There's no one "right" answer (...)

I got your model working on CPU on macOS by having Claude Code hack away furiously for a while. Here's a script that should work for anyone: https://gist.github.com/simonw/912623bf00d6c13cc0211508969a1...

You can run it like this:

  cd /tmp
  git clone https://huggingface.co/sdobson/nanochat
  uv run https://gist.githubusercontent.com/simonw/912623bf00d6c13cc0211508969a100a/raw/80f79c6a6f1e1b5d4485368ef3ddafa5ce853131/generate_cpu.py \
    --model-dir /tmp/nanochat \
    --prompt "Tell me about dogs."

Re: NanoChat – The best ChatGPT that $100 can buy

#179

Post by danielmarkbruce »

nl wrote: post #174
danielmarkbruce wrote: post #127

Earlier quoted context omitted.

You wildly misunderstand pytorch.

I suspect the OP's issues might be mostly related to the ROCM version of PyTorch. AMD still can't get this right.

Probably - but the answer is to avoid ROCM, not pytorch.

Re: NanoChat – The best ChatGPT that $100 can buy

#180

Post by chairmansteve »

notatoad wrote: post #151
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

I agree. I am "writing" simple crud apps for my own convenience and entertainment. I can use unfamiliar frameworks and languaged for extra fun and education.

Good times!

Post reply on HN