Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

281–290 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#281

Post by notatoad »

mrugge wrote: post #258
pja wrote: post #237

Earlier quoted context omitted.

Yep. I find the hype around AI to be wildly overblown, but that doesn’t mean that what it can do right now isn’t interesting & useful. If you told me a decade ago that I could have a fuzzy search engine on my desktop that I could use to vaguely describe some program that I needed & it would go out into the universe of publicly available source code & return something that looks as close to the thing I’ve asked for as…

Why do you think what you describe being excited about does not warrant the current level of AI hype? I agree with your assessment and sometimes I think there is too much cynicism and not enough excitement.

the current level of AI hype amongst a lot of people, but especially investors and bosses, is that you can already give an AI a simple prompt and get it to spit out a fully functional, user-ready application for you. and we're so incredibly far off that.

the things that AI is able to do are incredible, but hype levels are just totally detached from reality.

Re: NanoChat – The best ChatGPT that $100 can buy

#282

Post by JLC443 »

ggsp wrote: post #235
vunderba wrote: post #171

Earlier quoted context omitted.

100%. I find the "LLMs are completely useless" and the "LLMs will usher in a new era of messianic programming" camps to be rather reductive. I've already built some pretty large projects [1] with the assistance of agentic tooling like Claude Code. When it comes to the more squirrely algorithms and logic, they can fall down pretty hard. But as somebody who is just dreadful at UI/UX, having it hammer out all the web de…

Hey, thank you for making this—I really enjoyed playing it and it feels like it fits the mental-reward-between-work-tasks need. It did spin up my M1's fans after a few minutes which is a rather rare occurrence, but I'm guessing that's par for the course when you're working with a bunch of video on canvas. Either way, hope I remember it the next time I'm looking for a puzzle to solve while I take a break :)

Just thought I'd add to this thread that I also had a lot of fun playing this game, and I don't normally enjoy puzzles on the computer!

A couple of very minor pieces of feedback, if you're open to it: The camera momentum when dragging felt a little unnatural. The videos seemed to have a slightly jumpy framerate and were a bit low-resolution when zoomed in.

Honestly though, those are minor nitpicks. It's a really fun and polished experience. Thanks for sharing!

Re: NanoChat – The best ChatGPT that $100 can buy

#283

Post by bravura »

sammyd56 wrote: post #241
bravura wrote: post #185

Earlier quoted context omitted.

The measures that drop exponentially like val/bpb and train/loss you should put the x-axis in log-scale. That will better show you if it's converged

Great call, thankyou - I switched to log scale for those metrics - agree that it is much clearer.

Sorry fat fingers. It should be the y axis that is log scale, not x axis. (Sometimes both is good.)

Did you notice the inflection point in which the loss drops faster than expected in the top graph? Maybe you should let it run more…

Re: NanoChat – The best ChatGPT that $100 can buy

#284

Post by zoba »

chipsrafferty wrote: post #157

Would love to hear some metrics on training it on your personal computer rather than a "cloud GPU box". I don't care if it takes 3 months to train if I have something good, offline, and free(ish, but just pay electric bills)

I’d also be interested in this. Especially for Macs

Re: NanoChat – The best ChatGPT that $100 can buy

#285

Post by nurettin »

tehnub wrote: post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

Backprop and transformers isn't exactly off the grid coding, but I can see how it would require a lot of patience to force claude into writing this.

Re: NanoChat – The best ChatGPT that $100 can buy

#286

Post by danielmarkbruce »

yorwba wrote: post #238
danielmarkbruce wrote: post #179

Earlier quoted context omitted.

Probably - but the answer is to avoid ROCM, not pytorch.

Avoiding ROCm means buying a new Nvidia GPU. Some people would like to keep using the hardware they already have.

The cost to deal with rocm is > cost of a consumer nvidia gpu by orders of magnitude.

Re: NanoChat – The best ChatGPT that $100 can buy

#287

Post by pmarreck »

notatoad wrote: post #281
mrugge wrote: post #258

Earlier quoted context omitted.

Why do you think what you describe being excited about does not warrant the current level of AI hype? I agree with your assessment and sometimes I think there is too much cynicism and not enough excitement.

the current level of AI hype amongst a lot of people, but especially investors and bosses, is that you can already give an AI a simple prompt and get it to spit out a fully functional, user-ready application for you. and we're so incredibly far off that. the things that AI is able to do are incredible, but hype levels are just totally detached from reality.

> is that you can already give an AI a simple prompt and get it to spit out a fully functional, user-ready application for you.

But it can already do that. Isn't that the whole "one-shotting" thing?

The problem is, of course, that it won't be optimized, maintainable or have anyone responsible you can point to if something with it goes wrong. It almost certainly (unless you carefully prompted it to) won't have a test suite, which means any changes (even fixes) to it are risky.

So it's basically a working mockup generator.

I am so, so tired of "semi-technical" youtubers showing off new models with one-shots. The vast majority of actual devs who use this stuff need it to work over long-term context windows and over multiple iterations.

Re: NanoChat – The best ChatGPT that $100 can buy

#288

Post by pmarreck »

CapsAdmin wrote: post #232
gyomu wrote: post #121

Earlier quoted context omitted.

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

I work on this typed lua language in lua, and sometimes use llms to help fix internal analyzer stuff, which works 30% of the time for complex, and sometimes not at all, but helps me find a solution in the end. However when I ask an llm to generate my typed lua code, with examples and all, on how the syntax is supposed to be, it mostly gets it wrong. my syntax for tables/objects is: local x: {foo = boolean} but an llm…

I've had success in the past with getting it to write YueScript/Moonscript (which is not a very large part of its training data) by pointing it to the root URL for the language docs and thus making that part of the context.

If your typed version of Lua has a syntax checker, you could also have it try to use that first on any code it's generated

Re: NanoChat – The best ChatGPT that $100 can buy

#289

Post by hirako2000 »

kasey_junk wrote: post #245
troupo wrote: post #227

Earlier quoted context omitted.

Indeed. And only Karpathy is entitled to say that AI tools produce wrong code for him. And he's only entitled to say it for this project only. If anyone else says this, "the skepticism is exhausting", and their experience is completely irrelevant.

Go look at the comments on HN whenever someone posts about their AI coding workflow. It will be littered with negative comments that either imply or outright say that the poster is either shilling, ignorant or working only on toy examples. The grievance attitude seems to exist in both directions and is actually what is exhausting.

posts about yet another ai workflow, typically presented with hyperbole is exhausting. The backfires are rather appeasing, entertaining at the least.

Re: NanoChat – The best ChatGPT that $100 can buy

#290

Post by kouteiheika »

varunneal wrote: post #102
montebicyclelo wrote: post #91

> nanochat is also inspired by modded-nanoGPT Nice synergy here, the lineage is: Karpathy's nano-GPT -> Keller Jordan's modded-nanoGPT (a speedrun of training nanoGPT) -> NanoChat modded-nanoGPT [1] is a great project, well worth checking out, it's all about massively speeding up the training of a small GPT model. Notably it uses the author's Muon optimizer [2], rather than AdamW, (for the linear layers). [1] https:/…

Muon was invented by Keller Jordan (and then optimized by others) for the sake of this speedrunning competition. Even though it was invented less than a year ago, it has already been widely adopted as SOTA for model training

The most exciting thing about Muon for me is that it requires half the state of Adam while having either equivalent or better performance. That's amazing if you are VRAM limited! And just like Adam, you can also quantize it. I can get it to work relatively well as low as 4-bit, which essentially cuts down the memory requirements from full 32-bit Adam by a factor of 16x! (And by a factor of 4x vs 8-bit Adam).
Post reply on HN