Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

311–320 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#311
post #121

Earlier quoted context omitted.

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

Everything is unless your app is a React todolist or leatcode questions.

I've had some success with a multi-threaded software defined radio (SDR) app in Rust that does signal processing. It's been useful for trying something out that's beyond my experience. Which isn't to say it's been easy. It's been a learning experience to figure out how to work around Claude's limitations.

Re: NanoChat – The best ChatGPT that $100 can buy

#312

So could I in practice train it on all my psychology books, materials, reports, case study and research papers and then run it on demand on a 1xH100 node - https://getdeploying.com/reference/cloud-gpu/nvidia-h100 whenever I have a specialised question?

You could! But just like others have mentioned, the performance would be negligible. If you really wanted to see more of a performance boost by pretraining you could try to create a bigger chunk of data to train off of. This would be done by either creating synthetic data off of your material, or finding adjacent information to your material. Here's a good paper about it: https://arxiv.org/abs/2409.07431>

Re: NanoChat – The best ChatGPT that $100 can buy

#313
post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

The llama.cpp maintainers working on supporting Qwen3-next are also not enthused by LLM output. They had to go over everything and fix it up.

https://github.com/ggml-org/llama.cpp/pull/16095#issuecommen...

Re: NanoChat – The best ChatGPT that $100 can buy

#314
post #258
post #237

Earlier quoted context omitted.

Yep. I find the hype around AI to be wildly overblown, but that doesn’t mean that what it can do right now isn’t interesting & useful. If you told me a decade ago that I could have a fuzzy search engine on my desktop that I could use to vaguely describe some program that I needed & it would go out into the universe of publicly available source code & return something that looks as close to the thing I’ve asked for as…

Why do you think what you describe being excited about does not warrant the current level of AI hype? I agree with your assessment and sometimes I think there is too much cynicism and not enough excitement.

Because to justify the current hype and spending, these companies have to have a product that will generate trillions of dollars and create mass unemployment. Which they don't have.

Re: NanoChat – The best ChatGPT that $100 can buy

#315
post #244

Earlier quoted context omitted.

For anyone curious this is the error when running uv sync on macos, > uv sync Resolved 88 packages in 3ms error: Distribution `torch==2.8.0+cu128 @ registry+ https://download.pytorch.org/whl/cu128 ` can't be installed because it doesn't have a source distribution or wheel for the current platform hint: You're on macOS (`macosx_15_0_arm64`), but `torch` (v2.8.0+cu128) only has wheels for the following platforms: `many…

Yeah, that's because cuda on a mac isn't a thing - it could be swapped to the normal torch package but you'd have to do some code patching to make sure it's running on mps, even then some of the code may need rewriting/patching if there's no mps version of the cuda kernals.

Isn't there a common PyTorch API interface that could chose OS/hardware specific backend automatically? Or this project is hard coding cuda variant of PyTorch as a requirement?

Re: NanoChat – The best ChatGPT that $100 can buy

#316

Earlier quoted context omitted.

This makes sense, right? It's a relatively novel thing to be writing. I don't find it to be a damning remark like other comments here seem to be concluding. If anything, the fact that Karpathy reached towards Claude/Codex in an attempt to gain value is indicative that, in previous coding efforts, those tools were helpful to him.

It's not _that_ far off distribution though. The math and concepts are well understood.

That's not really how LLM work though. It's fundamentally next word prediction, based on statistics of the context. Reordering ideas (which can drastically change the outcome) can result in a statistically rare context. The silly failures of simple riddles [1], and the like, demonstrate this well.

The riddle issue is putting trivial ideas together, but combined them in a statistically rare way, giving low quality output that tends towards the statistically significant answer, even if it's incorrect. The same things happens with coding, when combining well known things together in uncommon ways.

Worse (as with the riddle problem), nearby concepts that have strong statistics are going to act like attractors, with the LLM always trending towards those, removing and rewriting bits of code to better accommodate those, even if they're the opposite of what you want. I have this happen all the time in my somewhat obscure work. It'll rewrite key maths in my code to be the statistically significant textbook example, which is not what I need. I'll fix it manually or point it out, and a few changes later, it'll rewrite it again. A reasonable way around this is to "pin" the concept with a very strongly worded negative comment, like "DO NOT USE DOT PRODUCT. THIS IS NOT VECTOR PROJECTION. DO NOT modify the next line of code.".

[1] Claude 4.1 Opus:

> Prompt: "A son and his mother are in a car accident. They’re both rushed to the hospital. The doctor is about to operate on the son. But, before he does, he looks down and says "This is my son! I can’t operate on this boy!". How can this be?"

> Response: "The doctor is the boy's father.

This is a classic riddle that plays on assumptions people might make about gender and professions. The boy has two parents - his mother (who was in the car with him) and his father (the doctor at the hospital). The riddle works because some people automatically assume the doctor must be male, forgetting that the boy's father could be the doctor since his mother was already mentioned as being in the accident."

Another, with output that doesn't match the goal, statistically attracted to the riddle:

> Prompt: "A man, a sheep, and a wolf are on one side of the river, with a boat that can only hold two. How can the man safely get the boat to the other side of the river, without the sheep being eaten?"

Re: NanoChat – The best ChatGPT that $100 can buy

#317

Earlier quoted context omitted.

HN's cynicism towards AI coding (and everything else ever) is exhausting. Karpathy would probably cringe reading this.

First, it's not cynicism but a more realistic approach than just following SV marketing blindly, and second, it's not "everything else", just GenAI, NFTs/ICOs/Web3, "Metaverse" (or Zucks interpretation of it), delf-driving cars ready today, maybe a bit Theranos.

The thing is cryptocurrency and metaverse stuff was obvious bullshit from day one while even GPT-3 was clearly a marvel from day one. It's a false pattern match.

Re: NanoChat – The best ChatGPT that $100 can buy

#318

Earlier quoted context omitted.

HN's cynicism towards AI coding (and everything else ever) is exhausting. Karpathy would probably cringe reading this.

I mean Karpathy himself wrote that he could not use the AI tools for the project, so he had to handwrite most of it. I wonder why.

That's exactly why I said he would cringe at it. Seeing someone look at him saying "it's not able to make a good GPT clone" and going "yeah it's useless for anything besides React todo list demos" would definitely evoke some kind of reaction. He understands AI coding agents are neither geniuses nor worthless CRUD monkeys.

Re: NanoChat – The best ChatGPT that $100 can buy

#320

Earlier quoted context omitted.

I mean Karpathy himself wrote that he could not use the AI tools for the project, so he had to handwrite most of it. I wonder why.

That's exactly why I said he would cringe at it. Seeing someone look at him saying "it's not able to make a good GPT clone" and going "yeah it's useless for anything besides React todo list demos" would definitely evoke some kind of reaction. He understands AI coding agents are neither geniuses nor worthless CRUD monkeys.

Hm, interesting point. So if he and other GenAI hotshots understand that, why do they keep seeling the tools as precisely no less than geniuses? Often with a bit of fear mongering about all the jobs that would be lost soon etc.?
Post reply on HN