Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

231–240 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#231

Post by Arisaka1 »

notatoad wrote: post #151
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

Before copilot what I'd do is diagnose and identify the feature that resembles the one that I'm about to build, and then I'd copy the files over before I start tweaking.

Boilerplate generation was never, ever the bottleneck.

Re: NanoChat – The best ChatGPT that $100 can buy

#232

Post by CapsAdmin »

gyomu wrote: post #121
tehnub wrote: post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

I work on this typed lua language in lua, and sometimes use llms to help fix internal analyzer stuff, which works 30% of the time for complex, and sometimes not at all, but helps me find a solution in the end.

However when I ask an llm to generate my typed lua code, with examples and all, on how the syntax is supposed to be, it mostly gets it wrong.

my syntax for tables/objects is: local x: {foo = boolean}

but an llm will most likely gloss over this and always use : instead of = local x: {foo: boolean}

Re: NanoChat – The best ChatGPT that $100 can buy

#233

Post by ComplexSystems »

varunneal wrote: post #102
montebicyclelo wrote: post #91

> nanochat is also inspired by modded-nanoGPT Nice synergy here, the lineage is: Karpathy's nano-GPT -> Keller Jordan's modded-nanoGPT (a speedrun of training nanoGPT) -> NanoChat modded-nanoGPT [1] is a great project, well worth checking out, it's all about massively speeding up the training of a small GPT model. Notably it uses the author's Muon optimizer [2], rather than AdamW, (for the linear layers). [1] https:/…

Muon was invented by Keller Jordan (and then optimized by others) for the sake of this speedrunning competition. Even though it was invented less than a year ago, it has already been widely adopted as SOTA for model training

I haven't heard of this before. Has Muon dethroned Adam and AdamW as the standard general purpose optimizer for deep learning?

Re: NanoChat – The best ChatGPT that $100 can buy

#234

Post by Yizahi »

wordpad wrote: post #222
Yizahi wrote: post #66

Earlier quoted context omitted.

I would adjust your formula to the: number of people you help x how much you help them x number of people you harm x how much you harm them For example - harming a little bit all content creators of the world, by stealing their work without compensation or permission. How much does that cost globally every year after year? How do we even quantify long term consequences of that? Stuff like that.

If you consider the cost of hiring a human professional to over using multimodal AI for something, its very realize literally thousands of dollars of value per chat. Multiply that by many billions of chats per day. Lawyers and other professionals charge a lot. So do artists, especially when you want to do a million revisions. LLMs hand it out for free, making many knowledge and art professions affordable and accessib…

I wonder what people will do, when they will realize that LLM lawyers produce insufficient results, but "suddenly" all cheap bottom rung lawyers are gone and switched professions.

As for the LLM "creative" content, have you seen it or read it? Well, same problem. After you will need a quality content, good luck finding some cheap creator. Pay full price for an experienced one and likely wait.

PS: I don't doubt that LLMs are here to stay. They will se a lot of usage and pervade all industries. It's just that future will be pretty shit. Talking on phone with LLMs, reading LLM slop, seeing LLM lop everywhere, receiving generated emails and using LLMs to reverse parse them to search for an actual content, major economy downturn, rapidly slowing salary growth (not that it was big before), etc.

Re: NanoChat – The best ChatGPT that $100 can buy

#235

Post by ggsp »

vunderba wrote: post #171
rootusrootus wrote: post #135

Earlier quoted context omitted.

That is a good thing to hear from someone as reputable as Karpathy. The folks who think we're on the cusp of AGI may want to temper their expectations a bit. I do love Claude Code, because one thing I periodically need to do is write some web code, which is not my favorite type of coding but happens to have incredibly good coverage in the training data. Claude is a much better web developer than I am. But for digging…

100%. I find the "LLMs are completely useless" and the "LLMs will usher in a new era of messianic programming" camps to be rather reductive. I've already built some pretty large projects [1] with the assistance of agentic tooling like Claude Code. When it comes to the more squirrely algorithms and logic, they can fall down pretty hard. But as somebody who is just dreadful at UI/UX, having it hammer out all the web de…

Hey, thank you for making this—I really enjoyed playing it and it feels like it fits the mental-reward-between-work-tasks need. It did spin up my M1's fans after a few minutes which is a rather rare occurrence, but I'm guessing that's par for the course when you're working with a bunch of video on canvas. Either way, hope I remember it the next time I'm looking for a puzzle to solve while I take a break :)

Re: NanoChat – The best ChatGPT that $100 can buy

#236

Post by eternauta3k »

squeaky-clean wrote: post #188
goalieca wrote: post #176

Earlier quoted context omitted.

Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.

Wysiwyg kind of fell apart once we had to stop assuming everyone had an 800x600 or 1024x768 screen, because what you saw was no longer what others got.

Not a big issue with QT layouts (still have to test the result though)

Re: NanoChat – The best ChatGPT that $100 can buy

#237

Post by pja »

notatoad wrote: post #151
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

Yep. I find the hype around AI to be wildly overblown, but that doesn’t mean that what it can do right now isn’t interesting & useful.

If you told me a decade ago that I could have a fuzzy search engine on my desktop that I could use to vaguely describe some program that I needed & it would go out into the universe of publicly available source code & return something that looks as close to the thing I’ve asked for as it can find then that would have been mindblowing. Suddenly I have (slightly lossy) access to all the code ever written, if I can describe it.

Same for every other field of human endeavour! Who cares if AI can “think“ or “do new things”? What it can do is amazing & sometimes extremely powerful. (Sometimes not, but that’s the joy of new technology!)

Re: NanoChat – The best ChatGPT that $100 can buy

#238

Post by yorwba »

danielmarkbruce wrote: post #179
nl wrote: post #174

Earlier quoted context omitted.

I suspect the OP's issues might be mostly related to the ROCM version of PyTorch. AMD still can't get this right.

Probably - but the answer is to avoid ROCM, not pytorch.

Avoiding ROCm means buying a new Nvidia GPU. Some people would like to keep using the hardware they already have.

Re: NanoChat – The best ChatGPT that $100 can buy

#239

Post by random_cynic »

gyomu wrote: post #121
tehnub wrote: post #107

Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

[dead]

Re: NanoChat – The best ChatGPT that $100 can buy

#240

Post by andai »

vessenes wrote: post #106
echelon wrote: post #105

Earlier quoted context omitted.

8xH100 is pretty wild for a single inference node. Is this what production frontier LLMs are running inference with, or do they consume even more VRAM/compute? At ~$8/hr, assuming a request takes 5 seconds to fulfill, you can service roughly 700ish requests. About $0.01 per request. Is my math wrong?

This is the spec for a training node. The inference requires 80GB of VRAM, so significantly less compute.

The default model is ~0.5B params right?
Post reply on HN