Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

201–210 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#201

Post by bgwalter »

simonw wrote: post #115
infinitezest wrote: post #113

Earlier quoted context omitted.

1. According to who? Open AI? 2. Its current state is "basically free and containing no ads". I don't think this will remain true given that, as far as I know, the product is very much not making money.

Yes, that number is according to OpenAI. They released that 800m number at DevDay last week. The most recent leaked annualized revenue rate was $12bn/year. They're spending a lot more than that but convincing customers to hand over $12bn is still a very strong indicator of demand. https://www.theinformation.com/articles/openai-hits-12-billi...

Part of that comes from Microsoft API deals. Part of that will most certainly come because the vast network of companies buy subscriptions to help "Open" "AI" [1].

Given the rest of circular deals, I'd also scrutinize if it applies to the revenue. The entanglement with the Microsoft investments and the fact that "Open" "AI" is a private company makes that difficult to research.

[1] In a U.S. startup, I went through three CEOs and three HR apps, which mysteriously had to change for no reason but to accommodate the new CEO's friends and their startups.

Re: NanoChat – The best ChatGPT that $100 can buy

#202

Post by SeanAnderson »

astrange wrote: post #186
SeanAnderson wrote: post #71

Earlier quoted context omitted.

would love more details on this. this is exactly the type of project I'd like to dabble in to get more up to speed.

People have been doing this for a while. https://x.com/roborosewater https://bsky.app/profile/roborosewaterm.bsky.social You can see the invention of RLHF/ChatGPT here because text generation suddenly became much more coherent and also much less interesting. You have to go back to older tech for surrealism because nobody will let you see the good stuff (the base models).

I guess I was much more interested in being able to work with an LLM to create good, synergistic Commander decks and less interested in generating custom Magic cards.

I'm sure I can dig up info on how to do this and piece it together, but I thought OP might have a guide specifically for it.

Re: NanoChat – The best ChatGPT that $100 can buy

#203

Post by trial3 »

meowface wrote: post #199
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

HN's cynicism towards AI coding (and everything else ever) is exhausting. Karpathy would probably cringe reading this.

okay but he literally does have a bridge that non-deterministically might take you to the wrong place to sell you

Re: NanoChat – The best ChatGPT that $100 can buy

#204

Post by handfuloflight »

meowface wrote: post #200
vunderba wrote: post #171

Earlier quoted context omitted.

100%. I find the "LLMs are completely useless" and the "LLMs will usher in a new era of messianic programming" camps to be rather reductive. I've already built some pretty large projects [1] with the assistance of agentic tooling like Claude Code. When it comes to the more squirrely algorithms and logic, they can fall down pretty hard. But as somebody who is just dreadful at UI/UX, having it hammer out all the web de…

>and the "LLMs will usher in a new era of messianic programming" camps Well, this one might still be borne out. It's just silly to think it's the case right now. Check in again in 10 years and it may be a very different story. Maybe even in 5 years.

What do we build now to reap the coming of the messianic era?

Re: NanoChat – The best ChatGPT that $100 can buy

#205

Post by Ono-Sendai »

dinkblam wrote: post #57

from their promotional material: >> Why is the sky blue? > The sky is blue due to an optical illusion called the Rayleigh Scattering Rayleigh Scattering is not an illusion but an effect. > […] particles are made up of tiny blue and violet particles that cause the light to bend in a particular way. ugh. no, there are no "tiny blue" particles in the sky.

not sure why you are being downvoted. That 'explanation' of Rayleigh scattering is just wrong.

Re: NanoChat – The best ChatGPT that $100 can buy

#206

Post by jmspring »

simonw wrote: post #198
jmspring wrote: post #197

8XH100 nodes start at ~$450ish/day. Not sure about the $100 part. I need to dig into the post.

The quoted $100 price is for 4 hours at $24/hour. 450 / 24 = $18.75 so your numbers roughly match that.

Thanks. Working on platforms - days are more interesting than hours.

Re: NanoChat – The best ChatGPT that $100 can buy

#207

Post by mcmoor »

goalieca wrote: post #176
notatoad wrote: post #151

Earlier quoted context omitted.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.

I still miss my days of programming Visual Basic 6. Nothing since then ever compares.

Re: NanoChat – The best ChatGPT that $100 can buy

#208

Post by simonw »

Ono-Sendai wrote: post #205
dinkblam wrote: post #57

from their promotional material: >> Why is the sky blue? > The sky is blue due to an optical illusion called the Rayleigh Scattering Rayleigh Scattering is not an illusion but an effect. > […] particles are made up of tiny blue and violet particles that cause the light to bend in a particular way. ugh. no, there are no "tiny blue" particles in the sky.

not sure why you are being downvoted. That 'explanation' of Rayleigh scattering is just wrong.

Because that explanation is expected to be wrong. This is a partially trained tiny model, Andrej shared that obviously incorrect explanation to emphasize that the model is not trustworthy or useful at that stage.

Re: NanoChat – The best ChatGPT that $100 can buy

#210

Post by JonathanFly »

Havoc wrote: post #25

>If your GPU(s) have less than 80GB, you'll have to tune some of the hyperparameters or you will OOM / run out of VRAM. Look for --device_batch_size in the scripts and reduce it until things fit. E.g. from 32 (default) to 16, 8, 4, 2, or even 1. That sounds like it could run on a 24gb GPU. Batch size of 8 would imply 20gb mem, no? ...presumably just takes forever

> Batch size of 8 would imply 20gb mem, no?

I'm running it now and I had to go down to 4 instead of 8, and that 4 is using around 22-23GB of GPU memory. Not sure if something is wrong or if batch is only scaling part of the memory requirements. (Edit: I restarted running the training script directly instead of torch run, and 8 still doesn't fit, but 4 is now using 16-17 instead.)

On my 4090 the tok/sec is 523, which is 1/2000 of the 1,000,000 tok/sec of the 8 80GB H100s. That feels too slow so maybe something is wrong. The 4090 is about 1/3 of the raw compute. I'm sure there's other losses from less batching but even if it were 1/10ths as fast, I'd expected something more like 1,000,000 / 10 / 8 so at least 10,000 tok/sec.

Post reply on HN