Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NanoChat – The best ChatGPT that $100 can buy

github.com

Post reply on HN
Print view View original

291–300 of 326 posts

Re: NanoChat – The best ChatGPT that $100 can buy

#291

Post by philipallstar »

squeaky-clean wrote: post #188
goalieca wrote: post #176

Earlier quoted context omitted.

Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.

Wysiwyg kind of fell apart once we had to stop assuming everyone had an 800x600 or 1024x768 screen, because what you saw was no longer what others got.

I can imagine adding breakpoints to a wysiwyg editor being not terribly difficult. They decouple presentation from logic pretty well.

Re: NanoChat – The best ChatGPT that $100 can buy

#292

Post by dahcryn »

SchemaLoad wrote: post #139
gyomu wrote: post #121

Earlier quoted context omitted.

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

Everything is unless your app is a React todolist or leatcode questions.

simple CRUD, is as common in many many business applications or backend portals, are a good fit for AI assistance imho. And fix some designs here and there, where you can't be bothered to keep track of the latest JS/CSS framework

Re: NanoChat – The best ChatGPT that $100 can buy

#294

Post by spyder »

ComplexSystems wrote: post #233
varunneal wrote: post #102

Earlier quoted context omitted.

Muon was invented by Keller Jordan (and then optimized by others) for the sake of this speedrunning competition. Even though it was invented less than a year ago, it has already been widely adopted as SOTA for model training

I haven't heard of this before. Has Muon dethroned Adam and AdamW as the standard general purpose optimizer for deep learning?

It's for hidden layers and not for every parameter: From Keller's Muon github page:

"Muon is an optimizer for the hidden weights of a neural network. Other parameters, such as embeddings, classifier heads, and hidden gains/biases should be optimized using standard AdamW."

And I just looked into this nanochat repo and it's also how it's used here.

https://github.com/karpathy/nanochat/blob/dd6ff9a1cc23b38ce6...

Re: NanoChat – The best ChatGPT that $100 can buy

#295

Post by Mabusto »

notatoad wrote: post #151
SchemaLoad wrote: post #139

Earlier quoted context omitted.

Everything is unless your app is a React todolist or leatcode questions.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

I've been using AI like this as well. The code-complete / 'randomly pop up a block of code while typing' feature was cool for a bit but soon became annoying. I just use it to generate a block of boilerplate code or to ask it questions, I do 90% of the 'typing the code' bit myself, but that's not where most programmers time is spent.

Re: NanoChat – The best ChatGPT that $100 can buy

#296

Post by llbbdd »

bgwalter wrote: post #201
simonw wrote: post #115

Earlier quoted context omitted.

Yes, that number is according to OpenAI. They released that 800m number at DevDay last week. The most recent leaked annualized revenue rate was $12bn/year. They're spending a lot more than that but convincing customers to hand over $12bn is still a very strong indicator of demand. https://www.theinformation.com/articles/openai-hits-12-billi...

Part of that comes from Microsoft API deals. Part of that will most certainly come because the vast network of companies buy subscriptions to help "Open" "AI" [1]. Given the rest of circular deals, I'd also scrutinize if it applies to the revenue. The entanglement with the Microsoft investments and the fact that "Open" "AI" is a private company makes that difficult to research. [1] In a U.S. startup, I went through t…

When people take the time to virtue signal "Open" "AI" or the annoyingly common M$ on here I wonder often why they are wasting their precious time on earth doing that

Re: NanoChat – The best ChatGPT that $100 can buy

#297

Post by notatoad »

Mabusto wrote: post #295
notatoad wrote: post #151

Earlier quoted context omitted.

people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.

I've been using AI like this as well. The code-complete / 'randomly pop up a block of code while typing' feature was cool for a bit but soon became annoying. I just use it to generate a block of boilerplate code or to ask it questions, I do 90% of the 'typing the code' bit myself, but that's not where most programmers time is spent.

i'm not sure when you tried it, but if you've had copilot disabled it might be worth giving it another go. in my totally anecdotal experience, over the last few months it's gotten significantly better at shutting up when it can't provide anything useful.

Re: NanoChat – The best ChatGPT that $100 can buy

#298

Post by notatoad »

pmarreck wrote: post #287
notatoad wrote: post #281

Earlier quoted context omitted.

the current level of AI hype amongst a lot of people, but especially investors and bosses, is that you can already give an AI a simple prompt and get it to spit out a fully functional, user-ready application for you. and we're so incredibly far off that. the things that AI is able to do are incredible, but hype levels are just totally detached from reality.

> is that you can already give an AI a simple prompt and get it to spit out a fully functional, user-ready application for you. But it can already do that. Isn't that the whole "one-shotting" thing? The problem is, of course, that it won't be optimized , maintainable or have anyone responsible you can point to if something with it goes wrong. It almost certainly (unless you carefully prompted it to) won't have a test…

from what i can tell, the one-shot thing only works on youtube.

you might produce something that looks usable at first, but the actual application functionality will be significantly broken in most ways. it maybe works enough to do a demo for your video, but it won't work enough to actually distribute to end-users. and of course, as you say, it's not testable or maintainable in any way, so fixing what's broken is a bigger project than just writing it properly in the first place.

Re: NanoChat – The best ChatGPT that $100 can buy

#299

Post by varunneal »

tbalsam wrote: post #104
varunneal wrote: post #102

Earlier quoted context omitted.

Muon was invented by Keller Jordan (and then optimized by others) for the sake of this speedrunning competition. Even though it was invented less than a year ago, it has already been widely adopted as SOTA for model training

This is the common belief but not quite correct! The Muon update was proposed by Bernstein as the result of a theoretical paper suggesting concrete realizations of the theory, and Keller implemented it and added practical things to get it to work well (input/output AdamW, aggressive coefficients, post-Nesterov, etc). Both share equal credit I feel (also, the paper's co-authors!), both put in a lot of hard work for it…

I think it's good to bring up Bernstein & Newhouse as well as Yuchen Jin, Jiacheng You and the other speedrunners who helped iterate on Muon. But I think it's very fair to call Keller Jordan the main author of Muon of its current form. I'm also in the speedrunning community though maybe not as long as you have

Re: NanoChat – The best ChatGPT that $100 can buy

#300

Post by lukev »

SchemaLoad wrote: post #139
gyomu wrote: post #121

Earlier quoted context omitted.

> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!

Everything is unless your app is a React todolist or leatcode questions.

Generative AI for coding isn't your new junior programmer, it's the next generation of app framework.
Post reply on HN