Earlier quoted context omitted.
Now to wait for Sonnet 5 and GPT-6, and ask them to build that, and see what they come up with.
Why would you expect an improvement?
because they'll be trained on karpathy's implementation
NanoChat – The best ChatGPT that $100 can buy
261–270 of 326 posts
Re: NanoChat – The best ChatGPT that $100 can buy
#262Earlier quoted context omitted.
HN's cynicism towards AI coding (and everything else ever) is exhausting. Karpathy would probably cringe reading this.
First, it's not cynicism but a more realistic approach than just following SV marketing blindly, and second, it's not "everything else", just GenAI, NFTs/ICOs/Web3, "Metaverse" (or Zucks interpretation of it), delf-driving cars ready today, maybe a bit Theranos.
I’ve recently written a message queue database connector in Go using Claude Code, checkpointing, recovery, all that stuff built in.
I’d say it made me around 2x as productive.
I don’t think the cynicism of HN is justified, but I think what people forget is that it takes several months of really investing a lot of time into learning how to use AI well. If I see some of the prompts people give, and expect it to work, yeah no wonder that only works for React-like apps.
Re: NanoChat – The best ChatGPT that $100 can buy
#263I am building a product similar to DataGPT https://datagpt.com/ and Julius.ai - will this help in that?
Re: NanoChat – The best ChatGPT that $100 can buy
#264Earlier quoted context omitted.
people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.
Yep. I find the hype around AI to be wildly overblown, but that doesn’t mean that what it can do right now isn’t interesting & useful. If you told me a decade ago that I could have a fuzzy search engine on my desktop that I could use to vaguely describe some program that I needed & it would go out into the universe of publicly available source code & return something that looks as close to the thing I’ve asked for as…
They go beyond merely "return something that looks as close to the thing I’ve asked for as it can find". Eg: Say we asked for "A todo app that has 4 buttons on the right that each play a different animal sound effect for no good reason and also you can spin a wheel and pick a random task to do". That isn't something that already exists, so in order to build that, the LLM has to break that down, look for appropriate libraries and source and decide on a framework to use, and then glue those pieces together cohesively. That didn't come from a singular repo off GitHub. The machine had to write new code in order to fulfill my request. Yeah, some if it existed in the training data somewhere, but not arranged exactly like that. The LLM had to do something in order to glue those together in that way.
Some people can't see past how the trick is done (take training data and do a bunch of math/statistics on it), but the fact that LLMs are able to build the thing is in-and-of-itself interesting and useful (and fun!).
Re: NanoChat – The best ChatGPT that $100 can buy
#265Earlier quoted context omitted.
First, it's not cynicism but a more realistic approach than just following SV marketing blindly, and second, it's not "everything else", just GenAI, NFTs/ICOs/Web3, "Metaverse" (or Zucks interpretation of it), delf-driving cars ready today, maybe a bit Theranos.
I’ve recently written a message queue database connector in Go using Claude Code, checkpointing, recovery, all that stuff built in. I’d say it made me around 2x as productive. I don’t think the cynicism of HN is justified, but I think what people forget is that it takes several months of really investing a lot of time into learning how to use AI well. If I see some of the prompts people give, and expect it to work, y…
I asked AI to create a basic autoencoder based deep learning architecture for classifying time series data. This AI is a boon.
Re: NanoChat – The best ChatGPT that $100 can buy
#266Earlier quoted context omitted.
Are you using a coding agent or just an llm chat interface? Do you have a linter or compiler that will catch the misuse that you’ve hooked up to the agent?
I've dabbled with claude code in this particular project, but not much. My short experience with it is that it's slow, costly and goes off the rails easily. I prefer to work with more isolated parts of the code. But again, I don't really know all that much about agents. One thing I wanted to do on my project is reorganize all the tests, which sounds like an agent job. But I'd imagine I need to define some hard progra…
Agents aren’t magic. They are loops with tool calls in them that help keep agents on track. And most of the agent systems have some manner of hook that you can put your own tools in to enforce things like types and styles.
I’ve had good experiences writing small scripts and linters to enforce things that agents get wrong frequently. What’s nice about those is that the agents are very good at writing them and they are easy to verify. Plus they are valuable for new humans devs as well.
Re: NanoChat – The best ChatGPT that $100 can buy
#267Earlier quoted context omitted.
Muon was invented by Keller Jordan (and then optimized by others) for the sake of this speedrunning competition. Even though it was invented less than a year ago, it has already been widely adopted as SOTA for model training
sharing some useful resrources for learning Muon (since I'm also just catching up on it) - https://x.com/leloykun/status/1846842883967692926 - https://www.yacinemahdid.com/p/muon-optimizer-explained-to-a...
This Simple Optimizer Is Revolutionizing How We Train AI [Muon]
https://www.youtube.com/watch?v=bO5nvE289ec
I found the above video as a good introduction.
Re: NanoChat – The best ChatGPT that $100 can buy
#268Earlier quoted context omitted.
people say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.
Back in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.
[deleted]
Re: NanoChat – The best ChatGPT that $100 can buy
#269Earlier quoted context omitted.
Go look at the comments on HN whenever someone posts about their AI coding workflow. It will be littered with negative comments that either imply or outright say that the poster is either shilling, ignorant or working only on toy examples. The grievance attitude seems to exist in both directions and is actually what is exhausting.
> It will be littered with negative comments that either imply or outright say that the poster is either shilling, ignorant or working only on toy examples. And they would be often be right. Coupled with the fact that most of the glowing "omg I only code with AI" posts don't even try to show what code or products they are working on. And yes, the absolute vast majority of people who are skeptical are skeptical precis…
Just so we are clear, you are upset by people dismissing your experience gained skepticism but have no problem dismissing every positive comment as a shill, ignorant or simple?
You don’t see any dissonance in that? It’s only the positive people that are exhausting?
Re: NanoChat – The best ChatGPT that $100 can buy
#270I created this PR to make it easier for folks to train and serve it on any cloud (or their own K8s): https://github.com/karpathy/nanochat/pull/18