[dead]
Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
121–126 of 126 posts
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#122> Scale Autoresearch on your own GPU cluster
Who's got a cluster of H100s and H200s just lying around?
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#123> Scale Autoresearch on your own GPU cluster Who's got a cluster of H100s and H200s just lying around?
The entire discussion is about rented cloud clusters, so I guess anyone with the money to rent one?
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#124Earlier quoted context omitted.
Wrong and short-sighted take given that the LLM explores serially learning along the way, and can tool use and change code arbitrarily. It seems to currently default to something resembling hyperparameter tuning in absence of more specific instructions. I briefly considered calling the project “autotune” at first but I think “autoresearch” will prove to be the significantly more appropriate name.
I think we need to separate theory from practice. In theory, it can edit the training loop and come up with novel techniques. That is interesting. In practice, the vast majority of the changes that auto research actually made would have been found much faster with BO if properly parameterized. You do not need an LLM to find a better batch size or learning rate.
[flagged]
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#125Earlier quoted context omitted.
Fame doesn’t enter it - the point is Karpathy has about as strong a claim as anyone to having “actually used LLMs for non trivial tasks”.
That is not the case at all, considering that he himself started using and tweeting about llms for coding fairly recently. He's probably less experienced in that area than most people who started using claude cli last year. He is a researcher who understands neural networks and their architectures exceptionally well. That is all.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#126[dead]