[dead]
Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
51–60 of 126 posts
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#52[dead]
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#53Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#54Earlier quoted context omitted.
Out of curiosity, what sort of things have you seen it do that better fit 'autoresearch' than 'autotune' thus far? Optimizations it made that wouldn't be been surfaced by an autotune system, I suppose.
I see this critique about autoresearch online often, but I think it’s misplaced. Here’s a use case that may illuminate the difference, from my own work at Nvidia. Im currently training some large sparse autoencoders, and there are issues with dead latents. Several solutions exit to help here, such as auxk, which I can certainly include and tune the relevant params as you describe. However, I have several other ideas…
It wasn't meant as a critique, I'm legitimately interested in knowing more about where it can push boundaries and where it struggles. I agree that in general it's a truism that "Claude can try a number of new ideas" etc., but the question remains as to where in particular it actually takes advantage of this to push the envelope in a way other tools don't -- since that informs when it makes sense to use something like this.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#55Earlier quoted context omitted.
Why surely? Have you never seen an LLM try something new?
I honestly don't think I have. In this case, using a cheap(er) signal or heuristic as an initial filter before spending more resources on cases that pass the filter is a pattern that shows up all over the place, and LLMs are good at picking up on patterns like that and generalizing them. AFAICT.
I'm not sure how people say this so confidently. I have a rather esoteric haskell library that I've written and published for years. ChatGPT and Claude both know about it and frequently help me improve it, and propose completely novel approaches. I'm really not sure how people are so confident that they can't think of anything new. This seems like wishful confirmation bias.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#56Earlier quoted context omitted.
Do you realise who you’re replying to?
Why should we care that he’s famous?
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#57Earlier quoted context omitted.
You said "surely", and I asked: > Why surely? Have you never seen an LLM try something new? I'm afraid I can't make it any simpler than this. And I still don't know the answer to how you're so sure. To me there's several explanations, and it seems to you there's only one. I'm pretty happy with my communication style.
Seems to me the commenter was asking: what observations led us to conclude that original affirmative statement that “the AI did this entirely on its own”. Given that this is a common technique and not a novel invention, it’s probably present in the training set. The “surely” reads like it’s referring to the presence of that information in the training set. But your response casts it as saying “surely the AI has not i…
There is no burden of proof on me, because I'm not asserting that AI has invented something on its own. I haven't told you what my view is or whether I ever have a view.
The problem with the reasoning of the person I was responding to is that it's assuming "if X is in the training set and LLM outputs X, then it did so because X is in the training set". That does not follow. Conceivably it's possible that X is in the training set and LLM outputs X, but if X hadn't been in the training set the LLM also would've output X.
Lets look at that phrase again:
> Why do we think this emerged “on its own”? Surely this technique has been discussed in research papers that are in the training set.
This phrase implies "if X was in the training set, then LLM couldn't have come up with X on its own". This is false. In fact, my claim that the implication is false is testable, in the following manner: Have two training sets, T and T'. In T, X is present. In T' you've removed X but left X-adjacent things. Train LLM A on T and A' on T'. Find a prompt that requires that A outputs X. If on the same prompt A' also outputs X, that's an example of my claim. To repeat, my claim is "it's possible that X is in the training set and LLM outputs X, but if X hadn't been in the training set the LLM also would've output X."
In fact, I've just realized I even have a method for constructing (T, T') that guarantees what I've described. Not sure if it's worth a paper on its own though.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#58Earlier quoted context omitted.
Seems to me the commenter was asking: what observations led us to conclude that original affirmative statement that “the AI did this entirely on its own”. Given that this is a common technique and not a novel invention, it’s probably present in the training set. The “surely” reads like it’s referring to the presence of that information in the training set. But your response casts it as saying “surely the AI has not i…
There is no burden of proof on me, because I'm not asserting that AI has invented something on its own. I haven't told you what my view is or whether I ever have a view. The problem with the reasoning of the person I was responding to is that it's assuming "if X is in the training set and LLM outputs X, then it did so because X is in the training set". That does not follow. Conceivably it's possible that X is in the…
Your pure logic is probably right; I do not have the time or interest to dissect it.
But you’re missing the context and implication: “doing new stuff” is the major achievement we’re looking for next from LLMs. Seeing something that is “new” and is not in the training set is interesting in a way that something contained in the training set is not.
We cannot introspect LLMs meaningfully yet, so the difference between “came up with myself and it’s in the training set incidentally” and “applied a concept in the training set” is not meaningful.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#59Earlier quoted context omitted.
Why should we care that he’s famous?
Shades of https://news.ycombinator.com/item?id=35079
That whole thread is just amazing, if you back up a couple of levels from ground zero. Great perspectives from a lot of thoughtful posters.
E.g., you can see a post from a user named dhouston, who mentioned that he was thinking about starting an online file sync/backup service of some sort.
Re: Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
#60Earlier quoted context omitted.
Out of curiosity, what sort of things have you seen it do that better fit 'autoresearch' than 'autotune' thus far? Optimizations it made that wouldn't be been surfaced by an autotune system, I suppose.
The most recent round of autoresearch (round 2) which decreased "time to GPT-2" from 1.8 hours to 1.65 hours had some examples. I adjusted the program.md to "look at modded nanogpt project and draw inspirations from there for things to try" and it came back with a bunch of tuning, but also tried and implemented new architecture changes, some of which actually helped including the smear gate and the backout skip conne…
Do you have a sense of whether these validation loss improvements are leading to generalized performance uplifts? From afar I can't tell whether these are broadly useful new ideas or just industrialized overfitting on a particular (model, dataset, hardware) tuple.