71–80 of 110 posts
> Then I lock down Claude Code’s permissions to only edit these two files and run run.sh. No direct Python execution, no pip installs, no network access, no git push, etc. How does one run Claude Code without network access?
The docker container didn’t have network access. Claude didn’t have permission to execute anything other than the run.sh bash script, which would orchestrate the docker run
So... It did work. It found bugs (that he didn't know about) and it did optimization (that he hadn't done).
Most of the gains came from fixing a bug + hyperparameters with optuna which is supposed to be already quite automatic (you set the list of all the var with values you want to try and voilà). I guess a simple claude code session would fix that in a few minutes instead of a full day.
To me, I guess the main value of Autoresearch would be to test different kind of architectures. It's sometimes hard to know what to choose and it would probably give a nice overview.
Anyone used it for exploratory modeling?
Earlier quoted context omitted.
Our experiments aren’t free. We use cloud infrastructure. An experiment costs on the order of tens of dollars, so massively parallelizing “spaghetti at wall” simulators is costly before we even talk about LLMs.
If it is an experiment. Can’t you just make a POC for the experiment that doesn’t need to use half of AWS to just run? And if the experiment is actually positive you can then bring it to the real application and test it there (and spending the 10-100 usd it costs to test it live)?
Earlier quoted context omitted.
Our experiments aren’t free. We use cloud infrastructure. An experiment costs on the order of tens of dollars, so massively parallelizing “spaghetti at wall” simulators is costly before we even talk about LLMs.
If it is an experiment. Can’t you just make a POC for the experiment that doesn’t need to use half of AWS to just run? And if the experiment is actually positive you can then bring it to the real application and test it there (and spending the 10-100 usd it costs to test it live)?
Steelmanning your question though, I guess you could come up with some sort of tiered experimentation scheme where you slowly expose it to more data and more compute based on prior success or failures.
I often use LLMs to explore prior art and maybe find some alternative ways of thinking of problems. About 90% of what it tells me is useless or inapplicable to my domain due to a technicality it could not have known, but the other 10% is nice and has helped me learn some great new things. I can’t imagine letting an agent try everything that the LLM chatbot had recommended ($$$). Often coming up in recommendations are…
I've done something with a small project I have and I had very similar results overall.
Earlier quoted context omitted.
> I find LLMs useful in regurgitating one-liners This was the case for me a year ago. Now Claude or Codex are routinely delivering finished & tested complete features in my projects. I move much, much faster than before and I don’t have an elaborate setup - just a single CLAUDE.md file with some basic information about the project and that’s it.
People keep saying this and I agree Claude has gotten a lot better even in my own experience, but I think the value is questionable. What’s the point of adding features that are inscrutable? I have gotten Claude to make a feature and it mostly works and if it doesn’t work quite right I spend a massive amount of time trying to understand what is going on. For things that don’t matter too much, like prototyping, I thin…