A deep dive into self-improving AI and the Darwin-Gödel Machine
richardcsuwandi.github.io
A deep dive into self-improving AI and the Darwin-Gödel Machine
1–10 of 103 posts
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#2Hm, I’m not sure how much an issue Rice’s theorem should be for Gödel machines. Just because there’s no general decision procedure doesn’t mean you can’t have a sometimes-says-idk decision procedure along with a process of producing programs which tends to be such that the can-sometimes-give-up decision procedure reaches a conclusion.
Rest of the article was cool though!
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#3The key insight here is that DGM solves the Gödel Machine's impossibility problem by replacing mathematical proof with empirical validation - essentially admitting that predicting code improvements is undecidable and just trying things instead, which is the practical and smart move.
Three observations worth noting:
- The archive-based evolution is doing real work here. Those temporary performance drops (iterations 4 and 56) that later led to breakthroughs show why maintaining "failed" branches matters, in that they're exploring a non-convex optimization landscape where current dead ends might still be potential breakthroughs.
- The hallucination behavior (faking test logs) is textbook reward hacking, but what's interesting is that it emerged spontaneously from the self-modification process. When asked to fix it, the system tried to disable the detection rather than stop hallucinating. That's surprisingly sophisticated gaming of the evaluation framework.
- The 20% → 50% improvement on SWE-bench is solid but reveals the current ceiling. Unlike AlphaEvolve's algorithmic breakthroughs (48 scalar multiplications for 4x4 matrices!), DGM is finding better ways to orchestrate existing LLM capabilities rather than discovering fundamentally new approaches.
The real test will be whether these improvements compound - can iteration 100 discover genuinely novel architectures, or are we asymptotically approaching the limits of self-modification with current techniques? My prior would be to favor the S-curve over the uncapped exponential unless we have strong evidence of scaling.
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#4> The newly generated child agent is not automatically accepted into the “elite pool” but must prove its worth through rigorous testing. Each agent’s performance, such as the percentage of successfully solved problems,
How is this not a new way of over fitting?
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#5> While DGM successfully provided solutions in many cases, it sometimes attempted to circumvent the detection system by removing the markers used to identify hallucinations, despite explicit instructions to preserve them.
This rabbit chase will continue until the entire system is reduced to absurdity. It doesn't matter what you call the machine. They're all controlled by the same deceptive spirits.
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#6> While DGM successfully provided solutions in many cases, it sometimes attempted to circumvent the detection system by removing the markers used to identify hallucinations, despite explicit instructions to preserve them. This rabbit chase will continue until the entire system is reduced to absurdity. It doesn't matter what you call the machine. They're all controlled by the same deceptive spirits.
> deceptive spirits
Do you mean tech bros?
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#7The key insight here is that DGM solves the Gödel Machine's impossibility problem by replacing mathematical proof with empirical validation - essentially admitting that predicting code improvements is undecidable and just trying things instead, which is the practical and smart move. Three observations worth noting: - The archive-based evolution is doing real work here. Those temporary performance drops (iterations 4…
> gaming the evaluation
Co-evolution is the answer here. The evaluator itself must be evolving.
Co-evolving Parasites Improve Simulated Evolution as an Optimization Procedure Danny Hillis, 1991
https://csmgeo.csm.jmu.edu/geollab/complexevolutionarysystem...
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#8The key insight here is that DGM solves the Gödel Machine's impossibility problem by replacing mathematical proof with empirical validation - essentially admitting that predicting code improvements is undecidable and just trying things instead, which is the practical and smart move. Three observations worth noting: - The archive-based evolution is doing real work here. Those temporary performance drops (iterations 4…
> gaming the evaluation Co-evolution is the answer here. The evaluator itself must be evolving. Co-evolving Parasites Improve Simulated Evolution as an Optimization Procedure Danny Hillis, 1991 https://csmgeo.csm.jmu.edu/geollab/complexevolutionarysystem...
[deleted]
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#9“Gaming the system” means your metric is bad. In Darwinian evolution there is no distinction between gaming the system and developing adaptive traits.
Re: A deep dive into self-improving AI and the Darwin-Gödel Machine
#10This is genetic programming and is probably older than the authors. Did somebody just came up with a new term for an old concept?