Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A deep dive into self-improving AI and the Darwin-Gödel Machine

richardcsuwandi.github.io

Post reply on HN
Print view View original

41–50 of 103 posts

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#41

Post by godelski »

dgb23 wrote: post #37
godelski wrote: post #20

We realize test driven development doesn't work, right? Any scientist worth... any salt will tell you that fitting data is the easy part. In fact, there's a very famous conversation between Enrico Fermi and Freeman Dyson talking about just this. It's something we've known about in physics for centuries Edit: Guys, I'm not saying "no tests", the "Driven Development" part is important. I'm talking about this[0]. | Test…

Test first programming has its use and can be quite peoductive. I believe the issue with „TDD“ is the notion that it should drive design and more importantly that it‘s always applied. I disagree with both if those. Given a problem where test first makes sense, I prefer roughly this procedure: 1. Figure out assumptions and guarantees. 2. Design an interface 3. Produce some input and output data (coupled) 4. Write a te…

You and I are in agreement for the most part.

Especially steps 1-2 are not things easy to hand off in the first place.

Step 6 it's important: reflect on your work and challenge it. I'm distinguishing this from 4 because you need to take the part of a strong adversary.

I'm not quite sure this is hire evolutionary programs work, having written plenty myself. I'd lean on no. I'm certain this is not the fill of my work as a researcher.

As a researcher you can't just put ideas together and follow some algorithm. There's no clear way to continue except in the incremental works. Don't get me wrong, those can do a lot of good, but they'll never get you anything groundbreaking. To do really novel things you need to understand details of what went on before. It's extremely beneficial to reproduce because you want to verify. When doing that you want to look carefully at assumptions and find what you're taking for granted. Maybe that's step 1 for you but step 1 is ongoing. The vast majority of people I meet fail to check their assumptions at even a basic level. Very few people want to play that game of 20 questions over and over being highly pedantic. Instead I hear "from first principles" and know what's about to follow is not a set of axioms. Carl Sagan bakes a pie from first principles. That's too far tbh, but you should probably mill your own flower (and that's still a long way from first)

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#42

Post by meindnoch »

looofooo0 wrote: post #40

"Mathematical breakthroughs: Most notably, it discovered an algorithm for multiplying 4x4 complex-valued matrices using just 48 scalar multiplications, surpassing Strassen’s 1969 algorithm" Again despite all the AI no one found the paper which gives the best bound to this (46): https://ieeexplore.ieee.org/document/1671519

>just 48 scalar multiplications

48 complex scalar multiplications. Which is at least 3 real multiplications.

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#43

Post by sdl »

yubblegum wrote: post #7
xianshou wrote: post #3

The key insight here is that DGM solves the Gödel Machine's impossibility problem by replacing mathematical proof with empirical validation - essentially admitting that predicting code improvements is undecidable and just trying things instead, which is the practical and smart move. Three observations worth noting: - The archive-based evolution is doing real work here. Those temporary performance drops (iterations 4…

> gaming the evaluation Co-evolution is the answer here. The evaluator itself must be evolving. Co-evolving Parasites Improve Simulated Evolution as an Optimization Procedure Danny Hillis, 1991 https://csmgeo.csm.jmu.edu/geollab/complexevolutionarysystem...

And in Reinforcement Learning:

POET (Paired Open-Ended Trailblazer): https://www.uber.com/en-DE/blog/poet-open-ended-deep-learnin...

SCoE (Scenario co-evolution): https://dl.acm.org/doi/10.1145/3321707.3321831

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#44

Post by upghost »

grg0 wrote: post #10

This is genetic programming and is probably older than the authors. Did somebody just came up with a new term for an old concept?

> More precisely, the metacode that controls its behavior and ability

Footnote one validates your assumption.

It seems like the key contribution here is the discovery that anthropomorphizing genetic programming is more optimal for clicks/funding.

Saying it is optimizing some code sounds way less interesting than it is optimizing its own code.

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#45

Post by Xmd5a »

sgt101 wrote: post #33

I spent a lot of time last summer trying to get prompts to optimise using various techniques and I found that the search space was just too big to make real progress. Sure - I found a few little improvements in various iterations, but actual optimisation, not so much. So I am pretty skeptical of using such unsophisticated methods to create or improve such sophisticated artifacts.

This is exactly what I'm doing. Some papers I'm studying:

TextGrad: Automatic "Differentiation" via Text: https://arxiv.org/abs/2406.07496

LLM-AutoDiff: Auto-Differentiate Any LLM Workflow : https://arxiv.org/abs/2501.16673

Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs: https://arxiv.org/abs/2406.16218

GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers: https://arxiv.org/abs/2412.09722

PromptWizard: Task-Aware Prompt Optimization Framework: https://arxiv.org/abs/2405.18369

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#47

Post by pydry »

godelski wrote: post #20

We realize test driven development doesn't work, right? Any scientist worth... any salt will tell you that fitting data is the easy part. In fact, there's a very famous conversation between Enrico Fermi and Freeman Dyson talking about just this. It's something we've known about in physics for centuries Edit: Guys, I'm not saying "no tests", the "Driven Development" part is important. I'm talking about this[0]. | Test…

>But it is insane to write tests first and then just write code to complete tests. It completely ignores the larger picture. It ignores how things will change and it has no context of what is good code and bad code

If you TDD outside in and tend to test from the edges of your stack, being conservative about moving your test coupling lower down the stack then it provides you with the freedom to change things underneath the hood and still have a body of tests that let you know you didnt break anything.

If you TDD inside out then yes you can and probably will create an enormous inflexible mess of tests that dont tell you if your code worked.

Sadly many tutorials teach it as "you want to write a class "Ball" you should write a test for that class first" which is wrongheaded.

Thats just writing tests badly though, it's not intrinsic to red-green-refactor.

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#48

Post by looofooo0 »

meindnoch wrote: post #42
looofooo0 wrote: post #40

"Mathematical breakthroughs: Most notably, it discovered an algorithm for multiplying 4x4 complex-valued matrices using just 48 scalar multiplications, surpassing Strassen’s 1969 algorithm" Again despite all the AI no one found the paper which gives the best bound to this (46): https://ieeexplore.ieee.org/document/1671519

>just 48 scalar multiplications 48 complex scalar multiplications. Which is at least 3 real multiplications.

I think they completely misstated in the original paper what they did. It was a tensor decomposition of complex of 4x4 matrices up to the factor 0.5. Which is a nice result, but it is not really anything practical for a computer program doing 4x4 complex matrix multiplication.

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#49

Post by bonoboTP »

rustcleaner wrote: post #16

>Darwin-Gödel Machine First time I'm hearing abaut this. Feels like I'm always the last to know. Where else are the more bleeding edge publishing points for this and ML in general?

The bleeding edge is very noisy, they by definition haven't stood the test of time and there is a competition for attention and overinflated claims similar to social media attention economy.

About where to find them: arxiv. You can set up Google Scholar alerts for keywords, or use one of many recommendation platforms, such as https://scholar-inbox.com/

Re: A deep dive into self-improving AI and the Darwin-Gödel Machine

#50

Post by chriswarbo »

xianshou wrote: post #3

The key insight here is that DGM solves the Gödel Machine's impossibility problem by replacing mathematical proof with empirical validation - essentially admitting that predicting code improvements is undecidable and just trying things instead, which is the practical and smart move. Three observations worth noting: - The archive-based evolution is doing real work here. Those temporary performance drops (iterations 4…

The "Goedel Machine" is an interesting definition, but wildly impractical (though I wouldn't say it's impossible, since it only has to find some improvement, not "the best" improvement; e.g. it could optimise its search procedure in a way that's largely orthogonal to the predicted rewards).

Schmidhuber later defined "PowerPlay" as a framework for building up capabilities in a more practical way, which is more adaptive than just measuring the score on a fixed benchmark. A PowerPlay system searches for (problem, replacement) pairs, where it switches to the replacement if (a) the current system cannot solve that problem, (b) the replacement can solve that problem, and (c) the replacement can also solve all the problems that caused previous replacements (maintained in a list).

I formalised that in Coq many years ago ( http://www.chriswarbo.net/projects/powerplay ), and the general idea can be extended to (a) include these genetic-programming approaches, rather than using a single instance; and (b) could be seeded with desirable benchmarks, etc. to guide the system in a useful direction (so it's "self-invented" problems can include things like "achieves X% on benchmark Y")

Post reply on HN