221–230 of 298 posts
It optimized
initializers.normal (0.0
initializers.normal (0 + 1j * 0,
Anyway, impressive results. That's why OpenAI and Elon were so frightened about Hassabi.
Did you see that halluzination in the paper? It optimized initializers.normal (0.0 to initializers.normal (0 + 1j * 0, I thought the results were being reviewed? Anyway, impressive results. That's why OpenAI and Elon were so frightened about Hassabi.
Interestingly, it seems alphaevolve has already been in use for a year, and it is just now being publicly shown. The paper also mentions that it uses Gemini 2.0 (pro and flash), which creates a situation where Gemini 2.0 was used in a way to train Gemini 2.5. I don't know if I would call this the fabled "self improving feedback loop", but it seems to have some degree of it. It also begs the question if Alphaevolve wa…
> which creates a situation where Gemini 2.0 was used in a way to train Gemini 2.5. The use of synthetic data from prior models to create both superior models and distilled models has been going on since at least OpenAI's introduction of RLHF, and probably before that too.
That’s distinct from those prior models providing actual code to improve the next model
Earlier quoted context omitted.
It is not some very explicit threshold beyond which AI will take job but before it won't. What's already happening is long drawn attrition where tools at different level of code, low code , no code will keep creeping up. And it will start with people are not respected or valued for their work, so they can leave, once left, they will not be replaced or replaced lower skilled folks and at some point that position stop…
> And it will start with people are not respected or valued for their work, so they can leave, once left, they will not be replaced or replaced lower skilled folks and at some point that position stop existing altogether. Automation changed farming for the worse? Farmers today are not respected / valued for their work? Farmers were replaced with low skilled labor? Do you think the job of a farmer (aka "food grower")…
Pretty much most of them feel underpaid for the amount of hard work and I hear they're having problem recruiting younger people to the business, so many foreigners take those jobs.
This looks like something that can (and should) be reimplemented open-source. It doesn't look like a particularly daunting project.
If it goes well, I could open source it.
What are the things you would want to optimize with such a framework? (So far I've been focusing on optimizing ML training and architecture search itself). Hearing other ideas would help motivate me to open source if there's real demand for something like this.
[1] Blog: https://deepmind.google/discover/blog/discovering-novel-algo...
[2] Paper: https://www.nature.com/articles/s41586-022-05172-4
[3] arxiv.org/pdf/2210.04045
[4] arxiv.org/abs/2212.01175 Flip graphs for matrix multiplication
(Reposted from here, where I made a mini deep-dive into this: https://x.com/friederrrr/status/1922846803420119410?t=7jZ34P...)
This looks like something that can (and should) be reimplemented open-source. It doesn't look like a particularly daunting project.
Had mentioned the same on X: https://x.com/friederrrr/status/1922850981181784152?t=usXpK1...
This looks like something that can (and should) be reimplemented open-source. It doesn't look like a particularly daunting project.
I've been working on something very similar as a tool for my own AI research -- though I don't have the success they claim. Mine often plateaus on the optimization metric. I think there's secret sauce in the meta-prompting and meta-heuristic comments from the paper that are quite vague, but it makes sense -- it changes the dynamics of the search space and helps the LLM get out of ruts. I'm now going to try to integra…
In my case, I'd mainly be interested in mathematics: I'd provide a mathematical problem and a baseline algorithm for it and would want an open source framework to be able to improve on that.
This looks like something that can (and should) be reimplemented open-source. It doesn't look like a particularly daunting project.
I've been working on something very similar as a tool for my own AI research -- though I don't have the success they claim. Mine often plateaus on the optimization metric. I think there's secret sauce in the meta-prompting and meta-heuristic comments from the paper that are quite vague, but it makes sense -- it changes the dynamics of the search space and helps the LLM get out of ruts. I'm now going to try to integra…