81–90 of 189 posts
Proving a generality seems much more difficult since you don't know what you are trying to build, although I suppose in some cases you can prove it by proving that it's impossible to construct a counter-example.
This is really an argument about test-time scaling, even though the post never uses the term. These days "test-time scaling" mostly means letting the model talk to itself for longer, but the first genuinely surprising results came from plain sampling. Google's AlphaCode generated millions of candidate programs and filtered them down to a handful of submissions, which beat the average human programmer in 2022, before…
for somebody who's out of the loop: what's the fuss over the ABC conjecture?
It doesn't help that the author is rather withdrawn and not willing to spend any effort in making it more approachable.
Some tried, and said they found gaps in the proof, to which the author responded, but they were not convinced.
And that's essentially the situation since 2018.
Earlier quoted context omitted.
Yes, one of the obvious ways to me to use these models is to tell them to write such a program. It can then go figure out data extraction and normalization. This is "the harness improving itself". Have it write tools to do its tasks.
I think you missing the problem with composition itself. The tool generation is the easy part. Knowing which tool is needed, specifying it correctly, validating it against current context, knowing why it failed, and deciding when it needs replacement are separate tasks, and they compose, and composition compounds and fails.
Disclaimer: I only scanned the article quickly; I might be re-stating something already in the article. We have just got some very strong evidence about the way in which LLM-based systems solve mathematical problems and this evidence supports what many have already suspected including myself. Here's what I'm talking about. On 10 August Anthropic released an article [1] claiming that: An unreleased research version of…
Yes this is the "Universal Problem Solving Algorithm". It's actually the same algorithm used by Evolution.
Also this algorithm is vastly different than "monkeys with typewriters". Monkeys don't learn or evolve their writing. There is no memory, no constraints, no learning-curve. At each iteration they freshly sample from a Uniform Distribution. Expected time for a solution is infinitely long.
"The Universal Algorithm" on the other hand is incredibly fast. Humans (designers, researchers) also use the same algorithm but they are much slower to iterate than computers. Instead of trying 650 different ideas at a single run, we have 100s of researchers each try few different ideas independently.
Earlier quoted context omitted.
> the human brain unconsciously does calculus and linear algebra We're not unconciously doing calculus and linear algebra. If you're arguing you 'do calculus' to predict how to catch a ball, I'm sorry but it's not supported by the data.
[dead]
I think "modeling the trajectory" is not necessarily what we are doing either. It's more likely we are using much simpler heuristics. If you are trying to catch a ball flying through the air, you can just look at the ball and modulate your running speed to keep your eyes at a fixed angle until you catch the ball. It's much more analogous to a PID controller than a model of the trajectory.
Earlier quoted context omitted.
Would you trust a bridge built by it with zero human interference? A train? A plane? A skyscraper? What about fight in a war? Unless you can, I wouldn't consider it AGI, because you're actually trusting human intelligence to verify the bridge or train or plane or skyscraper is safe or that the robot is following orders. And even then, you're trusting human-influenced guardrails etc. I would consider it AGI when an AI…
Would you trust a bridge built by a single human? In reality, we have lots of guardrails to ensure that we don't screw up and kill a lot of people by deploying defective bridges (or cars to drive on them). Those guardrails often are written in blood, and still occasionally fail.
AGI checking other AGI should give you that same trust, no? Deepseek says my ChatGPT bridge is stable, you should trust it. Claude says it's stable. The humans say it isn't, but they aren't AGI. You can trust this bridge because it's been vetted by AGI. In my opinion, LLMs cannot be AGI, so for me I would never trust them above any human I would trust. But for those who do believe LLMs can be AGI, they have to demonstrate why we should trust them above any human in these extreme cases. Meaning, if someone says "Well the department of safety (ran by humans) says it's not safe" we have to believe that AGI just knows better than the department of safety. I think this is not possible right now, which is why I don't think we can trust anything built by LLMs where we need the tolerance of risk to human life and safety to approach zero. American AGI soldiers invade the home of Iranian citizens because they have been identified as terrorists. Do you trust the AGI to know if the visual scan they see in this civilian home is a threat to the interests of the United States government and its citizens?
> If they were, then their big speed advantage over us would mean that there would be much more of a flood of results. Is this true right now? Just recently Jarred Sumner tweeted [1] that he managed to make some progress on the Riemann hypothesis while on a jog. Managed to get somewhere by encouraging the llm to “keep going” and “believe in yourself”. This raised a few questions for me. Had no one at Anthropic though…
> Still not sure what that means, but some analytic number theorists seem excited
ie, I prompted AI and it put out a giant pile of tokens. I dont know what it means, but I hope someone gets excited. Mathematicians are now the priests and shamans chanting incantations and taking the holy blessings from the AI gods.
LLMs are good at pattern recognition, so its less a type of math that they'll be good at, and more that when you provide documentation or text that can be easily parsed/compared to its training data/reasoning ability, the better answers you get from an LLM.
Also, you need to be knowledgeable at the same thing you are asking the LLM to do, to verify the answer it gives you (at least for the time being).