Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?

Everyone is impressed when the model does something they don't understand deeply. But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply.

I do think it's slightly better than the mean across all topics. But I also strongly suspect it'll soon serve as a great example for regressing to the mean.

This is the bit that I think you should focus on a bit more. I don't think it's the case you need complicated, clever algorithms and architectures in order to get complicated, clever behavior.

If you start with particle physics, then work your way up to chemistry, and then biology, you can see how we start with very, very simple rules, but at each level there is more and more complexity. The universe "running" physics is the epitome of a brute-force approach. It would be a mistake to say that because the rules of particle physics are so simple, that nothing made of those particles could ever think.

Likewise, even though these models are just big arrays of numbers that we stir in the right way to make them spit out something closer to what we want over and over again, I think it's a mistake to say that out of that, can never arise something much more capable than humans.

> You can’t just train an LLM with more and more parameters and more and more tokens and expect it to be smarter than the data it was trained on. And such models don’t bring us any real understanding of what it would take to make super-intelligent machines.

Others have addressed the first point here, but for the second -- yes, that's true. As for the second, I think we'll have a bit of warning before we get to true superintelligence, but even now, I it seems to me like we have half of an AGI in large LLM models. They don't seem to be conscious, can't really evaluate its their thoughts except by printing them out and reading them in again, and are only superintelligent in terms of knowing lots of facts about lots of things. But I think we are probably going to figure out how to create the other parts and we'll be there.

I am worried that humanity is on a bit of very-high-inertia train of "more and more progress" without enough safeguards. It was ok in the past, but as our world gets more and more connected and new inventions get spread far and wide in less time than ever before, it's possible for damage to be done on a very wide scale before we can figure out how to counteract it. It also means that good things can spread in the same way -- but the problem is that it's not just the average that matters, it's the variance. It doesn't matter if you create and disseminate 9 out of 10 new technologies that are massively beneficial if the other 1 ends up with humanity gone or completely disempowered, and you can't take advantage of the good stuff.

Have you considered that the evidence just isn't convincing yet?

If you view popular llms as text prediction machines, they are in fact much better than spell check or auto correct from a few years ago. But if you actually ask it to solve the problem with nuance it will not use nuance. That's the part that would impress me.

As a recent example if you ask chat GPT how to use ffmpeg to slice out a video. and you tell it that you only want 3 seconds of video. somebody who deeply understands how ffmpeg works, (or even someone who deeply read the documentation) would point out that you have to be aware that it can only cut to keyframes. so if the keyframe is not aligned to the time you ask for you will not get the video that you expect.

another example ask it to play 20 questions with you, it will cheat at the end. even if you give it very specific instructions it still is unable to follow them to the fair conclusion of the game. (or it will make a mistake and understanding about some of the semantics of the question, but I don't fault it for a difference in context)

I'd caution you that just because you're impressed for the subjects that you understand deeply does not mean that chat gpt is good at all subjects. and therefore I assert that it is disrespectful to be so dismissive of people who have different opinions than yours. I believe the default should be to assume good faith rather than dismissiveness "they just don't understand"

So whatever you think you're criticising isn't the same thing that I'm using. Kind of demonstrates my point that you're either using an older version or choosing to ignore evidence for some reason.