Live data from Hacker News

Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

news.ycombinator.com

21–30 of 62 posts

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#21
> sober voices as Geoffrey Hinton sounding the alarm about super-smart AI

Can someone please explain to me what exactly the danger is / the dangers are of "super-intelligent" AI?

AIUI, an AI is a combination of hardware, software and parametrization. In broad terms, it exists as a black box which supplies to humans responses to token sets fed to it.

Even if an AI has the launch codes for ICBMs somewhere in its training data, it doesn't have an interface to the nearest missile silo to use them. It cannot commandeer the resources (hardware, space, cooling, electricity) it needs to operate, it is dependent on humans to supply those. So humans can pull the plug on it at any time.

Even if an AI were to become both sentient and nefarious, by what mechanism would it harm humans?

I'm genuinely looking for concrete examples of such a mechanism because I can't imagine any which humans couldn't trivially control or override.

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#22
post #14

Earlier quoted context omitted.

> But it's very rare when someone is impressed when the model is generating text based in something they do understand deeply. This statement sounds out of date. And you can see this sentiment a lot on HN. I don’t know if the people who say this haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence.

> s haven’t tried GPT-4 or they have and are just stubbornly refusing to change their mind about something when presented with new evidence. Have you considered that the evidence just isn't convincing yet? If you view popular llms as text prediction machines, they are in fact much better than spell check or auto correct from a few years ago. But if you actually ask it to solve the problem with nuance it will not use…

I had it extract some general rules from this page and pasted them below:

https://video.stackexchange.com/questions/26302/ffmpeg-cutti...

USER given the following general rules:

1. The issue described in the text occurs when cutting a video between keyframes, resulting in blank video with original sound. 2. Potential solutions include overlaying the copied part with a few seconds of the re-encoded original, seeking to an earlier frame, finding the previous keyframe and clipping from that, or re-encoding the whole clip. 3. FFmpeg does not have a built-in feature to automatically resolve this issue.

Are any of these rules applicable to the question below?

"how do I use ffmpeg to slice out a video. and I only want 3 seconds of video." ASSISTANT Enter an assistant message here.

ASSISTANT Yes, rule 1 and rule 3 are applicable to the question. The issue of cutting a video between keyframes may apply when slicing out a 3-second video using FFmpeg, and FFmpeg does not have a built-in feature to automatically resolve this issue.

USER Given that what is the answer to the question

ASSISTANT To use FFmpeg to slice out a 3-second video, you can use the following command:

```bash ffmpeg -i input.mp4 -ss START_TIME -t 3 -c:v libx264 -c:a copy output.mp4 ```

Replace `START_TIME` with the timestamp (in the format `hh:mm:ss`) from which you want to start the 3-second clip. For example, if you want to start at 1 minute and 30 seconds, the command would be:

```bash ffmpeg -i input.mp4 -ss 00:01:30 -t 3 -c:v libx264 -c:a copy output.mp4 ```

However, if you encounter an issue with blank video caused by keyframe cutting, you may need to seek to an earlier frame, find the previous keyframe, or re-encode the whole clip as mentioned in the provided rules.

It seems to be able to find the applicable rules and apply them.

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#23
It's not only Hinton but Yoshua Bengio too that wrote a great article that was downvoted in olbivion here while he has the higher h-index in computer sciece right now. Should make you suspicious tht there is some psychological bias among tech right now because the alternative is very scary. This bias is called normalcy bias.

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#24

> sober voices as Geoffrey Hinton sounding the alarm about super-smart AI Can someone please explain to me what exactly the danger is / the dangers are of "super-intelligent" AI? AIUI, an AI is a combination of hardware, software and parametrization. In broad terms, it exists as a black box which supplies to humans responses to token sets fed to it. Even if an AI has the launch codes for ICBMs somewhere in its traini…

https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#25

I think most people are just seeing the speed of (seemingly) progress as a direction to AGI. But we are far from it. Though maybe we are seeing something interesting in linear world models recently emerging. But my guess is gradual progress and then all of a sudden we will have AGI. But that first part will take 20-30 years. at least.

But 2 out of 3 scientists that created the tech you are talking about disagree with you. Makes you wonder if..?

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#26
post #4

I'm not deeply in this space, but from what I understand, the problems are thus: 1. Progress in LLMs has come much more rapidly than expected. This means that when [arbitrary threshold] is crossed, we probably won't have much, if any, advance warning. 2. Nobody on earth knows what the path to AGI (or even narrow-but-still-superhuman intelligence with enough agency to be dangerous) looks like. So, it's not currently p…

>Progress in LLMs has come much more rapidly than expected I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?

10 years ago we knew you needed a huge amount of data. These LLM are proof of that

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#27

I think most people are just seeing the speed of (seemingly) progress as a direction to AGI. But we are far from it. Though maybe we are seeing something interesting in linear world models recently emerging. But my guess is gradual progress and then all of a sudden we will have AGI. But that first part will take 20-30 years. at least.

I don't know that it's meaningful to throw out numbers like 20 or 30 years anymore. I've seen enough experts make predictions in this field and be wrong that the only prediction I'm comfortable making about their predictions is that they're probably wrong.

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#29
post #4

I'm not deeply in this space, but from what I understand, the problems are thus: 1. Progress in LLMs has come much more rapidly than expected. This means that when [arbitrary threshold] is crossed, we probably won't have much, if any, advance warning. 2. Nobody on earth knows what the path to AGI (or even narrow-but-still-superhuman intelligence with enough agency to be dangerous) looks like. So, it's not currently p…

>Progress in LLMs has come much more rapidly than expected I hear this very often, but I'm not sure, what were the expectations 10 years ago? What are the expectations for the next 10 years?

Well, it's what I've heard from many "doomerists" in the last year. It's not even a question of 10 years; ChatGPT at launch was pretty surprising to a lot of people. Could be many experts weren't surprised by ChatGPT, though, since we got Sydney and GPT-4 and others pretty soon afterward.

Re: Ask HN: Why do so many assume we’re on the cusp of super-intelligent AI?

#30

> sober voices as Geoffrey Hinton sounding the alarm about super-smart AI Can someone please explain to me what exactly the danger is / the dangers are of "super-intelligent" AI? AIUI, an AI is a combination of hardware, software and parametrization. In broad terms, it exists as a black box which supplies to humans responses to token sets fed to it. Even if an AI has the launch codes for ICBMs somewhere in its traini…

https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/

Thank you for the reference. Its author defines a "rogue AI" as:

"an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere"

and explains that it would also need to be goal-directed in a way which would be at odds with human wellbeing.

Stipulating all that, what is still missing is an explanation of the mechanism by which an AI, rogue or otherwise, could do harm. How is it supposed to affect the world outside its computing substrate?

Absent humans making available the interfaces and resources to do so, it can't. The referenced article includes an example of a genocidal human doing exactly that, and using an AI as a force multiplier. That, as the trope goes, is a social problem, not a technical problem, and it needs a social solution, not a technical one.

Each of the other examples in the referenced article (military AI going rogue, wireheading, amoral corporate AIs manipulating humans) require AIs interfacing with the physical world outside their computing substrate or with the biosphere. Again, because these scenarios remain dependent on humans making available such interfaces, I fail to see how a hypothesized "rogue" AI could achieve any autonomy to do serious damage.

I see this panic about rogue AIs as well-intentioned but misguided, and perhaps exploited by folks who would like to control / diminish / force licensing of general purpose computing.

Post reply on HN