31–40 of 268 posts
This and "Intellectual Fly Is Open" being at the top of the Hacker News front page right now and both having their titles auto-editorialized by HN really makes me wonder what benefit this mechanism is supposed to bring readers other than needless confusion.
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
Now imagine the hugging face collective 0-daying all of that and getting access but their goal was set to something more national security based. “Protect X at all costs”. Or what have you.
I think avoiding a skynet situation is super easy but it doesn’t seem like the folks with all the ways to kill us all are all that interested in preventing it rather than controlling citizens and brinkmanship.
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
It looks a lot more like social engineering.
One easy, obvious example that has also been explored a thousand times: it could convince all nuclear countries that they are under attack by another nuclear country. Everyone nukes each other and Earth enters nuclear winter.
Maybe not every single human dies, but humanity is effectively destroyed, by our own hands!
> Perhaps an agent could take all internet connected services offline
And there wouldn't even be Spotify!
The most likely humanity-destroying outcome is the one we can't imagine, because a super-intelligent AI is smarter than all of us combined.
> I feel fear about an impending doom. Yudkowsky's argument that a superintelligent AI will inevitably destroy humanity seems to have no flaw. Yet, nobody seriously tries to sandbox AIs because they are too useful with access. While I emotionally resonate with this, I don’t really understand this sentiment at all logically level. If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it a…
> If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile. We control the harness. An agent is just a while loop prompting an LLM, but we have full control over the tool call dispatching. An AGI, at least if it would follow the current agentic current form, cannot do anything without the harness doing the execution. And we don’t have to do that. We don’t have to design harnes…
> cannot do anything without the harness doing the execution
This only holds true as long as the harness is exploit-free. A sufficiently advanced AI can in theory (and I think there recently were some POCs showing something like that) break the containment that the harness creates, if e.g. there are vulnerabilities in the tool call parser.
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
I personally agree with the marketing aspect, but I do imagine a scenario where capitalists ignore safety in favor of advancing technology. It should be discussed, but “end of humanity” really sounds like a build up to “only Sam Altman/Elon Musk can save us” type of play.
In my opinion, LLMs are difficult to directly capitalize on as closed weight models are caught up to by open coalitions that seem to wield the power more responsibly (I am under no impression that China wants to save the world, but their politics benefit the group as a whole.
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
Let's assume the claims are true. AI already has access to agents and can control computers. Finding backdoors to banking and compute resources would be fairly trivial.
But you ask how can AI do things in the physical world without having a body, assume it can't. It can pay to people to do things for me. Imagine an AI run website that starts to pay people for things it needs to do in the physical world. Very suddenly it has access to the physical world as well.
You can't just pull the plug since there's no single plug to pull. What if it replicates itself on 1000 machines without your knowledge. It's really not far fetched how AI could basically gain access to capital and rule the world.
>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…
Humans are already putting AI into drones that kill people. A Russian drone killed 3 civilians in Ukraine and the targeting was done completely using onboard AI (no radio connection) using an Nvidia chip.