Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leo's first encyclical attacks technological messianism

economist.com

Post reply on HN
Print view View original

191–200 of 330 posts

Re: Leo's first encyclical attacks technological messianism

#191

Post by biztos »

amanaplanacanal wrote: post #190
oezi wrote: post #189

Earlier quoted context omitted.

Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?

I don't think we have any evidence that LLMs are even the right path for AGI. It's possible that it is, and it's possible that it isn't. If I was a betting human, I'd bet on "isn't", but what do I know?

On the other hand, it's pretty easy to imagine the full range of Skynet activities being done by a supercharged (but non-AGI) agentic LLM. Meanwhile, every company that can say so with a straight face is trying to become Cyberdyne Systems, and there's no shortage of hackers like us lining up to work for them.

Re: Leo's first encyclical attacks technological messianism

#192

Post by alecco »

oezi wrote: post #189
alecco wrote: post #188

They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…

Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?

> Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown.

Because the real world is not a Hollywood movie. An LLM could try to do something along the lines if either it gets fine-tuned to do it, or somebody instructs it to do it.

I see extremely more likely a small group of humans using a powerful LLM to "take over" critical parts of the world economy. But it wouldn't be like pressing a button. I'm talking of NSA/CIA/Pentagon/Wall St. kind of evil people. And I bet they would do it surreptitiously.

Re: Leo's first encyclical attacks technological messianism

#194

Post by basisword »

NitpickLawyer wrote: post #102
ses1984 wrote: post #96

Earlier quoted context omitted.

People especially wanted uber because uber charged below market rates by subsidizing rides with vc money.

Maybe. But the fact that they're still in business shows that different people value different things. Be it rating schemes, payment alternatives, choosing their music, choosing their cars, one click hailing and so on. The people have spoken, the social contract has changed.

>> But the fact that they're still in business shows that different people value different things.

No it doesn't. It shows they could undercut the market, monopolise it, and then charge more once they'd killed the competition.

Re: Leo's first encyclical attacks technological messianism

#195

Post by basisword »

knollimar wrote: post #106
notahacker wrote: post #77

Earlier quoted context omitted.

But would governments defined by a cadre of techno-authoritarians disproportionately close to Mr Tariff do a better job?

Mr Tariff is probably slightly better to keep those around than not, but ideally I'd rather the government curate experts they employ to advise them. I guess I kinda walked right into handing technocrats power, but I really just want the government to understand the tech they regulate. We'd rather have populist zingers on TV though.

Unfortunately they're choosing to surround themselves with experts who have a massive financial interest in things going their way.

Re: Leo's first encyclical attacks technological messianism

#196

Post by basisword »

miyoji wrote: post #74
amelius wrote: post #63

Earlier quoted context omitted.

It basically says "we have no control over it", and "we don't know how it works".

Both of those things happen to be entirely true, though.

And if someone without billions of dollars in backing released a product like that they'd actually be held responsible for the consequences.

Re: Leo's first encyclical attacks technological messianism

#197

Post by api »

oezi wrote: post #189
alecco wrote: post #188

They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…

Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?

It has no volition. Why would it do this unless someone told it to?

Re: Leo's first encyclical attacks technological messianism

#198

Post by api »

alecco wrote: post #192
oezi wrote: post #189

Earlier quoted context omitted.

Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?

> Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. Because the real world is not a Hollywood movie. An LLM could try to do something along the lines if either it gets fine-tuned to do it, or somebody instructs it to do it. I see extremely more likely a small group of humans using a powerful LLM t…

Something like your takeover scenario already happened, but not through AI. It happened through atomic weapons.

Nations who have them are in a different class from nations who don’t. Nations who have a lot of them, delivery systems, and systems that might be able to shoot down some of a counterattack are superpowers.

Using this leverage these nations and their leaders have been able to dictate world policies. Through the last half of the 20th century this was the US and the USSR. Now it’s the US, EU, Russia, and China, and a few smaller nations with a few nukes. The club is a little bigger but not much.

This kind of thing could happen in the 21st century with AI. If so it will probably be the US and China who control the most powerful AI “agency amplifiers.”

Re: Leo's first encyclical attacks technological messianism

#199

Post by CamperBob2 »

alecco wrote: post #188

They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…

but they are not true AI [2]

Let's ask the operator of a Chinese room to give us a novel math proof.

Go on, I'll wait.

Re: Leo's first encyclical attacks technological messianism

#200

Post by Self-Perfection »

alecco wrote: post #188

They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…

I am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. You are saying this like an argument which allows somehow estimate upper bound of possible influence of these entities. And I do not see how this help to make predictions. It sounds to me like saying "but air is just molecules bobbing around". OK, that's true, but does it help calculate wing aerodynamic profile?

Yup, it makes sequence of symbols. We already have seen that producing specific sequences of symbols is mindbogglingly powerful: merely DNA producers somehow has flown to the Moon!

And yes I am quite aware of Chinese room analogy. Perfectly fine applies to humans as well: single neurons in my head do not understand language, yet I as a whole I would say do understand. Just like applying Chinese room to humans does not help to estimate what humans can do I do not see how it helps to estimate what LLM can do.

Post reply on HN