Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

101–110 of 280 posts

Re: AI 2040 and the cult of intelligence

#101

Post by vinay_ys »

baq wrote: post #4

Extremely optimistic take IMHO that anyone will be allowed to own a powerful box of artificial intelligence

Right now, almost all the incentives are to build very large models that run across many machines in huge datacenters. There is very little incentive to build models that can run well on a small machine under your desk.

So it is less about whether people will be “allowed” to own AI, and more about whether there will be anything useful to own in the first place.

The incentive for local models is mostly to make them good orchestrators or user agents. They may give you some privacy and control, but they will still depend on much larger models running in datacenters for anything difficult.

I remember all the excitement around OpenMoko and other open-source “BlackBerry killer” projects. BlackBerry did get killed, but not by any of those individual-first projects.

For AI regulation, I think we should focus on normal commercial rules: consumer protection, privacy, antitrust, liability, and so on. In other words, focus on where money changes hands and where companies have power over users.

Military and offensive use is different. There, regulation is not much of a defense. The real defense is having enough capability and strength of your own.

Restricting AI because it can give dangerous knowledge to ordinary people is like restricting the printing press because it can be used to spread radical ideas.

Re: AI 2040 and the cult of intelligence

#102

Post by majormajor »

skybrian wrote: post #34
Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

"Consistently wrong" seems a bit much. Seems like being directionally right early that AI would be a Big Deal and scary should count for something? It doesn't mean any details or other predictions are right, though.

It's not a particularly novel claim since at least Terminator (and even moreso Terminator 2) made it EXTREMELY mainstream.

It was a topic in less-mainstream sci-fi well before that. And some more mainstream stuff like Star Trek TOS.

Frankly it seems more common than not in the last 40 years. I don't really remember a big wave of claims "Terminator is silly, no sort of AI could ever be malevolent!"

Re: AI 2040 and the cult of intelligence

#103

Post by johnfn »

fasterik wrote: post #91
johnfn wrote: post #60

Earlier quoted context omitted.

The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility. How is that in any way "consistently wrong with all their predictions"?

>The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility That's not accurate at all. AI research has been around since the 1950s and pioneers of the field identified risks early on, including Alan Turing, Norbert Wiener, and I. J. Good. The problem with Yudkowsky is that he lays out elaborate doomsday scenarios with extreme confidence, ex…

You are saying 1) other people have said the same thing and 2) you don't like the particular way he said it. Those can both be true, but that isn't even close to "consistently wrong with all their predictions".

Re: AI 2040 and the cult of intelligence

#104

Post by chatmasta »

> Plan L, for local

> Your AI is aligned with you. It never refuses a request, and it is always working on your behalf. Just like my gun, if I want my AI to help me kill my stepmother, it does. The fact that we are even discussing something else should be so far outside the Overton window.

I have been saying this since ChatGPT launched. It seems so obvious. We have never stopped the acceleration of knowledge or technology. It’s impossible to slow down progress, other than within some regulatory siloes that will end up worse in the long run.

I’m not scared of superintelligence. I’m scared of the people in control of super intelligence.

The only defense is parity and diffusion of power. We need an AGI behind every blade of grass.

If you think superintelligence is a weapon, then you should also think every citizen should have one because otherwise they’ll have no way to defend themselves against a tyrannical government or corporatocracy.

Artificial intelligence, and the hardware powering it, needs to be protected under the 2nd Amendment.

Re: AI 2040 and the cult of intelligence

#105

Post by exitb »

piker wrote: post #26

What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”

Notably, AI IS generally permitted to that, as there are easily obtainable models that will play along. ChatGPT won’t because OpenAI chose and implemented that limitation.

Re: AI 2040 and the cult of intelligence

#106

Post by c1ccccc1 »

piker wrote: post #26

What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”

Just a terminology note: Alignment does not mean the AI will help its owner kill people. (Indeed, an AI aligned to value human life would generally try to prevent murders.) The word for an AI that follows all instructions of its owner, as that owner intended them to be understood, is "corrigible" or "controllable".

Re: AI 2040 and the cult of intelligence

#107

Post by Jtarii »

jchw wrote: post #77
Jtarii wrote: post #48

>Like we either live in a world with freedom or we don’t This is something I would expect a 12 year old to say. Constraints on your freedom are literally everywhere in every interaction you have with every part of society.

That's a bit literal of an interpretation. Even acknowledging freedom as a spectrum that can never truly be absolute, there's gotta be a threshold you cross where you'd say "OK, we no longer have basic freedoms." And not only are we definitely screaming towards that threshold, but people seem thrilled about it, for a variety of reasons. All I can say to that is I guess I hope everyone enjoys the world they think they…

The article seems to be taking a very literal interpretation of "freedom" as in any restriction on what a local LLM can do is totalitarian dystopia.

I don't think it should be legal to get an LLM to research how to make a bioweapon in your basement, that seems like a bad idea.

Re: AI 2040 and the cult of intelligence

#108

Post by fasterik »

johnfn wrote: post #103
fasterik wrote: post #91

Earlier quoted context omitted.

>The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility That's not accurate at all. AI research has been around since the 1950s and pioneers of the field identified risks early on, including Alan Turing, Norbert Wiener, and I. J. Good. The problem with Yudkowsky is that he lays out elaborate doomsday scenarios with extreme confidence, ex…

You are saying 1) other people have said the same thing and 2) you don't like the particular way he said it. Those can both be true, but that isn't even close to "consistently wrong with all their predictions".

No, they didn't say the same thing. Identifying potential risks is not the same thing as confidently predicting the extinction of humanity.

Re: AI 2040 and the cult of intelligence

#109

Post by johnfn »

fasterik wrote: post #108
johnfn wrote: post #103

Earlier quoted context omitted.

You are saying 1) other people have said the same thing and 2) you don't like the particular way he said it. Those can both be true, but that isn't even close to "consistently wrong with all their predictions".

No, they didn't say the same thing. Identifying potential risks is not the same thing as confidently predicting the extinction of humanity.

We agree the guy roughly said AI will be dangerous. We agree AI is dangerous. Not sure what more there is to say here.

Re: AI 2040 and the cult of intelligence

#110

Post by ben_w »

andy99 wrote: post #45

It will be interesting to see how the discussion shakes out, this is all stuff that would get you down modded and talked down to if you said if on HN, so far the reactions are positive but I think there’s a massive authoritarian / safetyism contingent here that will disagree hard with almost everything.

As someone who considers themselves in the "safetyism contingent", the only thing I find myself agreeing with about the rollout of LLMs has been the argument that it has to be shown to and used by the public while under development.

Every specific prediction not only sounds like science fiction before it arrives, it tautologically is science fiction because such predictions are narratives about something science-y that has yet to exist: https://www.youtube.com/shorts/wt-fLWxkrfs

Even when we accurately predict something in the future, we often have no clue what the implications are for society until it is in our hands: https://youtu.be/2Pw_7vAK9k8?si=X5t4tcxsXiuHEfBi

Go too fast, suddenly drop in tech with too many novel implications all at the same time, and we likely hit something relevant without having the means to cope with all the change. However, "too many" and "likely" are weasel words with no predictive power about what the danger threshold might be. I may be entirely wrong to think there is one, or we may have been bouncing off it since the industrial revolution, or I may just be echoing the same thing everyone feels once reaching their 40s: https://www.youtube.com/watch?v=fkUwXenBokU

Post reply on HN