Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

171–180 of 280 posts

Re: AI 2040 and the cult of intelligence

#171

Post by ryeights »

chatmasta wrote: post #104

> Plan L, for local > Your AI is aligned with you. It never refuses a request, and it is always working on your behalf. Just like my gun, if I want my AI to help me kill my stepmother, it does. The fact that we are even discussing something else should be so far outside the Overton window. I have been saying this since ChatGPT launched. It seems so obvious. We have never stopped the acceleration of knowledge or techn…

Of course. Everyone should have an ASI assistant that will happily tutor them, step-by-step, through synthesizing mass killing agents from household chemicals.

Then the good guys can just ask their ASI how to construct a hermetically sealed bubble to live out the rest of their lives in!

Re: AI 2040 and the cult of intelligence

#172

Post by daishi55 »

Borealid wrote: post #79
mlinsey wrote: post #51

I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction…

"The AI" cannot "refuse" any more than a hammer can refuse. If you use a hammer to kill someone, you commit a crime and the hammer is unprosecuted. If you design a hammer to kill people and then give it away, you are partially liable for the deaths it causes. If you make a normal hammer and someone uses it to kill, you bear no blame. I don't see why these standards should change when the hammer also emits text messag…

The AI can absolutely refuse, what are you talking about?

These things can and do say “sorry I won’t help you with that” based on the nature of your request. What do you call that if not a refusal?

Re: AI 2040 and the cult of intelligence

#173

Post by solid_fuel »

JumpCrisscross wrote: post #169
mortenjorck wrote: post #66

Earlier quoted context omitted.

I would have been very hesitant to run the “just killed wife“ test given that ChatGPT will indeed flag your account and if a certain threshold is crossed, escalate to humans, who will presumably contact authorities to avoid this happening again: https://www.bbc.com/news/articles/cq6je7e80r7o Obviously a certain percentage of the user base is running lurid hypotheticals through the system all the time, but I don’t dou…

> ChatGPT will indeed flag your account Oh no, I'll have to spin up a new e-mail address...

Maybe finish reading the sentence before you respond.

Re: AI 2040 and the cult of intelligence

#174

Post by ericpauley »

rmbyrro wrote: post #165
ericpauley wrote: post #164

Earlier quoted context omitted.

Am I missing something here? Bike tires are pretty much the easiest tires to change. You don’t even really need irons and it involves a few very basic steps.

And our robots still look far from capable of changing bike tires...

That's not OP's contention though; it's that models are incapable of even surfacing sufficient information. This doesn't pass the smell test.

To wit, here is ChatGPT's "superintelligent" answer: https://chatgpt.com/s/t_6a52b4a322bc81919308ec864a0cc056

Seems like a perfectly cromulent description by which a beginner could probably change a tire, perhaps with a few follow-ups to understand what certain components are.

Re: AI 2040 and the cult of intelligence

#175

Post by simoncion »

SiempreViernes wrote: post #167
Aurornis wrote: post #160

Earlier quoted context omitted.

> The AI can and should refuse it. This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes. Which is precisely what we’re going through right now with frontier models and it’s terrible. These proposals always assume some perfect mechanism for identifying the thought crime with triggering on the normal requests. The people who want to commit crimes…

> This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes. Oh no, of only we had some sort of ideas about automated authentication and authorisation! (If big AI actually make a model that would reliably reject dangerous queries in the first place, they can just use auth tech to serve the current ones to vetted customers)

> If big AI actually make a model that would reliably reject dangerous queries in the first place...

Not happening, for two reasons

1) LLMs mix attacker-controlled data into their instruction stream, and it doesn't look at all like the big LLM providers consider making that impossible to be an actual priority.

2) Intent is what distinguishes "dangerous" queries from prosocial ones. Reliably determining intent is so complex that not even humans can do it.

Re: AI 2040 and the cult of intelligence

#176

Post by Aurornis »

SiempreViernes wrote: post #167
Aurornis wrote: post #160

Earlier quoted context omitted.

> The AI can and should refuse it. This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes. Which is precisely what we’re going through right now with frontier models and it’s terrible. These proposals always assume some perfect mechanism for identifying the thought crime with triggering on the normal requests. The people who want to commit crimes…

> This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes. Oh no, of only we had some sort of ideas about automated authentication and authorisation! (If big AI actually make a model that would reliably reject dangerous queries in the first place, they can just use auth tech to serve the current ones to vetted customers)

> they can just use auth tech to serve the current ones to vetted customers

Your solution is to require everyone to surrender their privacy and then allow some company or government decide who is worthy of being allowed to use AI?

It’s crazy how authoritarian, controlling, and anti-privacy the anti-LLM conversation is getting.

Re: AI 2040 and the cult of intelligence

#177

Post by perching_aix »

reinitctxoffset wrote: post #58

Your AI should be at least as aligned with you as your lawyer. It is 100% it's job to help you get away with it if you murdered someone. A lawyer who does less can be disbarred. There is only one sensible alignment for an AI system to which we do not confer the status of moral patient (different conversation): operator aligned .

> It is 100% it's job to help you get away with it if you murdered someone. A lawyer who does less can be disbarred.

That is not what defense lawyers do. They're there to ensure the power of the "justice" system isn't being misused, to the best of their ability. They'll not even want to know if the person really did the crime per se.

https://youtube.com/shorts/WgCjWluc4Mk

> There is only one sensible alignment for an AI system to which we do not confer the status of moral patient (different conversation): operator aligned.

If the operator has a moral code of any kind, this is immediately rendered logically incoherent. Not that the letter of the law would be a moral standard to begin with.

Re: AI 2040 and the cult of intelligence

#178

Post by johnfn »

plorkyeran wrote: post #157
johnfn wrote: post #109

Earlier quoted context omitted.

We agree the guy roughly said AI will be dangerous. We agree AI is dangerous. Not sure what more there is to say here.

He made specific predictions about what problems AI would cause, not just a vague "AI will be dangerous". Those specific predictions were entirely incorrect; they both said that things would happen that didn't, and failed to foresee any of the actual problems with AI that have happened.

Yudkowsky said AI would be a problem when very few people taken seriously in the mainstream were saying it was going to be a problem. Saying "specific predictions are wrong" is missing the forest for the trees here.

Re: AI 2040 and the cult of intelligence

#179

Post by tangenter »

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming.

What you’re seeing are people who live in their heads. We all do it from time to time, but we snap out of it when we need to actually get shit done. These people are tenured in bullshitting and they sound as if they know what they are talking about, but are in reality inept and incapable. They’ve practiced rhetoric and prose here and there, but they profoundly lack experience in things outside of their immediate field.

Re: AI 2040 and the cult of intelligence

#180

Post by Aurornis »

ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments.

There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments.

It all starts to feel like a side discussion about a total fantasy world where the government is authoritarian, but benevolent and correct.

You don’t write the laws and regulations for a perfectly faultless government that doesn’t exist. You write them for the governments as they do exist, which means they must be written first in a way that makes them resilient to bad governments abusing them.

Every time someone proposes one of these regulations where we control speech or force government centralized ID checking on to social websites, I wish they would pause and imagine how a law might be misused if someone they didn’t like got elected. We have years of evidence of government officials declaring things “domestic terrorism” to use justify using laws and powers for almost any purpose they want. Why would anyone think this wouldn’t happen if governments were given control of LLMs, too?

Post reply on HN