Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

41–50 of 280 posts

Re: AI 2040 and the cult of intelligence

#41

Post by reinitctxoffset »

baq wrote: post #4

Extremely optimistic take IMHO that anyone will be allowed to own a powerful box of artificial intelligence

It'll be like DNS or dynamic linking or torrents of media or watching TLS on your own box: only people motivated to be free from corporate surveillance will be. By default a megacorp will get everything you see online or ask online, they will run code on your computer underneath you without telling you, they'll collaborate with arbitrarily totalitarian governments, we already know what they do by default.

But we also know how hopeless it is against motivated people who are willing to learn about computers. The Equation Group, eh, if they want you they're gonna get you. But some mid-level guy at a FAANG? He can't do shit to people who set up their own rig.

And AI is massively assymetrical in its ability to speed run a corpo-durable rig, we have long since passed the point where open weights and commodity equipment are enough to bootstrap arbitrary capability. They missed their chance to keep it in the bottle.

It's another turn of the crank on Late Soviet America asking for your papers comrade, but in the bitter end if they really want to oppress your ass they'll have to send a thug with a gun like those ICE guys terrorizing all the Latino guys.

So remember, they came for this other guy, and I said nothing...

Re: AI 2040 and the cult of intelligence

#42

Post by softwaredoug »

ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

We already have organizations like Internet Watch Foundation advocating against e2e encryption in an environment where governments have demonstrated a willingness to collude with private companies to surveillance the most vulnerable

Re: AI 2040 and the cult of intelligence

#43

Post by iandanforth »

Fight extremism with extremism I guess? Maybe we don't have to be either "world govt" or "help me get away with murder". It's ok to say things will be messy and complicated and that limited regulation is probably good even if we agree totalitarian control is bad.

Re: AI 2040 and the cult of intelligence

#44

Post by skybrian »

piker wrote: post #26

What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”

If you say you're doing research for a novel, should it consider that plausible? How much does it need to know about its users to vet them?

I think part of the answer is that AI chat doesn't need to be general-purpose. It turned out that people really liked using a chat UI that seems to be general purpose, but you don't need to make answering any question a user asks your business. You don't need to provide therapy if you're not in the therapy business. It should be possible to specialize.

But in order for that to work, a company needs to explain to its customers what business it's in.

Re: AI 2040 and the cult of intelligence

#45

Post by andy99 »

It will be interesting to see how the discussion shakes out, this is all stuff that would get you down modded and talked down to if you said if on HN, so far the reactions are positive but I think there’s a massive authoritarian / safetyism contingent here that will disagree hard with almost everything.

Re: AI 2040 and the cult of intelligence

#46

Post by gilrain »

skybrian wrote: post #34
Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

"Consistently wrong" seems a bit much. Seems like being directionally right early that AI would be a Big Deal and scary should count for something? It doesn't mean any details or other predictions are right, though.

> Seems like being directionally right early that AI would be a Big Deal and scary should count for something?

Only if it was an uncommon prediction; otherwise, it’s just evidence of common sense.

Re: AI 2040 and the cult of intelligence

#48

Post by Jtarii »

>Like we either live in a world with freedom or we don’t

This is something I would expect a 12 year old to say. Constraints on your freedom are literally everywhere in every interaction you have with every part of society.

Re: AI 2040 and the cult of intelligence

#49

Post by reinitctxoffset »

drc500free wrote: post #32
Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

If you replace "AI" with "Adonai" in EY's framing, it reduces to biblical parables from his childhood about arguing with God to negotiate a new covenant. He's clearly a bright guy, but a lot of his work seems to be trying to reconcile Old Testament narrative patterns with atheism, and simply slotting an omnipotent AI in as a replacement.

I once heard TESCREAL/EA/etc. called "Calvinism for programmers".

Having been raised substantially by a Calvinist grandmother, oh yeah.

It's a super weird religion.

Re: AI 2040 and the cult of intelligence

#50

Post by awakeasleep »

slopinthebag wrote: post #14

The scariest thing about LLM's (imo) is some central power, such as a corporation or government, manipulating the truth for their own benefit. Open and locally runnable models are the only solution I can see. They're not without risks of course, but the risk of the centralisation of intelligence seems far greater to me. That being said, could this local intelligence empower bad actors to do very bad things? Like, exi…

It's not simply that people will get a warped answer when they ask AI a question.

It's that AI will be given the command, and then generate the stories, post the comments to reddit, contact the PR firms, put marketing contracts out for influencer media, and more, like an army of yes men to rewrite the story.

Post reply on HN