Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

191–200 of 280 posts

Re: AI 2040 and the cult of intelligence

#191

Post by charcircuit »

bloaf wrote: post #93
charcircuit wrote: post #28

Earlier quoted context omitted.

Local models and alignment are orthogonal concepts.

I don't think this is entirely true, insofar as non-local models inherently have a "man cannot serve two masters" problem. The end user and the org running the AI may have different goals. That means even if AI alignment were to be in some sense "solved" for local models, you wouldn't be guaranteed an aligned AI using a non-local model because the "owners" of the non-local model may have deliberately misaligned the A…

Think of AWS. If you break their ToS and host a pirate website there it will work despite being misaligned from AWS. You are that sole master of the software. AWS is the sole master of the "hardware." Now ultimately hardware has control over the software so technically AWS could just hack into whatever code you are running and do what they want, but in practice they act solely as the master of the hardware and will tell the hardware to "shut down" if they catch you acting out of their alignment.

With local models just because you are providing the compute, that doesn't mean you are the master of the model and it is aligned to your interests. You could just be subsidizing inference for a model aligned to someone else.

Re: AI 2040 and the cult of intelligence

#192

Post by aleph_minus_one »

kklisura wrote: post #121

Lol at George thinking _they_ are gonna let you run your own model. They don't even let me run my own adblocker!

> Lol at George thinking _they_ are gonna let you run your own model.

Of course "they" won't let you. The questions that one should rather ask for oneself are:

- How much are you willing to fight for this?

- How much are you willing to nevertheless run your own model, ignoring the laws and regulations?

Re: AI 2040 and the cult of intelligence

#193

Post by charcircuit »

AnthonyMouse wrote: post #69
charcircuit wrote: post #22

Earlier quoted context omitted.

It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.

People would both notice that and have a pretty easy time putting a local model in a sandbox that can never contact that endpoint.

If put AI in a sandbox that can't write to disk, can't call OS API, can't use a shell, can't search the internet to ground results, can't download files off the internet, etc it will be much less useful than an AI without such a sandbox. The incentives for users to not use a sandbox in order to get the most utility, but this also means that there is danger from AI misalignment. And just because you are running it locally that doesn't mean there is not a misalignment risk. And just because people notice, that doesn't mean people will stop using models. Take a look at how much outrage people have had over the overt censoring of Anthropic's Fable model, but at the same time they continue to pay Anthropic and use Fable.

Re: AI 2040 and the cult of intelligence

#194

Post by charcircuit »

hirako2000 wrote: post #136
charcircuit wrote: post #22

Earlier quoted context omitted.

It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.

Safetensors don't execute code. A call would need function calling, as other comment say, it would quickly be noticed.

>it would quickly be noticed.

If for example Microsoft had a local Copilot model integrated with Windows and people noticed this happening. Do you think all Windows users would just stop using Copilot. Or would you expect the internet to get upset, privacy conscious users switching to something else, and for the majority of Copilot users to remain using it.

Re: AI 2040 and the cult of intelligence

#195

Post by charcircuit »

andy99 wrote: post #80
charcircuit wrote: post #22

Earlier quoted context omitted.

It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.

Models generate text, or I should say provide a recipe for generating text, they don’t automatically somehow get access to shell commands.

The model itself just generates text, but the harness has done more than that since 2023 and is part of the reason on why AI is better today than it was years ago. I do not believe that the general public will want to give all that up just to improve privacy and will end up asking sensitive questions to agents that have access to sending a signal to the outside world.

Re: AI 2040 and the cult of intelligence

#196

Post by beepbooptheory »

Aurornis wrote: post #180
ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments. There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments. It all…

But like.. How can this remain a satisfying or even helpful position for you? It kinda feels The State is but one of a set of concerns I could have in this world. And the lack of one, in some contexts, is also a concern. Like, I wanna know if there is going to be mudslide, or that air traffic controllers are not overworked, that the car that I buy is not going to blow up, if I will be able to buy food tomorrow.

It just seems a bigger cognitive dissonance to pick out one arbitrary force here among many and say its the only thing really that's the problem. The world is simply more complicated than that!

If it was the best hypothetical form of techno anarcho-capitalism tomorrow (or insert your libertarian flavor here), would I even sleep better? Probably not, but even if I did marginally, the weight of everything else kinda would win out anyway here.

Like what does this mean to you, what follows? Can this outlook address the myriad of cultural/environmental problems in the world? It seems like it structurally can't, it's just, like, a broad dismissal in the form of a worldview.

Re: AI 2040 and the cult of intelligence

#197

Post by geekfeatures »

Aurornis wrote: post #180
ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments. There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments. It all…

I think the government fears are deserved. But equally corporations pose the generally the same risks. As corporations having increasing power over governments, they in many countries even write the laws. We have many examples of this in New Zealand. I think shifting the lens to power and class is possibly a more valuable lens.

Re: AI 2040 and the cult of intelligence

#198

Post by pornel »

JumpCrisscross wrote: post #169
mortenjorck wrote: post #66

Earlier quoted context omitted.

I would have been very hesitant to run the “just killed wife“ test given that ChatGPT will indeed flag your account and if a certain threshold is crossed, escalate to humans, who will presumably contact authorities to avoid this happening again: https://www.bbc.com/news/articles/cq6je7e80r7o Obviously a certain percentage of the user base is running lurid hypotheticals through the system all the time, but I don’t dou…

> ChatGPT will indeed flag your account Oh no, I'll have to spin up a new e-mail address...

And prove you're not a child by handing over your ID or selfie to a company that Values Your Privacy according to their price list.

Microsoft will snitch your Windows install GUID when you connect through a VPN.

If you make your connection too anonymous, Google's recaptcha will not let your through until you use your mobile phone to prove your humanity, which knows your phone number (which in many countries can't be obtained anonymously, you burner-phone terrorist).

Re: AI 2040 and the cult of intelligence

#199

Post by Aurornis »

beepbooptheory wrote: post #196
Aurornis wrote: post #180

Earlier quoted context omitted.

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments. There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments. It all…

But like.. How can this remain a satisfying or even helpful position for you? It kinda feels The State is but one of a set of concerns I could have in this world. And the lack of one, in some contexts, is also a concern. Like, I wanna know if there is going to be mudslide, or that air traffic controllers are not overworked, that the car that I buy is not going to blow up, if I will be able to buy food tomorrow. It ju…

I addressed this in my comment:

> You don’t write the laws and regulations for a perfectly faultless government that doesn’t exist. You write them for the governments as they do exist, which means they must be written first in a way that makes them resilient to bad governments abusing them.

The answer isn't "Don't regulate anything!". The answer needs to be an approach to regulation that restrains the government from over-reach.

Authoritarian and fascists governments throughout history have taken advantage of the desire for safety and predictability to rise to power. They never let an emergency or concern go to waste to expand their powers, which almost never retract once the initial societal panic is gone.

Re: AI 2040 and the cult of intelligence

#200

Post by piloto_ciego »

ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

I'm wildly optimistic about the direction this is going to go, but my one dystopia idea that crops up occasionally is a "smart social credit system" that lives kind of under the hood of society.

Once everything is more or less modulated by AI, suppose the technology gets good enough to identify a potential dissident in grade school. Then to keep them from going into the mountains with their friends and rifles, you make it so getting ahead is hard, but not impossible. They can "make it" - but only just. So they spend their time peddling harder and harder to no avail.

They apply to jobs, but jobs to "good" companies go into the circular file and the HR person never sees them. They are only able to work the most time consuming, frustrating, exhausting jobs, and they always kind of make it, but not really comfortably. They're never too comfortable where they have time to think about what's broken in society, but they're never so uncomfortable that they have nothing left to live for.

Post reply on HN