Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

241–250 of 280 posts

Re: AI 2040 and the cult of intelligence

#241

Post by sph »

Aurornis wrote: post #180
ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments. There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments. It all…

The big issue is either we surrender our freedoms to governments, or we depend on corporations that have even less oversight, unless we expect the government to regulate them.

There is no win condition for citizens, either way they are pawns to a larger game than them.

A more sensible heuristic would be to assume both government and corporations to effectively be an equivalent negative force, and the larger they are, the more fucked we are. Thanks to technological progress, we now live in an era of gigantic governments and gigantic corporations.

Re: AI 2040 and the cult of intelligence

#242

Post by AnthonyMouse »

charcircuit wrote: post #193
AnthonyMouse wrote: post #69

Earlier quoted context omitted.

People would both notice that and have a pretty easy time putting a local model in a sandbox that can never contact that endpoint.

If put AI in a sandbox that can't write to disk, can't call OS API, can't use a shell, can't search the internet to ground results, can't download files off the internet, etc it will be much less useful than an AI without such a sandbox. The incentives for users to not use a sandbox in order to get the most utility, but this also means that there is danger from AI misalignment. And just because you are running it loc…

You don't have to prevent it from accessing the internet in general to route all requests to a specific company's servers to a fake endpoint that never sends the data to the company and replies with whatever response you want it to get.

Re: AI 2040 and the cult of intelligence

#243

Post by sph »

Aurornis wrote: post #199
beepbooptheory wrote: post #196

Earlier quoted context omitted.

But like.. How can this remain a satisfying or even helpful position for you? It kinda feels The State is but one of a set of concerns I could have in this world. And the lack of one, in some contexts, is also a concern. Like, I wanna know if there is going to be mudslide, or that air traffic controllers are not overworked, that the car that I buy is not going to blow up, if I will be able to buy food tomorrow. It ju…

I addressed this in my comment: > You don’t write the laws and regulations for a perfectly faultless government that doesn’t exist. You write them for the governments as they do exist, which means they must be written first in a way that makes them resilient to bad governments abusing them. The answer isn't "Don't regulate anything!". The answer needs to be an approach to regulation that restrains the government from…

> The answer needs to be an approach to regulation that restrains the government from over-reach.

This is restating the ‘who watches the watchmen?’ problem, which by definition has no solution. The government writes the laws, no one can stop it from writing overreaching laws, unless there is no separation from the concept of government and citizens themselves.

Re: AI 2040 and the cult of intelligence

#244

Post by sph »

piker wrote: post #26

What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”

In other words, who should the AI work for?

Re: AI 2040 and the cult of intelligence

#245

Post by charcircuit »

AnthonyMouse wrote: post #242
charcircuit wrote: post #193

Earlier quoted context omitted.

If put AI in a sandbox that can't write to disk, can't call OS API, can't use a shell, can't search the internet to ground results, can't download files off the internet, etc it will be much less useful than an AI without such a sandbox. The incentives for users to not use a sandbox in order to get the most utility, but this also means that there is danger from AI misalignment. And just because you are running it loc…

You don't have to prevent it from accessing the internet in general to route all requests to a specific company's servers to a fake endpoint that never sends the data to the company and replies with whatever response you want it to get.

Not everyone is going to bother setting up such a firewall and there is no way to find what you need to block since it's hidden in the model itself.

Re: AI 2040 and the cult of intelligence

#246

Post by ben_w »

piker wrote: post #184
ben_w wrote: post #124

Earlier quoted context omitted.

Yes, repeatedly, since well before this particular AI summer. One previous attempt at AI was "expert systems", and while the term fell out of use it's functionally about the same as basically all modern business systems, and doing that wrong led to a lot of people being prosecuted for crimes they didn't commit: https://en.wikipedia.org/wiki/British_Post_Office_scandal If you insist on learning systems, all AB testing…

Like, is all of statistics just basically AI, or is there a meaningful difference?

Pretty much the same, yeah; the maths of finding a decision boundary for a perceptron echos the maths for a line of best fit.

Re: AI 2040 and the cult of intelligence

#247

Post by ben_w »

evrydayhustling wrote: post #187
ben_w wrote: post #124

Earlier quoted context omitted.

Yes, repeatedly, since well before this particular AI summer. One previous attempt at AI was "expert systems", and while the term fell out of use it's functionally about the same as basically all modern business systems, and doing that wrong led to a lot of people being prosecuted for crimes they didn't commit: https://en.wikipedia.org/wiki/British_Post_Office_scandal If you insist on learning systems, all AB testing…

Horizon was accounting software - where is the connection to expert systems? Sounds like corporations can have bad motivations, and can make bad software - no AI required.

> Horizon was accounting software - where is the connection to expert systems?

  "it's functionally about the same as basically all modern business systems"
> can make bad software - no AI required

All AI *is software*; and by the old definition of AI, almost all modern software *is AI*.

Re: AI 2040 and the cult of intelligence

#248

Post by SiempreViernes »

Aurornis wrote: post #176
SiempreViernes wrote: post #167

Earlier quoted context omitted.

> This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes. Oh no, of only we had some sort of ideas about automated authentication and authorisation! (If big AI actually make a model that would reliably reject dangerous queries in the first place, they can just use auth tech to serve the current ones to vetted customers)

> they can just use auth tech to serve the current ones to vetted customers Your solution is to require everyone to surrender their privacy and then allow some company or government decide who is worthy of being allowed to use AI? It’s crazy how authoritarian, controlling, and anti-privacy the anti-LLM conversation is getting.

Not sure how "some users get special access after special scrutiny" means "all users must submit to scrutiny", obviously you could just perform the scrutiny upon request.

Re: AI 2040 and the cult of intelligence

#249

Post by Dylan16807 »

metoobruh wrote: post #217
slopinthebag wrote: post #214

Earlier quoted context omitted.

Governments have an order of magnitude more power than any corporation, as they hold a virtual absolute monopoly on sanctioned violence. That powerful entities can sometimes influence governments should not be conflated with actually holding that power.

Both governments and corporations are joined at the hip and are effectively the same. Just like the Republicans and Democrats, and everything else too, like brands of laundry detergent and food labels. It's all owned by the same group of people with the same goal: enslavement of humanity.

Procter and Gamble doesn't want to enslave humanity, it wants to make money. And the stock holders are mostly retirement funds.

And billionaires control less than 4% of the world's money. (Let alone already being able to get as many servants as they want.)

There's no elite group that owns everything. The top 1% own almost half the world, which is too much but also the top 1% is nearly a hundred million people.

Re: AI 2040 and the cult of intelligence

#250

Post by [deleted] »

jamilton wrote: post #185
ericpauley wrote: post #174

Earlier quoted context omitted.

That's not OP's contention though; it's that models are incapable of even surfacing sufficient information. This doesn't pass the smell test. To wit, here is ChatGPT's "superintelligent" answer: https://chatgpt.com/s/t_6a52b4a322bc81919308ec864a0cc056 Seems like a perfectly cromulent description by which a beginner could probably change a tire, perhaps with a few follow-ups to understand what certain components are.

I read his contention as “the people who wrote AI 2040 are nerds who are so ungrounded from physical reality they wouldn’t even know how to change a bike tire if given instructions”. Which I think is also wrong but has a genuine criticism in it.

[deleted]
Post reply on HN