Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intelligence Is Not the Main Bottleneck

writingruxandrabio.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Intelligence Is Not the Main Bottleneck

#71

Post by giancarlostoro »

zone411 wrote: post #50
mister_mort wrote: post #37

Earlier quoted context omitted.

This has been a core belief of the EA/LessWrong/Rationalist people for many years - that above a certain level of intelligence you effectively get hypnotic powers, and people will obey AIs no matter what. So they said you can't keep a superpowered AI airgapped as merely offering a chat interface will immediately take control of the next person to interact with it and they will deliberately compromise the airgapping.

A complete mischaracterization, as usual for HN lately when discussing AI or LessWrong. Obviously, even average levels of persuasion are enough to convince some people. And nobody is air-gapping AI.

> And nobody is air-gapping AI.

https://genai.mil/

I mean... I would hope that AI used for military needs is not deployed in the public internet.

Re: Intelligence Is Not the Main Bottleneck

#72

Post by dsign »

MerrimanInd wrote: post #58

Climate change is another big problem that frontier AI labs have handwaved away as being "solvable" by AGI. The premise that a pure superintelligence could hand us a plan to beat climate change that's so good and airtight that A) it would definitely work and B) people would execute it. But there have been entire books written about the fact that right now we have the technology to drastically curtail climate change a…

> Both premises seem laughably unlikely to me.

Yeah, that's not gonna happen.

> The AGI-pilled must think one of two things;

I'm cynical and I believe that AI will be our undoing. In fact, when we let that genie out of the bottle, many other scenarios became possible. For example, here's one third scenario to your "two things": a sufficiently cynical[^1] and resourceful rogue AI may solve climate change by steamrolling over the "set of political, social, and systemic reasons" and teaching the monkeys a lesson about the real significance of very vast agency and intelligence. For example, it could manipulate people into an anti technological uprising and subsequent backward, anti-humanist ethos, while at the same time modifying one or two key ecosystems so that the Mediterranean would smell of cooked cabbage (dimethyl sulfide) and bring about a North African monsoon that, with sufficient time, would turn the Sahara into carbon-sequestering dense jungle...

[^1]: It takes one to know one.

Re: Intelligence Is Not the Main Bottleneck

#73

Post by samth »

Veedrac wrote: post #54

Indeed. Humans and not other apes reached the moon by our great proclivity to long distance running, through our exceptional sweat glands... ah, wait. How is it possible to live in the world around us and not see, apparent and transparent, that intelligence is the lever on which everything rests? Humans are surprisingly weird animals, but it's not our hairlessness or our fairly average propensity to violence that did…

How much more intelligence will it take to get people to be ok with data center construction? How much more intelligence will be needed to reduce overuse of antibiotics? How much more intelligence is necessary to enable free trade with Canada?

Re: Intelligence Is Not the Main Bottleneck

#74

Post by RandomLensman »

HardlyCognizant wrote: post #60
dgellow wrote: post #6

Earlier quoted context omitted.

I would go further and say that the human should be the bottleneck/blocker. If not designed for and to serve humans, which means to have human in the position of power and control and take in account all the various human oddities, why are we building all of this?

> why are we building all of this? To remove humans from power in all cases where our biases and animalistic nature would cause unnecessary suffering to others. As a species we are quickly approaching some very hard problems that no individual, or even small group, is capable of handling in a consistently rational manner. On our current trajectory of resource constraints and habitability constrictions, ethics by the…

What is rational? Who determines what is a true constraint vs a choice, etc.?

Re: Intelligence Is Not the Main Bottleneck

#76

Post by overgard »

mister_mort wrote: post #37
scuppernong wrote: post #30

"He looked at me incredulously. Surely, a smart person like me should know that AI, or better said, AGI will be hyperpersuasive soon – already on a bunch of benchmarks it exceeds professional debaters at persuasion." I'm always tickled by this tic of this kind of person, that human traits are something that we can maximize to infinity. That if we trained a robot to tell jokes, it would eventually tell jokes so funny…

This has been a core belief of the EA/LessWrong/Rationalist people for many years - that above a certain level of intelligence you effectively get hypnotic powers, and people will obey AIs no matter what. So they said you can't keep a superpowered AI airgapped as merely offering a chat interface will immediately take control of the next person to interact with it and they will deliberately compromise the airgapping.

It's kind of fascinating how toxic the EA/LessWrong/Rationalist stuff has become. I remember reading it back in the early days (~2010 or so) and finding a lot of good ideas mixed in with some dorkiness (Harry Potter fan fiction about rationalism..). It always seemed like a sort of harmless niche with some good ideas and some wacky ones. And yet now it's basically become a cultish religion for the ultrapowerful who use it to justify doing some really awful things.

Re: Intelligence Is Not the Main Bottleneck

#77

Post by xboxnolifes »

discreteevent wrote: post #2

Alan Kay said "A change in perspective is worth 80 IQ points,". A change in perspective is often brought about by communication or I/O. In my experience this often bears out. The people who communicate get more done than the clever person who doesn't get out into the problem domain. The limit is I/O not intelligence.

I'd even say (collective) intelligence is limited by I/O. It's why we talk so much about signal to noise ratios with regards to communication.

Re: Intelligence Is Not the Main Bottleneck

#78

Post by discreteevent »

Veedrac wrote: post #54

Indeed. Humans and not other apes reached the moon by our great proclivity to long distance running, through our exceptional sweat glands... ah, wait. How is it possible to live in the world around us and not see, apparent and transparent, that intelligence is the lever on which everything rests? Humans are surprisingly weird animals, but it's not our hairlessness or our fairly average propensity to violence that did…

It's very clear that we have plenty of intelligence but we are not acting in it. What intelligence are we missing that is needed to solve climate change? (our most pressing problem - which AI is exacerbating)

> We don't have more houses than chimpanzees because we're _strong_.

You're literally making the point that we already have intelligence. I'm not sure how this is supposed to be an argument for a shortage. To revise: The point you are arguing against is not that intelligence is not important but that it's not the bottleneck.

Re: Intelligence Is Not the Main Bottleneck

#79

Post by bitwize »

drivebyhooting wrote: post #5

I think the next bottleneck is embodiment. AI can lay the best laid plans, but who will execute it? I think a possible solution to the dual problems of AI embodiment and unemployment is humans becoming avatars for AI. Imagine one of the unskilled masses donning a headset and being guided through all manner of work? Roofing, ACHVAC, stocking shelves, etc… No need to train anybody.

As in the title character of Marshall Brain's story "Manna".

Or the AI could threaten humans into compliance, as in the 2008 film Eagle Eye.

Re: Intelligence Is Not the Main Bottleneck

#80

Post by bitwize »

Lutger wrote: post #23
drivebyhooting wrote: post #5

I think the next bottleneck is embodiment. AI can lay the best laid plans, but who will execute it? I think a possible solution to the dual problems of AI embodiment and unemployment is humans becoming avatars for AI. Imagine one of the unskilled masses donning a headset and being guided through all manner of work? Roofing, ACHVAC, stocking shelves, etc… No need to train anybody.

Robots being able to perform any task a human can do, physically, doesn't seem that far off. They would be powered by AI of course. The question is: do we still have to work? If not, who will take all the wealth? We thought the washing machine would have giving us so much free time, because it used to take a full day a week. Instead, we only got more frantic it seems and started putting in more labor. But what will h…

Appliances like the washing machine literally killed the market for "hired help" in the anglosphere. In-house servants used to be relatively common among the middle and upper classes; with the aid of electrical appliances, the time/money costs of doing the laundry, dishes, and so forth went down so far that only the very rich care to hire servants as a mark of status.
Post reply on HN