Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

201–210 of 253 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#201

Post by mbgerring »

antonvs wrote: post #195
mbgerring wrote: post #63

Earlier quoted context omitted.

Physics allows this, and actually taking advantage of it requires billions of dollars of unprecedented infrastructure buildout that is already destabilizing the power grid . The only reason that infrastructure buildout is happening at all is the ideological capture of a small handful of obscenely wealthy people, who are fueling this buildout by spreading the extreme paranoia you’re echoing here. I do not understand w…

> There is nothing inevitable about tying up all of this productive capital in the pursuit of AI. In the US capitalist context, it's certainly inevitable, because AI is the biggest and most attractive source of profit and power out there right now. In that context, the broad strokes of what's happening currently, including the financial bubble, are predictable and inevitable. What are the concrete steps which would a…

- Legislation

- Labor organizing

I grew up in a union household, and my dad and my grandfather fighting for better wages, healthcare and working conditions are the reason why I got a good education and work in Silicon Valley surrounded by Stanford assholes.

All of us who work for a paycheck can get together and say, “no, we will not allow you to record keystrokes and mouse movements to train our replacements. No, we will not have our performance or future employment based on an AI leaderboard.”

Previous generations fought and died for our right to do that, but in 2026 we just sit on our hands and complain on this forum. We can and should do better.

The U.S. is absolutely on fire right now with opposition to data centers. We, collectively, can extract concessions or ban their construction altogether.

These things aren’t “easy”. They are also eminently possible.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#202

Post by famouswaffles »

fragmede wrote: post #160
emtel wrote: post #155

Earlier quoted context omitted.

How is drugging an animal's food a "brute force tactic"? What would qualify as a non-brute-force tactic in your mind?

Learning to speak cat and then reasoning with the cat, in cat-ese, to get into the carrier of its own volition.

There is no cat-ese. Cats do not have language.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#203

Post by famouswaffles »

jjulius wrote: post #158
emtel wrote: post #155

Earlier quoted context omitted.

How is drugging an animal's food a "brute force tactic"? What would qualify as a non-brute-force tactic in your mind?

Well now we're just ignoring the point for the sake of a semantics debate.

It's just a poor point. Cats are a bad analogy anyhow because there is no cat language. You can't actually speak to them or have a drawn out discussion. But you can with humans and many have inspired thousands or even millions to action doing just that, no super-intelligence required.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#204

Post by pixl97 »

CGMthrowaway wrote: post #198
My_Name wrote: post #174

I note for premise 2 that the people who doubt the premise are the very stupid, but more importantly the very smart. The former is no challenge to the premise, but the latter? That is a different story. EDIT : For S&G I asked Claude about it. It replied : The talk groups Penrose with the religious doubter, as if the two objections were the same species and could be dispatched by the same gesture: most of us find this…

Idk whether I fall into the former or latter bucket, but "smarter minds see further, judge better" and "there is a soul that taps into a cosmic godhead" are one and the same statement to me

So you think "smarter minds see further, judge better" == "there is a soul that taps into a cosmic godhead"

Good golly, that's the silliest statement completely ignoring that our ancestors wiped most large mammals off the planet by seeing further and judging better by using tools, traps, and the environment around them because of their larger brain size.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#205

Post by pixl97 »

optimalsolver wrote: post #187
famouswaffles wrote: post #95

Earlier quoted context omitted.

Yeah the whole thing is strange. He does such a good job in the first bit outlining all the reasons for caution that I was intrigued to see what he had to say against it, but it's just one bad argument after another. Even Hawking's cat is bad. Make some money and pay someone to get the cat in the box.

I bet Schrödinger could get a cat into a box.

Yea, but getting out alive is the hard part.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#206

Post by pixl97 »

jokoon wrote: post #185

This is just wishful thinking We don't have a good definition of intelligence Also, the premise that this thing would take over, it's hard to reason why it would do so Anthropocentrism is also problematic in this article.

> the premise that this thing would take over, it's hard to reason why it would do so

I mean, it's not very hard to reason why it would at all.

Think about the things we're already using LLMs for, computer security being a big one. Being a defender is difficult, you have to cover all your bases. Preemptive attacks on those that would attack you can be effective. The same goes for all the military uses of AI that are already occurring now.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#207

Post by TremendousJudge »

Jach wrote: post #161

This was such a terrible piece of work back when it came out, it hasn't aged any better. By the way, it was responded to by MIRI: https://intelligence.org/2017/01/13/response-to-ceglowski-on...

the comment about MIRI being a doomsday cult was exactly right. yudkowsky lost what little credibility he had left when he argued for the bombing of datacenters. as is (correctly, imo) argued by the essay, reading what they have to say is like reading arguments by heaven's gate

Re: Superintelligence: The Idea That Eats Smart People (2016)

#208

Post by altruios »

slopinthebag wrote: post #135
altruios wrote: post #110

Earlier quoted context omitted.

What do even mean by 'soul'? What do you mean by 'supernatural' - and (assuming your definition is the standard one of 'not detectable by any measurement') by what mechanism that could possibly affect physical matter? (the onus is on you to prove the positive claim that there exists the supernatural or soul to begin with, which there is currently no evidence for). The concept is self defeating by its own definition,…

Not OP but I'll take a shot since I have somewhat similar sounding views. (I assume OP is talking about consciousness and it's origins when they use the term "soul"). > What do you mean by 'supernatural' I would just say something outside our current capacity for understanding. How does that quote go...something like "sufficiently advanced technology is indistinguishable from magic". "Not detectable by any measuremen…

> I would just say something outside our current capacity for understanding. How does that quote go...something like "sufficiently advanced technology is indistinguishable from magic"

So your definition is merely that the supernatural is the natural we have not been able to measure yet? That's just the 'god of the gaps' by a different name.

> isn't right because we clearly detect it in some way since we are discussing it now.

We could be discussing invisible pink unicorns. so those must be real since we are able to discuss them, right? (obviously not: the same reasoning holds true for why the soul {probably} doesn't exist).

> In this domain the idea of "proof" becomes less relevant.

This is counter to your earlier stance that the supernatural is the natural we haven't yet been able to measure. either proof exists or you have to accept things on faith: you can't have it both ways. this is poor reasoning.

> My guess is our brains act as a receiver for some "field" of consciousness

Not impossible, and could in theory be testable and falsifiable.

There is a lot of conflicting thinking here. Very muddy.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#209

Post by pixl97 »

jandrese wrote: post #30
blamestross wrote: post #6

AI Superintelligence doesn't scare me for the same reasons "grey goo" doesn't scare me. We are awash in self-replicating machines. The biosphere is already a grey-goo apocalypse. Any new competitors have a serious moat to cross to out compete any existing self-replicators. We are awash in intelligent agents. Our society (and meta society) is full of superhuman agents already. There is a huge moat for any new intellig…

I have the same feeling. I'm not worried about superintelligent AI because we are only training them on human level intelligence. By what mechanism does our current AI technology take the leap to technologies that humans have never conceived of? Our current AI is more like a fancy Google search than some kind of machine God.

>By what mechanism does our current human brains take the leap that other mammals have never conceived of?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#210

Post by antonvs »

mbgerring wrote: post #201
antonvs wrote: post #195

Earlier quoted context omitted.

> There is nothing inevitable about tying up all of this productive capital in the pursuit of AI. In the US capitalist context, it's certainly inevitable, because AI is the biggest and most attractive source of profit and power out there right now. In that context, the broad strokes of what's happening currently, including the financial bubble, are predictable and inevitable. What are the concrete steps which would a…

- Legislation - Labor organizing I grew up in a union household, and my dad and my grandfather fighting for better wages, healthcare and working conditions are the reason why I got a good education and work in Silicon Valley surrounded by Stanford assholes. All of us who work for a paycheck can get together and say, “no, we will not allow you to record keystrokes and mouse movements to train our replacements. No, we…

I think you're in denial about the reality of the situation.

The reality is that the US has been a story of increasing concentration of wealth and power. The people who "fought and died" bought some important (from a human rights perspective) but ultimately minor (from a capitalist perspective) concessions from the capital class. The battle you describe is one of defense of rights, not of gaining control.

The overall system of capitalist control remains unaffected, and it's why the buildout of AI is, in fact, inevitable under the current system.

You're essentially saying no, you want public control of the means of production instead. That might be great, if even a sizable fraction of the US population agreed with you. But due in no small part to decades of propaganda, they don't.

Post reply on HN