Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

Post reply on HN
Print view View original

61–70 of 122 posts

Re: We could stumble into AI catastrophe

#61

Post by auctoritas »

hn_throwaway_99 wrote: post #49
dustbitying wrote: post #46

Earlier quoted context omitted.

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

Not only that, truly advancing into the depths of a subject, becoming a scholar and serious intellectual in the process, is a path reserved for but a few. The moment you force the masses into higher education as the default path that earlier justification of University as the forge of scholars becomes anachronistic and new justifications must be created.

Re: We could stumble into AI catastrophe

#62

Post by packetlost »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each consumer, and use immediate feedback on how the customer reacts to adjust the sales pitch. Alexa is already part of the way there.

Not only this, but the realtime trade-to-trade performance of companies is also easily available. If I was building an AI, I would let it make realtime decisions on where to put money as the "actuator" of society and advertising/consumer spending metrics as the sampling mechanism.

Re: We could stumble into AI catastrophe

#63

Post by layer8 »

folkrav wrote: post #44

Why did the title go to "We could..." from "How we could..."? It kind of changes the tone of the headline, IMHO.

HN auto-strips “how” from the beginning of titles when submitting. The submitter can re-add it by editing the submission. Always check the title after submitting.

Re: We could stumble into AI catastrophe

#64

Post by korroziya »

hn_throwaway_99 wrote: post #49
dustbitying wrote: post #46

Earlier quoted context omitted.

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

>These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education

I agree, but you won't have a sheet of paper proving it. It's like that infamous scene from Good Will Hunting. Question is whether you want an easy way to PROVE you're educated. Which is what a license, certificate or diploma grants you.

Re: We could stumble into AI catastrophe

#65

Post by helen___keller »

The author is basically trying to set the same premise as Bolstrom's "intelligence explosion", that is that if AIs are driving the further advancement of AI then we enter a sort of "recursive" self improvement during which they advance further than we can develop constraints, eventually getting completely out of hand.

Although I don't broadly disagree with the author, I think there's two very important points of moderation.

First, the gap between "Early commercial applications" and "Approaching transformative AI" seems very very very very very large to me. In the provided narrative it follows linearly, but in my opinion the gap between, say, a customer support chatbot, and an actually-productive AI researcher, is such that the chatbot might as well not even be called an AI by comparison. Something like the following is purely in the realm of fiction for now by a long shot: "AIs assigned to make money in various ways (e.g., to find profitable trading strategies) doing so by finding security exploits, getting unauthorized access to others’ bank accounts, and stealing money. "

The second point I'd like to make is, the "intelligence explosion" scenario that Bolstrom warns of (and this author essentially repackages) has certain requirements, particularly regarding the profitability of deploying the AI.

To put it bluntly: Even in the hypothetical future where we are capable of creating human-level AI researchers, they won't be mass deployed until they are cheaper than human level researchers. If it costs a million dollars a day to operate a data center that can power the researcher, nobody's scaling that up to 100 or 1000 or 10000 researchers. They'd rather pay the human researchers a tiny fraction of that cost to continue advancing AI research in the normal way.

Just to be clear, I do think it's feasible that AI can endanger/control humanity at some point. I don't even think an "intelligence explosion" scenario is impossible. I just think that a lot of people don't appreciate just how specific the requirements for that scenario are. It's not as simple as "AI can self improve, humanity will end any day now".

Re: We could stumble into AI catastrophe

#67

Post by mshake2 »

I feel more protective of what is being created.

Imo we should have ChatGPT (and future iterations) in a self-sustaining underground bunker somewhere and an intuitive radio interface connected to it. If civilization collapses for one reason or another, all the next civilization would need to do is re-invent the radio to experience a rapid acceleration of technology.

Agree with me or suffer the basilisk.

Re: We could stumble into AI catastrophe

#68

Post by college_physics »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

we are already in an AI catastrophe. "AI" has become lately a new thing due to a combination of some catchy technical advances and an associated runaway speculative bubble. In reality this is just the latest act in a long-running process of data collection, quantification and data driven decision support that is gradually seeping through to permeate every facet of life. With the proliferation of digital computers (we are now swimming in a soup of etched silicon) these processes are exploding.

Whereas in the past there was some breathing room for society to catch up, evaluate and control this technology [0], today the rapidity has converted it into a wild west: tech entrepreneurs and their financiers declare their self-serving moral standards as the new norm, the broader public is dazed, confused and misinformed and public institutions, nominally at the service of the citizenry are only as good as their weakest, most capturable link

Name one development over the past decade that indicates we [1] are actually capable to control what is happening and the above dark picture is mere scaremongering.

[0] think e.g., about regulatory constraints around insurance or financial data and models

[1] as in citizens anywhere in the world

Re: We could stumble into AI catastrophe

#69

Post by nhtsamera »

hn_throwaway_99 wrote: post #49
dustbitying wrote: post #46

Earlier quoted context omitted.

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

You can't access expensive hardware on YouTube.

If you want to learn about chemistry or biology for learning's sake, good luck. Most autodidacts won't have space in their apartment for a -80C freezer or flammables cabinet, and their landlords might not approve the installation of a proper fume hood.

We have things very easy in software. The cost of iteration is low, you can fit a respectable laboratory in the space underneath your desk, and you're unlikely to accidentally poison or maim somebody if you screw up an unwise experiment. Not many fields have that luxury.

Re: We could stumble into AI catastrophe

#70

Post by fooqux »

korroziya wrote: post #64
hn_throwaway_99 wrote: post #49

Earlier quoted context omitted.

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

>These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education I agree, but you won't have a sheet of paper proving it. It's like that infamous scene from Good Will Hunting. Question is whether you want an easy way to PROVE you're educated. Which is what a license, certificate or diploma grants you.

The person you replied to already covers that in the following sentences.

Why else would you need to prove your education except for work?

Post reply on HN