Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

Post reply on HN
Print view View original

71–80 of 122 posts

Re: We could stumble into AI catastrophe

#71

Post by idontpost »

korroziya wrote: post #64
hn_throwaway_99 wrote: post #49

Earlier quoted context omitted.

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education. The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

>These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education I agree, but you won't have a sheet of paper proving it. It's like that infamous scene from Good Will Hunting. Question is whether you want an easy way to PROVE you're educated. Which is what a license, certificate or diploma grants you.

[dead]

Re: We could stumble into AI catastrophe

#72

Post by zmmmmm »

Animats wrote: post #30
dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

At least corporations are currently limited by requiring humans who tend to have at least some limits to their actions in terms of moral, ethical considerations and can be physically constrained by imprisoning them etc.

An AI could be completely untethered from that and set new thresholds for evil corporate behaviour.

Re: We could stumble into AI catastrophe

#73

Post by yamtaddle »

Animats wrote: post #57
amanaplanacanal wrote: post #53

Earlier quoted context omitted.

We have been framing it that way for a long time though. How many times have you heard someone talk about the average income of high school graduates vs college graduates as a reason to go to college? I agree with you though. Education should be about personal enrichment, not personal “enrichment”.

We have been framing it that way for a long time though. Yes. That changed on February 28, 1967.[1][2] [1] https://www.chronicle.com/article/the-day-the-purpose-of-col... [2] https://archive.is/G11YT

Fussell (among others) blamed the GI Bill for a marked decline in the average quality of college education, and the diminution of what "college educated" meant (without further qualifiers).

The spike in demand drove a ton of "normal" schools (teachers' colleges) to become proper colleges and even universities, plus a bunch of new ones to pop up, but rather than increasing the supply of (previously) college-level education, it mostly created a kind of knock-off, lower-tier product—there just weren't enough excellent professors to meet demand, not enough good, experienced college administrators, not enough anything, including, arguably, students who were fit for college (as it had previously been) in the numbers that were now attending.

Re: We could stumble into AI catastrophe

#74

Post by pessimizer »

The AI catastrophe will be how our oligarchs will choose to use it. As far as I can tell, there are no moral qualms amongst the powerful against using it for total surveillance and continual automated policing (can't wait to get a phone call from an AI asking me about the content of a tweet.)

Star Cops thought about it more than our popular journalists or philosophers seem to think we should:

-----

  Michael: ...it was slightly sensitive material, and in my field one is aware of the vulnerability of carrier waves.

  Nathan: Is that what you do here?

  Michael: It wouldn't be wise to read too much into a casual remark, Commander.

  Nathan: That's your principal research area here, isn't it? Communications monitoring.

  David: Not more buggers and supersniffers...

  Nathan: More impressive than that. We've always been able to hear more than we can listen to. Are you developing intelligent listening systems?

  David: Sounds impressive.

  Michael: I must suggest that to go on could be a foolish risk.

  David: Threats, Doctor?

  Michael: Observation, merely.

  Nathan: Machines that listen to everything and decide for themselves what's worth passing on? Is that where your BHG information came from? Did you do a test running? Did one of your new computers pick it up from the babble of all the world?

  Michael: Commander, it's a pleasure meeting you.

  David: Well, tireless attention to every word spoken? No possibility of human error. You can tell it to listen to the bad guys, it'll listen to everyone and identifies the ones you [won't like.]
Star Cops, Episode 3 [1987]

-----

https://youtu.be/G3RVPQyMCoY?t=713

Re: We could stumble into AI catastrophe

#75

Post by yamtaddle »

dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

> An important point is that AI does not need to be sentient or conscious to do a lot of damage.

It's not 100% clear that consciousness is necessary even for things like creative problem solving or goal-setting (i.e. choosing which problems to solve). That is, advanced intelligences that can do all we can and more—and could even "decide", in the worst case, that they'd be better off without humans—may not need consciousness as a precondition for that.

Cf. Peter Watts' Blindsight

http://www.rifters.com/real/Blindsight.htm

Re: We could stumble into AI catastrophe

#76

Post by yamtaddle »

DerekBickerton wrote: post #5

AI will always need humans in the loop to intervene. Remember the old IBM motto: 'A computer isn't accountable so a computer should never make a management decision'.

Noooooo. Absolutely not. The second it's a significant advantage in business or war to hand the reigns over to AI, no human in the loop, it will happen. Because the side that doesn't do it will lose, badly.

Re: We could stumble into AI catastrophe

#77

Post by Joeri »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

I’m expecting even worse: heavily weaponized political influencing by AI bots. Brexit was won in 2016 by microtargeting ads on facebook based on individual profiles, and since then this has become a common strategy. Think of what a game changer it will be to have convincing AI bots engaging one on one with people online to give them exactly the arguments needed to make them vote the right way. The candidate with the best AI team is all but guaranteed to win.

This is not a matter of “if”, it is a matter of “when”.

Re: We could stumble into AI catastrophe

#78

Post by yamtaddle »

Joeri wrote: post #77
Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

I’m expecting even worse: heavily weaponized political influencing by AI bots. Brexit was won in 2016 by microtargeting ads on facebook based on individual profiles, and since then this has become a common strategy. Think of what a game changer it will be to have convincing AI bots engaging one on one with people online to give them exactly the arguments needed to make them vote the right way. The candidate with the…

We're gonna have to choose between a free, semi-anonymous, international communication network, and democracy. Probably very soon.

I've long suspected China had the right idea here, and am, in this case, not happy to be proven right.

Re: We could stumble into AI catastrophe

#79

Post by haswell »

fooqux wrote: post #70
korroziya wrote: post #64

Earlier quoted context omitted.

>These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education I agree, but you won't have a sheet of paper proving it. It's like that infamous scene from Good Will Hunting. Question is whether you want an easy way to PROVE you're educated. Which is what a license, certificate or diploma grants you.

The person you replied to already covers that in the following sentences. Why else would you need to prove your education except for work?

In a hypothetical post-work era where people no longer build their reputation via career, it’s not hard to imagine group dynamics emerging such that education level is no longer a means to an end, but an end unto itself.

Essentially a new way to establish hierarchies.

Re: We could stumble into AI catastrophe

#80

Post by bumby »

dustbitying wrote: post #46
Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

>college is not a 'factory' (or any sort of industry) that 'manufactures' workers

Unfortunately, the vocation focus of college has been a driving force since the early days of the industrial revolution, dating to the 1850s. The Morrill Land-Grant Act explicitly states that colleges (at least in the case of land grant schools) are there to train people for work:

>without excluding other scientific and classical studies and including military tactic, to teach such branches of learning as are related to agriculture and the mechanic arts, in such manner as the legislatures of the States may respectively prescribe, in order to promote the liberal and practical education of the industrial classes in the several pursuits and professions in life.

I respect and value the other traditional purpose of college, but we can also acknowledge that these purposes can change as society does.

Post reply on HN