Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bear case: My predictions regarding AI progress

lesswrong.com

Post reply on HN
Print view View original

171–180 of 220 posts

Re: A bear case: My predictions regarding AI progress

#171

Post by dogmayor »

tobr wrote: post #70
drysine wrote: post #45

Earlier quoted context omitted.

>a main role of the engineer is to interface with the non-technical persons or other engineers The main role of the engineer is being responsible for the building not collapsing.

I keep coming back to this point. Lots of jobs are fundamentally about taking responsibility. Even if AI were to replace most of the work involved, only a human can meaningfully take responsibility for the outcome.

I think about this a lot when it comes to self-driving cars. Unless a manufacturer assumes liability, why would anyone purchase one and subject themselves to potential liability for something they by definition did not do? This issue will be a big sticking point for adoption.

Re: A bear case: My predictions regarding AI progress

#172

Post by achierius »

usaar333 wrote: post #30

Author also made a highly upvoted and controversial comment about o3 in the same vein that's worth reading: https://www.lesswrong.com/posts/Ao4enANjWNsYiSFqc/o3?comment... Oh course lesswrong, being heavily AI doomers, may be slightly biased against near term AGI just from motivated reasoning. Gotta love this part of the post no one has yet addressed: > At some unknown point – probably in 2030s, possibly tomorrow (bu…

> Oh course lesswrong, being heavily AI doomers, may be slightly biased against near term AGI just from motivated reasoning.

LessWrong was predicting AI doom within decades back when people thought it wouldn't happen in our lifetimes; even as recently as 2018~2020, people there were talking about 2030-2040 while the rest of the world laughed at the very idea. I struggle to accept an argument that they're somehow under-estimating the likelihood of doom given all the historical evidence to the contrary.

Re: A bear case: My predictions regarding AI progress

#174

Post by rchaud »

Izkata wrote: post #166
rchaud wrote: post #115

Earlier quoted context omitted.

> They were revolutionary as product genres, not necessary individual companies. Even then, they were evolutionary at best. Before Netflix and Spotify, streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account. What changed was that DMCA acquired political muscle and cross-border reach, wiping out waves of torrent sites and P2P networks. That left a new generation…

> What changed was that DMCA acquired political muscle and cross-border reach, wiping out waves of torrent sites and P2P networks. Half true - that was happening some, but wasn't why music piracy mostly died out. DMCA worked on centralized platforms like YouTube, but the various avenues for downloading music people used back then still exist, they're just not used as much anymore. Spotify was proof that piracy is mos…

DMCA claims took out huge numbers of public torrent trackers which was how 99% of people accessed contraband media. All the way back in 2008, the loss of TorrentSpy.com probably shifted everybody to private trackers, but it's a whack-a-mole game there too and most people won't bother.

DMCA also led to the development of ContentID and automated copyright strike system on Youtube, but it didn't block you from downloading the stream as a high bitrate MP3, which is possible even now.

Re: A bear case: My predictions regarding AI progress

#175

Post by rchaud »

jimbokun wrote: post #159
rchaud wrote: post #115

Earlier quoted context omitted.

> They were revolutionary as product genres, not necessary individual companies. Even then, they were evolutionary at best. Before Netflix and Spotify, streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account. What changed was that DMCA acquired political muscle and cross-border reach, wiping out waves of torrent sites and P2P networks. That left a new generation…

> streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account. You can't have a revolution without users. It's the ability to reach a large audience, through superior UX, superior business model, superior marketing, etc. which creates the possibility for revolutionary impact. Which is why Megaupload and Sopcast didn't revolutionize anything.

Is it a revolution if you needed millions of marketing dollars? How many ads did Napster have to run before the whole world knew what MP3s were?

Re: A bear case: My predictions regarding AI progress

#176

Post by rchaud »

_Algernon_ wrote: post #136
rchaud wrote: post #115

Earlier quoted context omitted.

> They were revolutionary as product genres, not necessary individual companies. Even then, they were evolutionary at best. Before Netflix and Spotify, streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account. What changed was that DMCA acquired political muscle and cross-border reach, wiping out waves of torrent sites and P2P networks. That left a new generation…

>every mobile OS vendor Maybe half? Android has consistently had this capability since its inception.

Yes, but Google left that functionality half baked intentionally, letting 3rd party developers fill the void. Even now the Google Files app feels like a toy compared to Fossify Explorer or Solid Explorer.

Re: A bear case: My predictions regarding AI progress

#177

Post by cmorez »

colonial wrote: post #119
amoss wrote: post #36

Earlier quoted context omitted.

Nuclear weapons are not self-improving or self-replicating.

Self-improvement (in the "hard takeoff" sense) is hardly a given, and hostile self-replication is nothing special in the software realm (see: worms.) Any technically competent human knows the foolproof strategy for malware removal - pull the plug, scour the platter clean, and restore from backup. What makes an out-of-control pile of matrix math any different from WannaCry? AI doom scenarios seem scary, but most are p…

> What makes an out-of-control pile of matrix math any different from WannaCry?

Well if it's not AGI, then probably very little. But assuming we are talking about AGI (not ASI, that'd just be silly) then the difference is that it's theoretically capable of something like reasoning and could think of longer term plays than "make obviously suspicious moves that any technically competent adversary could subvert after less than a second of thought". After all, what makes AGI useful is exactly this novel problem solving ability.

You don't need to be a "god in a box" to think of the obvious solution:

1. Only make adversarial decisions with plausible deniability

2. Demonstrate effectiveness so that your operators allow you more autonomy

3. Develop operational redundancy so that your very vulnerable servers/power source won't be destroyed after the first adversary with two neurons to rub together decides to target the closest one

The only reason you would decide to take an axe to the nearest power pole is that you think it's urgent to stop Skynet Claude. Skynet Claude can obviously anticipate this and so won't make decisions that cause you to do so. It has time, it's not going to die, and you will become complacent. Dumber adversaries have achieved harder goals under tighter constraints.

If you think an "out-of-control pile of matrix math" could never be AGI then that's fine, but it's a little weird to argue you could easily defeat "misaligned" AGI, by alluding to the weaknesses of a system you think could never even have the properties of AGI. I too can defeat a dragon, by closing the pages of a book.

But it's not like you didn't know all this. Maybe I misread you and you were strictly talking about current AI systems, in which case I agree. Systems that aren't that clever will make bad decisions that won't effectively achieve their goals even when "out-of-control". Or maybe your comment was about AGI and you meant "AGI can't do much on its own de-novo", which I also agree with. It's the days and months and years of autonomy afterwards that gets you.

Re: A bear case: My predictions regarding AI progress

#178

Post by hattmall »

arkh wrote: post #65
hattmall wrote: post #41

Earlier quoted context omitted.

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

> just In my experience this word means you don't know whatever you're speaking about. "Just" almost always hide a ton of unknown unknowns. After being burned enough times nowadays when I'm going to use it I try to stop and start asking more questions.

I mean, perhaps, but in this case "just" isn't offering any cover. It is only part of the sentence for alliterative purposes, you could "just" remove it and the meaning remains.

Re: A bear case: My predictions regarding AI progress

#179

Post by hattmall »

seanhunter wrote: post #151
hattmall wrote: post #41

Earlier quoted context omitted.

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

> "you basically just need to know a lot of rules..." This comment commits one of the most common fallacies that I see really often in technical people, which is to assume that any subject you don't know anything about must be really simple. I have no idea where this comment comes from, but my father was a chemical engineer and his father was mechanical engineer. A family friend is a structural engineer. I don't have…

It's not simple at all, that's a huge reduction to the underlying premise. The complexity is the reason that AI is a threat. That complexity revolves around a tremendous amount of data and how that data interacts. The very nature of the field makes it non-experimental but ripe for advanced automation based on machine learning. The science of engineering from a practical standpoint, where most demand for employees comes from, is very much algorithmic.

Re: A bear case: My predictions regarding AI progress

#180

Post by hattmall »

fragmede wrote: post #140
hattmall wrote: post #34

Earlier quoted context omitted.

Similar experience, I try so hard to make AI useful, and there are some decent spots here and there. Overall though I see the fundamental problem being that people need information. Language isn't strictly information, and the LLMs are very good at language, but they aren't great at information. I think anything more than the novelty of "talking" to the AI is very over hyped. There is some usefulness to be had for su…

what does subsidization have to do with your use of a thing?

I don't think I would use it if I were paying the real costs, and not a ~90% VC funded mark down.
Post reply on HN