Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bear case: My predictions regarding AI progress

lesswrong.com

Post reply on HN
Print view View original

151–160 of 220 posts

Re: A bear case: My predictions regarding AI progress

#151

Post by seanhunter »

hattmall wrote: post #41
cglace wrote: post #32

The thing I can't wrap my head around is that I work on extremely complex AI agents every day and I know how far they are from actually replacing anyone. But then I step away from my work and I'm constantly bombarded with “agents will replace us”. I wasted a few days trying to incorporate aider and other tools into my workflow. I had a simple screen I was working on for configuring an AI Agent. I gave screenshots of…

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

   > "you basically just need to know a lot of rules..."
This comment commits one of the most common fallacies that I see really often in technical people, which is to assume that any subject you don't know anything about must be really simple.

I have no idea where this comment comes from, but my father was a chemical engineer and his father was mechanical engineer. A family friend is a structural engineer. I don't have a perspective about AI replacing people's jobs in general that is any more valuable than anyone elses, but I can say with a great deal of confidence that in those three engineering disciplines specifically literally none of any of their jobs are about knowing a bunch of rules and best practices.

Don't make the mistake of thinking that just because you don't know what someone does, that their job is easy and/or unnecessary or you could pick it up quickly. It may or may not be true but assuming it to be the case is unlikely to take you anywhere good.

Re: A bear case: My predictions regarding AI progress

#152

Post by immibis »

HelloMcFly wrote: post #94
immibis wrote: post #86

Earlier quoted context omitted.

"Do something existing with a different mechanism" is innovative, but not revolutionary, and certainly not a new "product genre". My parents used to order pizza by phone calls, then a website, then an app. It's the same thing. (The friction is a little bit less, but maybe forcing another human to bring food to you because you're feeling lazy should have a little friction. And as a side effect, we all stopped being as…

> innovative, but not revolutionary The experience of Netflix, Spotify, and Uber were revolutionary. It felt like the future, and it worked as expected. Sure, we didn't realize the poison these products were introducing into many creative and labor ecosystems, nor did we fully appreciate how they would operate as means to widen the income inequality gap by concentrating more profits to executives. But they fit cleanl…

Making simple, small improvements feel revolutionary is good marketing.

Re: A bear case: My predictions regarding AI progress

#153

Post by dsign »

colonial wrote: post #119
amoss wrote: post #36

Earlier quoted context omitted.

Nuclear weapons are not self-improving or self-replicating.

Self-improvement (in the "hard takeoff" sense) is hardly a given, and hostile self-replication is nothing special in the software realm (see: worms.) Any technically competent human knows the foolproof strategy for malware removal - pull the plug, scour the platter clean, and restore from backup. What makes an out-of-control pile of matrix math any different from WannaCry? AI doom scenarios seem scary, but most are p…

You have a point that a powerful malicious AI can still be unplugged, if you are close to each and every power cord that would feed it, and react and do the right thing each and every time. Our world is far too big and too complicated to guarantee that.

Re: A bear case: My predictions regarding AI progress

#154

Post by lackoftactics »

ath3nd wrote: post #148
lackoftactics wrote: post #116

Earlier quoted context omitted.

I appreciate the pushback and acknowledge that my earlier comment might have conveyed too much certainty—skepticism here is justified and healthy. However, I'd like to clarify why optimism regarding AGI isn't merely wishful thinking. Historical parallels such as heavier-than-air flight, Go, and protein folding illustrate how sustained incremental progress combined with competition can result in surprising breakthroug…

I am with you that when smart people combine their efforts together and build on previous research + learnings, nothing is impossible.

I started the conversation off on the wrong foot. Commenting with “ad hominem” shuts down open discussion.

I hope we can have a nice talk in future conversations.

Re: A bear case: My predictions regarding AI progress

#155

Post by colonial »

dsign wrote: post #153
colonial wrote: post #119

Earlier quoted context omitted.

Self-improvement (in the "hard takeoff" sense) is hardly a given, and hostile self-replication is nothing special in the software realm (see: worms.) Any technically competent human knows the foolproof strategy for malware removal - pull the plug, scour the platter clean, and restore from backup. What makes an out-of-control pile of matrix math any different from WannaCry? AI doom scenarios seem scary, but most are p…

You have a point that a powerful malicious AI can still be unplugged, if you are close to each and every power cord that would feed it, and react and do the right thing each and every time. Our world is far too big and too complicated to guarantee that.

Again, that's the "god in a box" premise. In the real world, you wouldn't need a perfectly timed and coordinated response, just like we haven't needed one for human-programmed worms.

Any threat can be physically isolated case-by-case at the link layer, neutered, and destroyed. Sure, it could cause some destruction in the meantime, but our digital infrastructure can take a lot of heat and bounce back - the CrowdStrike outages didn't destroy the world, now did they?

Re: A bear case: My predictions regarding AI progress

#156

Post by red-iron-pine »

_huayra_ wrote: post #92
csomar wrote: post #22

> LLMs still seem as terrible at this as they'd been in the GPT-3.5 age. Software agents break down once the codebase becomes complex enough, game-playing agents get stuck in loops out of which they break out only by accident, etc. This has been my observation. I got into Github Copilot as early as it launched back when GPT-3 was the model. By that time (late 2021) copilot can already write tests for my Rust function…

Ultimately, every AI thing I've tried in this era seems to want to make me happy, even if it's wrong, instead of helping me. I describe it like "an eager intern who can summarize a 20-min web search session instantly, but ultimately has insufficient insight to actually help you". (Note to current interns: I'm mostly describing myself some years ago; you may be fantastic so don't take it personally!) Most of my intera…

interns can generally also tell me "tbh i have no damn idea", while AI just talks out it's virtual ass, and I can't read from it's voice or behavior that maybe it's not sure.

interns can also be clever and think outside the box. this is mostly not good, but sometimes they will surprise you in a good way. the AI by definition can only copy what someone else has done.

Re: A bear case: My predictions regarding AI progress

#157

Post by zero_bias »

nyrikki wrote: post #150
danielbln wrote: post #78

Earlier quoted context omitted.

We don't know what exactly makes us humans as intelligent as we are. And while I don't think that LLMs will be general intelligent without some other advancements, I don't get the confident statements that "clearly pattern matching can't lead to intelligence" when we don't really know what leads to intelligence to begin with.

We can't even define what intelligence is. We know or have strong hints at the limits of math/computation related to LLMs + CoT Note how PARITY and MEDIAN is hard here: https://arxiv.org/abs/2502.02393 We also know HALT == open frame == symbol grounding == system identification problems. The definition of AGI is also not well defined, but given the following: > Strong AI, also called artificial general intelligence,…

Humans can’t solve NP-hard problems either, so definition of intelligence shouldn’t lie here, and these particular limits shouldn’t matter too

Re: A bear case: My predictions regarding AI progress

#158

Post by jimbokun »

roenxi wrote: post #60
roncesvalles wrote: post #56

Earlier quoted context omitted.

The last line has been my experience as well. I only trust what I've verified firsthand now because the Internet is just so rife with people trying to influence your thoughts in a way that benefits them, over a good faith sharing of the truth. I just recently heard this quote from a clip of Jeff Bezos: "When the data and the anecdotes disagree, the anecdotes are usually right.", and I was like... wow. That quote is t…

> I knew Uber, Netflix, Spotify were revolutionary the first time I used them. Maybe re-tune your revolution sensor. None of those are revolutionary companies. Profitable and well executed, sure, but those turn up all the time. Uber's entire business model was running over the legal system so quickly that taxi licenses didn't have time to catch up. Other than that it was a pretty obvious idea. It is a taxi service. T…

> The innovations they made were almost completely legal ones; figuring out how to skirt employment and taxi law.

The impact of this was quite revolutionary.

> except for the fact that they have a pretty traditional content creator lab tacked on the side to do their own programs.

The way in which they did this was quite innovative, if not "revolutionary". They used the data they had from the watching habits of their large user base to decide what kinds of content to invest in creating.

Re: A bear case: My predictions regarding AI progress

#159

Post by jimbokun »

rchaud wrote: post #115
roncesvalles wrote: post #64

Earlier quoted context omitted.

They were revolutionary as product genres, not necessary individual companies. Ordering a cab without making a phone call was revolutionary. Netflix at least with its initial promise of having all the world's movies and TV was revolutionary, but it didn't live up to that. Spotify because of how cheap and easy it was to have access to all the music, this was the era when people were paying 99c per song on iTunes. I've…

> They were revolutionary as product genres, not necessary individual companies. Even then, they were evolutionary at best. Before Netflix and Spotify, streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account. What changed was that DMCA acquired political muscle and cross-border reach, wiping out waves of torrent sites and P2P networks. That left a new generation…

> streaming movies and music were already there as a technology, ask anybody with a Megaupload or Sopcast account.

You can't have a revolution without users. It's the ability to reach a large audience, through superior UX, superior business model, superior marketing, etc. which creates the possibility for revolutionary impact.

Which is why Megaupload and Sopcast didn't revolutionize anything.

Re: A bear case: My predictions regarding AI progress

#160

Post by red-iron-pine »

gerikson wrote: post #85
hattmall wrote: post #41

Earlier quoted context omitted.

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

Most engineering fields are de jure professional, which means they can and probably will enforce limitations on the use of GenAI or its successor tech before giving up that kind of job security. Same goes for the legal profession. Software development does not have that kind of protection.

for ~3 decades IT could pretend it didn't need unions because wages and opportunities were good. now the pendulum is swinging back -- maybe they do need those kinds of protections.

and professional orgs are more than just union-ish cartels, they exist to ensure standards, and enforce responsibility on their members. you do shitty unethical stuff as a lawyer and you get disbarred; doctors lose medical licenses, etc.

Post reply on HN