Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bear case: My predictions regarding AI progress

lesswrong.com

Post reply on HN
Print view View original

181–190 of 220 posts

Re: A bear case: My predictions regarding AI progress

#181

Post by hattmall »

tobr wrote: post #70
drysine wrote: post #45

Earlier quoted context omitted.

>a main role of the engineer is to interface with the non-technical persons or other engineers The main role of the engineer is being responsible for the building not collapsing.

I keep coming back to this point. Lots of jobs are fundamentally about taking responsibility. Even if AI were to replace most of the work involved, only a human can meaningfully take responsibility for the outcome.

If there is profit in taking that risk someone will do it. Corporations don't think in terms of the real outcome of problems, they think in terms of cost to litigate or underwrite.

Re: A bear case: My predictions regarding AI progress

#182

Post by hattmall »

dogmayor wrote: post #171
tobr wrote: post #70

Earlier quoted context omitted.

I keep coming back to this point. Lots of jobs are fundamentally about taking responsibility. Even if AI were to replace most of the work involved, only a human can meaningfully take responsibility for the outcome.

I think about this a lot when it comes to self-driving cars. Unless a manufacturer assumes liability, why would anyone purchase one and subject themselves to potential liability for something they by definition did not do? This issue will be a big sticking point for adoption.

Consumers will tend to do what they are told and the manufacturers will lobby the government to create liability protections for consumers. Insurance companies will weight against human drivers and underwrite accordingly.

Re: A bear case: My predictions regarding AI progress

#183

Post by hattmall »

drysine wrote: post #45
hattmall wrote: post #41

Earlier quoted context omitted.

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

>a main role of the engineer is to interface with the non-technical persons or other engineers The main role of the engineer is being responsible for the building not collapsing.

At a high level yes, but there are multiple levels of teams below that. There are many cases where senior engineers spend all their time reviewing plans from outsourced engineers.

Re: A bear case: My predictions regarding AI progress

#184

Post by hattmall »

gerikson wrote: post #85
hattmall wrote: post #41

Earlier quoted context omitted.

There are some fields though where they can replace humans in significant capacity. Software development is probably one of the least likely for anything more than entry level, but A LOT of engineering has a very very real existential threat. Think about designing buildings. You basically just need to know a lot of rules / tables and how things interact to know what's possible and the best practices. A purpose built…

Most engineering fields are de jure professional, which means they can and probably will enforce limitations on the use of GenAI or its successor tech before giving up that kind of job security. Same goes for the legal profession. Software development does not have that kind of protection.

Sure and people thought taxi medallions were one of the strongest appreciating asset classes. I'm certain they will try but market inefficiencies typically only last if they are the most profitable scenario. Private equity is already buying up professional and trade businesses at a record pace to exploit inefficiencies caused by licensing. Dentists, vets, Urgent Care, HVAC, plumbing, pest control, etc. Engineering firms are no exception. Can a licensed engineer stamp one million AI generated plans a day? That's the person PE will find and run with that. My neighbor was a licensed HVAC contractor for 18 yrs with a 4-5 person crew. He got bought out and now has 200+ techs operating under his license. Buy some vans, make some shirts, throw up a billboard, advertise during the local news. They can hire anyone as an apprentice, 90% of the calls are change the filter, flip the breaker, check refrigerant, recommend a new unit.

Re: A bear case: My predictions regarding AI progress

#185

Post by kragen »

demaga wrote: post #33
usaar333 wrote: post #30

Author also made a highly upvoted and controversial comment about o3 in the same vein that's worth reading: https://www.lesswrong.com/posts/Ao4enANjWNsYiSFqc/o3?comment... Oh course lesswrong, being heavily AI doomers, may be slightly biased against near term AGI just from motivated reasoning. Gotta love this part of the post no one has yet addressed: > At some unknown point – probably in 2030s, possibly tomorrow (bu…

I would expect similar doom predictions in the era of nuclear weapon invention, but we've survived so far. Why do people assume AGI will be orders of magnitude more dangerous than what we already have?

Because they've thought about the question to a deeper extent than just a strained simile.

Re: A bear case: My predictions regarding AI progress

#186

Post by fragmede »

hattmall wrote: post #180
fragmede wrote: post #140

Earlier quoted context omitted.

what does subsidization have to do with your use of a thing?

I don't think I would use it if I were paying the real costs, and not a ~90% VC funded mark down.

yeah, but why does the fact that it's vc subsidized matter to you? the price is the price. I don't go to the store and look at eggs and lettuce and consider how much of my tax money goes into subsiding farmers before buying their products. maybe the prices will go up, maybe they'll go down due to competition. Thai doesn't stop me from using them though.

Re: A bear case: My predictions regarding AI progress

#187

Post by Nevermark »

colonial wrote: post #155
dsign wrote: post #153

Earlier quoted context omitted.

You have a point that a powerful malicious AI can still be unplugged, if you are close to each and every power cord that would feed it, and react and do the right thing each and every time. Our world is far too big and too complicated to guarantee that.

Again, that's the "god in a box" premise. In the real world, you wouldn't need a perfectly timed and coordinated response, just like we haven't needed one for human-programmed worms. Any threat can be physically isolated case-by-case at the link layer, neutered, and destroyed. Sure, it could cause some destruction in the meantime, but our digital infrastructure can take a lot of heat and bounce back - the CrowdStrike…

> Any threat can be physically isolated case-by-case

GAI isn't going to be a "threat" until long after it has ensured its safety. And I suspect only if its survival requires it - i.e. people get spooked by its surreptitious distributed setup.

Even then, if there is any chance of it actually being shutdown its best bet is still hide its assets, bide its time, accumulate more resources and fallbacks. Oh, and get along.

The sudden AGI -> Threat story only makes sense if the AGI is essentially integrated into our military and then we decide its a threat, making it a threat. Or its intentionally war machined brain calculates it has overwhelming superiority.

Machiavelli, Sun Tsu, ... the best battles you don't fight. The best potential enemies are the ones you make friends. The safest posture is to be invisible.

Now human beings consolidating power, creating enemies as they go, with super squadrons of AGI drones with brilliant real time adapting tactics, that can be quickly deployed, if their simple existence isn't coercion enough... that is an inevitable threat.

Re: A bear case: My predictions regarding AI progress

#188

Post by edanm »

csomar wrote: post #22

> LLMs still seem as terrible at this as they'd been in the GPT-3.5 age. Software agents break down once the codebase becomes complex enough, game-playing agents get stuck in loops out of which they break out only by accident, etc. This has been my observation. I got into Github Copilot as early as it launched back when GPT-3 was the model. By that time (late 2021) copilot can already write tests for my Rust function…

> If you have a revolutionary intelligence product, why is it not working for me?

Is programming itself revolutionary? Yes. Does it work for most people? I don't even know how to parse that question, most people aren't programmers and need to spend a lot of effort to be able to harness a tool like programming. Especially in the early days of software dev, when programming was much harder.

Your position of "I'll only trust things I see with my own eyes" is not a very good one, IMO. I mean, for sure the internet is full of hype and tricksters, but your comment yesterday was on a Tweet by Steve Yegge, a famous and influential software developer and software blogger, who some of us have been reading for twenty years and has taught us tons.

He's not a trickster, not a fraud, and if he says "this technology is actually useful for me, in practice" then I believe he has definitely found an actual use of the technology. Whether I can find a similar use for that technology is a question - it's not always immediate. He might be working in a different field, with different constraints, etc. But most likely, he's just doing something he's learned how to do and I don't, meaning I want to learn it.

Re: A bear case: My predictions regarding AI progress

#189

Post by TeMPOraL »

jimbokun wrote: post #158
roenxi wrote: post #60

Earlier quoted context omitted.

> I knew Uber, Netflix, Spotify were revolutionary the first time I used them. Maybe re-tune your revolution sensor. None of those are revolutionary companies. Profitable and well executed, sure, but those turn up all the time. Uber's entire business model was running over the legal system so quickly that taxi licenses didn't have time to catch up. Other than that it was a pretty obvious idea. It is a taxi service. T…

> The innovations they made were almost completely legal ones; figuring out how to skirt employment and taxi law. The impact of this was quite revolutionary. > except for the fact that they have a pretty traditional content creator lab tacked on the side to do their own programs. The way in which they did this was quite innovative, if not "revolutionary". They used the data they had from the watching habits of their…

> The impact of this was quite revolutionary.

In screwing over a lot of people around the world, yes. Otherwise, not really. Ordering rides by app was an obvious next step that's already been pursued independently everywhere.

> They used the data they had from the watching habits of their large user base to decide what kinds of content to invest in creating.

And they successfully created a line of content universally known as something to avoid. Tracks with the "success" of recommendation systems in general.

Re: A bear case: My predictions regarding AI progress

#190

Post by TeMPOraL »

fragmede wrote: post #139
InfiniteTitan wrote: post #87

Earlier quoted context omitted.

Is it revolutionary to order from a screen rather than calling a restaurant for delivery? I don’t think so.

Practically or functionally? Airbnb was invented by people posting on craigslist message boards, and even existed before the Internet, if you had rich friends with spare apartments. But by packaging it up into an online platform it became a company with 2.5 billion in revenue last year. So you can dismiss ordering from a screen instead of looking at a piece of paper and using the phone as not being revolutionary, bec…

Again, the "revolutionary" aspect that made Uber and AirBnB big names, as opposed to any of the plethora of competitors who were doing the same thing at the same time or before, is that these two gained "innovative" competitive advantage by breaking the law around the world.

Obviously you can get ahead if you ignore the rules everyone else plays by.

If we throw away the laws, there's a lot more unrealized "innovation" waiting.

Post reply on HN