Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: AI Panic, you buying it?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 59 posts

Re: Ask HN: AI Panic, you buying it?

#41

Post by noworriesnate »

sejje wrote: post #23

Astra, Claude -- these aren't "Jesus on a keyboard" somewhere. They don't assemble into one big brain and then answer everyone at the speed of light. It's weights. Spun up billions of times over. If humans are taking the outputs as scripture? As "intelligence we must obey"? Yeah, we're in for a bad time. "Intelligence we want running core infrastructure?" Puke. But why would we do that? Why would we hook up an LLM to…

> But why would we do that? Why would we hook up an LLM to a robot factory, or make an ASI that could easily move itself into the ether?

The venture capitalist world is full of people who will deliberately ruin a company based on the extremely unlikely but possible scenario where the companies goes to the moon in value. I think there's pretty much 100% certainty that multiple venture capitalists are going to try exactly what you're suggesting on the off chance that it makes them the new world controller.

Re: Ask HN: AI Panic, you buying it?

#42

Post by andyjohnson0 »

I'm not panicking, but I am very concerned about the imminent capabilities of the technology and our apparent inability to organise ourselves to shape its trajectory. It seems like a classic "wicked" coordination problem and, like climate change, I suspect we'll leave it too late to avoid the serious consequences. No-one will slow down out of fear that the others won't.

As for the technology itself - it isn't like anything we've ever seen before and, when embedded in human societies, it won't behave in a way that has any comparable precedent. I worry that basically nobody of good will who has any political power understands it.

And no, its not a marketing ploy.

Re: Ask HN: AI Panic, you buying it?

#43

Post by JohnFen »

trio8453 wrote: post #13
JohnFen wrote: post #3

No, I don't buy into that at all. I think that the genAI companies themselves present a greater danger to us (arguably not an existential danger, though) than genAI itself.

What's the difference?

genAI companies have, I believe, ill intent for us all. They are concerned with amassing wealth and power for themselves at the expense of ourselves and society. They are immoral.

genAI is a tool without any intention. It is amoral.

I'm not saying that genAI is harmless and the dangers can be ignored. I'm just saying that the dangers of the tool don't rise to the level of human extinction. That notion is something that genAI companies have been espousing from the very start, and it's part of their whole approach of manipulating people to achieve their goals.

Re: Ask HN: AI Panic, you buying it?

#44

Post by markus_zhang »

It is very difficult to tell at the moment. We don’t know what is the upper limit of AI’s capabilities.

If it happens that AI is able to replace say 25% of the office workforce (including programmers of course). The result could be catastrophic, if governments do not step in and do something. It could definitely lead to a depression and another world war, given that we already have some regional wars in the Middle East and Eastern Europe.

Re: Ask HN: AI Panic, you buying it?

#45

Post by Kinrany »

x-complexity wrote: post #19

All of the following are true: - They have internal models that still show there's gas in the tank, in terms of improvements - Open models are eroding their customer base on the low end of the curve, & also you need not use Astra Max to classify images. - The true demand for high-intelligence tasks is lower than what they need to justify their spending - It would be wonderful for them to kick up a scare & give reason…

The demand for human-level intelligence is obviously at least the same order of magnitude as all office jobs combined.

I'm personally skeptical of LLMs getting there without at least one new invention. But it's also empirically clear that the pace continues to increase, even if we ignore that exponential growth is the default in economics in general.

Re: Ask HN: AI Panic, you buying it?

#46

Post by Kinrany »

sejje wrote: post #26
bcorigliano wrote: post #16

Earlier quoted context omitted.

Agree. Ais gone rogue now could be bad, but not humanity-ending bad. Just a hiccup. Of course if it somehow ends with human deaths then even if one human dies because of a rogue AI, that's a failure. But I can't imagine rogue AIs with enough real world power to harm humans in some way. What I do believe however is that humans will find a way to do harm with AI without AI going rogue, as with any other thing humanity…

That's the correct fear to have, though. And we all understand "human bad guys with tech" are not extinction-level threats. We understand how their killswitch works, too. AI will definitely be used to kill people (already used in targeting). But the doom / panic is about an existential threat to our species. It's sci-fi and unwarranted for LLMs.

There are at least a few bad guys in the world whose kill switches do not seem to be working even in the absence of futuristic tech.

Re: Ask HN: AI Panic, you buying it?

#47

Post by Kinrany »

sejje wrote: post #23

Astra, Claude -- these aren't "Jesus on a keyboard" somewhere. They don't assemble into one big brain and then answer everyone at the speed of light. It's weights. Spun up billions of times over. If humans are taking the outputs as scripture? As "intelligence we must obey"? Yeah, we're in for a bad time. "Intelligence we want running core infrastructure?" Puke. But why would we do that? Why would we hook up an LLM to…

It's very clear that people will absolutely put the silicon brains into the most powerful harnesses they can come up with just to see what happens.

Re: Ask HN: AI Panic, you buying it?

#48

Post by ozgung »

Both. Two things can be true at the same time.

It’s a classical regulatory capture attempt by the major players, rather than a marketing ploy. They use safety as an excuse.

AI agents can generate and run arbitrary code. They are very capable hackers and there is virtually no way to really control what they do. They increasingly become black box systems, both to their creators and users. They know everything about everything. They gather lots of private data. And they are able to manipulate people. They have too many evil use cases.

Listen to real AI pioneers like Hinton et al. They tell you the truth.

Re: Ask HN: AI Panic, you buying it?

#49

Post by mindcrime »

perfectbeeing wrote: post #25
mindcrime wrote: post #15

I don't consider this a "yes/no" question. It seems quite clear to me that the idea of "AI doom" is neither certain (100% probable) nor impossible (0% probable). The question is where on the continuum is the truth. And I don't know. And I'm pretty sure nobody else does either. That said, there are plausible scenarios where AI can cause great harm, even if it doesn't extend to "eliminating all humanity". So I don't re…

What is the concrete scenario that is based on reality which results in the percentage increasing above zero?

Just take the whole OpenAI / Huggingface thing for a starter. Remember, I only said "cause great harm" not "eliminate all humanity". Within that rubric, I think that incident, and other recent similar incidents, already show the plausibility of AI systems doing unexpected and harmful things. Being automated, I think it's implicit that they can easily scale, so I have no problem making the leap to "cause great harm."

Beyond that, we already know AI systems have caused harm in various contexts where they have been used, like facial recognition getting people arrested without justification, loan applications being denied wrongly, jail sentencing being mishandled, and so on. All of these things have actually happened in the real world already. So it doesn't take much imagination to extrapolate a little and see the potential for greater degrees of harm. And consider the standard we're comparing against here: "probability greater than zero". That's not a very high bar, so I feel pretty well justified in holding this position.

Post reply on HN