Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

611–620 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#611

Post by chii »

AlexandrB wrote: post #377
revelio wrote: post #145

Earlier quoted context omitted.

Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…

> What have we got now? ChatGPT is trained to give all sides of the issue and not express strong opinions, which is better than 90% of journalists and academics manage. I think we're experiencing the "golden age" of AI at the moment. We'll see what kind of monetization OpenAI and others will land on, but I would be shocked if messing with the model's output for commercial gain is not in the cards in the future.

which is exactly why it's important to have multiple sources of models, trained by different people/groups, and competing against each other.

Single source monopoly is almost always bad for society. Unless there's some sort of natural monopoly, in which case gov't regulation and transparency is required. But i dont think ai models are something that has natural monopoly unlike cables, or pipelines.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#612

Post by chii »

fatherzine wrote: post #327
revelio wrote: post #145

Earlier quoted context omitted.

Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…

"AI will make things much better, just as the Internet did." We must be living in very different worlds. I sometimes wonder if the numbers behind https://en.wikipedia.org/wiki/Disease_of_despair (roughly tripled in 20 years of Internet) are just the first steps of a hockey stick.

How would you know if these disease of despair wouldn't have been worse had there not been internet?

How come the same despair from places like russia (where death from alcoholism is almost epidemic), isn't being attributed to the internet there?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#613

Post by ChatGTP »

ethanbond wrote: post #569
anon-3988 wrote: post #562

Earlier quoted context omitted.

Obviously you haven't read or skimmed the article because the article makes no mention of AGI. However, it is very bland and predictable. I am not sure why any technologists or pioneer would think their technology wouldn't be used for bad. You can probably replace any mention of AI, ML or NN in the article with any other invention in the past 1 billion years and it will still make sense. What technology/inventions ou…

What’s different in the downside case of AI is exactly what’s different in the upside case of AI: immense power that we have never seen before. We now have a few “live x-risks,” each one representing a high wire we cannot fall off of even once, but nor can we just choose to step off of them safely. AI is an additional potential doom we are suspended above, and if it lives up to its positive potential it’ll also be ab…

Wow, what a mess we’ve created for ourselves. It’s kind of tragic but I can’t help but laugh.

I don’t think the situation we’re in at the moment gives me much reason to believe we’re actually intelligent. Maybe we’re intelligent but we completely lack wisdom ?

Birds don’t sit around all day creating such huge problems for themselves. Only we seem to do that…time for a rethink?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#614

Post by MacsHeadroom »

ben_w wrote: post #51
saynay wrote: post #10

Earlier quoted context omitted.

Most still believe that "really intelligent AI" is still a long way off, from what I have seen. Many have started to believe there can be a lot of harm caused by the systems well before then, however.

It depends what you mean by "intelligence". For any given definition so far, when the AI can do that, we have changed our minds about if that counts. So, when I was a kid, "intelligence" meant being good at chess and maths, having a good memory, knowing a lot of trivia, and being able to speak a second language. On all of these things except language, a raspberry pi and a cheap memory card beats essentially all human…

>On all of these things except language, a raspberry pi and a cheap memory card beats essentially all humans.

llama.cpp runs quite fast on a raspberry pi 8GB, beating most humans at language.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#615

Post by chii »

slowmovintarget wrote: post #186
revelio wrote: post #145

Earlier quoted context omitted.

Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…

Social media algorithms on "the internet" have caused wars, supported genocides, created extreme societal polarization, have led to dramatically increased suicide rates among teens, especially teen girls, and more. But I got to share baby pics with my mom. How will a far noisier information flow help? Generative AI will only help us do what we've been doing in far greater quantity. Just like calculators can only help…

none of the problems you mentioned are caused by the internet.

These are human problems. Humans cause them, not the tool. I would not give up the tool, just because said tool could be misused by some people to do harm. Just like i don't stop driving just because there's some people who run others over.

May be some regulation is important - but only _after_ it has been shown to have caused harm, and that the harm is not outweighed by the good.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#617

Post by danShumway »

theptip wrote: post #563
danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

I find this position hard to grok. You’re complaining about people worrying about AGI because you view the short-run implications of this tech to be quite bad. To me, a lack of prompt security in the short term bodes poorly for our safety in N generations when these systems are actually powerful. Like, sure, someone is gonna get swatted by an AI in the next year or two, and that sucks, but that is a tiny speck of dus…

> Is it that you just think P(AGI) is really low, so worrying about an unlikely future outcome bothers you when there is actual harm now?

Having now gotten a few opportunities to really use GPT-4 in-depth, I am much more bearish about the intelligence potential of LLMs than many other people are. This is not something I lose sleep over. But I don't like to focus on that because I'm not sure it matters.

I commented the same elsewhere, but there is no world where AI alignment is solved if prompt injection isn't. If you can't get an AI to reliably avoid swearing, how on earth can it be aligned?

So if you want to look though a long-term perspective and you're really worried about existential risks, the attitude towards prompt injection -- the willingness of the entire tech sector to say "we can't control it but we're going to deploy it anyway" -- should terrify you. Because how prompt injection gets handled is how general alignment will get handled.

The companies will display the same exact attitudes in both cases. They won't move carefully. They are proving to you right now that they will not be responsible. And at every step of the process there will be a bunch of people on HN saying, "okay, the AI goes a little rogue sometimes, but the problem is exaggerated, stop making such a big deal of it."

There is no point in talking about the long-term consequences of unaligned AI if we can't solve short-term alignment and short-term task derailing, because if threats like prompt injection are not taken seriously, long-term alignment is not going to happen.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#618

Post by chii »

partiallypro wrote: post #546
m3kw9 wrote: post #534

Earlier quoted context omitted.

Which is fine, humans will adapt to this info noise rather than going crazy, Hinton is way underestimating human intelligence

I think the problem is that the internet created a ton of new jobs, even while taking some. So far, I can't think of an example of AI creating jobs...only taking them. When you have a lot of newly unemployed people, drowned in debt, unable to know what to believe (AI lies and generations will become more prominent)...I can see that as becoming a massive political problem. It's not quite like robots on an assembly flo…

> I can't think of an example of AI creating jobs...only taking them.

future jobs which doesn't exist today will not be in your vocabulary or thoughts, which is why you cannot think of them. Does not mean such jobs will not exist.

The play today for the concerned, is to start owning capital as well as selling their labour. People who only rely solely on labour as their source of income will be disadvantaged, as labour is increasingly less useful.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#619

Post by int_19h »

hn_throwaway_99 wrote: post #241
archerx wrote: post #36

There is part of me that thinks that this A.I. fear-mongering is some kind of tactic by Google to get everybody to pause training their A.I.s so they can secretly catch up in the background. If I was to do some quick game theory in my mind this would be the result. Imagine being Google, leading the way in A.I. for years, create the frameworks (tensorflow), create custom hardware for A.I. (TPUs), fund a ton of researc…

I think a comment on the reddit thread about this is somewhat appropriate, though I don't mean the imply the same harshness: > Godfather of AI - I have concerns. > Reddit - This old guy doesn't know shit. Here's my opinion that will be upvoted by nitwits. Point being, if you're saying that the guy who literally wrote the paper on back propagation is "fear mongering", but who is now questioning the value of his life's…

I think there are two distinct points here that need to be clearly separated.

When Hinton gives an estimate on how fast things are going to move and how far they can go, that is the part where his background gives his estimates much higher credibility than any random person on the Internet.

But how dangerous that level is to humanity as a whole is a separate question, and one that he is not an expert on.

Post reply on HN