Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

341–350 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#341

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

[deleted]

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#342

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

Regardless of incentives, I don’t see any particular reason to think he has a more informed view than other experts on the trajectory of AI. He’s made several incorrect bets (capsule networks). I’m sure he’s smart and all. His contributions were valuable. But he’s not special in this particular moment.

What's wrong with capsule networks?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#343
post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#344

How is this different from what we have now? His immediate concern is that the internet will be flooded with false photos, videos and text, and the average person will “not be able to know what is true anymore.” maybe it's just me, but seems like this isn't a problem with technology but rather with how we organize society He is also worried that A.I. technologies will in time upend the job market. Today, chatbots lik…

I don't know why but I m pumped for the public internet to be littered with fake photos, so that people no longer lose their jobs over dumb things they did 10 years ago, and so that governments can no longer spy on their people reliably

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#345

Earlier quoted context omitted.

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

An AI doesn't have to be good in every domain to outcompete humans in many domains (which AI's already do). Besides, AI's can farm work out to other systems to do what they need. This has already been shown to work in practice with existing systems that do this.

Sure.. it also has to have sensors that outperform humans and require very little maintenance, or is maintenance that the platform can do itself without any renewable items, or with items that it can craft itself.

It has to have a power density which outperforms humans with a reliable source of charging that it can defend, maintain and repair if necessary. Fatigue is an issue here too, if you can cause the machine to use more power in an interval than it could possibly regain by charging in that same interval, you've got them entirely on the back foot.

Communications are an issue. Reliable distributed RF communications requires constant network coordination. These would be particularly easy to disrupt and for transmissions to be highly frustrated.

Finally.. we have nuclear weapons. Detente is a real strategy.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#346
post #158

Earlier quoted context omitted.

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

This comment appears to be getting upvotes, so I am going to jump in and say that it is very clear to anyone who knows that this commenter has no idea how AI is being used in the military or how military targeting works in the West. These things are generally classified, but algorithms are nowhere close to being considered decision-capable and human-in-the-loop targeting will be the norm for quite some time. Even nex…

I don't know the first thing about autonomous weapons systems, but nothing the GP said sounds technically infeasible. And what does the game theory look like once one party has AI that can make decisions without humans? Wouldn't that be a huge "advantage" in combat?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#347
post #317

Earlier quoted context omitted.

In addition to what people clarified in this thread, you probably will be interested in this: Neural network was not a popular research area before 2005. In fact, the AI nuclear winter in the 90s left such a bitter taste that most people thought that NN is a dead end, so much so that Hinton could not even get enough funding for his research. If it were not for Canada's (I forgot the institution's name) miraculous dec…

I remember in 2010 a postdoc came to teach a course on model checking and the classroom was just packed with CS students. I never took it but it will be interesting to see what kind of synthesis between traditional logic and neural network paradigms can be achieved.

"it will be interesting to see what kind of synthesis between traditional logic and neural network paradigms can be achieved"

Ben Goertzel talks about his work on something like this at around the 16 minute mark in this video:

https://m.youtube.com/watch?v=MVWzwIg4Adw

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#348

Earlier quoted context omitted.

somebody has a no disparage

Trying to be diplomatic, but this is such an unnecessary snarky, useless response. Google obviously did go slow with their rollout of AI, to the point where most of the world criticized them to no end for "being caught flat footed" on AI (myself included, so mea culpa). I don't necessarily think they did it "right", and I think the way they set up their "Ethical AI" team was doomed to fail, but at least they did clea…

> Google obviously did go slow with their rollout of AI, to the point where most of the world criticized them to no end for "being caught flat footed" on AI (myself included, so mea culpa).

they were criticized because they are losing competition not because of rollout, their current tech is weaker than ChatGPT.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#349
post #321

Earlier quoted context omitted.

That statement seems like such science fiction that it's kind of baffling an AI expert said it. What does it even mean for the AI to be smarter than people? I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data. And even the best case interactions I've seen online still rely on human intelligence to guide the AI to good outcomes instead of bad ones. Writing is a harder tas…

> I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data. By combining contexts from different fields. People are already using it with non-English languages and it responds in that language with something they couldn't previously find in that language.

Automatic translation is impressive, to be sure.

But looking up information and translating it into other languages is well within the realm of human skill. And the information it's translating came from people to begin with.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#350
post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

That reminds me of this, more optimistically: Matter and energy had ended and with it space and time. Even AC [Automated Computer] existed only for the sake of the one last question that it had never answered from the time a half-drunken computer technician ten trillion years before had asked the question of a computer that was to AC far less than was a man to Man. All other questions had been answered, and until thi…

> reverse the direction of entropy

This is unironically my spiritual belief in a greater power and purpose for living even if I can’t directly do anything to affect it. I think it is one of the most fundamental dogmas of any religion, that ultimately there is order.

Post reply on HN