[dead]
Geoffrey Hinton leaves Google and warns of danger ahead
301–310 of 1001 posts
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#302We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…
Military application of AI drones isn't even the worst possible use, it's nations using them to completely subjugate their own population(although the same tech could be used against non-peer nations). Combination of things like Gorgon Stare to direct smaller AI controlled drones like what they are using in Ukraine would be a police state nightmare. https://en.wikipedia.org/wiki/Gorgon_Stare https://longreads.com/201…
Yup.
[1]
> Pentagon testing mass surveillance balloons across the US
[2]
> For instance, using Gorgon Stare, a 24-hour aerial surveillance system, the U.S. Air Force had been able to plot back from a car bomb explosion in Kabul in 2019, which killed 126 civilians, to find the location of safe houses used to execute the attack.
[1] - https://www.theguardian.com/us-news/2019/aug/02/pentagon-bal...
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#303The flip-flopping of AI critics is completely explainable by flip-flopping morals of the architects. > Dr. Hinton said that when people used to ask him how he could work on technology that was potentially dangerous, he would paraphrase Robert Oppenheimer, who led the U.S. effort to build the atomic bomb: “When you see something that is technically sweet, you go ahead and do it.” If anyone outside the core architects…
Would the world be better off without MAD?
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#304I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…
Is there any indication that current methods could lead to a model that generates text as if it had an IQ of 200? These are trained on texts written by humans who are, quite overwhelmingly, much lower in IQ than 200. Where's the research on developing models that don't just produce better or faster facsimiles of broadly average-IQ text?
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#305“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?
This quote is the first thing I've seen that really makes me worried. I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.
He is far from the only one.
If you're interested in exploring this further I can really recommend taking a look at some of the papers that explore GPT-4's capabilities. Most prominent among them are the "Sparks of AGI" paper from Microsoft, as well as the technical report from openai. Both of them are obviously to be taken with a grain of salt, but they serve as a pretty good jumping off point.
There are some pretty good Videos on Youtube exploring these papers if you don't want to read them yourself.
Also take a look at the stuff that Rob Miles has published over on Computerphile, as well as his own channel. He's an Alignment Researcher with a knack for explaining. He covers not just the theoretical dangers, but also real examples of misaligned ai, that alignment researchers have predicted would occur as capabilities grow.
Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. It's still unclear that it will get us all the way there, but recent developments have made a lot of ai researchers rethink that possibility, with many of them significantly shortening their own estimates as to when and how we will get there. It's very unusual that the people that are better informed and closer to the research are more worried than the rest of the world and it's worth keeping this in mind as you explore the topic.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#306> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.
Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…
Whether society (here I'm referring to "Representative democracy with general elections;" YMMV if you're under an authoritarian or totalitarian state where someone is already filtering the truth for you) will be fine will be heavily dependent upon whether two things happen:
1. The public, in general, comes to understand in an in-their-bones way that they currently do not understand that most of what they see online is hogwash. I.E. the bozo bit has to flip all the way to "My neighbor says there's a missing dog on the block... but is that really my neighbor?"
2. Some other mechanism of truth-pedigree that has not yet been invented comes along to allow for communication of the current state of the world to work.
Without (1) we know democracies are easily led by credible, subtle propaganda, and a well-tuned network of hostile actors will drive wedges at the friction points in representative democracies and crack them into warring subcultures.
Without (2) voters will have insufficient tools at their disposal to understand country-scale issues and their ability to effect positive outcomes with their vote will collapse into noise, which is a ripe environment for authoritarians to swoop in and seize power (and a ripe environment for centralized authoritarian states to outmaneuver the representative democracies on the world stage and gain power).
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#307We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…
Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…
People have gotten it to play chess. The quality of play you get is very sensitive to prompt details, and may also be affected by OpenAI's ongoing model tuning, but here's one experiment in which it won 11 games of 19 against amateurs (treating any illegal move it tried to make as an immediate forfeit): https://dkb.blog/p/chatgpts-chess-elo-is-1400
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#308Earlier quoted context omitted.
My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers whose primary purpose seems to be to over-complicate projects. Imagine the same teams that make planes which need to be hard rebooted every few days, now they're making advanced AI to dynamically target and kill people.
This reminds me of the turret robots from Futurama https://youtu.be/MsY0PVqTleY https://youtu.be/WzbT0Q2jh_w
Well let's just build a robot Santa too. What could possibly go wrong
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#309Earlier quoted context omitted.
I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…
>I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories When has he said this? For the most part I feel Chomsky has been misunderstood when it comes to LLMs. As best as I can tell what Chomsky has said is that LLMs do not provide any insight into how language works, it's not really a scientific advancement so much as it's an engineering breakthrough. The fact that LLMs exist and can mimic natur…
Chomsky said that LLMs are statistical regurgitators which means LLMs can never actually reason and explain which language understanding requires. That they are a wrong model of computation by definition.
It's an interesting position and I'm sympathetic toward it, he could be partly right in the end.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#310Earlier quoted context omitted.
I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…
I've thought about this now, and I think that: - the scientists people know about are generally older - older people are often thought of as wiser, or may indeed be so - when a famous scientist - who is already likely to be older, and who has a history of getting things right - gets something wrong, then it's more jarring and noticeable My theory then is that it isn't true, but we notice such cases more. Also, exampl…
And Chomsky is in touch with other colleagues who agree with him, it's not as if his disagreement stems from being an old, isolated hermit. At the least you'd have to argue his colleagues are also mistaken.