Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…
Geoffrey Hinton leaves Google and warns of danger ahead
711–720 of 1001 posts
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#712Earlier quoted context omitted.
Yeah, it seems that driving this fact home may in fact be beneficial. Right now a lot of people still assume that everyone on the internet is truthful with good intentions. Making it very clear that this isn't true may be helpful to reset this frame of mind.
Does anyone else hate the fact that we are actively encouraging making a low trust society? That can't lead to good things...
Maybe I am too pessimistic and we will be able to fulfill everyone's needs one day. But as the rich keep getting richer and focusing on power for themselves that doesn't seem like a direction that we are going to move towards soon.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#713Earlier quoted context omitted.
Cade Metz is the same hack who tried to smear Scott Alexander. This guy is the personification of journalistic malpractice.
> Cade Metz is the same hack who tried to smear Scott Alexander. This guy is the personification of journalistic malpractice. He didn't "smear" Scott Alexander. That's just the hit-job framing pushed by Alexander's fans, who were mad he didn't write a puff piece and they couldn't just make up rules on about stuff on their websites (e.g. about using people's self-disclosed real names) and have the rest of the world be…
In his previous work as a psychiatrist as a hospital, he went by Scott Siskind (which seems to be the name he uses with family, on identity documents etc), whereas in rationalist circles and his old blog SSC he went by Scott Alexander. He has explained why it is a problem for a psychiatrist to have a known identity beyond the usual "blank slate", that his clients can project onto. Indeed, his real name being associated with his online one led to him having to leave his former job (albeit more "by mutual agreement" rather than "fired" as I understand).
It seems that the NYT, despite having a "real names" policy, is more than willing to bend it when it is convenient to them (for example they are more than happy not to mention birth names of transgender people even if they haven't gone through a full legal name change process). But they weren't willing to do it in this case, just referring to "the rules". The cost of this was that Scott lost his job (although he now makes more on Substack than he used to as a full-time psychiatrist), and the benefit ... I really don't see what benefit there is for NYT readers to know the surname of Scott-the-psychiatrist, when the article was about Scott-the-rationalist.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#714Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#715It's important to have a discussion about AI safety, and the ethics surrounding LLMs. But I'm really tired of all this sensationalism. It completely muddies the waters; it almost seems intentional at this point.
But what I rarely see discussed is the opportunity costs in not having this progress at as fast a pace as possible.
The pie chart of existential threats for humanity definitely has rouge AI on it.
But that's a slice amidst many human driven threats ranging from nuclear war to oceans dying.
What there's not very many human driven slices of pie for is realistic solutions to these issues.
On that pie chart, some sort of AGI deus ex machina is at the current rate of progress probably the only realistic hope we have.
People who have been in the AI industry for a long time are still operating on thought experiments that barely resemble the actual emergence we are watching.
So while you have old AI thought leaders going on about cold heartless AI not being able to stop itself from making paperclips because of its innate design from training, we have humans unable to stop themselves from literally making paperclips (and many other things) because of our own evolutionary brain shortcomings, even as it dooms us.
From what I've seen so far, it seems far easier to align AI to care about the lives and well-being of humans different from it than it seems to be to align most humans to care about the lives of fellow humans different from them.
The opportunity cost of leaving the world in human hands as opposed to accelerating a handoff to something better at adapting to the emerging world and leaving the quirks of its design from generations of training behind seems far more dangerous than the threat of that new intelligence in isolation.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#716It's important to have a discussion about AI safety, and the ethics surrounding LLMs. But I'm really tired of all this sensationalism. It completely muddies the waters; it almost seems intentional at this point.
Did we have some ethics discussion when the lightbulb was invented? Or when the car was invented? No, and if we did, we couldn't have foreshadowed all the positive and negative impacts. My grandfather always told me that back in the days, "smart" people said that no train should be allowed to go faster than 50km/h because the heart would explode. Nobody here can say that he wasn't impressed by ChatGPT. How could we e…
"There was some wonderful stuff about [railway trains] too in the U.S., that women's bodies were not designed to go at 50 miles an hour. Our uteruses would fly out of our bodies as they were accelerated to that speed."
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#717It's important to have a discussion about AI safety, and the ethics surrounding LLMs. But I'm really tired of all this sensationalism. It completely muddies the waters; it almost seems intentional at this point.
Did we have some ethics discussion when the lightbulb was invented? Or when the car was invented? No, and if we did, we couldn't have foreshadowed all the positive and negative impacts. My grandfather always told me that back in the days, "smart" people said that no train should be allowed to go faster than 50km/h because the heart would explode. Nobody here can say that he wasn't impressed by ChatGPT. How could we e…
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#718“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?
I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#719Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…
You simply add another 'reviewer' layer that does a clarification task on the input and response to detect it.
The problem is this over doubles the cost of implementation to prevent something no one actually cares about fixing for a chatbot.
"Oh no, the user got it to say silly things to themselves."
This isn't impacting other people's experiences or any critical infrastructure.
And in any applications that do, quality analysis by another GPT-4 layer will be incredibly robust, halting malicious behavior in its tracks without sophisticated reflection techniques that I'm skeptical could successfully both trick the responding AI to answer but evade the classifying AI in detecting it.
Re: Geoffrey Hinton leaves Google and warns of danger ahead
#720Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…
There is one system, also widely-deployed, other than LLMs, that's well-known to be vulnerable to prompt injection: humans . Prompt injection isn't something you can solve . Security people are sometimes pushing things beyond sense or reason, but even they won't be able to fix that one - not without overhauling our understanding of fundamental reality in the process. The distinction between "code" and "data", between…
Hello LISP :)