It’s somehow fitting that he’s being interviewed by Q.
Douglas Hofstadter changes his mind on Deep Learning and AI risk
41–50 of 520 posts
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#42I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…
I think they are used to that these fields progress relatively slowly. Also there are some things these systems already know that they though would come much later. Hofstadter even says this in the interview. In the 90s, early 2000s I also thought that AGI will come one day, but it will take centuries. Nowadays I think it can be just decades. The progress is very fast compared to my old worldview. I don't think they are mislead at all. I don't think they mischaracterize the current capabalities of these systems. They just think that if progress is this fast (compared to their previous estimates), AGI can come soon, where soon is 5-10-20 years. Younger guys, who are more used to the state of the art, and did not live when things currently possible seemed far far away are less impressed.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#43Earlier quoted context omitted.
No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…
There were plenty of well-off people who flew to Syria to go behead other people.
Anyway this doesn't matter that much. Sure, you can imagine a world totally different from ours where there would be zero differential risk between a chess-playing computer and a language-speaking computer. But we live in this world, and the risk profile is not the same.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#44I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.
If it's a positive interaction, then what is your fear?
I suspect that the OP meant that it triggers a stimulus reaction which is mistaken for a positive interaction.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#45Earlier quoted context omitted.
No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…
The circumstances are very active, and AI has real potential to act as a force multiplier for them.
We've already seen this with troll farms and campaigns of seeded insanity like the Q Cult.
Existing AI tools can make similar efforts cheaper and more effective, and future AI tools are likely to be even more powerful.
There's a huge difference between how an experienced technical researcher sees AI and how a politician, war lord, or dark media baron sees it.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#46Earlier quoted context omitted.
No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
Nobody has been convinced to fly an airplane into a building by a language model either.
"This might be a risk"
"No it's not, it hasn't materialized"
Risk: Possibility of loss or injury
Risks, definitionally, are things that have not happened yet.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#47I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…
Consider the comparatively simple case of these LLMs providing decent enough chatbots to fool a majority.
If you deploy them on social media in a coordinated fashion, you can easily sway public opinion.
You only need to train them to adhere to psyops techniques not sufficiently well known nor easily detectable. Of which there are many.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#48Earlier quoted context omitted.
No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.
Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…
Quite so, which is why retrospective analysis like "The CIA helped start The Paris Review and that made literature friendly to neoliberal ideology" are confections of confirmation bias. Nothing is ever that pat. But tidy little conspiracies are also never the goal. A nudge is both all that is realistic to aim for and a few successes are all you need to shift public perception.
Arming every ambitious cult leader wannabe from some retrograde backwater with an information war WMD deserves some caution.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#49> If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches.
This seems like the reaction of an atheist who already overvalued human intelligence.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#50Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logi…
People overly impressed by LLMs haven't spent a lot of time trying to make them actually useful. When you do, you learn that they're talented mimics but still quite limited.
This also applies to humans.