Earlier quoted context omitted.
Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield. You actually need someone with vision and track record of doing right predictions and placing right technology bets. Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of…
> You actually need someone with vision and track record of doing right predictions and placing right technology bets. Hofstadter does not exactly fit this description.
Douglas Hofstadter changes his mind on Deep Learning and AI risk
351–360 of 520 posts
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#352Earlier quoted context omitted.
You may be far better informed than many you're arguing with, but casual dismissal IS foolish. The biggest potential / likely issues here aren't mere capabilities of systems and simple replacement of humans here and there. It's acceleration of "truth decay", accelerating and ever-more dramatic economic, social, and political upheaval, etc. You do not need "Terminator" for there to be dramatic downsides and damage fro…
I'm dismissing the survey, and I think you should as well. That's an orthogonal issue to actual x-risk, and I covered a bit more here: https://news.ycombinator.com/item?id=36577523 > I'd personally suggest keeping an eye on white papers from RAND and the like in trying to get some sense of the actual implications in the real world I think this is excellent advice and we're on the same page.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#353"And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior." Although he did real work in physics, Hofstadter's fame comes from writing popular books about science which explai…
The parent comment strikes me as an uninformed smear. Have you read any Hofstadter? It's hardly mass market scientific summary as you seem to think. And his career was in philosophy of mind and computer science, not just as a somewhat popular author. His writing is a bit more complex than the hallucinations of an llm.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#354Earlier quoted context omitted.
In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No…
Plenty of people have been talking about that, from my memory?
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#355Earlier quoted context omitted.
> One side seems to have much better quality of evidence. Since I AM an expert, I care a lot less about what surveys say. I have a lot more experience working on AI Safety than 99% of the ICM/NeurIPS 2021 authors, and probably close to 100% of the respondents. In fact, I think that reviewing community (ICML/NeurIPS c. 2020) is particularly ineffective and inexperienced at selecting and evaluating good safety research…
Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Similarly, you think that x-risk dismissal among the experts you agree with is not self-interested - but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would…
Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why.
> but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would likely say, as you realize, "No, my dismissal of the concerns of other experts is legitimate!"
Okay. But I have more to gain than to lose by advocating for AI Safety, since that's what I work on.
> The thing is, it doesn't require a lot of expertise to understand that there is actually an enormous risk
My comment was about X-risk and AGI. Find any of a myriad of comments here where I agree there are real risks that should be taken seriously. Those risks -- the reasonable ones -- are not existential and have nothing to do with AGI.
> Systems as smart or smarter than we are, that are not controlled for the benefit of humanity, are an existential risk...
I am not an adherent to this religious dogma and have seen no empirical evidence that the powerful rationalisms people use to talk themselves into these positions has any basis in current or future reality.
I want the LessWrong cult to have fewer followers in the halls of power precisely because I actually do give a damn about preventing the worst-case FEASIBLE outcomes, which have nothing to do with AGI bullshit.
BTW: if we're all too busy worrying about AGI who's going prevent the pile of ridiculous should-be-illegal bullshit that's about to cause a bunch of real harm? No one. If you want conspiracies about intent, look at who's funding all this AGI x-risk bullshit.
And you can be sure as hell of one thing: no one in big tech wants me giving Congress advice on what our AI and data privacy regulations should look like. They would MUCH prefer endless hearings on "risks" they might even know are bullshit nonsense.
But why focus any effort on 100% risks of data privacy instrusions, copyright infringement, wanton anti-trust violations, and large-scale disinfo campaigns when there's a 1% risk of extinction, right?
But anyways. I'm just some fool who has actually spent over a decade in the trenches trying to prevent specific bad outcomes. I'm sure the sci-fi essays and prognostications from famous CEOs are far more persuasive and entertaining and convicting than the ramblings of some nobody actually try to fix real problems with mostly boring solutions.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#356Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#357This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…
I work in a AI research lab for a large tech company and my observations are completely opposed to yours. Almost every researcher I have spoken believes that real risk exists, to some degree or other. Recent surveys of people in industry have largely borne this out - your anecdote sounds more like an anomaly to me.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#358Earlier quoted context omitted.
Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Similarly, you think that x-risk dismissal among the experts you agree with is not self-interested - but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would…
> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested…
In the simplest form:
1. AI could become smarter than humanity.
2. AI is rapidly progressing towards super-human intelligence.
3. If AI is smarter than humanity it could destroy or dominate humanity.
4. We aren't certain AI won't destroy or dominate humanity.
Therefore, there is some non-negligible existential risk from AI.
Which of these points do you disagree with, or do you think the conclusion doesn't follow?
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#359Earlier quoted context omitted.
Plenty of people have been talking about that, from my memory?
Every time it comes up on the news it's some MIT doctor or VC telling us about how AI is going to replace us or try to take over the world.
Again, I think ego is probably a better explanation than nefarious intent, but it's a nice side-effect.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#360Earlier quoted context omitted.
As well as these models will all be used to launder content. In the near future you will likely be able to ask an AI to create the next 15 sequels to the movie war games starring a young matthew broderick. Episode 12 will have a karate kid cross over. But how much of what is there will be an actually new thing? How much will be scraped from my War Games fan fic website that contains scripts of 27 sequels to War Games…
Here is ChatGPT 4’s summary of a sequel. I don’t think it’s very good. Maybe another AI would enjoy it though. > Create a summary of a sequel to the movie “War Games” Title: "War Games: Firewall" In "War Games: Firewall", now-adult computer genius David Lightman (Matthew Broderick) has been working as a cyber security consultant for the Pentagon. When an advanced, multi-national cyber-terrorist organization emerges,…