Earlier quoted context omitted.
People ascribe way too much non-programatic nature to humans imo Just look up the Name a Color and a Tool question Did you think of one? Red hammer? Or blue screwdriver?
hmmm, I chose a yellow saw
Douglas Hofstadter changes his mind on Deep Learning and AI risk
511–520 of 520 posts
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#512Earlier quoted context omitted.
>These are things you can just look up. The LLM contributes nothing that the internet didn't already provide The internet doesn't allow one to design _new_ weapons, possibly way more effective (which if you read carefully the first story, this one does). >I'm not sure that LLMs significantly increase the attack surface here. Being able to ask an AI to develop new deadly varieties which we'll not be able to detect or…
This entire category falls into my "numerical siulations at national labs" category of "don't care". If you wanted to use a bioweapon to kill a bunch of people, you would ignore the DeepCE paper and use weapons that have existed for decades. Existing weapons would be easier to design, easier to manufacture, easier to deploy, and more effective at killing. Computational drug discovery is not new, to put it mildly, and…
>There are huge barriers to entry between "know how to design/build/deploy a nuke/bioweapon" and "can actually do it".
>public models don't provide them with a tool more or less powerful than the internet
> I'm saying that AI doesn't meaningfully change that landscape because the barrier to operationalizing is high enough that by the time you can operationalize it's unclear why you need to model -- that you couldn't have made the a similar discovery with a bit of extra time or even just used something off the shelf to the same effect.
Your expertise is in AI, but the issues here aren't just AI, they involve (for example) chemistry and biology, and I suggest speaking with chemists and biologists on the difference AI makes to their work. You may discover the huge barrier isn't that huge, and that AI can make discoveries easier in ways that 'a little extra time' is strongly underselling (most humans would take a very long time searching throughout possibility-space, such a search may well be detectable since it will require repeated synthesis and experiment...). Also, to borrow an old Marxist chestnut: A sufficient difference in quantity is a qualitative difference*. Make creating weapons easy enough and you get an entirely different world.
I get your issues with the 'LessWrong cult', I have quite a few of my own. However, that doesn't make the risks nonexistent, even if we were to discount AGI completely. Given what I see from current industry leaders (often easily bypassed blacklisting) I'm not so impressed with the current safety record. I fear it will crack on the first serious test with disastrous consequences.
* There's a smarter phrasing which I can't find or remember.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#513Earlier quoted context omitted.
AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.
> A million jobs are begging for HUDS that guide the worker [..] I have a pretty dull, comfortable desk job and lack the imagination to come up with any. Can you name some? (I've been wrong before when I was skeptical when everyone was hyped about iPad, Deep Learning, etc. so please convince me about Apple Vision, Mark's metaverse, or google's glasses and paint what might be in 5 years.)
Work: A 3D room-scale representation of GDB would help me keep a lot more of a program’s state in my head.
Fun: I want a HUD for skiing that marks where my friends are and warns me of rocks / cliffs / incoming weather.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#514This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…
In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No…
> We are about to enter a couple of decades of people using these pithy model systems to make real decisions that impact lots of people,
If we're extraordinarily lucky. If not, you can take comfort in that fact that redlining won't survive us either.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#515Earlier quoted context omitted.
In the "scientists complacent about dangers of the systems they are working on" file, I think we are all familiar with https://en.wikipedia.org/wiki/Demon_core . Now, admittedly, this was scientists being complacent about a thing they knew was dangerous, whereas here we are saying scientists don't think their thing is dangerous. But very clearly, AI could be dangerous, so it's more that these scientists don't think t…
> Presumably the scientists and engineers behind the https://en.wikipedia.org/wiki/Therac-25 didn't think it would kill people. Therac-25 is an excellent example, but of EXACTLY the opposite point. On the contrary, abstract AGI safety nonsense taking such a strong grip on academic and industrial AI Safety research is what would most frighten me. In the intervening decades, people concerned about software safety provi…
>while sci-fi about killer robots was entirely unhelpful in preventing software bugs.
Unless we are now talking about actually building AI and killer robots, which we are assuredly doing.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#516Earlier quoted context omitted.
What? I never proposed that dichotomy and don't believe in it. You did and you're tripping over it a bit. You can just discard that model and engage with the substance of the topic, you know!
The substance of this discussion is that a language model has never convinced anyone to crash a plane into a building and your position is that pointing out that fact is to ignore the material reality of the possibility of some theoretically advanced technology (which could maybe be adjacent to language models) at some point convincing a hypothetical person to crash a plane into a building. That is to say, the substa…
Why aren't you mentioning that an LLM has already convinced someone to commit suicide?
Why are you being deliberately obtuse about the unknowable-ness of the exponential curve the enthusiasts on one hand salivate over, and on the other confidently assert optimism just as you're doing?
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#517Earlier quoted context omitted.
I suspect OP that your respnding to thinks something like … Hofstadter is not known to have made himself uber-rich therefore he’s not worth taking advice from or something along those lines
Enlighten me. What has his research agenda resulted in? What did Hofstadter get right that others did not?
Well, you got that guy...
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#518Earlier quoted context omitted.
Yeah, I don’t see a reason for any AI to be able to translate all concepts to human thoughtspace. If an AI is able to have exponentially more possible thoughts than a human, then only a tiny subset would be understood by humans. It’s be like trying to fit GPT-4 onto a floppy disk.
A floppy disk is a fixed size. The number of thoughts human language can convey is infinite. English Wikipedia has 6.6M articles. The AI could drop a Wikipedia-sized batch of articles, expertly written and cross-referenced, every day, forever. At the same interval they could drop 100 million YouTube videos, expertly authored and hyper-clear. So yes there might be an infinite amount they cannot convey, but there is al…
There are an infinite number of rational numbers, but if you only understood the rationals then you wouldn’t understand pi or e. Or even sqrt(2).
Infinities can be very constraining.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#519Earlier quoted context omitted.
I love how you jumped straight from "people weirdly into eugenics should be disavowed" to "dysgenics is desirable". That is the epitome of black-and-white thinking if I ever saw it.
If a third alternative existed that would be a meaningful objection.
The lines are being drawn such that counseling to abort fetuses with Down syndrome is called eugenics, rhetorically equal to Nazi racial purges.
"In a 1985 book titled Should the Baby Live?, Singer and his coauthor warn their audience that “this book contains conclusions which some readers will find disturbing. We think that some infants with severe disabilities should be killed.” Why? In part because of the burden they’d place on society.
This is eugenics of the darkest sort"*
-- https://www.truthdig.com/articles/nick-bostrom-longtermism-a...
The funny thing is that these people are pro abortion. They fully support your right to abort a baby for any reason, even if you just want to go drinking, unless you're doing it because the baby is actually unhealthy. And then, oh no, you have to keep it.
Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk
#520Earlier quoted context omitted.
Here is ChatGPT 4’s summary of a sequel. I don’t think it’s very good. Maybe another AI would enjoy it though. > Create a summary of a sequel to the movie “War Games” Title: "War Games: Firewall" In "War Games: Firewall", now-adult computer genius David Lightman (Matthew Broderick) has been working as a cyber security consultant for the Pentagon. When an advanced, multi-national cyber-terrorist organization emerges,…
I feel this example points to a key issue regarding LLM AI. I've read the work on AI story generation from early days to now, and GPT-4's output shows a jump in performance that leaves me both elated and dismayed. The fluency and coherence of the generated narrative is incredible, frankly better than most student writing. On the other hand, I get an overwhelming sense of banality. Looking back on historical speculati…
The examples you're looking at are coming from systems that have undergone long courses of reinforcement learning specifically teaching them to be banal.