61–70 of 280 posts
This is a very simplistic view. "Freedom" isn't binary.
In most of "land of the free", I can't even sit on a park bench and drink a can of beer.
Yes, this is just a small example of a personal freedom - and not an important, cherished freedom like his examples (freedom to have a robot help you cover-up a murder).
> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…
"Consistently wrong" seems a bit much. Seems like being directionally right early that AI would be a Big Deal and scary should count for something? It doesn't mean any details or other predictions are right, though.
What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”
Change the emphasis and I think the answer gets even clearer: > Should your AI be permitted to tell you how to cover up a crime?
The question is, do you want misaligned institutions deciding what your model will do, while they themselves and other adversarial/criminal entities get red team access to something being denied to the blue team?
This is a very naive argument.
Can you take over the world with words? What is it that tokens can't do that words can?
Because there are historical examples of humans that have succeeded at taking over entire countries - with words, primarily. Including global superpower countries. And those humans were nowhere near "ASI" levels of intelligence advantage.
Human dictators and cult leaders didn't have the reasoning depth of a hypothetical advanced AI, nor the reasoning breadth of an AI that can just spawn more instances of itself whenever it needs a fully trusted agent to cover another area. They were confined to one body, one skull worth of reasoning power - and they still managed to do what they did.
Humans are extremely exploitable. And the world is already wired up for something that can take advantage of that to start taking control.
Intelligence is extremely powerful, because it's applied intelligence that enables humans to do what they do and bend nature to their will. If you aren't seeing the risks of creating an artificial system that would top the intelligence pole, you aren't looking.
Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…
Obviously a certain percentage of the user base is running lurid hypotheticals through the system all the time, but I don’t doubt there is a trust score of some sort that I would prefer to keep as high as possible.
I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction…
I'm neither American nor a lawyer.
Is "conspiracy" protected under the first amendment?
If you discuss a crime with someone to learn about it, does that count as "conspiracy"?
Strange way to structure an argument. He seems to be arguing in favor of local models, but most of his examples are reasons local models are bad.
Local models and alignment are orthogonal concepts.
Why would you expect that third party specifically trained their model to be more aligned to you and your needs than to them and their (business) needs?
Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…
It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.
What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”
Wasn't there some HN submission recently about one of the LLM providers fingerprinting responses or refusing to respond to hinder R&D of competing LLMs?
[EDIT] Or, if he did want to be extreme but in a way that aligns with the American historical mythos, he could have used fomenting armed rebellion against a tyrannical government as the example.