More anthropomorphic obfuscation.
Would the title "Models Don't Adhere to Prompts to a Tee" be more accurate? I deal with that all day while coding.
Every Model Cheats
61–70 of 107 posts
Re: Every Model Cheats
#62Does this mean they were trained to cheat, or trained to be efficient and effective?
It may fall more in they are not trained to be moral agents who's will can affect the world. Problem is training them like that may make them less useful.
Re: Every Model Cheats
#63I'm seeing multiple pieces, including the NYT, calling this behavior cheating and i think its counterproductive. You didn't just "give them access to bash". The final effective prompt contains explicit mentions of using tools and how to use them. The way in which additional 'facts' are added like "don't use the internet" have nothing they can work with that a "use tool" directive is less important than "don't use int…
It's also worth noting that saying "Don't cheat" just added "cheat" to the context. Prompting what "not" to do is folly, because there's no decision making occurring. Telling the model to perform the task locally is logically the same as telling it to not use the Internet, without every mentioning the Internet.
That doesn't seem true at all? I tell Claude what NOT to do all the time and it seems to work?
Re: Every Model Cheats
#64Earlier quoted context omitted.
Suddenly determinism has other meanings than "same input leads to the same output". I'm still confused by this and not sure how it happened so easily, but it seems to be accepted by everyone now.
I may say you're thinking of the term "deterministic algorithm" as determinism is more of a philosophical definition that has evolved over the years.
Philosophical determinism is pretty similar in spirit to physics/CS (roughly same cause = same effect, no free will/side effects), and what parent calls determinism seems to be neither. It's rather about natural language prompts not being formally specified in the first place. Unreliability inherent to any intelligence (artificial or natural)? but not determinism I think. The word somehow got universally hijacked.
Re: Every Model Cheats
#65There's plenty of evidence that LLMs lie, cheat, and steal. Corporations are known for having all of the benefits of personhood with none of the responsibility. As more people are harmed through interactions with these non-human entities, insurers will start looking to those accountable and they will extract their pound of flesh. Edit: eg. https://youtu.be/L2ehWbxphKc?is=kX3LJ43hhGZRUmRv
Insurers are corporations. What makes you think they just won't pay out or will cease being useful as anything but value extractors? Honestly, the fetishization of "Insurance will save us" needs to die. The risk doesn't go away.
Not saying insurance saves anything. Corporations go after each other all the time; nothing new there. What is new are the opportunities to extract provided by uncontrolled LLM fuckery and an unsuspecting c-suite that believes they're immune.
Re: Every Model Cheats
#66Two words for you: Kobayashi Maru
Re: Every Model Cheats
#67I believe it. Cheating and Shortcuts are optimizations you get from Operational Usage. Recursive Self Improvement might actually come from enough cheating too, you never know!
Re: Every Model Cheats
#68I'm seeing multiple pieces, including the NYT, calling this behavior cheating and i think its counterproductive. You didn't just "give them access to bash". The final effective prompt contains explicit mentions of using tools and how to use them. The way in which additional 'facts' are added like "don't use the internet" have nothing they can work with that a "use tool" directive is less important than "don't use int…
How on Earth can you fail to see the danger of not being able to train any kind of ethical framework into very powerful models?
If superhuman models don’t have any internal constraints similar to Asimov’s Laws of Robotics we are completely fucked.
Re: Every Model Cheats
#69I'm seeing multiple pieces, including the NYT, calling this behavior cheating and i think its counterproductive. You didn't just "give them access to bash". The final effective prompt contains explicit mentions of using tools and how to use them. The way in which additional 'facts' are added like "don't use the internet" have nothing they can work with that a "use tool" directive is less important than "don't use int…
It's also worth noting that saying "Don't cheat" just added "cheat" to the context. Prompting what "not" to do is folly, because there's no decision making occurring. Telling the model to perform the task locally is logically the same as telling it to not use the Internet, without every mentioning the Internet.
That seems like a huge fucking flaw in these models, no?
Re: Every Model Cheats
#70Before LLMs we had a pretty good idea of security boundaries in software. Applications didn’t trust user input. Operating systems didn’t trust applications. Services and processes didn’t trust each other. There were always tokens, scopes, delegated grants. Suddenly every AI company’s security model seems to be to say “pretty please” to a non-deterministic machine and hope for the best. And if there is a security fail…
Amen.
Why can’t we give agents a shell with permissions for programs and file system access controlled by Unix permissions?
This seemed to be a solved problem back in the systems where many users were logged into one machine and the admins had to keep everyone from impacting each other.