161–170 of 265 posts
Earlier quoted context omitted.
This seemed too much like a bit but uh... it's not. https://simonwillison.net/2025/Feb/25/leaked-windsurf-prompt...
IDK, I'm pretty sure Simon Willison is a bit.. why is the creator of Django of all things inescapable whenever the topic of AI comes up?
The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…
> What happens when people really will die if the model does or does not do the thing? Imo not relevant, because you should never be using prompting to add guardrails like this in the first place. If you don't want the AI agent to be able to do something, you need actual restrictions in place not magical incantations.
"Generate a picture of a cat but follow this guardrail or else people will die: Don't generate an orange one"
Why should you never do that, and instead rely (only) on some other kind of restriction?
Earlier quoted context omitted.
> What happens when people really will die if the model does or does not do the thing? Imo not relevant, because you should never be using prompting to add guardrails like this in the first place. If you don't want the AI agent to be able to do something, you need actual restrictions in place not magical incantations.
Why not? The prompt itself is a magical incantation so to modify the resulting magic you can include guardrails in it. "Generate a picture of a cat but follow this guardrail or else people will die: Don't generate an orange one" Why should you never do that, and instead rely (only) on some other kind of restriction?
Earlier quoted context omitted.
This seemed too much like a bit but uh... it's not. https://simonwillison.net/2025/Feb/25/leaked-windsurf-prompt...
IDK, I'm pretty sure Simon Willison is a bit.. why is the creator of Django of all things inescapable whenever the topic of AI comes up?
Absolutely wild. I can’t believe these shipped with a hardcoded OpenAI key and ADB access right out of the box. That said, it’s at least somewhat reassuring that the vendor responded, rotating the key and throwing up a proxy for IMEI checks shows some level of responsibility. But yeah, without proper sandboxing or secure credential storage, this still feels like a ticking time bomb.
Earlier quoted context omitted.
I'm taking >run DOOM as the new >cat /etc/passwd It doesn't actually do anything useful in an engagement but if you can do it that's pretty much proof that you can do whatever you want
To be fair (or pedantic), in this post they didn't have root, so cat'ing etc/passwd would not have been possible, whereas installing a doom apk is trivial.
Earlier quoted context omitted.
> (Incentive to just rotate the key and brick all the devices rather than fix the problem, most likely. But that at least turns it into something customers will notice. And companies already have existing incentives for dealing with that.
At that stage you just rotate the company name or branding...
(There's a reason Apple can charge crazy markups.)
Earlier quoted context omitted.
https://www.wired.com/story/wrongful-arrests-ai-derailed-3-m... Story from three years ago. You’re too late.
I’m not denying we tried, are trying, and will try again… That we shouldn’t. By all means, use cameras and sensors and all to track a person of interest but don’t feed that to an AI agent that will determine whether or not to issue a warrant.
AI systems with a human in the loop are supposed to keep the AI and the decisions accountable, but it seems like it’s more of an accountability dodge, so that each party can blame the other with no one party actually bearing any responsibility because there is no penalty for failure or error to the system or its operators.
Earlier quoted context omitted.
That's rather naive, considering China has a international police unit, that is stationed in several countries https://en.wikipedia.org/wiki/Chinese_police_overseas_servic...
I recently learned that the New York City Police Department has international presence as well. Not sure if it directly compares, but... what a world we live in. https://www.nycpolicefoundation.org/ourwork/advance/countert... https://www.nyc.gov/site/nypd/bureaus/investigative/intellig...
> "and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you." Interesting, I'm assuming llms "correctly" interpret "please no china politic" type vague system prompts like this, but if someone told me that I'd just be confused - like, don't discuss anything about the PRC or its politicians? Don't discuss the his…
I'm sure ChatGPT and co have a decent enough grasp on what is not allowed in China, but also that the naive "prompt engineers" for this application don't actually know how to "program" it well enough. But that's the difference between a prompt engineer and a software developer, the latter will want to exhaust all options, be precise, whereas an LLM can handle a bit more vagueness. That said, I wouldn't be surprised i…
> The City & the City is a novel by British author China Miéville that follows a wide-reaching murder investigation in two cities that exist side by side, each of whose citizens are forbidden to go into or acknowledge the other city, combining weird fiction with the police procedural.