201–210 of 265 posts
Earlier quoted context omitted.
To be fair (or pedantic), in this post they didn't have root, so cat'ing etc/passwd would not have been possible, whereas installing a doom apk is trivial.
/etc/passwd is world readable by default.
Their email responses all show telltale signs of AI too which is pretty funny.
Earlier quoted context omitted.
Why not? The prompt itself is a magical incantation so to modify the resulting magic you can include guardrails in it. "Generate a picture of a cat but follow this guardrail or else people will die: Don't generate an orange one" Why should you never do that, and instead rely (only) on some other kind of restriction?
Are people going to die if your AI generates an orange cat? If so, reconsider. If not, it's beside the discussion.
Earlier quoted context omitted.
Why not? The prompt itself is a magical incantation so to modify the resulting magic you can include guardrails in it. "Generate a picture of a cat but follow this guardrail or else people will die: Don't generate an orange one" Why should you never do that, and instead rely (only) on some other kind of restriction?
Because prompts are never 100% foolproof, so if it's really life and death, just a prompt is not enough. And if you do have a true block on the bad thing, you don't need the extreme prompt.
Earlier quoted context omitted.
This seemed too much like a bit but uh... it's not. https://simonwillison.net/2025/Feb/25/leaked-windsurf-prompt...
IDK, I'm pretty sure Simon Willison is a bit.. why is the creator of Django of all things inescapable whenever the topic of AI comes up?
- oh, it's that guy again
+ prodigiously writes and shares insights in the open
+ builds some awesome tools, free - llm cli, datasette
+ not trying to sell any vendor/model/service
On balance, the world would be better of with more simonw shaped people
Earlier quoted context omitted.
Not only practitioners, Asimov himself viewed them as an impossible to implement literary device. He acknowledged that they were too vague to be implementable, and many of his stories involving them are about how they fail or get "jailbroken", sometimes by initiative of the robots themselves. So yeah, it's quite sad that close to a century later, with AI alignment becoming relevant, we don't have anything substantial…
Not sad, before it was SciFi and now we are actually thinking about it.
Honestly, getting into the whole AI alignment thing before it was hot[0], I imagined problems like Evil People building AI first, or just failing to align the AI enough before it was too late, and other obvious/standard scenarios. I don't think I thought of, even for a moment, the situation in which we're today: that alignment becomes a free-for-all battle at every scale.
After all, if you look at the general population (or at least the subset that's interested), what are the two[1] main meanings of "AI alignment"? I'd say:
1) The business and political issues where everyone argues in a way that lets them come up on top of the future regulations;
2) Means of censorship and vendor lock-in.
It's number 2) that turns this into a "free-for-all" - AI vendors trying to keep high level control over models they serve via APIs; third parties - everyone from Figma to Zapier to Windsurf and Cursor to those earbuds from TFA - trying to work around the limits of the AI vendors, while preventing unintended use by users and especially competitors, and then finally the general population that tries to jailbreak this stuff for fun and profit.
Feels like we're in big trouble now - how can we expect people to align future stronger AIs to not harm us, when right now "alignment" means "what the vendor upstream does to stop me from doing what I want to do"?
--
[0] - Binged on LessWrong a decade ago, basically.
[1] - The third one is, "the thing people in the same intellectual circles as Eliezer Yudkowsky and Nick Bostrom talked about for decades", but that's much less known; in fact, the world took the whole AI safety thing and ran with it in every possible direction, but still treat the people behind those ideas as crackpots. ¯\_(ツ)_/¯
Earlier quoted context omitted.
This seemed too much like a bit but uh... it's not. https://simonwillison.net/2025/Feb/25/leaked-windsurf-prompt...
IDK, I'm pretty sure Simon Willison is a bit.. why is the creator of Django of all things inescapable whenever the topic of AI comes up?
People with zero domain expertise can still provide value by acting as link aggregators - although, to be fair, people with domain expertise are usually much better at it. But some value is better than none.
Earlier quoted context omitted.
> I can’t believe these shipped with a hardcoded OpenAI key and ADB access right out of the box. As someone with a lot of experience in the mobile app space, and tangentially in the IoT space, I can most definitely believe this, and I am not surprised in the slightest. Our industry may "move fast", but we also "break things" frequently and don't have nearly the engineering rigor found in other domains.
It was a good thing for user privacy that the keys were directly on the device, it is only in DAN mode that a copy of the chats were sent. So eventually if they remove the keys from the device, messages will have to go through their servers instead.
You want to think through that one again? With the OpenAI key on device it means anyone could use that key to call (and bill) OpenAI's APIs. It's absolutely not feasible to ship the OpenAI keys on device.
Earlier quoted context omitted.
There is no difference to other countries. In France if you say bad things about certain groups of people then you can literally go to jail (but the censorship is directly IN the models)
You don't feel there's a difference between a State banning criticism of the State, and a State passing anti-hate speech laws to protect people from, e.g., nazis?