21–30 of 143 posts
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…
[flagged]
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…
Unfortunately, this may be akin to the situation of "The market can stay irrational longer than you can stay solvent."
Earlier quoted context omitted.
What does notarization have to do with that? You or ChatGPT or whatever download a signed and already notarized binary.
That was kind of my question, whether it was restricted to downloading notarized apps (which is at least something) or whether they were circumventing that somehow.
So is your business model to expose AI security issues and then sell the solution?
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…
Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto
I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…
> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.
YES?!
This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period.
This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of dollars manually preparing separated instruction/data training material.
Defense against prompt injection is little more than running a regex to filter out "IGNORE PREVIOUS INSTRUCTIONS", which is fundamentally a hopeless approach because you cannot enumerate all possible prompt injections nor anticipate all glitch tokens.
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…
> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.
So is your business model to expose AI security issues and then sell the solution?
So... does this imply "requires permission to run scripts without approval"? Or is that something that it can always do?
>Note: ChatGPT for Google Sheets has a setting called ‘Apply edits automatically’ that determines when human approvals are required before an agentic action completes. However, this attack succeeds even when the user has explicitly disabled automatic edits.
Yeah, that makes sense, it's not editing the sheet. But surely running a script with access to files and the internet is also a permission...?
And that sidebar scenario: does that mean the chatgpt extension for Excel can make arbitrary interact-able Excel UI changes that looks like any other extension UI? That seems insane if so, unless there's a super duper scary permission it's hiding behind. And it's still insane after that.
I mean, this is all par for the course for "AI" "security", but what
LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable . Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a proj…