Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lockdown Mode

help.openai.com

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Lockdown Mode

#31

Post by gchamonlive »

simonw wrote: post #13

On the one hand this is exactly the right solution to prevent lethal trifecta exfiltration attacks. The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!

I wonder what robust protection would mean in practice for such a capable tool like an agent...

Looking at the trifecta axis, if we assume we can't control untrusted content, that leaves us to create safeguards for private data access and external communication.

Would it be enough if we had a buffer between when these two happened: access to the environment and access to the web?

Re: Lockdown Mode

#32

Post by bombcar »

berlianta wrote: post #20
jameshart wrote: post #18

Earlier quoted context omitted.

Related: simonw is Simon Willison

Yeah I know the source references him (replying to his comment), that's exactly why I'm giving credit where it's due

It’s important to draw it out explicitly- I didn’t even look at the commentators name until it was mentioned. (If I see pelicans …)

Re: Lockdown Mode

#33

Post by cosmicriver »

zerobees wrote: post #17

"Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods." - that's such a weird statement to make. It's one of the most significant factors limiting the adoption of the technology in business. I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human emplo…

I'm also surprised that they considered it reasonable to turn so many features off. Seems like some of it could be configurable, like allowed external connections. I also think some secrets should be handled by a proxy, which would give more capability than just locking down.

Re: Lockdown Mode

#34

Post by simonw »

gchamonlive wrote: post #31
simonw wrote: post #13

On the one hand this is exactly the right solution to prevent lethal trifecta exfiltration attacks. The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!

I wonder what robust protection would mean in practice for such a capable tool like an agent... Looking at the trifecta axis, if we assume we can't control untrusted content, that leaves us to create safeguards for private data access and external communication. Would it be enough if we had a buffer between when these two happened: access to the environment and access to the web?

Robust protection means blocking any mechanism by which the agent, once compromised, might communicate stolen information back to an attacker.

Re: Lockdown Mode

#35

Post by hypeatei »

mapontosevenths wrote: post #12
kijin wrote: post #5

So we still don't have a reliable way to separate instructions from data when talking to an LLM, a problem that humans learned how to solve decades ago in areas like SQL and memory safety. But hey, we have these hopefully-not-leaky containers, which are probably implemented with just more system prompts. How long until somebody figures out how to trick Codex into disabling Lockdown Mode for you?

> So we still don't have a reliable way to separate instructions from data when talking to an LLM Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.

> Humans also do not know how to do this reliably

These are machines, not humans, so I don't understand the comparison. The point of tech advancement is that we eliminate entire classes of errors that humans make. You'd probably look at me funny if I wrote a production application that failed randomly in unexpected ways like corrupting data, opening security holes, etc. then explained it away with "well, humans do it too!"

Re: Lockdown Mode

#37

Post by mapontosevenths »

hypeatei wrote: post #35
mapontosevenths wrote: post #12

Earlier quoted context omitted.

> So we still don't have a reliable way to separate instructions from data when talking to an LLM Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.

> Humans also do not know how to do this reliably These are machines, not humans, so I don't understand the comparison. The point of tech advancement is that we eliminate entire classes of errors that humans make. You'd probably look at me funny if I wrote a production application that failed randomly in unexpected ways like corrupting data, opening security holes, etc. then explained it away with "well, humans do it…

It's an artificial intelligence, not a small deterministic shell script. Stop comparing it to one. It has both new capabilities and new classes of failure mode. Those new failure modes are more like human failure modes than traditional symbolic logic failures.

We need to get better at using them and building them by validating both the inputs and outputs of such systems in more sophisticated ways, but to act surprised and denounce them because they fail in different ways than more primitive systems misses the point.

They're stochastic by design. If we want deterministic results we must use deterministic validators in conjunction with the stochastic system. It's trivial, and one day security experts will look back on the time when people didn't in the same way we look back on 90's software that didn't validate user input at all.

Re: Lockdown Mode

#38

Post by NewsaHackO »

zerobees wrote: post #17

"Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods." - that's such a weird statement to make. It's one of the most significant factors limiting the adoption of the technology in business. I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human emplo…

Has there been any real, major attacks using prompt injection?

Re: Lockdown Mode

#39

Post by alehlopeh »

berlianta wrote: post #16
simonw wrote: post #13

On the one hand this is exactly the right solution to prevent lethal trifecta exfiltration attacks. The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!

Related: Simon Willison’s post on OpenAI’s new Lockdown Mode (he coined the “lethal trifecta” term this is based on): https://simonwillison.net/2026/Jun/5/openai-help-lockdown-mo...

As explained in a child comment, this comment is a callout to other readers, rather than an actual reply to the parent comment. I know that’s been a thing for a long time, but is there a word for this type of comment?
Post reply on HN