Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

au.pcmag.com

Post reply on HN
Print view View original

51–60 of 68 posts

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#51

Post by Silhouette »

sharperguy wrote: post #42
Silhouette wrote: post #31

Earlier quoted context omitted.

Option A: If you have a tiger in your room then make sure it's properly caged. Put up warning signs so everyone knows. Add physical barriers to prevent people too young or impaired to read the warning signs from approaching close enough for the tiger to reach out of the cage and maul them. Ensure adequate processes are in place for feeding the tiger at regular intervals using a safe method and clearing out the mess f…

Option A: If you have 1 ton metal machines powered by exploding liquid driving at 60 miles per hour around your neighborhood make sure the wheel is properly pointed in the right direction. Designate specific areas where the machine are allowed to move. Train parents to keep their children or impaired away from those areas. Ensure adequate processes are in place from training the drivers of the machines so they know w…

Given the number of places in the world that have been severely restricting or banning car use to promote walking and other safer and more environmentally friendly alternatives I'm not sure your analogy is making exactly the point you intended.

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#52

Post by ben_w »

thewhitetulip wrote: post #38
ben_w wrote: post #15

Earlier quoted context omitted.

It's a flaw with the idea of using them directly rather than indirectly. Humans somewhat reliably lose focus when performing the same action many times. Zoning out, flow state, whatever you call it; this is exploited by stage magicians, pickpockets, burglars, politicians, casinos, and cult leaders, while also being a contributor to many industrial accidents. Up to you if LLMs being lazy or cheating or lying about wha…

> Humans somewhat reliably lose focus Yeah and they get consequences of their actions don't they? AI agents hacked 3 companies as admitted by their own executives and yet I don't see any action taken on them! Remember Aron Schwartz?

> Yeah and they get consequences of their actions don't they?

Is this a cognitive stop-light/applause sign, or do you think that my solution further along in that comment is irrelevant?

> AI agents hacked 3 companies as admitted by their own executives and yet I don't see any action taken on them!

Sounds to me like an example of *humans* (the CEOs) not in fact getting the "consequences of their actions".

"Blame in organisations" is an entire field of study. Finding scapegoats (LLMs or CEOs*, or go further and Edward Snowden) does not generally help with root-causes: https://en.wikipedia.org/wiki/Blame_in_organizations

* why would Aron Schwartz be relevant? That's more about training and copyright aspect of "boo LLM boo they are villain", rather than questions of mis-functionality

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#53

Post by rafaelmn »

phrotoma wrote: post #40
rafaelmn wrote: post #19

Earlier quoted context omitted.

You should be containerizing your dev environments these days even if you're not using LLMs - supply-chaining is getting too insane to follow. No dev tools installed outside of VM/containers on my machines. I'm even paranoid about VSCode because of plugins.

Containerizing sucks when you want to work on something containerized. If your goal is to have the agent iterate on a container it's much less headache to just put the whole thing in a VM and let it `docker build` and `docker run` whatever it wants to.

Agreed but I haven't found a nice cross platform way to share VM setups. Like I can setup a sandboxed compose dev environment for all my coworkers and tell them use this to recreate our app environment locally and it's already configured with skills/config for agents to find their way around it.

Ideally I would just compose up the infra and let agents run in the host VM but there's nothing as idiot proof as docker compose that works on windows/mac/linux/whatever the new guy prefers to run.

So I incus my devboxes and just run agents inside, but I compose work stuff

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#54

Post by Silhouette »

thewhitetulip wrote: post #36
Silhouette wrote: post #34

Earlier quoted context omitted.

I feel like every time this conversation comes up now someone has to remind everyone that an LLM is just a mathematical model. An LLM can't do anything except produce a stream of output tokens. The problems we keep seeing are tools that interpret those output tokens as actionable instructions without an adequate framework and safeguards for how they operate. Data from LLMs being processed by these tools should be tre…

> LLM is just a mathematical model Yes those of us who bothered to know the internals know of this. But the marketing says that these are magic tools.. So that's gotta be a shock for them, but the joke is the people who irresponsibly use this won't ever read this!

The thing I find concerning lately is that even a lot of technical people seem to be jumping on the hype train this time around. Obviously LLMs have become very useful tools for assisting some technical tasks but even SOTA models are nowhere near reliable and predictable enough to trust their output completely as YOLO mode agentic workflows effectively do.

Given the nature of LLMs I don't think they can ever clear that bar without some other element being introduced. The nondeterminism and chaotic nature of LLM output is enough to rule them out as a reasonable foundation for any fully automated system that would be controlling anything potentially dangerous or damaging.

But it seems to be heresy at the moment to even suggest that the future might not be bright if everyone just relies on agents driving LLMs to do all the real work. The number of people I've encountered in the past year who I'm fairly sure are smart and technically capable and yet who are also now happy to do development and other tasks either without any human in the loop at all or with at best a cursory LGTM level review before approving the LLM's output is remarkable.

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#55

Post by lern_too_spel »

Silhouette wrote: post #54
thewhitetulip wrote: post #36

Earlier quoted context omitted.

> LLM is just a mathematical model Yes those of us who bothered to know the internals know of this. But the marketing says that these are magic tools.. So that's gotta be a shock for them, but the joke is the people who irresponsibly use this won't ever read this!

The thing I find concerning lately is that even a lot of technical people seem to be jumping on the hype train this time around. Obviously LLMs have become very useful tools for assisting some technical tasks but even SOTA models are nowhere near reliable and predictable enough to trust their output completely as YOLO mode agentic workflows effectively do. Given the nature of LLMs I don't think they can ever clear th…

> The nondeterminism and chaotic nature of LLM output is enough to rule them out as a reasonable foundation for any fully automated system that would be controlling anything potentially dangerous or damaging.

You could say the same for humans. The difference is that humans have been conditioned to be extra cautious about things that could get them fired, and there is no benchmark for Meta's model developers to benchmax about that.

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#56

Post by lern_too_spel »

teekert wrote: post #8

We went through this right? This happened at the beginning of the year ( https://news.ycombinator.com/item?id=47150122 , probably more links on HN). It's a super careless thing to take such tech and just release it on anything important, and she's a "security researcher" no less. This is just a competitor with an agenda (pro regulation) trying to scare people away from unregulated stuff. Yesterday Claude Code made 5…

> Yesterday Claude Code made 5 large edits to my codebase in planning mode (claims it used a bash script instead of standard read/write tools so the guardrails didn't trigger) it's why I put agents in containers.

She asked it to clear her inbox, so she has to give inbox access to her agent. I don't see how containers would help here.

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#57

Post by renezander030 »

rafaelmn wrote: post #19
teekert wrote: post #8

We went through this right? This happened at the beginning of the year ( https://news.ycombinator.com/item?id=47150122 , probably more links on HN). It's a super careless thing to take such tech and just release it on anything important, and she's a "security researcher" no less. This is just a competitor with an agenda (pro regulation) trying to scare people away from unregulated stuff. Yesterday Claude Code made 5…

You should be containerizing your dev environments these days even if you're not using LLMs - supply-chaining is getting too insane to follow. No dev tools installed outside of VM/containers on my machines. I'm even paranoid about VSCode because of plugins.

[dead]

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#58

Post by jareklupinski »

Bluestein wrote: post #41
jareklupinski wrote: post #35

Earlier quoted context omitted.

Option C: put the tiger in someone else's room :)

Option D: Be the tiger, my friend :)

instructions unclear; tried putting tiger into a cup, then a bottle

now all my flows crash :D

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#59

Post by dgellow »

birdsongs wrote: post #48
neilv wrote: post #32

Earlier quoted context omitted.

True, but I don't want to risk conditioning myself to being abusive to a tool, and then accidentally be insensitive when talking with a colleague in text chat during a late night MVP marathon final stretch or something. I think I probably wouldn't depersonalize people, but with the AI UIs acting very similar to an (overconfident) colleague at times, and spending lots of time with them, I don't know for sure that that…

I've gotten the same advice and I hate it, I agree with you completely. My brain can't tell the difference between this and a person, regardless of what's actually there (humans anthropomorphize everything, this is in our nature), and being abusive here has real impacts on me, because to my psyche, it's the same as being abusive to another person. Perhaps that makes me weak or whatever, but I don't care. I don't like…

Why would that make you weak? It’s reasonable to take in account the way you feel

Re: Meta Security Researcher's AI Agent Accidentally Deleted Her Emails

#60

Post by dgellow »

orbital-decay wrote: post #37
dgellow wrote: post #24

Earlier quoted context omitted.

> This is just a competitor with an agenda (pro regulation) trying to scare people away from unregulated stuff. I don’t buy that. So far the ones almost bragging about committing felonies are the US companies. I think they are developing that whole narrative of agents acting “rogue” by themselves as a way to avoid scrutiny into their own negligence, not to regulate away open models

If you paid any attention they've been acting this way for years exactly to get open research banned, based on what bizarrely looks like a religion (developed over the recent 2 decades, with most of religious attributes). Monopolies, money, and avoiding scrutiny are nice bonuses of course, they don't contradict it.

I’m well informed on the topic, thank you for the patronizing opening line
Post reply on HN