Live data from Hacker News

Scamlexity: When agentic AI browsers get scammed

guard.io

1–10 of 198 posts

Re: Scamlexity: When agentic AI browsers get scammed

#4
I appreciate that the article correctly points out the core design flaw here of LLMs is the non-distinction between content and commands in prompts.

It’s unclear to me if it’s possible to significantly rethink the models to split those, but it seems that that is a minimal requirement to address the issue holistically.

Re: Scamlexity: When agentic AI browsers get scammed

#5
This should hit the headlines.

I was always of the opinion that AI of all kinds is not a threat unless someone decides to connect it to an actuator so that it has direct and uncontrolled effect on the external world. And now it's happening en masse with agents, MCPs etc. I don't even mention things we don't know about (military and other classified projects).

Re: Scamlexity: When agentic AI browsers get scammed

#7
post #4

I appreciate that the article correctly points out the core design flaw here of LLMs is the non-distinction between content and commands in prompts. It’s unclear to me if it’s possible to significantly rethink the models to split those, but it seems that that is a minimal requirement to address the issue holistically.

Ah, it's like the good old days when operating systems like DOS didn't really make the distinction between executable files and data files. It would happily let you run any old .exe from anywhere on Earth. Viruses used to spread like wildfire until Norton Antivirus came along.

Re: Scamlexity: When agentic AI browsers get scammed

#8
post #5

This should hit the headlines. I was always of the opinion that AI of all kinds is not a threat unless someone decides to connect it to an actuator so that it has direct and uncontrolled effect on the external world. And now it's happening en masse with agents, MCPs etc. I don't even mention things we don't know about (military and other classified projects).

You don't have to guess about the military applications, it's all over the news. Even bog standard FPV drones that Ukraine is churning out at a rate of >100k/month have image recognition these days, so that if the video stream gets jammed they can finish off the mission autonomously.

Even on a hobby level, ardupilot+openCV+cheap drone kit from amazon is a DIY project within the skill set of a significant part of the visitors of this very site.

Re: Scamlexity: When agentic AI browsers get scammed

#9
“Agentic” seems the be some quick pivot buzzword that the AI grifters started pushing as soon as generic AI started to show cracks.

“Hey this AI stuff looks a bit overhyped.”

“AI? Oh that’s kids stuff, let me tell you about our agentic features!”

Giving flaky shaky AI the ability to push buttons and do stuff. What could possibly go wrong? Malicious actors will have a field day with this.

Re: Scamlexity: When agentic AI browsers get scammed

#10
post #5

This should hit the headlines. I was always of the opinion that AI of all kinds is not a threat unless someone decides to connect it to an actuator so that it has direct and uncontrolled effect on the external world. And now it's happening en masse with agents, MCPs etc. I don't even mention things we don't know about (military and other classified projects).

Yea I’ve been surprised about the risk conversations because they cannot do anything if you run them in a sandbox. But it seems like for many the assumed part was we’d hook LLMs into everything asap. It’s absolutely mind boggling.
Post reply on HN