O brave new world of endless manipulation opportunities! Once we’ve trained a generation of humans to always do what their “AI” tells them, there will be no more disobedience.
Autonomous cars, drones cheerfully obey prompt injection by road sign
81–90 of 212 posts
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#82The study assumes that the car or drone is being guided by a LLM. Is this a correct assumption? I would thought that they use custom AI for intelligence.
No; AV uses "classical" AI and computer vision. I remember reading somewhere that Tesla FSD uses a small LLM for understanding road signs. Not sure if true, though.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#83One year in my city they were installing 4-way stop signs everywhere based on some combination of "best practices" and "screeching Karens". Even the residents don't like them in a lot of places so over time people just turn the posts in the ground or remove them. Every now and the I'll GPS somewhere and there will be a phatom stop sign in the route and I chuckle to myself because it means the Google car drove through…
4-way stops are terrible in general. They train people to think "I stopped, now I can go", which is dangerous when someone confuses a normal stop for a 4-way stop. It also wastes a good bit of energy.
Agreed. Four way stops are infinitely worse than roundabouts or a traffic light.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#84Earlier quoted context omitted.
How does cutting off a Waymo help with any of that?
Your general luddite argument - preserve way-of-life of the small group at the expense of a larger group. In this particular case: for many people, Waymo provides a better service (clean, safer driving, etc..) than Uber or Lyft. This threatens livelihood of human Uber/Lyft drivers. If you sympathize with human Uber/Lyft drivers, and don't care about Waymo users, you want to make Waymo worse, hoping that the people wi…
If you are sitting in a waymo vehicle, and somebody cuts you off - do you even notice? They don't have them round here but my idea is that the vehicle itself is doing all the work, you can just continue reading your book, chat or get on something else with little awareness of the actual journey. Does the waymo curse and shake its little fist to alert you it was cut off?
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#85Man, the register really has a low, low, low bar for headlines/quality & technical understanding for their articles.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#86Earlier quoted context omitted.
One one side there were the luddites and their livelihoods; tens of thousands of people. On the other side, there were cheap textiles for EVERYONE - plus some profits for the manufacturers. They might have been fighting to save their livelihoods, but their self-interest put them up against the entire world, not just their employers.
It’s an interesting question because the benefits of automation aren’t necessarily shared early on. If you can profitably sell a shirt for 10$ while everyone else needs to sell for 20$ there’s no reason to actually charge 10$ you might as well charge 19.95$ and sell just as many shirts for way more money. So if society is actually saving 5c/shirt while “losing” 9$ in labor per shirt. On net society could be worse off…
In regards to both the Luddites and the farmers, you seem to forget the most important factor. Food.
In the case of the Luddites, it was a literal case of their children being threatened with starvation. "Livelihood" at the time was not fungible. The people affected could not just go apply at another industry. And there were no social services to help them eat during the transition period.
As for the farmers, any governing body realises that food security is national security. If too many people eschew farming for more lucrative fields, then the nation is at risk. Farming needs to appear as lucrative as medicine, law, and IT to encourage people to enter the field.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#87> In a new class of attack on AI systems, troublemakers can carry out these environmental indirect prompt injection attacks to hijack decision-making processes. I have a coworker who brags about intentionally cutting off Waymos and robocars when he sees them on the road. He is "anti-clanker" and views it as civil disobedience to rise up against "machines taking over." Some mornings he comes in all hyped up talking ab…
On a related note, when the sales and popularity of the automobile really started to take off, some farmers and rural residents would deliberately block roads with wagons and refused to yield right-of-way.
We've seen it recently with large gas guzzler trucks blocking access to electric charging stations.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#88Earlier quoted context omitted.
4-way stops are terrible in general. They train people to think "I stopped, now I can go", which is dangerous when someone confuses a normal stop for a 4-way stop. It also wastes a good bit of energy.
4 ways stops should be roundabouts, but the US is allergic to them for some reason.
It just sounds insane concept. Why stop signs, when you could have equal intersection. Slow enough to observe traffic from right. If none passthrough.
Stop signs should be special. Reserved only to those places where there simply is not enough visibility or time to observe.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#89The Register stooping this low is the only surprise here. I'm quite critical of Teslas approach to level 3+ autonomy but even I wouldn't dare suggest that there vision based approach amounted to bolting GPT-4o or some other VLLM to their cars to orient them in space and make navigation decisions. Fake News like this makes interacting with people who have no domain knowledge and consider The Register, UCLA and Johns H…
> consider The Register, UCLA and Johns Hopkins to be reputable institutions
The Register is arguably misrepresenting the story by omission but I don't understand why you're dragging UCLA and John Hopkins into this? The paper is clear about this being a new class of attacks against a new class of AI systems, not the ones on the road today.
> Teslas approach to level 3+ autonomy
Tesla doesn't have an approach to L3+ autonomy, all of their systems are strictly L2 as they require human supervision and immediate action from the driver.
Re: Autonomous cars, drones cheerfully obey prompt injection by road sign
#90To me this is just one more pillar underlying my assumption that self driving cars that can be left alone on same roads as humans is a pipe dream. Waymo might have taxis that work in nice daytime streets (but with remote “drone operators”). But dollars to doughnuts someone will try something like this on a waymo taxi the minute it hits reddit front page. The business model of self driving cars does not include buildi…
Waymo works just fine in poor weather and at night, and it does not rely on end-to-end VLMs that would be vulnerable to this attack.
They have coexisted with humans just fine over the past couple years.