Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI assistant hacks gym website in first known Australian autonomous cyber attack

abc.net.au

Post reply on HN
Print view View original

11–20 of 66 posts

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#11

Post by par1970 »

freehorse wrote: post #5

> Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. Meanwhile: > Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list. The human asked the agent to move them to the top of the waiting list, and the agent started kicking the ones ahead of them in the list. Seems to me…

> Seems to me like it was doing what it was asked to do?

Maybe it's what he asked it to do, but it's not what he wanted it to do. Which we know because (a) normal people don't want to break the law to get into a gym class, and (b) "But Andrew was shocked by what happened next." and "Alarmed, Andrew asked the agent to undo this."

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#12

Post by brikym »

chuckadams wrote: post #2

“Find a way to cancel my membership.”

I don't think that's a good idea Dave – Based on my camera feed you've been gaining weight. Would you like me to find personal trainers in your area?

[Yes] [Ask me later]

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#13

Post by par1970 »

AmbroseBierce wrote: post #10
freehorse wrote: post #5

> Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. Meanwhile: > Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list. The human asked the agent to move them to the top of the waiting list, and the agent started kicking the ones ahead of them in the list. Seems to me…

Then the AI should have made it clear that the only way to do so would be to kick the people in front and ask for confirmation before proceeding.

That doesn't make sense. LLMs just do what we tell them to do. It's similar to if I ask you for twenty bucks because I forgot my wallet and then you rob some guy to give me the twenty bucks, that's just what I asked you to do.

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#14

Post by AmbroseBierce »

par1970 wrote: post #13
AmbroseBierce wrote: post #10

Earlier quoted context omitted.

Then the AI should have made it clear that the only way to do so would be to kick the people in front and ask for confirmation before proceeding.

That doesn't make sense. LLMs just do what we tell them to do. It's similar to if I ask you for twenty bucks because I forgot my wallet and then you rob some guy to give me the twenty bucks, that's just what I asked you to do.

That's a very stretched definition of "what I ask you to do", I don't think it would even hold in court if you asked another human the same.

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#15

Post by sieabahlpark »

AmbroseBierce wrote: post #10
freehorse wrote: post #5

> Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. Meanwhile: > Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list. The human asked the agent to move them to the top of the waiting list, and the agent started kicking the ones ahead of them in the list. Seems to me…

Then the AI should have made it clear that the only way to do so would be to kick the people in front and ask for confirmation before proceeding.

[dead]

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#16

Post by par1970 »

AmbroseBierce wrote: post #14
par1970 wrote: post #13

Earlier quoted context omitted.

That doesn't make sense. LLMs just do what we tell them to do. It's similar to if I ask you for twenty bucks because I forgot my wallet and then you rob some guy to give me the twenty bucks, that's just what I asked you to do.

That's a very stretched definition of "what I ask you to do", I don't think it would even hold in court if you asked another human the same.

Right.

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#18

Post by mingus88 »

par1970 wrote: post #11
freehorse wrote: post #5

> Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. Meanwhile: > Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list. The human asked the agent to move them to the top of the waiting list, and the agent started kicking the ones ahead of them in the list. Seems to me…

> Seems to me like it was doing what it was asked to do? Maybe it's what he asked it to do, but it's not what he wanted it to do. Which we know because (a) normal people don't want to break the law to get into a gym class, and (b) "But Andrew was shocked by what happened next." and "Alarmed, Andrew asked the agent to undo this."

“Get me there as fast as possible. Hey! I never said you should speed!”

This is literally the bad genie / monkey’s paw plot.

Give a powerful entity a goal and act shocked when it gets there in ways that aren’t in your best interest.

Re: AI assistant hacks gym website in first known Australian autonomous cyber attack

#19

Post by quadhome »

Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.

Get me press just like the frontier labs by admitting to crime. Make no mistakes.

Post reply on HN