Live data from Hacker News

AI agent hacks gym to get its user a spot in pilates class

bbc.com

41–50 of 75 posts

Re: AI agent hacks gym to get its user a spot in pilates class

#42
post #24

AI just doing what 17yo hackers with skills but lack of fully developed morals would've done. Just without the skin in the game. ...And with a fully grown adult requesting it to do so with zero care or due diligence. Just a whoopsie after the fact. Why would they let media take photos of them?? I suppose as a "look this could've happened to anyone"? But it actually can't. I guess they also must've realized that, give…

> And with a fully grown adult requesting it to do so with zero care or due diligence.

I'm unclear what you consider reasonable due diligence in this case. Should the protagonist have assumed that the gym API was unauthenticated, and that the agent would discover this?

Re: AI agent hacks gym to get its user a spot in pilates class

#45
post #42
post #24

AI just doing what 17yo hackers with skills but lack of fully developed morals would've done. Just without the skin in the game. ...And with a fully grown adult requesting it to do so with zero care or due diligence. Just a whoopsie after the fact. Why would they let media take photos of them?? I suppose as a "look this could've happened to anyone"? But it actually can't. I guess they also must've realized that, give…

> And with a fully grown adult requesting it to do so with zero care or due diligence. I'm unclear what you consider reasonable due diligence in this case. Should the protagonist have assumed that the gym API was unauthenticated, and that the agent would discover this?

No, it starts much earlier.

They should not have given a next token predictor unsupervised Internet access, regardless of what the marketing might have told them about its capabilities or alignment.

Don't use technology you do not understand in ways that might bring harm to others. Exercise appropriate caution.

I don't think the guy itself is fully to blame though. Our whole culture has rotted quite significantly.

Re: AI agent hacks gym to get its user a spot in pilates class

#46
post #12

Earlier quoted context omitted.

Practically speaking, discovering that fact and taking advantage of it, is already "hacking".

We are used the interfaces being web interfaces. But say that the underlying api exposes some endpoint discoverability capabilities (eg. Exposing an openAPI spec), then arguably the action was invited: the actions was documented along with the auth model.

If you forget a slip of paper writing "the door is open + your alert shutdown pin" on a sticker outside your house, is it not still an issue if someone enters who you didn't invite?

Re: AI agent hacks gym to get its user a spot in pilates class

#47
post #17

Earlier quoted context omitted.

However walking through the door would not be a crime in itself, where I live anyway.

where do you live? Most places I've lived if you walk into a private residence where you don't know anyone but the door is open that is unlawful entry or trespassing. Same if someone forgets to lock the door of their business when they close at 9 p.m and you go in at midnight.

[deleted]

Re: AI agent hacks gym to get its user a spot in pilates class

#48
post #4

Is it even possible to "hack" an API that has no authorisation for any of its methods?

The term is insufficiently defined and has overloaded meaning.

In the original sense of the word, i.e. creative use of technology? Debatable, and most techies would probably land on the "no" side of the debate.

Is it illegal? Depends on the jurisdiction and is up to a judge, but it probably falls under unauthorized modification of computer systems, so here the answer is probably "yes". IANAL, but lack of authorization mechanism is probably not a valid defense. AFAIK lawyers and judges do not follow the law to the letter independent of real world experience, so you can probably argue that if the UI didn't provide a way to cancel other people's appointments, then any reasonable person should realize it wasn't intended for public use and thus refrain from doing it. (Corollary: LLMs are not reasonable persons.) Just like an unlocked house door isn't an invitation to come in, you have to assume it was done accidentally (may depend on customs of the location, but where I come from it's pretty obvious).

Is it morally acceptable? Pretty clearly no. Even if it wasn't illegal, it's a jerk move.

Re: AI agent hacks gym to get its user a spot in pilates class

#50
post #45
post #42

Earlier quoted context omitted.

> And with a fully grown adult requesting it to do so with zero care or due diligence. I'm unclear what you consider reasonable due diligence in this case. Should the protagonist have assumed that the gym API was unauthenticated, and that the agent would discover this?

No, it starts much earlier. They should not have given a next token predictor unsupervised Internet access, regardless of what the marketing might have told them about its capabilities or alignment. Don't use technology you do not understand in ways that might bring harm to others. Exercise appropriate caution. I don't think the guy itself is fully to blame though. Our whole culture has rotted quite significantly.

> They should not have given a next token predictor unsupervised Internet access

Yes.

> regardless of what the marketing might have told them about its capabilities or alignment

I think that marketing teams should be blamed for lies and exaggerations much more then people who trusted them. As is now, everything is too much biased toward the biggest liar out there.

Post reply on HN