Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting AI agent use and abuse

stytch.com

Post reply on HN
Print view View original

11–20 of 106 posts

Re: Detecting AI agent use and abuse

#11

Post by deadbabe »

JohnMakin wrote: post #7

I've been flagged as a bot on pretty much every major platform. Most ridiculously lately, linkedin - I have to prove my identity using 2 different forms of ID, which they still won't accept, OR find a notary and somehow prove I own the account I no longer have access to. Maybe try refining this tech a little better before you start blasting legitimate users with it - I am extremely skeptical of the catch rate given w…

I asked a manager about this, the policy is that we do not need to differentiate between bots and people who sound similar to bots: both are considered low quality content/engagement. Delete them.

Seems like wherever they delete bots, they will in the end, delete human beings.

Re: Detecting AI agent use and abuse

#12

Post by mcstempel »

JohnMakin wrote: post #7

I've been flagged as a bot on pretty much every major platform. Most ridiculously lately, linkedin - I have to prove my identity using 2 different forms of ID, which they still won't accept, OR find a notary and somehow prove I own the account I no longer have access to. Maybe try refining this tech a little better before you start blasting legitimate users with it - I am extremely skeptical of the catch rate given w…

LinkedIn always hits me with those frustrating custom CAPTCHAs where you have to rotate the shape 65 degrees -- they've taken a pretty blunt, high-friction approach to bot detection

I think most apps should primarily start with just monitoring for agentic traffic so they can start to better understand the emergent behaviors they're performing (it might tell folks where they actually need real APIs for example), and then go from there

Re: Detecting AI agent use and abuse

#13

Post by deadbabe »

ceejayoz wrote: post #2

I fully expect captchas to incorporate "type the racial slur / death threat into the box" soon, as the widely available models will balk at it.

It’s ironic that some of the first intelligent chatbots very quickly became Nazis and racists, and now we’ve swung the other way.

Re: Detecting AI agent use and abuse

#14

Post by do_not_redeem »

mcstempel wrote: post #9
diggan wrote: post #3

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a b…

Hey there, I'm the author of the post. I'm actually pretty sympathetic to your viewpoint, and I wanted to clarify my stance. I actually spent years working at a "good bot" company (Plaid), which focused on making users' financial data portable. The main reason Plaid existed was that banks made it hard for users to permission their data to other apps -- typically not solely out of security concerns, but to also active…

It seems like cases (1) and (2) would both be better handled by letting the user give their user agent a separate security context if they choose, instead of trying to detect/guess what kind of browser made that http request. I'm thinking about things like oauth permissions, GitHub's sudo mode, etc. Otherwise your magic detection code will inevitably end up telling an ELinks user "sorry, you need to download chrome to view your payment info".

Re: Detecting AI agent use and abuse

#15

Post by bsnnkv »

I have personally opted out of the arms race for at least one service that I operate.[1]

If AI agents figure out how to buy a subscription and transfer money from their operators to me, they are more than welcome to scrape away.

[1]: https://lgug2z.com/articles/in-the-age-of-ai-crawlers-i-have...

Re: Detecting AI agent use and abuse

#16

Post by diggan »

Etheryte wrote: post #8
diggan wrote: post #3

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a b…

The problem is resource consumption. On some of my servers, scrapers, bots etc make up a vast majority of both the bandwidth and CPU usage when left unchecked. If I didn't block them, I would need to pay for a beefier server. All the while this doesn't give me or my regular visitors any benefit, it's just large corporations driving up my hosting costs.

> The problem is resource consumption. On some of my servers, scrapers, bots etc make up a vast majority of both the bandwidth and CPU usage when left unchecked

What are they downloading, like heavy videos and stuff? Initiating heavy processes or similar?

Re: Detecting AI agent use and abuse

#17

Post by mcstempel »

do_not_redeem wrote: post #14
mcstempel wrote: post #9

Earlier quoted context omitted.

Hey there, I'm the author of the post. I'm actually pretty sympathetic to your viewpoint, and I wanted to clarify my stance. I actually spent years working at a "good bot" company (Plaid), which focused on making users' financial data portable. The main reason Plaid existed was that banks made it hard for users to permission their data to other apps -- typically not solely out of security concerns, but to also active…

It seems like cases (1) and (2) would both be better handled by letting the user give their user agent a separate security context if they choose, instead of trying to detect/guess what kind of browser made that http request. I'm thinking about things like oauth permissions, GitHub's sudo mode, etc. Otherwise your magic detection code will inevitably end up telling an ELinks user "sorry, you need to download chrome t…

You read our mind! https://stytch.com/blog/the-age-of-agent-experience/

Very much agreed that's the long-term goal, but I think we'll live in a world where most apps don't support oauth for a while longer (though I'd love for all of them to -- we're actually announcing something next week that makes this easy for any app to do)

But we're also envisioning an interim period where users are delegating to unsanctioned external agents (e.g. OpenAI Operator, Anthropic Computer Use API, etc.) prior to apps catching up and offering proper oauth

Re: Detecting AI agent use and abuse

#18

Post by emgeee »

deadbabe wrote: post #11
JohnMakin wrote: post #7

I've been flagged as a bot on pretty much every major platform. Most ridiculously lately, linkedin - I have to prove my identity using 2 different forms of ID, which they still won't accept, OR find a notary and somehow prove I own the account I no longer have access to. Maybe try refining this tech a little better before you start blasting legitimate users with it - I am extremely skeptical of the catch rate given w…

I asked a manager about this, the policy is that we do not need to differentiate between bots and people who sound similar to bots: both are considered low quality content/engagement. Delete them. Seems like wherever they delete bots, they will in the end, delete human beings.

I never really thought about this perspective but in some ways it makes sense. I think the ironic part is that LinkedIn now provides built-in AI tools that make you sound more like a bot.

Maybe they could fingerprint slop generated with they tools and allow it through to incentivize upgrading

Re: Detecting AI agent use and abuse

#19

Post by diggan »

hansonkd wrote: post #4
diggan wrote: post #3

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a b…

> It seems like suddenly half the internet forgot about the term "user-agent", which up until recently was almost always our browsers, but sometimes feed readers, which was acceptable it seems. Was it really "suddenly"? it seems like for the past decade there has been an ongoing push to make everyone use "chromium" based browsers. I remember 10-15 years ago you would get blocked for not using IE or whatever, even tho…

> I remember 10-15 years ago you would get blocked for not using IE or whatever, even though the site worked fine and there was no technical reason for the block

Reminds me of when I discovered that Google Inbox worked in Firefox, even though Google decided to only allow Chrome to access it:

https://news.ycombinator.com/item?id=8606879 - "Why Is Google Blocking Inbox on Firefox?" - 213 points | Nov 14, 2014 | 208 comments

(correct link to the gist is https://gist.github.com/victorb/1d0f4ee6dc5ec0d6646e today)

I think that "ongoing" push you're talking about was/is accidental, because a lot of people use Chrome. What I'm seeing now seems to be intentional, because people disagree with the ethics/morals surrounding AI, or seeing a large impact on their servers because of resource consumption, so more philosophical and/or practical, rather than accidental.

But who knows, I won't claim to have exact insights into exactly what caused "Chrome is the new IE", could be it was very intentional and they never stopped.

Re: Detecting AI agent use and abuse

#20

Post by digitaltrees »

I think companies need to rethink this and go the opposite direction, rather than being hostile and blocking AI Agents--and losing millions or billions in revenue when people sending AI Agents to do tasks on their behalf cant get the task done---they should redesign their software for Agent use.

https://www.loop11.com/introducing-ai-browser-agents-a-new-w...

Post reply on HN