Live data from Hacker News

Detecting AI agent use and abuse

stytch.com

81–90 of 106 posts

Re: Detecting AI agent use and abuse

#81
post #45

Earlier quoted context omitted.

Web Environment Integrity. Eventually your hardware will rat you out via attestation.

And you think nobody (professional hackers?) can put together a "virtual TPM" that falsifies real hardware info? I think there are much simpler solutions, but the big tech wants to retain the control.

Oh I can think of dystopian arrangements between Cloudflare, Google, Intel and AMD that'll fix that.

Re: Detecting AI agent use and abuse

#82

Looks like detecting real humans apart from agents is going to be an arms race if the detection is based on browser/device fingerprinting or visual/audio captchas; AI will only get better. What are captcha alternatives that can block resource consumption by bots?

A credit card.

I can give a credit card to my local AI.

Re: Detecting AI agent use and abuse

#83
post #3

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a b…

Hey there, I'm the author of the post. I'm actually pretty sympathetic to your viewpoint, and I wanted to clarify my stance. I actually spent years working at a "good bot" company (Plaid), which focused on making users' financial data portable. The main reason Plaid existed was that banks made it hard for users to permission their data to other apps -- typically not solely out of security concerns, but to also active…

> That said, I think it’s reasonable for app developers to decide how their services are consumed

But it's a far step from that to (attempting to) control the user agent, or only allow blessed clients/devices.

Of course the site operator is concerned with limiting and preventing abuse by malicious users and agents, and an app developer should build for enabling that.

> Main thing I care about is that AI agent use remains safe and aligned with user intent

Nice and all. Keep a level perspective though: At scale, you can't keep control of your users not getting scammed/phished/hacked, or plain doing destructive uninformed actions on their own accord. Similar here: If you aim for 0, that will be to detriment to (at best, I believe) your growth.

I believe the kind of patterns you describe in the article are in fact anti-patterns. Look at the kind of web and internet they lead to. Look at what they do to individual agency in society. Across the board, abuse is increasing alongside negative side-effects from false positives of these kinds of counter-measures - which will invariably end up abused (by ignorance or intentionally) to exclude an increasing number of "undesireds". Systematic discrimimation is an apt term for the emergent consistent blocking of certain groups and individuals even if "it's just the stats playing out that way"?

Consider accessibility, and the diversity of humans. It is a folly to believe you can craft a singular user-experience that works satisfactory for everyone, or even catalogue and "officially support" what's in need by your entire target audience. By blocking access to screen readers and other accessibility agents you limit or prevent the use from those relying on these tools.

> My idealistic long-term view though is that supporting AI agent use cases will eventually become table stakes.

My optimistic long-term view is that accessing content on my own terms with an agent I compiled myself is still an option (without any need for dystopian centralized signing services a la apple/mozilla), and that companies are still legally allowed to offer that option.

Re: Detecting AI agent use and abuse

#84
post #65

Earlier quoted context omitted.

> You write as if someone held a gun to your head and force you to sign up for Plaid. Plaid doesn't require anyone to use it. There is not a physical gun pointed at my head, but an increasing amount of digital online interactions are solely gated by Plaid. I've run into plenty cases where I simply had no choice, for example dealing with landlords. And you already know how long it takes for financial systems to evolve…

I think this anger with Plaid is unwarranted. Without them, or before them, you had zero API access because the banks (including yours) don't give a rat's ass on your fancy access needs. Now Plaid managed to gather together some kind of access. Are they to blame because they managed that? Do you still have any alternative with the bank? I think no, and no. You can get back to the "standard" situation of no API, no gu…

> Are they to blame because they managed that?

Do you understand that the ends don't always justify the means? Do you understand that not trading security and privacy for convenience means putting up with inconvenience? My complaints are warranted because yes, they are to blame, no, I do not want to be forced to use their service.

And when the company is eventually sold and financial transaction data harvested (whether against the wishes of the founders or not, loopholes exist), apologists will turn around and blame the new company instead of Plaid, who opened the door for them.

> Or you can create your own middleman service if you like and everybody will appreciate your Plaid alternative.

I think the financial tech market is rapidly evolving, and I'll just wait. If I need financial automation and a service like Stripe is not available, I can always use a cryptocurrency which respects my autonomy and privacy.

Re: Detecting AI agent use and abuse

#85
post #64

Earlier quoted context omitted.

It’s ironic that some of the first intelligent chatbots very quickly became Nazis and racists, and now we’ve swung the other way.

I am quite sure the people developing the current chatbots were well aware of what happened with Tay etc. I'd bet it's part of the reason for the safety stuff.

There was also, of course, this: https://www.theverge.com/2023/2/15/23599072/microsoft-ai-bin...

Re: Detecting AI agent use and abuse

#86
post #3

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a b…

Because they make their money from showing ads to human eyeballs and not to AIs.

Re: Detecting AI agent use and abuse

#87

Earlier quoted context omitted.

Hey there, I'm the author of the post. I'm actually pretty sympathetic to your viewpoint, and I wanted to clarify my stance. I actually spent years working at a "good bot" company (Plaid), which focused on making users' financial data portable. The main reason Plaid existed was that banks made it hard for users to permission their data to other apps -- typically not solely out of security concerns, but to also active…

Plaid is not a "good bot" company. Despite posturing from leadership, it is fundamentally unethical to build a pervasive banking middle-man service which requires users to surrender their private account credentials in order to operate. What if every business operated this way? It's disgusting that companies like Plaid have considerably set back public discourse on acceptable privacy tradeoffs.

Well then banks should offer a proper API with tokens and permissions.

What's that? They don't? Guess I'll just have to give Plaid my password then. Stupid banks.

btw this is the exact same way Facebook got people to migrate off MySpace.

Re: Detecting AI agent use and abuse

#88
post #87

Earlier quoted context omitted.

Plaid is not a "good bot" company. Despite posturing from leadership, it is fundamentally unethical to build a pervasive banking middle-man service which requires users to surrender their private account credentials in order to operate. What if every business operated this way? It's disgusting that companies like Plaid have considerably set back public discourse on acceptable privacy tradeoffs.

Well then banks should offer a proper API with tokens and permissions. What's that? They don't? Guess I'll just have to give Plaid my password then. Stupid banks. btw this is the exact same way Facebook got people to migrate off MySpace.

I remember Facebook's shady user acquisition tactics, and I also do not use Facebook and similarly think their business model is morally bankrupt.

> Guess I'll just have to give Plaid my password then.

Learned helplessness, trading digital sovereignty for convenience. There is a larger war being fought here that is bigger than you or me. Had Plaid not been forced upon me, I would never have used it willingly.

Re: Detecting AI agent use and abuse

#89
post #78

Earlier quoted context omitted.

I'm building a service which needs to extract rss feeds from pages (hntorss.com if you're interested). Nothing else. From any rational point of view, website owner would actively want this parser to work as easily as possible — the whole point is for users to see the content you publish! Alas, I still get rate-limited, 400-ed and others because of user agent and other bot-detection mechanisms.

> the whole point is for users to see the content you publish! no, the whole point (for most sites) is to make money off the users visiting said site (currently via advertising). Another third party service which slurps the data, and redirect the users to a different site to consume the data means the original site lost the revenue, but paid the bandwidth cost. So it's understandable that many sites want to block suc…

Even if it is not for profit (or especially so), the point of any publication is not just to get people to know something—you at least want people to read what you wrote and appreciate you for this.

Using Web normally, with search and all, is well-behaved in this regard, but using attribution-stripping technology isn’t.

If your readers don’t know you exist and you don’t know who your readers are or if they even exist, you basically become a ghost writer, content producer for LLMs (and in many cases some commercial LLM operator also makes money off your work, too).

Re: Detecting AI agent use and abuse

#90
post #87

Earlier quoted context omitted.

Plaid is not a "good bot" company. Despite posturing from leadership, it is fundamentally unethical to build a pervasive banking middle-man service which requires users to surrender their private account credentials in order to operate. What if every business operated this way? It's disgusting that companies like Plaid have considerably set back public discourse on acceptable privacy tradeoffs.

Well then banks should offer a proper API with tokens and permissions. What's that? They don't? Guess I'll just have to give Plaid my password then. Stupid banks. btw this is the exact same way Facebook got people to migrate off MySpace.

So do you also expect the bank to back you up if you get hacked, given your exposure of the password to plaid or other services?

Not sure banks are the best example for this discussion, though, since banks have legitimate reasons to secure and promote security of their accounts that is beyond simple IT resource usage.

Post reply on HN