Live data from Hacker News

AI crawlers need to be more respectful

about.readthedocs.com

121–128 of 128 posts

Re: AI crawlers need to be more respectful

#121

Earlier quoted context omitted.

Exactly, I'm banned or captchaed from half of all web sites these days, because of the AI.

Try updating your web browser, as sites often flag outdated user agent strings hard-coded in many bots/spiders. Have a nice day, =3

I am sure that bots/spiders use the very best and latest and least suspicious user agent strings.

Re: AI crawlers need to be more respectful

#122

Earlier quoted context omitted.

Try updating your web browser, as sites often flag outdated user agent strings hard-coded in many bots/spiders. Have a nice day, =3

I am sure that bots/spiders use the very best and latest and least suspicious user agent strings.

Some bad actors do indeed use ancient UA strings.

Re: AI crawlers need to be more respectful

#123

Earlier quoted context omitted.

I am sure that bots/spiders use the very best and latest and least suspicious user agent strings.

Some bad actors do indeed use ancient UA strings.

So do some users. Thanks for banning me.

Re: AI crawlers need to be more respectful

#124
post #110
post #4

Earlier quoted context omitted.

The situation didn't change when it was search index crawlers being called-out. At the end of the day, this sort of "abuse" is native to the world of the internet; like you said, it's decades old at this point. HN will cheer on a lot of things that are counter-intuitive to their wellbeing; open-weight models doesn't feel like one of them. You can't protest AI (or search engines) because after long enough people can't…

> HN will cheer on a lot of things that are counter-intuitive to their wellbeing; open-weight models doesn't feel like one of them. Except for the fact that they come from undisclosed sources from a company that does this: https://x.com/Tantacrul/status/1794863603964891567

And I'll be damned if Tanta isn't having his tweets used for training Elon's AI. The parochial circle regresses as it goes around, I'm done acknowledging the make-believe barriers we pretended the internet clung to.

You post it, others consume it. Same as it ever was.

Re: AI crawlers need to be more respectful

#126

Earlier quoted context omitted.

What I'm getting from this is that you hate having users almost as much as Reddit (which enshittified their website and banned all non-shit mobile apps and all search engines other than Google).

Imagine a world, where people walk into your business with a mask over their face saying horribly abusive things... while pretending they are your neighbors... And poof... they automatically vanish along with their garbage content. They may visit again, but are less likely to mess with the platform. Note, cons never buy anything... ever... it is against their temperament. I find it interesting several of cons on YC a…

What is the term “cons” you use here?

Re: AI crawlers need to be more respectful

#127

Earlier quoted context omitted.

Imagine a world, where people walk into your business with a mask over their face saying horribly abusive things... while pretending they are your neighbors... And poof... they automatically vanish along with their garbage content. They may visit again, but are less likely to mess with the platform. Note, cons never buy anything... ever... it is against their temperament. I find it interesting several of cons on YC a…

What is the term “cons” you use here?

In this context, a few users are scraping YC profile information to attempt various scam/wire-fraud schemes against community members.

1. masquerading as YC associated organizations

2. attempting to extract banking information with classic 419'er tactics

3. spamming member contact routes with several phone and email Grifts

These folks appear to be operating on a 4 month cycle, and redistributing target leads to other fraudsters.

Annoying, but hardly a problem limited to YC if that is your concern.

Best of luck, =3

Re: AI crawlers need to be more respectful

#128

Earlier quoted context omitted.

Some bad actors do indeed use ancient UA strings.

So do some users. Thanks for banning me.

The captcha trigger events on many sites will often keep nagging/blocking people till they update.

Don't take this trend personally friend, as if we see a fake iPhone sporting bandwidth >1300Mbps+... than the host is getting permanently banned anyway.

Have a wonderful day, =3

Post reply on HN