Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nepenthes is a tarpit to catch AI web crawlers

zadzmo.org

Post reply on HN
Print view View original

121–130 of 290 posts

Re: Nepenthes is a tarpit to catch AI web crawlers

#121

Post by inetknght »

permo-w wrote: post #117
bflesch wrote: post #87

Earlier quoted context omitted.

Their security.txt email address replies and asks you to go on BugCrowd. BugCrowd staff is unwilling (or too incompetent) to run a bash curl command to reproduce the issue, while also refusing to forward it to OpenAI. The support@openai.com waits an hour before answering with ChatGPT answer. Issues raised on GitHub directly towards their engineers were not answered. Also Microsoft CERT & Azure security team do not re…

why try this hard for a private company that doesn't employ you?

Some people have passion.

Re: Nepenthes is a tarpit to catch AI web crawlers

#122

Post by JohnMakin »

bflesch wrote: post #84
michaelbuckbee wrote: post #68

Earlier quoted context omitted.

What is the https://chatgpt.com/backend-api/attributions endpoint doing (or responsible for when not crushing websites).

When ChatGPT cites web sources in it's output to the user, it will call `backend-api/attributions` with the URL and the API will return what the website is about. Basically it does HTTP request to fetch HTML ` ` tag. They don't check length of supplied `urls[]` array and also don't check if it contains the same URL over and over again (with minor variations). It's just bad engineering all around.

Even if you were unwilling to change this behavior on the application layer or server side, you could add a directive in the proxy to prevent such large payloads from being accepted as an immediate mitigation step, unless they seriously need that parameter to have unlimited number of urls in it (guessing they have it set to some default like 2mb and it will break at some limit, but I am afraid to play with this too much). Somehow I doubt they need that? I don't know though.

Re: Nepenthes is a tarpit to catch AI web crawlers

#123

Post by m3047 »

Having first run a bot motel in I think 2005, I'm thrilled and greatly entertained to see this taking off. When I first did it, I had crawlers lost in it literally for days; and you could tell that eventually some human would come back and try to suss the wreckage. After about a year I started seeing URLs like ../this-page-does-not-exist-hahaha.html. Sure it's an arms race but just like security is generally an afterthought these days, don't think that you can't be the woodpecker which destroys civilization. The comments are great too, this one in particular reflects my personal sentiments:

> the moment it becomes the basic default install ( ala adblocker in browsers for people ), it does not matter what the bigger players want to do

Re: Nepenthes is a tarpit to catch AI web crawlers

#124

Post by grahamj »

bflesch wrote: post #108
grahamj wrote: post #102

Earlier quoted context omitted.

Note I said outgoing, as in the crawlers should be throttling themselves

Sorry for misunderstanding your point. I agree it should be throttled. Maybe they don't need to throttle because they don't care about cost. Funny thing is that servers from AWS were trying to connect to my system when I played around with this - I assume OpenAI has not moved away from AWS yet. Also many different security scanners hitting my IP after every burst of incoming requests from the ChatGPT crawler Azure IP…

yeah it’s fun out on the wild internet! Thankfully I don’t manage something thing crawlable anymore but even so the endpoint traffic is pretty entertaining sometimes.

What would keep me up at night if I was still more on the ops side is “computer use” AI that’s virtually indistinguishable from a human with a browser. How do you keep the junk away then?

Re: Nepenthes is a tarpit to catch AI web crawlers

#125

Post by [deleted] »

marc wrote: post #78

OpenAI doesn’t take security seriously. I reported a vulnerability to them that allowed you to get IP addresses of their paying customers. OpenAI responded “Not applicable” indicating they don’t think it was a serious issue. The PoC was very easy to understand and simple to replicate. Edit: I guess I might as well disclose it here since they don’t consider it an issue. They were/are(?) hot linking logo images of thir…

[deleted]

Re: Nepenthes is a tarpit to catch AI web crawlers

#126

Post by jillyboel »

dewey wrote: post #92
JohnMakin wrote: post #66

Earlier quoted context omitted.

Nice find, I think one of my sites actually got recently hit by something like this. And yea, this kind of thing should be trivially preventable if they cared at all.

> And yea, this kind of thing should be trivially preventable if they cared at all. Most of the time when someone says something is "trivial" without knowing anything about the internals, it's never trivial. As someone working close to the b2c side of a business, I can’t count the amount of times I've heard that something should be trivial while it's something we've thought about for years.

now try to reply to the actual content instead of some generalizing grandstanding bullshit

Re: Nepenthes is a tarpit to catch AI web crawlers

#128

Post by jillyboel »

bflesch wrote: post #108
grahamj wrote: post #102

Earlier quoted context omitted.

Note I said outgoing, as in the crawlers should be throttling themselves

Sorry for misunderstanding your point. I agree it should be throttled. Maybe they don't need to throttle because they don't care about cost. Funny thing is that servers from AWS were trying to connect to my system when I played around with this - I assume OpenAI has not moved away from AWS yet. Also many different security scanners hitting my IP after every burst of incoming requests from the ChatGPT crawler Azure IP…

They need to throttle because otherwise they're simply a DDoS service. It's clear they don't give a fuck though, like any bigtech company. They'll spend millions on prosecuting anyone who dares to do what they perceive as a DoS attack against them, but they'll spit in your face and laugh at you if you even dare to claim they are DDoSing you.

Re: Nepenthes is a tarpit to catch AI web crawlers

#129

Post by canu7 »

NathanKP wrote: post #81

This looks extremely easy to detect and filter out. For example: https://i.imgur.com/hpMrLFT.png In short, if the creator of this thinks that it will actually trick AI web crawlers, in reality it would take about 5 mins of time to write a simple check that filters out and bans the site from crawling. With modern LLM workflows its actually fairly simple and cheap to burn just a little bit of GPU time to check if the d…

If they need to query a trained LLM for each page they crawl, I would guess that the training cost would scale up pretty badly...

Re: Nepenthes is a tarpit to catch AI web crawlers

#130

Post by ezrast »

bwfan123 wrote: post #50
rvz wrote: post #42

Good. We finally have a viable mouse trap for LLM scrapers for them to continuously scrape garbage forever, depleting the host of their resources whilst the LLM is fed garbage which the result will be unusable to the trainer, accelerating model collapse. It is like a never ending fast food restaurant for LLMs forced to eat garbage input and will destroy the quality of the model when used later. Hope to see this sort…

indeed. this will spur research on how to distinguish BS from legit content. which is the fundamental hallucination problem in llms. and all of us will benefit from this.

You can't programatically detect novel BS any more than you can programatically detect viruses or spam. You can only add the fingerprints of known badness into an ever-growing database. Viruses and spam are antagonistic to well-resourced institutions, and their databases get maintained reasonably well. LLM slop is being generated by those same well-resourced institutions. I don't think it fits into the same category as Nepenthes.
Post reply on HN