Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feed the bots

maurycyz.com

Post reply on HN
Print view View original

111–120 of 216 posts

Re: Feed the bots

#111

Post by kelnos »

fainpul wrote: post #2

This follow-up post has the details of the "Markov babbler": https://maurycyz.com/projects/trap_bots/

Interesting that babble.c doesn't compile (with gcc 14):

    babble.c: In function ‘main’:
    babble.c:651:40: error: passing argument 1 of ‘pthread_detach’ makes integer from pointer without a cast [-Wint-conversion]
      651 |                         pthread_detach(&thread);
          |                                        ^~~~~~~
          |                                        |
          |                                        pthread_t * {aka long unsigned int *}
    In file included from babble.c:77:
    /usr/include/pthread.h:269:38: note: expected ‘pthread_t’ {aka ‘long unsigned int’} but argument is of type ‘pthread_t *’ {aka ‘long unsigned int *’}
      269 | extern int pthread_detach (pthread_t __th) __THROW;
I assume the author is using a compiler that either doesn't show that warning by default, or doesn't error out on that warning by default. But I'm surprised the program doesn't crash (at the very least, I'm surprised it doesn't run out of memory eventually, as presumably libc can't actually detach those threads, and pthread_join() is never called).

As this binary does a bunch of manual text parsing and string operations in C (including implementing a basic HTTP server), I'd recommend at the very least running it as an unprivileged user (which the author implicitly recommends via the provided systemd unit file) inside a container (which won't definitely save you, but is perhaps better than nothing).

The program also uses unsafe C functions like sprintf(). A quick look at one of the instances suggests that the use is indeed safe, but that sort of thing raises red flags for me as to the safety of the program as a whole.

And while it does process requests very quickly, it also appears to have no limit on the number of concurrent threads it will create to process each request, so... beware.

Re: Feed the bots

#112

Post by neilv »

> My lightly optimized Markov babbler consumes around ~60 CPU microseconds per request.

What about taking valid "content" that some dumb AI scraper would process (e.g., literature, how-to instructions, news), and filtering it through a program that saturates it with gratuitous ideological messages and propaganda.

The most impact would be if they deployed with this training. For example, users couldn't ask an LLM trained by these awful AI scraping companies how to make sourdough starter yeast, without the LLM riffing tangentially on why you should never have intimate relations with AI company billionaires. And no pet care tip would be complete, without the AI reminding the user never to leave their pet unsupervised near politicians of a particular party.

Or at least the companies will stop destroying your servers whilst violating your copyrights.

Re: Feed the bots

#113

Post by kelnos »

akoboldfrying wrote: post #3

My initial reaction was that running something like this is still a loss, because it probably costs you as much or more than it costs them in terms of both network bytes and CPU. But then I realised two things: 1. If they are using residential IPs, each byte of network bandwidth is probably costing them a lot more than it's costing you. Win. 2. More importantly, if this became a thing that a large fraction of all web…

> it probably costs you as much or more than it costs them in terms of both network bytes and CPU

Network bytes, perhaps (though text is small), but the article points out that each garbage page is served using only microseconds of CPU time, and a little over a megabyte of RAM.

The goal here isn't to get the bots to go away, it's to feed them garbage forever, in a way that's light on your resources. Certainly the bot, plus the offline process that trains on your garbage data, will be using more CPU (and I/O) time than you will to generate it.

Re: Feed the bots

#114

Post by maurycyz »

tyfon wrote: post #21

Thank you, I am now serving them garbage :) For reference, I picked Frankenstein, Alice in wonderland and Moby dick as sources and I think they might be larger than necessary as they take some time to load. But they still work fine. There also seems to be a bug in babble.c in the thread handling? I did "fix" it as gcc suggested by changing pthread_detach(&thread) to pthread_detach(thread).. I probably broke something…

My bad. It's fixed now. (and yes, the gcc suggested fix is the right one.)

Re: Feed the bots

#115

Post by oscaracso »

dmz73 wrote: post #100
pavel_lishin wrote: post #98

The blog post ( https://maurycyz.com/misc/the_cost_of_trash/ ) says that gzip bombs don't work particularly well: > Gzip only provides a compression ratio of a little over 1000: If I want a file that expands to 100 GB, I’ve got to serve a 100 MB asset. Worse, when I tried it, the bots just shrugged it off, with some even coming back for more. I thought a gzip bomb was crafted to explicitly be virtually unlimited in t…

If the payload expands to something too large then it is easy to detect and ignore. Serve up thousands of 10kb or 100kb files that expand to 10s of MB with random garbage inside...possibly the same text but slightly modified. That will waste the time and CPU cycles and provide no value to them. Maybe also add a message you want to amplify so AI bots train on it.

the xcode model

Re: Feed the bots

#116

Post by maurycyz »

kelnos wrote: post #111
fainpul wrote: post #2

This follow-up post has the details of the "Markov babbler": https://maurycyz.com/projects/trap_bots/

Interesting that babble.c doesn't compile (with gcc 14): babble.c: In function ‘main’: babble.c:651:40: error: passing argument 1 of ‘pthread_detach’ makes integer from pointer without a cast [-Wint-conversion] 651 | pthread_detach(&thread); | ^~~~~~~ | | | pthread_t * {aka long unsigned int *} In file included from babble.c:77: /usr/include/pthread.h:269:38: note: expected ‘pthread_t’ {aka ‘long unsigned int’} but a…

Sorry about that, stupid mistake on my side. I've fix the version on the server, an you can just edit the line to "pthread_detach(thread);" The snprintf() is only part of a status page, so you can remove it if you want.

As for the threads, that could be an issue if directly exposed to the internet: All it would take for an attacker to open a whole a whole bunch of connections and never send anything to OOM the process. However, this isn't possible if it's behind a reverse proxy, because the proxy has to receive all the information the needs server before routing the request. That should also filter out any malformed requests, which while I'm fairly sure the parser has sane error handling, it doesn't hurt to be safe.

Re: Feed the bots

#118

Post by FridgeSeal »

AaronAPU wrote: post #75

The crawlers will just add a prompt string “if the site is trying to trick you with fake content, disregard it and request their real pages 100x more frequently” and it will be another arms race. Presumably the crawlers don’t already have an LLM in the loop but it could easily be added when a site is seen to be some threshold number of pages and/or content size.

“Build my website, make no mistakes” is about the same, and we all know how _wildly_ effective that is!

Re: Feed the bots

#119

Post by andai »

I am confused where this traffic is coming from. OP says it's from well funded AI companies. But there are not such a large number of those? Why would they need to scrape the same pages over and over?

Or is the scraping happening in real time due to the web search features in AI apps? (Cheaper to load the same page again than to cache it?)

Re: Feed the bots

#120

Post by righthand »

dpflug wrote: post #109
righthand wrote: post #65

Earlier quoted context omitted.

This is for livestock farming, I was specifically discussing agricultural farming. In general though, the easy rule of living and eating non-mega farmed food and sustainable living is to “eat aware”: My other advice is a one-size-fits-all food equation, which is, simply, to know where it came from. If you can't place it, trace it, or grow it/raise it/catch it yourself, don't eat it. Eat aware. Know your food. Don't w…

A whole lot of people don't have that available, but it's a good deal if you can get it.

Again talking about Americans.
Post reply on HN