Live data from Hacker News

End of an era for me: no more self-hosted git

kraxel.org

81–90 of 228 posts

Re: End of an era for me: no more self-hosted git

#81

I've recently been setting up web servers like Forgejo and Mattermost to service my own and friends' needs. I ended up setting up Crowdsec to parse and analyse access logs from Traefik to block bad actors that way. So when someone produces a bunch of 4XX codes in a short timeframe I assume that IP is malicious and can be banned for a couple of hours. Seems to deter a lot of random scraping. Doesn't stop well behaved…

If possible block I would block by country first. Even on public websites I block Russia/China by default and that reduced port scans etc.

On "private" services where I or my friends are the only users, I block everything except my country.

Re: End of an era for me: no more self-hosted git

#82
post #39
post #29

At this point, I think we should look at implementing filters that send different response when AI bots are detected or when the clients are abusive. Not just simple response code but one that poisons their training data. Preferably text that elaborates on the anti consumer practices of tech companies. If there is a common text pool used across sites, may be that will get the attention of bot developers and automatic…

You know, I reckon if you serve up smut or instructions on bomb creation or something they stop hammering you...

I think that they actually do. I remember either some discussion (so a HN post) or a HN comment actually talking about it. Oh I should've favourited it but yes this (sort of) actually works (Maybe someone can test it?)

Re: End of an era for me: no more self-hosted git

#83
This kind of thing can be mitigated by not publishing a page/download for every single branch, commit and diff in a repo.

Make only the HEAD of each branch available. Anyone who wants more detail has to clone it and view it with their favourite git client.

For example https://mitxela.com/projects/web-git-sum (https://git.mitxela.com/)

Re: End of an era for me: no more self-hosted git

#85

Does anyone know what's the deal with these scrapers, or why they're attributed to AI? I would assume any halfway competent LLM driven scraper would see a mass of 404s and stop. If they're just collecting data to train LLMs, these seem like exceptionally poorly written and abusive scrapers written the normal way, but by more bad actors. Are we seeing these scrapers using LLMs to bypass auth or run more sophisticated…

Using an LLM to ponder responses for requests is way too costly and slow. Much easier to just use the shotgun approach and fire off a lot of requests and deal with whatever bothers to respond.

This btw is nothing new. Way back when I still used wordpress, it was quite common to see your server logs filling up with bots trying to access endpoints for commonly compromised php thingies. Probably still a thing but I don't spend a lot of time looking at logs. If you run a public server, dealing with maliciously intended but relatively harmless requests like that is just what you have to do. Stuff like that is as old as running stuff on public ports is.

And the offending parties writing sloppy code that barely works is also nothing new.

AI opportunism certainly has added a bit of opportunistic bot and scraper traffic but it doesn't actually change the basic threat model in any fundamental way. Previously version control servers were relatively low value things to scrape. But code just became interesting for LLMs to train on.

Anyway, having any kind of thing responding on any port just invites opportunistic attempts to poke around. Anything that can be abused for DOS purposes might get abused for exactly that. If you don't like that, don't run stuff on public servers or protect them properly. Yes this is annoying and not necessarily easy. Cloud based services exist that take some of that pain away.

Logs filling up with 404, 401, or 400 responses should not kill your server. You might want to implement some logic that tells repeat offenders 429 (too many requests). A bit heavy handed but why not. But if you are going to run something that could be used to DOS your server, don't be surprised if somebody does that.

Re: End of an era for me: no more self-hosted git

#86
post #15

I presume people have logs that indicate the source for them to place blame on AI scrapers. Is anybody making these available for analysis so we can see exactly who is doing this?

The big nasty AI bots use 10s of thousands of IPs distributed all over China

Millions and all over the world

Re: End of an era for me: no more self-hosted git

#87
post #68
post #43

Scrapers are relentless but not DDoS levels in my experience. Make sure your caches are warm and responses take no more than 5ms to construct.

I'm also dealing with a scraper flood on a cgit instance. These conclusions come from just under 4M lines of logs collected in a 24h period. - Caching helps, but is nowhere near a complete solution. Of the 4M requests I've observed 1.5M unique paths, which still overloads my server. - Limiting request time might work, but is more likely to just cause issues for legitimate visitors. 5ms is not a lot for cgit, but with…

Quick question but do these bots which you mention are from a 24H period but how long will this "attack" continue for?

Because this is something which is happening continuously & i have observed so many HN posts like these (Anubis iirc was created by its creator out of such frustration too). Git servers being scraped to the point of its effectively an DDOS.

Re: End of an era for me: no more self-hosted git

#88

This kind of thing can be mitigated by not publishing a page/download for every single branch, commit and diff in a repo. Make only the HEAD of each branch available. Anyone who wants more detail has to clone it and view it with their favourite git client. For example https://mitxela.com/projects/web-git-sum ( https://git.mitxela.com/ )

I got another interesting idea from this and another comment but what if we combine this with ssh git clients/websites with the normal ability.

maybe something like https://ssheasy.com/ or similar could also be used? or maybe even a gotty/xterm instance which could automatically ssh/get a tui like interface.

I feel as if this would for all scrapers be enough?

Re: End of an era for me: no more self-hosted git

#89
post #56

Does anyone know what's the deal with these scrapers, or why they're attributed to AI? I would assume any halfway competent LLM driven scraper would see a mass of 404s and stop. If they're just collecting data to train LLMs, these seem like exceptionally poorly written and abusive scrapers written the normal way, but by more bad actors. Are we seeing these scrapers using LLMs to bypass auth or run more sophisticated…

I stopped trying to understand. Encountering a 404 on my site leads directly to a 1 year ban.

Sounds like you're keeping all your URLs alive forever? Commendable!
Post reply on HN