Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

End of an era for me: no more self-hosted git

kraxel.org

Post reply on HN
Print view View original

221–228 of 228 posts

Re: End of an era for me: no more self-hosted git

#221

Post by kristjank »

mschuster91 wrote: post #169
kristjank wrote: post #75

Is there a way to block it by shibboleth? Curious, since the recent Google hack where you add -(n-word) to the end of your query so the AI automatically shuts down works like a charm.

I have no doubt that it works and it's hilarious that it works, but is there a way that does not involve my Google search history look like I've applied for a KKK membership?

You can try to use other nasty words, but a lot of technical discussions seem to still include an occasional fck or sht, and would get consequently excluded from the search results. This is especially common on reddit where sometimes recommended article titles sneak into the indexed page text, which often contain profanity.

On the contrary, almost none of them ever include racial slurs (and even if they do, I doubt such a distinguished audience has the answer to my questions)

Re: End of an era for me: no more self-hosted git

#222

Post by olivia-banks »

JohnTHaller wrote: post #30

The Chinese AI scrapers/bots are killing quite a bit of the regular web now. YisouSpider absolutely pummeled my open source project's hosting for weeks. Like all Chinese AI scrapers, it ignores robots.txt. So forget about it respecting a Crawl-delay. If you block the user agent, it would calm down for a bit, then it would just come back again using a generic browser user agent from the same IP addresses. It does this…

I had no idea Yisou was still around, or did somebody buy them? I'm not as up to date on Chinese tech as I should be.

Re: End of an era for me: no more self-hosted git

#223

Post by yellowapple »

moebrowne wrote: post #211
yellowapple wrote: post #206

Earlier quoted context omitted.

I suspect bog-standard per-IP rate limiting would also mitigate this, no?

People report that the requests come from many thousands of IPs

Any firewall worth its salt (like PF) can measure which IPs are making more requests per minute than is reasonable and shove them into a list to be handled separately (whether by blocking them outright or by putting them in a slower / lower-priority queue). Putting all those thousands of IPs into a queue that only gets 1% of the available bandwidth would solve the issue quite nicely, I think.

Re: End of an era for me: no more self-hosted git

#224

Post by replooda »

kstrauser wrote: post #126
xorcist wrote: post #120

Earlier quoted context omitted.

> I set it and use JavaScript to tell them to reload the page While throwing out all users who opt-in to javascript, using Noscript or uBlock or something like it, may be acceptable collateral damage to you, it might be good to keep in mind that this plays right into Big Adtech's playbook. They spend over two decades to normalize the behavior of running a hundred or more programs of untrusted origin on every page loa…

Oh please. That ship has sailed. I'm marginally sympathetic to people who don't run JavaScript on their browsers for a variety of reasons, but they've deliberately opted out of the de facto modern web. JS is as fundamental to current design as CSS. If you turn it off, things might work, but almost no one is testing that setup, nor should they reasonably be expected to. This has zero to do with Adtech for 99.99% of us…

Just you wait, I'll get... What's the name of the LLM thing again? ClawFish or something? I forgot — and the LSP is down, so that's that. Anyways, I'll tell the MoltClawde (???) to skillfully vibe code a skill for generating vehement anti-blackpill diatribes, then equip it and reply to your post with a such vehement — but, you know, sort-of lyrical — anti-blackpill diatribe, your pill will shine so much, it will make post-balrog Gandalf look like pre-balrog Gandalf and your own LLMthingmaclaude will wax poetical about not surrendering to generalized societal stupidity while it publishes balrog-related CVEs about curl's GAND_ELF() preprocessor directive.

Re: End of an era for me: no more self-hosted git

#225

Post by Imustaskforhelp »

soulofmischief wrote: post #198
Imustaskforhelp wrote: post #189

Earlier quoted context omitted.

This is really not the type of legacy that I want to leave behind on hackernews but then again, I have been vocal that I just write what I think. Literally. It has its flaws but I am not sugar coating it. Sometimes I am unable to explain myself but the thing is that I write on HN to point out of some idea, some discussion. It's better written here than lost and yes most of my ideas might be incoherent but they make p…

Great response. > Teach me instead of such tone for I am interested in learning Here are some notes: Run on sentences and lack of punctuation make your writing hard to follow; brevity can be effective. For each sentence, choose a subject, verb, predicate, proposition, etc. to form a single clause, but don't compound multiple such clauses into a single sentence. Break sentences up with punctuation so that the eye rest…

Thanks, this helps. I hope you understand tho that it can be hard for me to shift my linguistics in a day but I will definitely try to take this into account if I ever write professionally.

I like writing in HN because its fun & spontaneous. Some people write really crafted messages and I love reading those too :D but I am not sure if I might enjoy the site without such spontaneity, but I also liked reading how you trimmed my message and thinking process behind it. So no guarantees but I have favourited this comment for reading it whenever I want :D

I feel this comment can/will definitely help me in professional writing contexts, something which I struggle with. [In concising what I want to say]

This suggestion is helpful. Do you have any normal blogs I can read about this or something to read/learn more in my free-time?

Also, I definitely appreciate you taking time out of your day to write this message so have a nice day!

Re: End of an era for me: no more self-hosted git

#226

Post by testrun »

zajio1am wrote: post #216
kstrauser wrote: post #93

I cut traffic to my Forgejo server from about 600K request per day to about 1000: https://honeypot.net/2025/12/22/i-read-yann-espositos-blog.h... 1. Anubis is a miracle. 2. Because most scrapers suck, I require all requests to include a shibboleth cookie, and if they don’t, I set it and use JavaScript to tell them to reload the page. Real browsers don’t bat an eye at this. Most scrapers can’t manage it. (This wasn’t…

600K request per day is ~ 400/minute. That is very low number. But seems to me that many webapps are so bad that even that small number causes significant load for them.

That is an average. The distribution of requests might be more in bursts.

Re: End of an era for me: no more self-hosted git

#227

Post by hattmall »

M95D wrote: post #57
hattmall wrote: post #28

Can we not charge for access? If I have a link, that says "By clicking this link you agree to pay $10 for each access" then sending the bill?

Send it where?

If you can identify the IP / User-agent of the bot then to the registered agent of that corporation. If the IP isn't one you can link to a crawler directly then subpoena the provider just like in an infringement action.
Post reply on HN