311–320 of 365 posts
Earlier quoted context omitted.
Improve infrastructure. Apparently they are trivially DDOSable by anyone renting a few servers and running git clone in a loop? That's a problem they should solve. And no, blocking by user agent is not adequate protection.
Infrastructure costs. So you'd be willing to pay more for that?
Earlier quoted context omitted.
I wouldn't expect or want an RSS aggregator to respect robots.txt for explicitly added feeds. That is effectively a human action asking for that feed to be monitored so robots.txt doesn't apply. What would be good is respecting `Cache-Control`, which unfortunately many RSS clients don't, and just pick a schedule and poll on it.
robots.txt was originally created to include such bots. That they think they don't need to respect it goes against the original intent. Eg: https://www.robotstxt.org/faq/kinds.html >"What's New" monitoring
Earlier quoted context omitted.
We can behave like adults, ask why it's not satisfactory, and come to a more agreeable mutual solution, or we can blithely offer an incomplete solution, muzzle the other party, and just continue our DDoS.
See, here you just did it again: "muzzle the other party", as if it was causally connected to your disagreement about how the module proxy should work, and not to the abuse you inflicted on members of that community.
The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…
Earlier quoted context omitted.
> rip it out of existence Come the fuck off this "mature personality" shit if you're going to write like this. He proposed freezing a module no one wanted to maintain in a library specifically meant to host stuff with weaker compat guarantees, he didn't hop in a DeLorean and kill Zimmermann's grandpa. Meanwhile, the critical project Drew insisted he keep it for is... deprecated and unmaintained!
[flagged]
According to some comments in the linked GitHub issue, including [1] from last May, Drew could have simply asked to be excluded from automatic refresh traffic from the mirror. If I understand correctly, that would still leave traffic from the mirror when it’s acting as a direct proxy for someone’s request, but that is traffic that would be going to sr.ht regardless. For some reason he did not do this and instead chos…
Should every person that hosts an instance of SourceHut, Gitea, or Forgejo have to opt-in to this? That just doesn't scale at all. Drew is standing up for all independent hosters as much as he is standing up for his own business interests.
Earlier quoted context omitted.
Seems like they could also let one request per hour through, say, and then serve up the rest a 429. Users trying to clone their project would hit an almost certainly up to date Google cache and thus be happy and sr.ht save on pretty much all that traffic and thus be also happy.
I thought that too, but then sr.ht needs to keep state which IP made how many requests already, and as the post mentions google makes these requests from many different places. So they would have to count requests from the specific user agent and by that point it is again special purpose work and load on sr.ht (whereas general rate limiting per IP might be a good idea anyway).
Setting up an IP based rate limiting rule in nftables takes a few minutes at max, and I am not a professional sysadmin.
https://git.sr.ht/robots.txt > YandexBot is a dickhead, too aggressive > User-agent: Yandex > Disallow: / Hahaha.
User-agent: Amazonbot
Disallow: /
I'll just rehost my git repos on a git host who doesn't mind actually doing their job as a git host by letting git clients clone git repos.
Some napkin math: - peak queries are 2500 requests per hour - some repos are 4gb in size - let’s round and say 2000 requests at 1gb = 2000 gb/hour = 48000 gb/day - AWS bandwidth at $0.02 / gb = $960 / day = $28,800 / month So, one of the richest companies in the world is charging you nearly $30k monthly because they cannot be bothered to be polite. Would you be ok with that situation?
[0] https://github.com/golang/go/issues/44577#issuecomment-78531...
[1] https://github.com/golang/go/issues/44577#issuecomment-85107...
Earlier quoted context omitted.
> rip it out of existence Come the fuck off this "mature personality" shit if you're going to write like this. He proposed freezing a module no one wanted to maintain in a library specifically meant to host stuff with weaker compat guarantees, he didn't hop in a DeLorean and kill Zimmermann's grandpa. Meanwhile, the critical project Drew insisted he keep it for is... deprecated and unmaintained!
[flagged]