Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

161–170 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#161

Post by Ventito »

thaway2839 wrote: post #87
szbalint wrote: post #84

Earlier quoted context omitted.

“DDoSed” He complains about outgoing traffic of 4GB/day. On a server. In 2022. If this is an issue he should not be hosting Go modules on his own servers.

4 GB/day is a different person facing the same issue who is hosting a single module they claim is likely only used by them.

Still whats 4GB a day?

I really really don't get the issue.

Is he io bound? Is there no reasonable way for git hosting services to actually cache git checkouts?

DDoS / DoS should only be used, in my opinion, when the server goes down complelty not if it has a little bit of load and i still haven't seen anything which indicates that this is a real issue?

And i don't want to come across negativly, i just don't get the issue.

Re: Google has been DDoSing Sourcehut for over a year

#162

Post by kcartlidge »

Regardless of whether or not Drew is abrasive (I've never dealt with him so have no opinion), this is on Google.

If a single service (Go's module crawler) is doing this to multiple third parties (and it is, by design) then requiring Drew to implement a workaround is the same thing as relying on every (relevant to the issue) third party to do the same thing. Which is bad design.

This should be fixed at source or switched off.

As for the workaround itself it's true that going on a crawler exclusion list will not stop SourceHut being usable, only make it a little stale, but that doesn't matter. The problem is not at Drew's end. And once again for this workaround to be an effective solution every (relevant to the issue) third party needs to do the same thing. Which is bad design.

As a general rule, with excessive polling the system doing the polling is the one at fault. And as a general rule if one system impacts many fix at the originator. If Google devs don't know this, then why on earth are they getting the salaries they do?

Re: Google has been DDoSing Sourcehut for over a year

#163

Post by downsidesabound »

threatofrain wrote: post #14

> I was banned from the Go issue tracker for mysterious reasons, so I cannot continue to nag them for a fix.¹ I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct). I tried to advocate for Linux distros to patch out GOPROXY by default, citing privacy reasons, but I was unsuccessful. I have no further re…

Guy’s public behavior is constantly abrasive, I’m honestly surprised at the minor following he has.

Being right isn’t the only most important thing in the world, nor does it wash away all transgressions.

Re: Google has been DDoSing Sourcehut for over a year

#164

Post by tptacek »

rurban wrote: post #154
tptacek wrote: post #148

Earlier quoted context omitted.

Can you articulate why it isn't a solution, and how it would be punitive? There are people on this thread who appear to believe Google's workaround would mean that repositories hosted on sr.ht would be unusable as Go modules, which is not at all the case.

drew articulated it very well why google's offer doesn't help at all. https://github.com/golang/go/issues/44577#issuecomment-85693... A full git clone just to DDOS a hoster to check if the user-experience is still first-class, and filling a proxy is not an acceptable solution for a module hoster who has the pay the hosting bills by himself. If they want to know if their proxy is still uptodate, a cheap latest change…

You didn't answer my question. What's the problem with the Go team's workaround? I get that DeVault would like to redesign the Go modules system to suit his own preferences, but that's not on the table.

Re: Google has been DDoSing Sourcehut for over a year

#165

Post by tptacek »

kcartlidge wrote: post #162

Regardless of whether or not Drew is abrasive (I've never dealt with him so have no opinion), this is on Google . If a single service (Go's module crawler) is doing this to multiple third parties (and it is, by design) then requiring Drew to implement a workaround is the same thing as relying on every (relevant to the issue) third party to do the same thing. Which is bad design. This should be fixed at source or swit…

Can you help me understand the problem with the workaround, which is precisely to have Google's proxy not excessively poll DeVault's service? It really seems like DeVault's real argument here isn't about the impact of this on his service, but that he doesn't like the design of the proxy. He has a lot of standing to complain about impacts on his service, but essentially no standing to complain about designs he finds sub-par.

Re: Google has been DDoSing Sourcehut for over a year

#166

Post by tptacek »

RealStickman_ wrote: post #160
tptacek wrote: post #128

Earlier quoted context omitted.

Obviously it does cache, because the simplest solution offered by the Go team is to exclude sr.ht from the cache refresh that DeVault is complaining about, and he hasn't taken them up on that.

If it's such a widespread problem that there is a process to exclude domains from the cache refresh, doesn't that point to it being an issue that should be fixed by the party responsible instead of everybody else separately?

So far as I know, DeVault and the one other person who chimed in on his ticket are the only people who have ever availed themselves of this process, which may well have been created specifically for them. So, no, I don't think that's a valid point. Also: it is being fixed by the party responsible.

Re: Google has been DDoSing Sourcehut for over a year

#167

Post by kevincox »

AshamedCaptain wrote: post #73
dmm wrote: post #4

> The Go team holds that this service is not a crawler, and thus they do not obey robots.txt This seems wrong. I guess I always assumed that robots.txt applied to non-humans.

Even the main Google search crawler does not respect Crawl-Delay from robots.txt (as of today), so it's a moot point anyway. I suspect it's just a cultural thing inside Google.

I don't think so. When I worked at Google most things that talked to the outside world were carefully rate limited with quite generous backoffs if there was signs of the receiving side being harmed (errors or slowdowns). Especially crawling and mail sending. But Google is a big company and it is very likely that the Go team didn't think of these things that are probably top-of-mind for someone on the crawling, ads or mail teams.

Re: Google has been DDoSing Sourcehut for over a year

#169

Post by u9000 »

tptacek wrote: post #166
RealStickman_ wrote: post #160

Earlier quoted context omitted.

If it's such a widespread problem that there is a process to exclude domains from the cache refresh, doesn't that point to it being an issue that should be fixed by the party responsible instead of everybody else separately?

So far as I know, DeVault and the one other person who chimed in on his ticket are the only people who have ever availed themselves of this process, which may well have been created specifically for them. So, no, I don't think that's a valid point. Also: it is being fixed by the party responsible.

was being fixed. They just marked the issue as resolved and blocked any more comments on the thread.

Re: Google has been DDoSing Sourcehut for over a year

#170

Post by nyberg »

tptacek wrote: post #164
rurban wrote: post #154

Earlier quoted context omitted.

drew articulated it very well why google's offer doesn't help at all. https://github.com/golang/go/issues/44577#issuecomment-85693... A full git clone just to DDOS a hoster to check if the user-experience is still first-class, and filling a proxy is not an acceptable solution for a module hoster who has the pay the hosting bills by himself. If they want to know if their proxy is still uptodate, a cheap latest change…

You didn't answer my question. What's the problem with the Go team's workaround? I get that DeVault would like to redesign the Go modules system to suit his own preferences, but that's not on the table.

The issue isn't the Go module system but rather their proxy which is not part of Go and should respect server resources but doesn't. The workaround makes any 3rd party host for go modules a bad choice as packages will always be stale.

The issue has nothing to do with DD other than him raising the issue and being ignored. The fault is with google.

Post reply on HN