Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

81–90 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#81

Post by mrighele »

toast0 wrote: post #10

If it were me, and I wasn't willing to just block the traffic, I might just set a 128 kbps limit on it and call it a day[1]. Eventually, the other side will figure out that their fetchers are all backed up and work out how to do their job without burning so much bandwidth. [1] Yeah, that can be a bit of a pain to setup depending on the server settings, but some people have to pay for bandwidth and server resources, s…

Why not just return a 429 (Too Many Requests) if the specific repo has been requested by google not too long ago ? (e.g. 1 hour earlier or).

It's a standard response code, and with a bit of luck google will scale the requests accordingly. If not, this will still allow the proxy to operate properly without burning too much server resources.

(I understand that this may leave some customers unhappy since the proxy may be up to 1 hour stale. If that's the case, the server could also add a check to see if there have been pushes since last request, though this is quite more complex)

Re: Google has been DDoSing Sourcehut for over a year

#82

Post by stefan_ »

klodolph wrote: post #33
toast0 wrote: post #10

If it were me, and I wasn't willing to just block the traffic, I might just set a 128 kbps limit on it and call it a day[1]. Eventually, the other side will figure out that their fetchers are all backed up and work out how to do their job without burning so much bandwidth. [1] Yeah, that can be a bit of a pain to setup depending on the server settings, but some people have to pay for bandwidth and server resources, s…

The rate limit is unlikely to cause a problem. Google has been crawling the web since its very start, and the internal services which fetch resources from external web servers are extremely resilient. Some request fails? Some request is slow? It's not going to slow down other requests. Maybe these services aren't being used for Go, but the expertise is on tap. (These kinds services are also supposed to rate-limit the…

You would think so, but has someone made a zip bomb with Git packs yet? ;)

Re: Google has been DDoSing Sourcehut for over a year

#83

Post by cycomanic »

rfoo wrote: post #62
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> and still tries to be helpful "Assuming everyone else have exactly same design choice and architecture as yourself, making suggestion on this ground and calling other people crazy because they can't implement what you suggest them to do" is not trying to be helpful. Well, or maybe I'm just frustrated reading his repeated "please keep a copy 'locally' somewhere and run git fetch". Just like how I'm frustrated arguin…

I often don't agree with DeVaults opinions, however the design decision here is actually costing other people real money and I have not seen a convincing argument why it is necessary. Instead they acknowledge it's an issue, but then continue that behavior for nearly a year, which IMO shows complete disregard for smaller services or individuals hosting their own repositories.

Re: Google has been DDoSing Sourcehut for over a year

#84

Post by szbalint »

cookiengineer wrote: post #56
FiloSottile wrote: post #46

Earlier quoted context omitted.

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

“DDoSed”

He complains about outgoing traffic of 4GB/day. On a server. In 2022.

If this is an issue he should not be hosting Go modules on his own servers.

Re: Google has been DDoSing Sourcehut for over a year

#85

Post by verdverm »

ocdtrekkie wrote: post #71
tptacek wrote: post #28

Earlier quoted context omitted.

You can phone anywhere you want; the proxy is configurable, and there are (from what I can see) several independent implementations of the proxy itself. Go defaults to Google's proxy; few people change the default because Google's proxy is very good.

This article strongly suggests Google's proxy is not "good", even if it "works". It sounds atrociously inefficient.

It brings a lot of security to the Golang dependency system. It is more than just a dumb proxy or cache

Re: Google has been DDoSing Sourcehut for over a year

#86

Post by thaway2839 »

FiloSottile wrote: post #46
threatofrain wrote: post #14

> I was banned from the Go issue tracker for mysterious reasons, so I cannot continue to nag them for a fix.¹ I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct). I tried to advocate for Linux distros to patch out GOPROXY by default, citing privacy reasons, but I was unsuccessful. I have no further re…

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

The second link is irrelevant to the issue at hand. I'm sure there are a lot of shitty Google devs who have behaved shittily with others. I don't think that's reason for, say, an ISP, to ignore any issues Google might face as an entity.

The relevant first link clearly lays out an implementation problem. It's not just git.sr.ht that's facing it, but another user also comes in to point out a tremendous amount of traffic. An amount of traffic that is fairly irresponsible to say the least.

The issue owner doesn't deny the problem. They simply say it requires work. It appears they are unwilling to do the work to resolve the unsavory behavior, and instead are asking the host of Go modules to disable the ability for their service to be used as a Go module host, or instead suck it up and deal with the cost and complications of Google not putting in the effort to fix their architecture's DDoSish behavior.

Google and Go have the market power to pull this off, but let's not pretend this isn't bad behavior. An appeal to Drew's terrible personal communication style does not change that.

Re: Google has been DDoSing Sourcehut for over a year

#87

Post by thaway2839 »

szbalint wrote: post #84
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

“DDoSed” He complains about outgoing traffic of 4GB/day. On a server. In 2022. If this is an issue he should not be hosting Go modules on his own servers.

4 GB/day is a different person facing the same issue who is hosting a single module they claim is likely only used by them.

Re: Google has been DDoSing Sourcehut for over a year

#88

Post by yjftsjthsd-h »

tptacek wrote: post #28
ocdtrekkie wrote: post #3

The fact that a programming language calls home to by Google by default should make it a non-starter for most sane developers. The fact that it calls home so it can DDoS other sites is low-key hilarious. And you'd think Google would know how to like... operate an efficient CDN, perhaps? Like, if this was managed by a competent company, you'd think this service would be akin to putting Cloudflare in front of your serv…

You can phone anywhere you want; the proxy is configurable, and there are (from what I can see) several independent implementations of the proxy itself. Go defaults to Google's proxy; few people change the default because Google's proxy is very good.

> Go defaults to Google's proxy; few people change the default because Google's proxy is very good.

What percentage of users do you think actually know that the proxy is even there?

Re: Google has been DDoSing Sourcehut for over a year

#89

Post by yjftsjthsd-h »

VWWHFSfQ wrote: post #55

No offense, but you built a git hosting website. and now you don't like the (pre-existing) ecosystem of things that use it? Maybe just block go packages altogether if it's that big of a deal

There's a difference between making something available and being okay with wasteful and excessive usage.

Re: Google has been DDoSing Sourcehut for over a year

#90

Post by thaway2839 »

rfoo wrote: post #62
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> and still tries to be helpful "Assuming everyone else have exactly same design choice and architecture as yourself, making suggestion on this ground and calling other people crazy because they can't implement what you suggest them to do" is not trying to be helpful. Well, or maybe I'm just frustrated reading his repeated "please keep a copy 'locally' somewhere and run git fetch". Just like how I'm frustrated arguin…

Suggesting ideas is not the same as assuming everyone else should have the exact same design choice and architecture. The problem is that the actual issue owner on the Google side does not appear to have any solutions (or is burying solutions because they may be too complex).

It's pretty clear that the issue owner agrees there is something wrong here. That's why the issue is kept open and their only reason for not fixing it is that it's a complicated fix which requires effort.

As a workaround, they offer Drew the opportunity to essentially disable his service from being used as a useful source of Go modules.

Post reply on HN