Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

71–80 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#71

Post by ocdtrekkie »

tptacek wrote: post #28
ocdtrekkie wrote: post #3

The fact that a programming language calls home to by Google by default should make it a non-starter for most sane developers. The fact that it calls home so it can DDoS other sites is low-key hilarious. And you'd think Google would know how to like... operate an efficient CDN, perhaps? Like, if this was managed by a competent company, you'd think this service would be akin to putting Cloudflare in front of your serv…

You can phone anywhere you want; the proxy is configurable, and there are (from what I can see) several independent implementations of the proxy itself. Go defaults to Google's proxy; few people change the default because Google's proxy is very good.

This article strongly suggests Google's proxy is not "good", even if it "works". It sounds atrociously inefficient.

Re: Google has been DDoSing Sourcehut for over a year

#72

Post by cycomanic »

FiloSottile wrote: post #46
threatofrain wrote: post #14

> I was banned from the Go issue tracker for mysterious reasons, so I cannot continue to nag them for a fix.¹ I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct). I tried to advocate for Linux distros to patch out GOPROXY by default, citing privacy reasons, but I was unsuccessful. I have no further re…

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

> Disclosure: I was on the Go team at Google until earlier this month. Dealing with DeVault's bad faith arguments is one of the few things I won't miss of that job.

So does or does not the problem persist? Second was or was he not banned from the commenting issue tracker. Third does the CoC require that a person gets notified by the moderator and was DeVault notified?

If the answers are yes to all those problems I wonder who is makeing bad faith arguments?

Note: I have absolutely no skin in this game, except for being a sway and gmail user.

Re: Google has been DDoSing Sourcehut for over a year

#73

Post by AshamedCaptain »

dmm wrote: post #4

> The Go team holds that this service is not a crawler, and thus they do not obey robots.txt This seems wrong. I guess I always assumed that robots.txt applied to non-humans.

Even the main Google search crawler does not respect Crawl-Delay from robots.txt (as of today), so it's a moot point anyway.

I suspect it's just a cultural thing inside Google.

Re: Google has been DDoSing Sourcehut for over a year

#74

Post by ocdtrekkie »

johnduhart wrote: post #69
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code. So yeah, I'd argue that the bad faith part is actually justified. https://github.com/golang/go/issues/44577#issuecomment-11378... > We did consider caching clones, but it has security implications and adds complexity, so we decided not to. It is certainly not trivial to do and not something we…

In this case, it's really hard to see thrashing other people's servers relentlessly to collect data you already have as anything but incredibly, incredibly poor engineering. Y'all should write him a check for that much resource waste.

Re: Google has been DDoSing Sourcehut for over a year

#75

Post by stefan_ »

johnduhart wrote: post #69
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code. So yeah, I'd argue that the bad faith part is actually justified. https://github.com/golang/go/issues/44577#issuecomment-11378... > We did consider caching clones, but it has security implications and adds complexity, so we decided not to. It is certainly not trivial to do and not something we…

> but it has security implications and adds complexity

Read: we prefer to use your servers for caching. Not good enough. Maybe the issue is people making silly evasive arguments like these while the server load piles on?

Re: Google has been DDoSing Sourcehut for over a year

#76

Post by ocdtrekkie »

rfoo wrote: post #62
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> and still tries to be helpful "Assuming everyone else have exactly same design choice and architecture as yourself, making suggestion on this ground and calling other people crazy because they can't implement what you suggest them to do" is not trying to be helpful. Well, or maybe I'm just frustrated reading his repeated "please keep a copy 'locally' somewhere and run git fetch". Just like how I'm frustrated arguin…

Cloudflare manages to cache things and then serve them to lots of people, without having to request the same content 50 times an hour. Maybe Google should move some of their services over to a better content delivery platform? o.O

This is a scenario where anyone who isn't a Googler can see that "being Google" is not a justification for bad engineering. Do better.

Re: Google has been DDoSing Sourcehut for over a year

#77

Post by verdverm »

klodolph wrote: post #33
toast0 wrote: post #10

If it were me, and I wasn't willing to just block the traffic, I might just set a 128 kbps limit on it and call it a day[1]. Eventually, the other side will figure out that their fetchers are all backed up and work out how to do their job without burning so much bandwidth. [1] Yeah, that can be a bit of a pain to setup depending on the server settings, but some people have to pay for bandwidth and server resources, s…

The rate limit is unlikely to cause a problem. Google has been crawling the web since its very start, and the internal services which fetch resources from external web servers are extremely resilient. Some request fails? Some request is slow? It's not going to slow down other requests. Maybe these services aren't being used for Go, but the expertise is on tap. (These kinds services are also supposed to rate-limit the…

This is completely different architecture and services, largely open sourced. This is limited to Go and the dev team.

Most likely these errors would manifest down to the users of Go and those who've a dependency which lives on sourcehut

Re: Google has been DDoSing Sourcehut for over a year

#78

Post by adamrezich »

ocdtrekkie wrote: post #74
johnduhart wrote: post #69

Earlier quoted context omitted.

> I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code. So yeah, I'd argue that the bad faith part is actually justified. https://github.com/golang/go/issues/44577#issuecomment-11378... > We did consider caching clones, but it has security implications and adds complexity, so we decided not to. It is certainly not trivial to do and not something we…

In this case, it's really hard to see thrashing other people's servers relentlessly to collect data you already have as anything but incredibly, incredibly poor engineering. Y'all should write him a check for that much resource waste.

without knowing anything about this situation outside of this thread and the post it links to, it comes across as willful negligence to screw over someone who was a bother in past community transgressions

Re: Google has been DDoSing Sourcehut for over a year

#79

Post by CaliforniaKarl »

tptacek wrote: post #50
Rantenki wrote: post #41

Earlier quoted context omitted.

Calling it .5 QPS to downplay the severity is willfully ignoring the complaint in the article. It's not just a query, it's a full Git clone of the entire repo with it's history. That's a huge difference.

That's what a "Q" means in this context. It's a git hosting service.

True, though I don't think a "Q" from a client doing a fetch involves as much work as a "Q" from a client doing a full clone.

It seems like this Git hosting service's workload-per-Q has increased due to the Go proxy change, which (IMO) is a cause for concern.

Re: Google has been DDoSing Sourcehut for over a year

#80

Post by verdverm »

captainmuon wrote: post #43

It sucks having to work around something like this, but maybe the following would work: only allow the first checkout from a given Go node, blackhole later accesses. If the repository is modified or a certain time elapsed, reset and allow a download again. Also, if you want to escalate, I wonder if there is a way to create a fake git repository that expands to a huge amount of data when cloning, but uses minimal band…

Malicious behavior would come back to byte the author and his users & business
Post reply on HN