Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

111–120 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#111

Post by Beltalowda »

thaway2839 wrote: post #86
FiloSottile wrote: post #46

Earlier quoted context omitted.

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

The second link is irrelevant to the issue at hand. I'm sure there are a lot of shitty Google devs who have behaved shittily with others. I don't think that's reason for, say, an ISP, to ignore any issues Google might face as an entity. The relevant first link clearly lays out an implementation problem. It's not just git.sr.ht that's facing it, but another user also comes in to point out a tremendous amount of traffi…

> The second link is irrelevant to the issue at hand.

It's relevant to the issue of the ban, which is what the (sub)thread is about. I think the reason it was posted was to demonstrate a pattern, and I think that's pretty relevant.

Every single disagreement I've had with Drew escalated and I don't think that's my fault since it never happens with anyone else (Never? Well, hardly ever) and I've seen it happen with various other people too. Not that I'm perfect by any means or couldn't have done things better, but there's certainly a pattern here.

As for the actual issue at hand: I mostly agree with Drew, however, I don't really have enough information to be sure here, and Drew has misrepresented things in the past and does so here in this post (a reader unfamiliar with Go would be left assuming that Go "phones home" just for the sake of "phoning home" after reading this article, which is really a misrepresentation IMHO), so there's not a lot of trust here (again, a pattern).

Re: Google has been DDoSing Sourcehut for over a year

#112

Post by klodolph »

verdverm wrote: post #77
klodolph wrote: post #33

Earlier quoted context omitted.

The rate limit is unlikely to cause a problem. Google has been crawling the web since its very start, and the internal services which fetch resources from external web servers are extremely resilient. Some request fails? Some request is slow? It's not going to slow down other requests. Maybe these services aren't being used for Go, but the expertise is on tap. (These kinds services are also supposed to rate-limit the…

This is completely different architecture and services, largely open sourced. This is limited to Go and the dev team. Most likely these errors would manifest down to the users of Go and those who've a dependency which lives on sourcehut

A big chunk of the Go core maintainers are Google employees, and they run Go services on top of Google infrastructure.

Re: Google has been DDoSing Sourcehut for over a year

#113

Post by tptacek »

cycomanic wrote: post #72
FiloSottile wrote: post #46

Earlier quoted context omitted.

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

> Disclosure: I was on the Go team at Google until earlier this month. Dealing with DeVault's bad faith arguments is one of the few things I won't miss of that job. So does or does not the problem persist? Second was or was he not banned from the commenting issue tracker. Third does the CoC require that a person gets notified by the moderator and was DeVault notified? If the answers are yes to all those problems I wo…

The subthread we're commenting on is about the ban, not about the proxy.

Re: Google has been DDoSing Sourcehut for over a year

#114

Post by [deleted] »

threatofrain wrote: post #14

> I was banned from the Go issue tracker for mysterious reasons, so I cannot continue to nag them for a fix.¹ I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct). I tried to advocate for Linux distros to patch out GOPROXY by default, citing privacy reasons, but I was unsuccessful. I have no further re…

[deleted]

Re: Google has been DDoSing Sourcehut for over a year

#115

Post by tptacek »

ziml77 wrote: post #96
johnduhart wrote: post #69

Earlier quoted context omitted.

> I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code. So yeah, I'd argue that the bad faith part is actually justified. https://github.com/golang/go/issues/44577#issuecomment-11378... > We did consider caching clones, but it has security implications and adds complexity, so we decided not to. It is certainly not trivial to do and not something we…

I don't particularly care for Drew, but the issue he's reported here seems totally valid. And if he requested that he be excluded from getting hit by the crawler, wouldn't that mean it would be impossible for people to use packages from sr.ht unless they change their config? Plus, it does seem reasonable to think that only one of the crawlers needs to hit the site. The global replication can happen at the FS level or…

No. According to the Go project, adding his site to the exclusion list would reduce traffic to his site at the cost of freshness of the data the proxy collects; it would not make it "impossible" for people to use packages from sr.ht.

This is all in the thread that DeVault linked to from his post.

Re: Google has been DDoSing Sourcehut for over a year

#116

Post by tptacek »

adamrezich wrote: post #78
ocdtrekkie wrote: post #74

Earlier quoted context omitted.

In this case, it's really hard to see thrashing other people's servers relentlessly to collect data you already have as anything but incredibly, incredibly poor engineering. Y'all should write him a check for that much resource waste.

without knowing anything about this situation outside of this thread and the post it links to, it comes across as willful negligence to screw over someone who was a bother in past community transgressions

That's a risible suggestion. Even DeVault doesn't say that.

Re: Google has been DDoSing Sourcehut for over a year

#117

Post by tptacek »

pengaru wrote: post #70
arccy wrote: post #63

Earlier quoted context omitted.

bad faith refers to his behavior on other issue threads. also he (used to) spam the issue tracker with ads for his services

> also he (used to) spam the issue tracker with ads for his services If there's any entity I'm totally ok with anyone spamming with ads, it's Google .

It's not "Google" you'd be spamming, it's the Go language community, most of which has nothing to do with Google. Please don't make high-drama arguments like this.

Re: Google has been DDoSing Sourcehut for over a year

#118

Post by ajford »

rfoo wrote: post #62
cookiengineer wrote: post #56

Earlier quoted context omitted.

How are his arguments in bad faith if he is the one that gets DDoSed by your software for over a year, and still tries to be helpful? Not sure if you realize the absurdity of this, but he has to pay traffic and server costs. Like everyone else, except probably Google as it seems!? I mean, you didn't even consider implementing a simple fetch of an already cloned repository in your mirroring server code . So yeah, I'd…

> and still tries to be helpful "Assuming everyone else have exactly same design choice and architecture as yourself, making suggestion on this ground and calling other people crazy because they can't implement what you suggest them to do" is not trying to be helpful. Well, or maybe I'm just frustrated reading his repeated "please keep a copy 'locally' somewhere and run git fetch". Just like how I'm frustrated arguin…

Just about every CDN out there manages to cache at a global scale from a single hit, so I don't understand what's so vastly different about the go mirror system that it can't do the same under the hood.

Re: Google has been DDoSing Sourcehut for over a year

#119

Post by theamk »

8organicbits wrote: post #44

From https://github.com/golang/go/issues/44577#issuecomment-85107... > Yesterday, GoModuleMirror downloaded 4 gigabytes of data from my server requesting a single module over 500 times (log attached). As far as I know, I am the only person in the world using this Go module. From https://github.com/golang/go/issues/44577#issuecomment-78924... > yes we make a fresh clone every time I like golang as a developer, but thi…

From https://github.com/golang/go/issues/44577#issuecomment-11378...

> Anyone who's receiving too much traffic from proxy.golang.org can request that they be excluded from the refresh traffic, as we did for git.lubar.me. Nobody asked for sr.ht be added to the exclusion set, so as far as it's concerned nothing has changed.

Re: Google has been DDoSing Sourcehut for over a year

#120

Post by ajford »

verdverm wrote: post #85
ocdtrekkie wrote: post #71

Earlier quoted context omitted.

This article strongly suggests Google's proxy is not "good", even if it "works". It sounds atrociously inefficient.

It brings a lot of security to the Golang dependency system. It is more than just a dumb proxy or cache

Smarter than your average proxy, but apparently dumb enough it doesn't actually cache at all, according to the reports.
Post reply on HN