Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

171–180 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#171

Post by tptacek »

nyberg wrote: post #170
tptacek wrote: post #164

Earlier quoted context omitted.

You didn't answer my question. What's the problem with the Go team's workaround? I get that DeVault would like to redesign the Go modules system to suit his own preferences, but that's not on the table.

The issue isn't the Go module system but rather their proxy which is not part of Go and should respect server resources but doesn't. The workaround makes any 3rd party host for go modules a bad choice as packages will always be stale. The issue has nothing to do with DD other than him raising the issue and being ignored. The fault is with google.

How exactly does the workaround make packages "always be stale"?

Re: Google has been DDoSing Sourcehut for over a year

#172

Post by derekzhouzhen »

With rubygems.org or hex.pm, the package maintainer must explicitly publish a package to the registry. The registry never poll the original source, so the whole issue would not exist in the first place. I found Go's package registry very strange and limited:

* What if I am using a source control program other than git? Mercury?

* What if I have files in my git repo that I don't want to show to the world, like tests?

Re: Google has been DDoSing Sourcehut for over a year

#173

Post by captainmuon »

verdverm wrote: post #80
captainmuon wrote: post #43

It sucks having to work around something like this, but maybe the following would work: only allow the first checkout from a given Go node, blackhole later accesses. If the repository is modified or a certain time elapsed, reset and allow a download again. Also, if you want to escalate, I wonder if there is a way to create a fake git repository that expands to a huge amount of data when cloning, but uses minimal band…

Malicious behavior would come back to byte the author and his users & business

Not neccesarily. I was thinking of this story: https://news.ycombinator.com/item?id=26074139

> At the height of the browser wars I once woke up to Microsoft hotlinking a small button for downloading our software from the MSN homepage. I tried to reach someone there for hours but nobody cared enough to do something about it. The image was small (no more than a few K), but the millions of requests that page got were enough to totally kill our server.

> Finally, I replaced the image on there with a 'Netscape Now' button. Within 15 minutes the matter was resolved.

And there was another case where some Indian app was using excessive traffic from Wikimedia, which was solved similarly. Sometimes you have to do something to get the attention of the right people. (And I could imagine if you do it right and are not too malicious, you might actually get a bug bounty on top.)

Re: Google has been DDoSing Sourcehut for over a year

#174

Post by kcartlidge »

tptacek wrote: post #165
kcartlidge wrote: post #162

Regardless of whether or not Drew is abrasive (I've never dealt with him so have no opinion), this is on Google . If a single service (Go's module crawler) is doing this to multiple third parties (and it is, by design) then requiring Drew to implement a workaround is the same thing as relying on every (relevant to the issue) third party to do the same thing. Which is bad design. This should be fixed at source or swit…

Can you help me understand the problem with the workaround, which is precisely to have Google's proxy not excessively poll DeVault's service? It really seems like DeVault's real argument here isn't about the impact of this on his service, but that he doesn't like the design of the proxy. He has a lot of standing to complain about impacts on his service, but essentially no standing to complain about designs he finds s…

I can't really speak for Drew's problem with the workaround, only my own from a design point of view - he may not even agree with my own issue at all for all I know, so I don't want to put words in his mouth.

From my own perspective, however, the issue is about impact and responsibility.

The excess traffic is their impact as it is caused by their design and is entirely of their choosing. Thus it becomes their responsibility.

The workaround they offer (as with so much in tech) means their stuff becomes opt-out and not opt-in. By which I mean their own decisions/designs cause an impact where they require everyone else suffering from it to specifically and individually opt-out of being excessively polled.

This is an outrageous requirement they can only impose due to the power imbalance.

Re: Google has been DDoSing Sourcehut for over a year

#175

Post by exikyut »

8organicbits wrote: post #68

I'm wondering how much load this is sending to github. Github has a ton of golang packages, including many non-popular ones that wouldn't otherwise get much traffic. A refresh job running full clones many times a day must be burning up bandwidth and compute over there as well. I suppose it's a drop in the bucket for Github's usage, but it's got to be a huge number.

I wouldn't be surprised if half the value of the Microsoft acquisition was to get GitHub's infra closer to the super-scale pipes and peering that Azure/Office/Update/etc leverage. I would expect they probably run tens to hundreds of PBs per month.

Re: Google has been DDoSing Sourcehut for over a year

#176

Post by tptacek »

kcartlidge wrote: post #174
tptacek wrote: post #165

Earlier quoted context omitted.

Can you help me understand the problem with the workaround, which is precisely to have Google's proxy not excessively poll DeVault's service? It really seems like DeVault's real argument here isn't about the impact of this on his service, but that he doesn't like the design of the proxy. He has a lot of standing to complain about impacts on his service, but essentially no standing to complain about designs he finds s…

I can't really speak for Drew's problem with the workaround, only my own from a design point of view - he may not even agree with my own issue at all for all I know, so I don't want to put words in his mouth. From my own perspective, however, the issue is about impact and responsibility. The excess traffic is their impact as it is caused by their design and is entirely of their choosing. Thus it becomes their respons…

It sounds like DeVault let them know their proxy was hitting his service too hard, and they immediately responded by saying they could dial it down. Seems like a pretty normal sequence of events? I feel like I have to be missing something here.

Re: Google has been DDoSing Sourcehut for over a year

#177

Post by akagusu »

What I find shocking is not Google doing this or even how Go team didn't care about the issue and apparently banished him, but how much people here think this is an acceptable behavior and how much people take their time to defend Google and to justify this behavior blaming not Google but Drew because "it is hard to deal with him".

Re: Google has been DDoSing Sourcehut for over a year

#178

Post by voxtech »

tptacek wrote: post #151
quadrifoliate wrote: post #150

Earlier quoted context omitted.

> They didn't brush him off. I disagree, the "for boring technical reasons..." is as close to a brush-off as I can see. This is a technical issue tracker, why not be open and honest about the reasons? I feel like these days people just seem to take it on faith that "Oh, it's Google, surely they know best when they say it's a mysterious technical issue that's too hard to solve". Also, banning someone from the issue tr…

This is how every issue tracker works. It takes time to explain things (and more time to generate all the facts needed to put together an accurate explanation). What matters is that they got him a workaround --- apparently, almost immediately. He hasn't taken them up on it; it's unclear why. There's no indication I can see that DeVault's "banning" from the issue tracker has anything to do with any of this. There's al…

This claim is outrageous- if disabling sourcehut is a valid workaround, then google should end the refresh behavior on the proxy since it isn't providing any valuable service. If the proxy provides a valuable service, then disabling sourcehut would degrade service for sourcehut users.

Re: Google has been DDoSing Sourcehut for over a year

#179

Post by philosopher1234 »

tptacek wrote: post #176
kcartlidge wrote: post #174

Earlier quoted context omitted.

I can't really speak for Drew's problem with the workaround, only my own from a design point of view - he may not even agree with my own issue at all for all I know, so I don't want to put words in his mouth. From my own perspective, however, the issue is about impact and responsibility. The excess traffic is their impact as it is caused by their design and is entirely of their choosing. Thus it becomes their respons…

It sounds like DeVault let them know their proxy was hitting his service too hard, and they immediately responded by saying they could dial it down. Seems like a pretty normal sequence of events? I feel like I have to be missing something here.

This is probably 80% peoples' existing animosity towards Google & Go, so they're ready to hate, regardless of how the facts land.

I think its primarily about the Go team having a different working philosophy than the standard HN soft. dev philosophy, and an intolerance of difference.

Re: Google has been DDoSing Sourcehut for over a year

#180

Post by tptacek »

voxtech wrote: post #178
tptacek wrote: post #151

Earlier quoted context omitted.

This is how every issue tracker works. It takes time to explain things (and more time to generate all the facts needed to put together an accurate explanation). What matters is that they got him a workaround --- apparently, almost immediately. He hasn't taken them up on it; it's unclear why. There's no indication I can see that DeVault's "banning" from the issue tracker has anything to do with any of this. There's al…

This claim is outrageous- if disabling sourcehut is a valid workaround, then google should end the refresh behavior on the proxy since it isn't providing any valuable service. If the proxy provides a valuable service, then disabling sourcehut would degrade service for sourcehut users.

That logic doesn't even cohere. If Github and Gitlab can sustain more aggressive queries from the module proxy, they may have fresher cache entries in that proxy. That's what being more capable gets you. The Go team isn't obligated to operate at the lowest common denominator of all source hosts.

If the Go team's workaround was to make sr.ht not viable for Go modules, that would be deeply problematic. But it isn't; in fact, nobody who's upset about this seems to be able to describe what the Go team's workaround would even impact. Can you?

Post reply on HN