Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google has been DDoSing Sourcehut for over a year

drewdevault.com

Post reply on HN
Print view View original

141–150 of 225 posts

Re: Google has been DDoSing Sourcehut for over a year

#141

Post by spacexsucks »

prepend wrote: post #130
res0nat0r wrote: post #105

Earlier quoted context omitted.

They offered to turn off refreshing of his domain it appears on Jun 8, 2021: https://github.com/golang/go/issues/44577#issuecomment-85692...

That doesn’t seem like a solution at all and is actually kind of punative as that would make srht bad for hosting go. I think this is just an example of Google being a jerk and not caring enough to do proper software engineering. Go seems really interesting but I have avoided using it because it’s so tied to Google. And I don’t trust Google to make good decisions for developers or users.

No post body was provided.

Re: Google has been DDoSing Sourcehut for over a year

#145

Post by quadrifoliate »

FiloSottile wrote: post #46
threatofrain wrote: post #14

> I was banned from the Go issue tracker for mysterious reasons, so I cannot continue to nag them for a fix.¹ I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct). I tried to advocate for Linux distros to patch out GOPROXY by default, citing privacy reasons, but I was unsuccessful. I have no further re…

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

I do not understand why the issue is not discussing DeVault's straightforward robots.txt suggestion:

> Have you considered the robots.txt approach, which would simply allow the sysadmin to tune the rate at which you will scrape their service? The best option puts the controls in the hands of the sysadmins you're affecting. This is what the rest of the internet does.

The only explanation I see later in the thread is:

> For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt, so rather than going to a bunch of work to do that, we implemented a trivial list and offered to add sr.ht to it.

This is...not a satisfactory technical explanation. Perhaps the Go team should consider providing more openness and transparency about why it's "a fair bit of extra work" to implement an internet standard like DeVault suggests.

Honestly, DeVault is an abrasive person and not necessarily someone I would go out of my way to work with, but I don't see how the Golang team isn't at least somewhat at fault here for brushing off a community member with "Eh, it's too complicated for you to understand". That's not how you build a thriving community around a language.

Re: Google has been DDoSing Sourcehut for over a year

#146

Post by quadrifoliate »

mrighele wrote: post #81
toast0 wrote: post #10

If it were me, and I wasn't willing to just block the traffic, I might just set a 128 kbps limit on it and call it a day[1]. Eventually, the other side will figure out that their fetchers are all backed up and work out how to do their job without burning so much bandwidth. [1] Yeah, that can be a bit of a pain to setup depending on the server settings, but some people have to pay for bandwidth and server resources, s…

Why not just return a 429 (Too Many Requests) if the specific repo has been requested by google not too long ago ? (e.g. 1 hour earlier or). It's a standard response code, and with a bit of luck google will scale the requests accordingly. If not, this will still allow the proxy to operate properly without burning too much server resources. (I understand that this may leave some customers unhappy since the proxy may b…

> It's a standard response code, and with a bit of luck google will scale the requests accordingly.

Google already says in the comments [1] that it would be "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". I would not necessarily rely on them to respect HTTP 429 either.

In fact, given the facts at hand with the current situation, I would guess that they probably wouldn't handle it correctly, and would cite more "boring technical reasons" when asked.

----------------------------------------

[1] https://github.com/golang/go/issues/44577#issuecomment-11378...

Re: Google has been DDoSing Sourcehut for over a year

#147

Post by tptacek »

quadrifoliate wrote: post #145
FiloSottile wrote: post #46

Earlier quoted context omitted.

> In the meantime, if you would prefer, we can turn off all refresh traffic for your domain while we continue to improve this on our end. That would mean that the only traffic you would receive from us would be the result of a request directly from a user. This may impact the freshness of your domain's data which users receive from our servers, since we need to have some caching on our end to prevent too frequent fet…

I do not understand why the issue is not discussing DeVault's straightforward robots.txt suggestion: > Have you considered the robots.txt approach, which would simply allow the sysadmin to tune the rate at which you will scrape their service? The best option puts the controls in the hands of the sysadmins you're affecting. This is what the rest of the internet does. The only explanation I see later in the thread is:…

They didn't brush him off. They gave him an immediate workaround, which --- contra some messages on this thread --- did not entail making sr.ht unusable for Go projects. At the time he posted this, he had not taken the Go team up on that workaround; doing so appears to involve only DeVault saying "go ahead" to the Go team.

Re: Google has been DDoSing Sourcehut for over a year

#148

Post by tptacek »

prepend wrote: post #130
res0nat0r wrote: post #105

Earlier quoted context omitted.

They offered to turn off refreshing of his domain it appears on Jun 8, 2021: https://github.com/golang/go/issues/44577#issuecomment-85692...

That doesn’t seem like a solution at all and is actually kind of punative as that would make srht bad for hosting go. I think this is just an example of Google being a jerk and not caring enough to do proper software engineering. Go seems really interesting but I have avoided using it because it’s so tied to Google. And I don’t trust Google to make good decisions for developers or users.

Can you articulate why it isn't a solution, and how it would be punitive? There are people on this thread who appear to believe Google's workaround would mean that repositories hosted on sr.ht would be unusable as Go modules, which is not at all the case.

Re: Google has been DDoSing Sourcehut for over a year

#150

Post by quadrifoliate »

tptacek wrote: post #147
quadrifoliate wrote: post #145

Earlier quoted context omitted.

I do not understand why the issue is not discussing DeVault's straightforward robots.txt suggestion: > Have you considered the robots.txt approach, which would simply allow the sysadmin to tune the rate at which you will scrape their service? The best option puts the controls in the hands of the sysadmins you're affecting. This is what the rest of the internet does. The only explanation I see later in the thread is:…

They didn't brush him off. They gave him an immediate workaround, which --- contra some messages on this thread --- did not entail making sr.ht unusable for Go projects. At the time he posted this, he had not taken the Go team up on that workaround; doing so appears to involve only DeVault saying "go ahead" to the Go team.

> They didn't brush him off.

I disagree, the "for boring technical reasons..." is as close to a brush-off as I can see. This is a technical issue tracker, why not be open and honest about the reasons? I feel like these days people just seem to take it on faith that "Oh, it's Google, surely they know best when they say it's a mysterious technical issue that's too hard to solve".

Also, banning someone from the issue tracker in violation of their own CoC does not seem like good faith behavior either.

Note that the message from 'FiloSottile does not clearly spell out whether DeVault's ban was following a CoC, it just obliquely quotes some nasty stuff that he's apparently said in the past. I think other commenters are correct to call it irrelevant, it's about as relevant as it would be if I started randomly quoting Rob Pike's notorious comments about syntax highlighting [1].

Overall, I totally acknowledge the Golang team is providing a workaround that will solve the (admittedly abrasively spoken) user's problems. What I'm saying is that that is not enough. If you want an open language that will not turn into .NET, you have to do better by your community and also be transparent as to how and when you're planning to solve your users' problems. And preferably also be transparent about CoC bans, especially if there is a technical discussion involved.

For example, this sounds like a reasonable response to me:

"We have written up a task to make our proxy code parse robots.txt correctly, but there are N other tasks above it, and it looks like we won't be able to work on it for approximately the next six months. Until then, a workaround is..."

----------------------------------------

[1] https://groups.google.com/g/golang-nuts/c/hJHCAaiL0so/m/E2mQ....

Post reply on HN