Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sourcehut will blacklist the Go module mirror

sourcehut.org

Post reply on HN
Print view View original

221–230 of 365 posts

Re: Sourcehut will blacklist the Go module mirror

#221

Post by josephcsible »

cratermoon wrote: post #168
josephcsible wrote: post #137

Earlier quoted context omitted.

Why? Google has quite a reputation for wrongfully banning people from things. This doesn't seem unusual for them at all.

> Google has quite a reputation for wrongfully banning people from things. Extraordinary claims require extraordinary evidence.

Here's some evidence:

https://news.ycombinator.com/item?id=3803568

https://news.ycombinator.com/item?id=15066518

https://news.ycombinator.com/item?id=19124324

https://news.ycombinator.com/item?id=20826618

https://news.ycombinator.com/item?id=20841586

https://news.ycombinator.com/item?id=21247759

https://news.ycombinator.com/item?id=24791357

https://news.ycombinator.com/item?id=24965432

https://news.ycombinator.com/item?id=26061935

https://news.ycombinator.com/item?id=26956077

https://news.ycombinator.com/item?id=26965146

https://news.ycombinator.com/item?id=28621412

https://news.ycombinator.com/item?id=30855065

Re: Sourcehut will blacklist the Go module mirror

#222

Post by yamtaddle »

Beltalowda wrote: post #216
ddevault wrote: post #160

Earlier quoted context omitted.

I was indeed in the wrong when I made this comment four years ago. I have since apologized for it. I don't intend to re-litigate anything on HN at this point, but I have good reason to believe that this incident is unrelated to the reason I am presently banned. The linked comment was indeed out of line, and perhaps you feel justified in thinking that it should be sufficient grounds for a permanent expulsion from the…

> I don't think it's reasonable to use it as grounds to suggest that anyone should have their servers DoSed by Google with no recourse Of course not; this entire thread isn't necessarily hugely on-topic here, but it got brought up, so ... well ... here we are. And in fairness, you did bring up your ban in the posted article. > The linked comment was indeed out of line, and perhaps you feel justified in thinking that…

> "oh, that's the same guy who called us a bunch of morons last year"

Wait, did some folks in the Go community write that EFAIL site that was referenced as a reason to drop OpenPGP? If so, that changes the context of the post a bit, but I didn't see anything indicating that was the case in the linked thread.

Re: Sourcehut will blacklist the Go module mirror

#223

Post by maximilianburke »

rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

I realize in the real world most modules are probably hosted by large providers that can absorb the bandwidth, like Github, but it seems incredibly discourteous to not prioritize the hammering of small providers, especially two years on when the response is still "maybe later this year".

I think Drew is right in that he shouldn't take a personalized Sourcehut-only exception because this doesn't address the core issue for any new small providers that pop up.

Between this and the response in the original thread that said, "For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt," it gives the impression that the Go team doesn't care. Sometimes what we _need_ to do to be good netizens is a fair bit of boring technical work but it's essential.

Re: Sourcehut will blacklist the Go module mirror

#224

Post by jlarocco »

psanford wrote: post #43
ComputerGuru wrote: post #18

From the GitHub issue, by a Googler: > For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt […] This is coming from one of the biggest, richest, most well-staffed companies on the planet. It’s too much work for them to read a robots.txt file like the rest of the world (and plenty of one-man teams) do before hammering a server with terabytes of requests. If this is too much for…

Why did sourcehut not take the offer to be added to the refresh exclusion list like the other two small hosting providers did? It seems like that would have resolved this issue last year.

Supposedly the operator of sourcehut has been banned from posting to the Go issue tracker: https://github.com/golang/go/issues/44577#issuecomment-11378...

So, obviously he's supposed to know the even more obscure and annoying method of opting out.

Re: Sourcehut will blacklist the Go module mirror

#225

Post by dpkirchner »

cstejerean wrote: post #217
rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

I suspect they have a problem with this DDoS by default unless you ask to opt out behavior. Why is anyone getting hit with these expensive background refreshes until you have a chance to do it right? Why is it still not done right 2 years after this was first reported? Maybe it should be an opt-in list where the big providers (such as github) can be hit by an army of bots and everyone else is safe by default.

And why isn't this a priority? These folks are offering a service that benefits the Go ecosystem.

It smells like Go is on its way out.

Re: Sourcehut will blacklist the Go module mirror

#226

Post by ddevault »

rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

Hi Russ! Thank you for sharing. I am pleased to hear that there is finally some progress towards a solution for this problem. If you or someone working on the issue can reach out via email (sir@cmpwn.com), I would be happy to discuss the issue further. What you described seems like an incomplete solution, and I would like to discuss some additional details with your team, but it is a good start. I'm also happy to postpone or cancel the planned ban on the Go proxy if there's active motion towards a fix from Google's end. I am, however, a bit uneasy that you mentioned that it's only prioritized for "this year" -- another year of enduring a DoS from Google does not sound great.

I cannot file an issue; as the article explains I was banned from the Go community without explanation or recourse; and the workaround is not satisfying for reasons I outlined in other HN comments and on GitHub. However, I would appreciate receiving a follow-up via email from someone knowledgeable on the matter, and so long as there is an open line of communication I can be much more patient. These things are easily solved when they're treated with mutual respect and collaboration between engineering teams, which has not been my experience so far. That said, I am looking forward to finally putting this issue behind us.

Re: Sourcehut will blacklist the Go module mirror

#227

Post by dpkirchner »

psanford wrote: post #66
_wolfie_ wrote: post #33

Earlier quoted context omitted.

But that is what should happen. The build should break. So that someone can fix it.

Why? If some open source maintainer goes off the rails and deletes all their packages, why should that break my builds? I still have a valid license to the code, I don't really care that a maintainer rage quit 4 dependencies down from my application. I certainly don't want to have to scramble to deal with that.

This is an argument for maintaining a local cache of necessary build dependencies, not to rely on a third party.

Re: Sourcehut will blacklist the Go module mirror

#228

Post by wmf »

throwaway894345 wrote: post #63
ComputerGuru wrote: post #18

From the GitHub issue, by a Googler: > For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt […] This is coming from one of the biggest, richest, most well-staffed companies on the planet. It’s too much work for them to read a robots.txt file like the rest of the world (and plenty of one-man teams) do before hammering a server with terabytes of requests. If this is too much for…

I don't have any particular affinity for Google, but they're still a business and they're already developing the Go language (and relevant infrastructure) at their own expense. It's not like the Go team at Google has access to the entire Alphabet war chest like your "biggest, richest, well-staffed companies on the planet" suggests.

This problem isn't really specific to Google. If some hobby project was DoSing sites it would get banned. "We don't have the resources to not DoS" is not a valid excuse. The Go team needs to scope their ambitions properly; if they can't make their proxy work safely they should not have bothered to develop it.

Re: Sourcehut will blacklist the Go module mirror

#229

Post by eadmund »

rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

This reminds me a bit of a disfunctional relationship: clearly Sourcehut wants Google to stop DDoS their servers; clearly Google don’t actually want to DDoS Sourcehut, but Sourcehut also doesn’t want to ask Google to stop, and Google also want to be asked to stop. And so nothing gets done.

The question is who will swallow their pride first: Sourcehut or Google.

Re: Sourcehut will blacklist the Go module mirror

#230

Post by vladharbuz »

rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

Thanks for the insight, Russ. Would you comment on what the potential consequences of opting out of background refreshes would be? Could there be any adverse effects for users?
Post reply on HN