Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sourcehut will blacklist the Go module mirror

sourcehut.org

Post reply on HN
Print view View original

101–110 of 365 posts

Re: Sourcehut will blacklist the Go module mirror

#101

Post by luckylion »

psanford wrote: post #66
_wolfie_ wrote: post #33

Earlier quoted context omitted.

But that is what should happen. The build should break. So that someone can fix it.

Why? If some open source maintainer goes off the rails and deletes all their packages, why should that break my builds? I still have a valid license to the code, I don't really care that a maintainer rage quit 4 dependencies down from my application. I certainly don't want to have to scramble to deal with that.

Do they change the behavior if the repo was dropped for legal issues, and no valid licenses could have been obtained because the repo owner didn't have one in the first place?

Re: Sourcehut will blacklist the Go module mirror

#102

Post by cbrozefsky »

tssva wrote: post #94
SuperSandro2000 wrote: post #65

So either you disable the proxy completely for your site or get overwhelmed by traffic from a not to well written service, similar to a small DDOS attack, which is run by google and they are not planning to fix this? Did I get anything wrong here?

There is a refresh exclusion list which you can request your site to be added to. The proxy will continue to process requests for modules from your site but will not perform the automatic refresh which caused issues for Sourcehut. The Go team extended an offer to add Sourcehut to the list if a request to do so was made. The request never came and instead Sourcehut blocked the proxy.

"Opt out of me DoSing" you is not legit behavior -- especially when the victim has raised it with you multiple times, suggested fixes, and you have then blocked them from communicating in your issue tracker.

That's some really entitled thinking on the part of the Go team at Google, and it's sad to see people stanning for them.

Re: Sourcehut will blacklist the Go module mirror

#103

Post by throwaway894345 »

xena wrote: post #25
pjmlp wrote: post #23

Earlier quoted context omitted.

TL;DR version. Go package management was quite poor for many real use cases outside Google, so the community railed around many competing proposals, and when it appeared there was a clear winner, the Go team came out with their own solution instead. This after kind of supporting the ongoing community efforts.

Even more fun, Google doesn't even use Go modules internally! They use their own homegrown build system abomination that requires teams of people to maintain.

Pretty sure virtually every big tech company does the same thing, because they have to integrate a bunch of languages, target any arbitrary platform/architecture tuple, and support use cases like code generation and so on. There aren't any good solutions to this problem that don't require teams of people to maintain--the best in class open source solutions seem to be Bazel and Nix and these are beyond my skill to manage even for pretty basic use cases. Further, Google's internal build system predates Go modules by a decade or more; why would they pivot to Go modules?

Re: Sourcehut will blacklist the Go module mirror

#104

Post by Beltalowda »

shortlived wrote: post #93
Beltalowda wrote: post #79

Earlier quoted context omitted.

https://github.com/golang/go/issues/30141#issuecomment-46427... I don't have a full list of all posts at hand (some of which may be removed), but I've seen some other similar stuff as well; it's not an isolated incident. I was reading through the previous thread on this issue (goproxy sending loads of requests) and this one was posted as an example there.

Since GH requires login to see minimized comments, here it is: ddevault on Feb 15, 2019 "EFAIL" is an alarmist puff piece written by morons to slander PGP and inflate their egos. The standards don't need to change to fix the problems it mentions. The proposals help... marginally. The problem is not and was never with OpenPGP, it's with poorly written email clients (e.g. all email clients).

Oh right, I didn't know you had to login to see that.

Re: Sourcehut will blacklist the Go module mirror

#105

Post by tylersmith »

fluidcruft wrote: post #62
psanford wrote: post #43

Earlier quoted context omitted.

Why did sourcehut not take the offer to be added to the refresh exclusion list like the other two small hosting providers did? It seems like that would have resolved this issue last year.

I wondered that too, but then I wondered if that's what sourcehut have actually done. I didn't notice any details about how go module mirror will be blocked. Wouldn't the effect on sourcehut users be identical?

> I didn't notice any details about how go module mirror will be blocked.

It says in the post they'll check the UserAgent for the Go proxy string and return a 429 code.

Re: Sourcehut will blacklist the Go module mirror

#106

Post by throwaway894345 »

jen20 wrote: post #72
xena wrote: post #25

Earlier quoted context omitted.

Even more fun, Google doesn't even use Go modules internally! They use their own homegrown build system abomination that requires teams of people to maintain.

Is this something other than Bazel (or the progenitor thereof)? If so, calling it an abomination compared to the unfinished go build system is rather strong.

Google's internal build system is called Blaze, and (as I understand it) it supports tons of use cases that the Go tool doesn't:

* Distributing build jobs across clusters

* Targeting any arbitrary platform, architecture combination

* Supporting multiple languages and toolchains

* Arbitrary build-time script execution (e.g., code generation)

Re: Sourcehut will blacklist the Go module mirror

#107

Post by icy »

FiloSottile wrote: post #78
throwaway_62022 wrote: post #67

Earlier quoted context omitted.

I think it is problematic that we are using github issues as "support forum" for asking a git host provider to be excluded from the refresh list. This should not have come to that. Whatever happened to "reasonable defaults", so as a random person hosting a single Go module doesn't get DOSed - https://github.com/golang/go/issues/44577#issuecomment-86087... ?

Everyone can make their own assessment of what is a reasonable default and what counts as a DoS (and they are welcome to opt-out of any traffic), but note that 4GB per day is 0.3704 Mbps.

> but note that 4GB per day is 0.3704 Mbps

That's per Go repository. That's a non-trivial amount of egress data and probably adds up to thousands of dollars a month.

Re: Sourcehut will blacklist the Go module mirror

#108

Post by blibble »

pdonis wrote: post #10
galleywest200 wrote: post #6

> It should not be necessary to fetch the same git module up to 2,000 times per day. Holy cow Google! Wouldn't it behoove us to check if any changes occurred before downloading an entire repo?

I don't understand why they have to do a fresh clone every time.

presumably the proxy backend is stateless

Re: Sourcehut will blacklist the Go module mirror

#109

Post by throwaway894345 »

nouveaux wrote: post #69
throwaway894345 wrote: post #63

Earlier quoted context omitted.

I don't have any particular affinity for Google, but they're still a business and they're already developing the Go language (and relevant infrastructure) at their own expense. It's not like the Go team at Google has access to the entire Alphabet war chest like your "biggest, richest, well-staffed companies on the planet" suggests.

Go since inception has always been well funded. It is authored by some of the biggest names in programming and they are on staff at Google. This is not a side hobby. Not sure why you're suggesting that Go is lacking in resources.

Yes, Google staffs its Go team, but the original comment invokes Google's vast wealth as though its entire market cap is available for the development of Go, which is of course absurd. Google probably spends single-digit millions of dollars on Go annually, and it seems they've determined that supporting Drew's use case would require a nontrivial share of that budget which they feel could be spent to greater effect elsewhere.

Go is not only a "side project" at Google, but one of its most trivial side projects.

Post reply on HN