Sourcehut will blacklist the Go module mirror • Hacker News Board

Re: Sourcehut will blacklist the Go module mirror

#161

Post by captainmuon » Mon, Jan 09, 2023, 5:24 PM UTC

politician wrote: post #88

psanford wrote: post #71

Earlier quoted context omitted.

No. The Go team offered to add sourcehat to a list that would stop background refreshes. It would still allow fetches initiated by end users. The change sourcehat is making is to break end users unless they set some environment variables. I've not seen any explanation about why the solution offered by the Go team was unacceptable. Its weird that that is completely left out of the blog post here.

sourcehut's recommendations seem absolutely reasonable: (1) obey the robots.txt, (2) do bare clones instead of full clones, (3) maintain a local cache. I could build a system that did this in a week without any support from Google using existing open source tech. It's mind boggling that Google isn't honoring robots.txt, is requesting full clones, and isn't maintaining a local cache.

What does a local cache even mean at Googles scale though? Some of the cache nodes are likely closer to Sourcehuts servers than to Google HQ. I guess local would mean here that Google pays for the traffic. But then it is not a technical problem, but a "financial" one.

If you disregard the question who pays for a moment and only look at what makes sense for the "bits", the stateless architecture seems not so bad. Just a pity that in reality somebody else has to foot the bill.

Re: Sourcehut will blacklist the Go module mirror

#162

Post by alpaca128 » Mon, Jan 09, 2023, 5:24 PM UTC

Fnoord wrote: post #40

ddevault wrote: post #24

Earlier quoted context omitted.

I would share their side as well, but I never heard it. This is a violation of their own code of conduct, which requires them to notify the affected person, explain why, and offer the opportunity to mediate the situation. This is not the first time I was banned from the Go community without notice or explanation, and the first time turned out to be frivolous -- the ban was overturned months later with an admission th…

DoSing SourceHut? And the perpetrator is known ? Why don't you try to sue them?

Ask Max Schrems how easy it is to sue tech giants.

Re: Sourcehut will blacklist the Go module mirror

#163

Post by necron99 » Mon, Jan 09, 2023, 5:25 PM UTC

Whether Drew's right or wrong, this is disappointing. At the very least, it's going to make recommending Sourcehut as an alternative to Github harder to justify.

This is really unfortunate, especially as sr.ht isn't free (any longer). As far as I know, it's the only remaining managed source hosting/CI option for Mercurial users.

What a depressing way to start out 2023.

Re: Sourcehut will blacklist the Go module mirror

#164

Post by [deleted] » Mon, Jan 09, 2023, 5:27 PM UTC

alphazard wrote: post #39

pjmlp wrote: post #23

Earlier quoted context omitted.

TL;DR version. Go package management was quite poor for many real use cases outside Google, so the community railed around many competing proposals, and when it appeared there was a clear winner, the Go team came out with their own solution instead. This after kind of supporting the ongoing community efforts.

This is actually one of the reasons I trust the Go project is in good hands. The Go leadership has smart engineers, with good taste, willing to say no. The Go module implementation and version selection mechanism are better than what the community had converged on. Go has demonstrated that its governance model is capable of making better decisions than a pure democracy. This is not the case for most open source proje…

[deleted]

Re: Sourcehut will blacklist the Go module mirror

#165

Post by adql » Mon, Jan 09, 2023, 5:28 PM UTC

pjmlp wrote: post #21

leetrout wrote: post #8

I feel like Drew has been in a pissing match with the Go team for a while so this outcome doesnt surprise me. I love Go but not Google's stewardship of it. The tracking proxy, Russ' takeover / squash of the package management work, the weird silence / stonewalling on other community issues... Drew has a valid complaint. I hate to hear he was banned from the issue tracker but that sounds about right. As a sibling said…

Many issues with Go design steem from "how Google does it", this is just yet another one.

For me it seems that it's more that the few people at helm didn't really bother to learn any modern language or update themselves on theory so it feels like slighty nicer 90's language, not something developed recently.

And their weird entrenchment in "doing things the way plan9 did"

Hell, how you design a statically typed language and go "nah, we don't need sum types! That's too complex!"...

Re: Sourcehut will blacklist the Go module mirror

#166

Post by rakoo » Mon, Jan 09, 2023, 5:29 PM UTC

FiloSottile wrote: post #78

throwaway_62022 wrote: post #67

Earlier quoted context omitted.

I think it is problematic that we are using github issues as "support forum" for asking a git host provider to be excluded from the refresh list. This should not have come to that. Whatever happened to "reasonable defaults", so as a random person hosting a single Go module doesn't get DOSed - https://github.com/golang/go/issues/44577#issuecomment-86087... ?

Everyone can make their own assessment of what is a reasonable default and what counts as a DoS (and they are welcome to opt-out of any traffic), but note that 4GB per day is 0.3704 Mbps.

This way of thinking is the exact opposite of what we need to do in the IT for reducing our impact on the environment. 4 GB is an enormous amount of data. It's enough to listen to 21k hours of music, or having a small dump of all french wikipedia (no pics, main paragraph). It's enough to completely travel off-line in Germany. It's enough to watch 3 to 4 movies in a good resolution. And that's per day.

We must absolutely reduce our resource usage.

Re: Sourcehut will blacklist the Go module mirror

#167

Post by TUSF » Mon, Jan 09, 2023, 5:30 PM UTC

Spivak wrote: post #119

Tomte wrote: post #77

Earlier quoted context omitted.

I think Google should DDoS noone. Not everyone until they opt out.

But it’s not Google DDoSing them, it’s every user downloading packages. Without the proxy it would just be millions of users hammering their servers. Edit: Uh okay, if it's not user traffic then why wasn't the "don't background refresh" not an option?

The issue isn't from user-initiated requests. It's from the hundreds of automatic refreshes that the proxy then performs over the course of the day and beyond. One person who was running a git server that hosts a Go repo only they use was hit with 4gb of traffic over the course of a few hours.

Thats a DDoS.

Re: Sourcehut will blacklist the Go module mirror

#168

Post by cratermoon » Mon, Jan 09, 2023, 5:30 PM UTC

josephcsible wrote: post #137

cratermoon wrote: post #61

> The situation remained so for over a year. In that time, I was banned from the Go issue tracker without explanation, and was unable to continue discussing the problem with Google Something tells me there's another side to this story that we aren't hearing.

Why? Google has quite a reputation for wrongfully banning people from things. This doesn't seem unusual for them at all.

> Google has quite a reputation for wrongfully banning people from things.

Extraordinary claims require extraordinary evidence.

Re: Sourcehut will blacklist the Go module mirror

#169

Post by necron99 » Mon, Jan 09, 2023, 5:31 PM UTC

zozbot234 wrote: post #38

Sr.ht should set up their own graceful rate-limiting anyway, or they're going to have problems with badly coded CI setups as the service becomes more popular.

Wouldn't this just make the failures more mysterious and harder to track down for users? Failing sometimes for no reason that's obvious or apparent to end users is worse than always failing, IMO.

E.g.: source-based packages on distributions where users may not be Go programmers, or even non-programmers, will compile and install Go software where some nested library dependency is on sr.ht. These packages will now fail and, sadly, this is going to cause widespread disruption. I think it'd be worse if those failures only happened occasionally, and not reliably repeatably.

Re: Sourcehut will blacklist the Go module mirror

#170

Post by politician » Mon, Jan 09, 2023, 5:33 PM UTC

captainmuon wrote: post #161

politician wrote: post #88

Earlier quoted context omitted.

sourcehut's recommendations seem absolutely reasonable: (1) obey the robots.txt, (2) do bare clones instead of full clones, (3) maintain a local cache. I could build a system that did this in a week without any support from Google using existing open source tech. It's mind boggling that Google isn't honoring robots.txt, is requesting full clones, and isn't maintaining a local cache.

What does a local cache even mean at Googles scale though? Some of the cache nodes are likely closer to Sourcehuts servers than to Google HQ. I guess local would mean here that Google pays for the traffic. But then it is not a technical problem, but a "financial" one. If you disregard the question who pays for a moment and only look at what makes sense for the "bits", the stateless architecture seems not so bad. Just…

Are you serious? Google Cloud Storage is a service that Google sells to folks using its cloud. If they can't use it for their own project, that would be shocking, no?