Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sourcehut will blacklist the Go module mirror

sourcehut.org

Post reply on HN
Print view View original

181–190 of 365 posts

Re: Sourcehut will blacklist the Go module mirror

#181

Post by throwaway894345 »

TeMPOraL wrote: post #157
monocasa wrote: post #130

Earlier quoted context omitted.

They wouldn't write significant parts of their backend in a side project.

Not to mention, it's likely a quite impactful form of marketing / developer relations gain for them. I think so because when I talk to people who start to learn Go, I usually see a transfer of positive feelings and excitement from Go itself to Google as its creator/backer - one of the clearest examples of "halo effect" I've seen first-hand.

Do you really imagine some significant number of Google's search, cloud, etc customers were driven to Google over a competitor because of "good vibes" derived from Go? Google only develops Go because it's a useful internal tool, and I'm pretty sure the marketing team nor the executives spend any meeting minutes discussing Go.

Re: Sourcehut will blacklist the Go module mirror

#182

Post by psanford »

rakoo wrote: post #152
psanford wrote: post #43

Earlier quoted context omitted.

Why did sourcehut not take the offer to be added to the refresh exclusion list like the other two small hosting providers did? It seems like that would have resolved this issue last year.

Why should you opt-in to a No-DDOS list ? Why is it not the default ?

You are basically arguing that sr.ht is taking a "principled stand" against google. If that is what they are doing they should just say that and not pretend like there were no other options.

I'm ok with saying "google should do better!" But the compromise solution from the Go team seems reasonable to solve the immediate issue in a way that doesn't harm end users. The author should at least address why they have chosen the more extreme solution.

Re: Sourcehut will blacklist the Go module mirror

#183

Post by TeMPOraL »

jdiff wrote: post #135
politician wrote: post #88

Earlier quoted context omitted.

sourcehut's recommendations seem absolutely reasonable: (1) obey the robots.txt, (2) do bare clones instead of full clones, (3) maintain a local cache. I could build a system that did this in a week without any support from Google using existing open source tech. It's mind boggling that Google isn't honoring robots.txt, is requesting full clones, and isn't maintaining a local cache.

Despite the issue, I'm not convinced that Go doesn't do shallow fetches vs deep clones. Other issues (like Gentoo's issue with the proxy, I don't have a link handy sadly) point to fetches being done normally, not clones.

It's not about what Go allows, it's about what Google's proxy does on its own schedule. If there was a knob sr.ht could use to change this, it would've come up in the two years since this issue was raised with the Go team.

Re: Sourcehut will blacklist the Go module mirror

#184

Post by stonogo »

comex wrote: post #27

According to some comments in the linked GitHub issue, including [1] from last May, Drew could have simply asked to be excluded from automatic refresh traffic from the mirror. If I understand correctly, that would still leave traffic from the mirror when it’s acting as a direct proxy for someone’s request, but that is traffic that would be going to sr.ht regardless. For some reason he did not do this and instead chos…

Seems like a weird default policy, until you remember Google's wifi mapping opt-out process: https://support.google.com/maps/answer/1725632?hl=en#zippy=%...

"All this would go away if other people would just do what they're told" is a pretty dystopian policy, but it seems to be a popular choice in Google.

Re: Sourcehut will blacklist the Go module mirror

#185

Post by xigoi »

psanford wrote: post #43
ComputerGuru wrote: post #18

From the GitHub issue, by a Googler: > For boring technical reasons, it would be a fair bit of extra work for us to read robots.txt […] This is coming from one of the biggest, richest, most well-staffed companies on the planet. It’s too much work for them to read a robots.txt file like the rest of the world (and plenty of one-man teams) do before hammering a server with terabytes of requests. If this is too much for…

Why did sourcehut not take the offer to be added to the refresh exclusion list like the other two small hosting providers did? It seems like that would have resolved this issue last year.

IIRC, because that would only help the official SourceHut instance, not other instances.

Re: Sourcehut will blacklist the Go module mirror

#186

Post by throwaway894345 »

Analemma_ wrote: post #129
throwaway894345 wrote: post #63

Earlier quoted context omitted.

I don't have any particular affinity for Google, but they're still a business and they're already developing the Go language (and relevant infrastructure) at their own expense. It's not like the Go team at Google has access to the entire Alphabet war chest like your "biggest, richest, well-staffed companies on the planet" suggests.

Surely Google of all places has the most tested, battle-hardened robots.txt library in existence, and they have a company-wide public monorepo to boot. There's no excuse for this.

I'm pretty sure parsing robots.txt isn't the challenge. The Go team asserts that there are technical difficulties to this traffic optimization, and I don't have any reason to disbelieve them (they're clearly not dumb people, and I certainly trust them more than Internet randoms when it comes to maintaining the Go module proxy). It's a bummer for Drew, but he isn't Google's top priority right now (it seems wild to me that you think there is "no excuse" for Google not to prioritize niche use cases like Drew's--how do you imagine large organizations choose what to work on?).

Re: Sourcehut will blacklist the Go module mirror

#187

Post by pjmlp »

throwaway894345 wrote: post #80
pjmlp wrote: post #21

Earlier quoted context omitted.

Many issues with Go design steem from "how Google does it", this is just yet another one.

I doubt this is true in any objective sense. "issues" above is almost certainly subjective, and I don't think even in this particular case it's so much "how Google does it" as much as "Google isn't prioritizing limited Go-team resources to this issue right now". In general, I've been very happy with Go's design for the last decade with relatively few, relatively minor exceptions. I specifically have enjoyed that the…

I guess following Turbo Pascal and Delphi tooling might feel novel to those that never used it.

Re: Sourcehut will blacklist the Go module mirror

#188

Post by throwaway894345 »

yamtaddle wrote: post #180
throwaway894345 wrote: post #173

Earlier quoted context omitted.

Of course, Google doesn't materially benefit from optimizing the module proxy for Drew's use case, and I doubt your startups would have made traffic optimization its top priority either under similar circumstances (which is to say "no ROI from traffic optimization").

Drew's use case ?!

"scenario"? Pick your synonym.

Re: Sourcehut will blacklist the Go module mirror

#189

Post by throwaway894345 »

pjmlp wrote: post #187
throwaway894345 wrote: post #80

Earlier quoted context omitted.

I doubt this is true in any objective sense. "issues" above is almost certainly subjective, and I don't think even in this particular case it's so much "how Google does it" as much as "Google isn't prioritizing limited Go-team resources to this issue right now". In general, I've been very happy with Go's design for the last decade with relatively few, relatively minor exceptions. I specifically have enjoyed that the…

I guess following Turbo Pascal and Delphi tooling might feel novel to those that never used it.

The pedants among us might prefer: "novel among mainstream programming languages of the 21st century which predated Go".

Re: Sourcehut will blacklist the Go module mirror

#190

Post by Zamicol »

throwaway894345 wrote: post #86
Zamicol wrote: post #57

Earlier quoted context omitted.

I've blocked many of the people that called for Peter's ban on Twitter. I don't want to be on their radar or a target of some sort of witch hunt. I consider myself a nice person and not inflammatory/offensive, but I'm a belt and suspenders type of person. It's more important that I can submit an issue on the tracker than interacting "socially" with people who have a higher chance of ostracizing targeted individuals.…

Why were people calling for Peter's ban? I'm having a hard time imagining behavior that is so toxic that it merits a ban when "publicly advocating for banning someone from the community" is apparently fair play.

For Peter: https://old.reddit.com/r/golang/comments/ppluux/peter_bourgo...

And Drew talks "harshly". Says things like "shitty usage of the GPG command line tool by applications" where, "shitty" is considered by some crossing a line https://github.com/golang/go/issues/30141#issuecomment-46427...

Both are abrasive, but in my mind, this is a cultural issue.

I grew up in a judgmental, holier-than-thou religion that I rejected at an early age and then was ex-communicated from on my 18th birthday. I've lived this pattern of judgement and ostracisation. It's dangerous, closed minded, and wrong.

One of the reasons I work in software is because I'm a "strange" person. I say things people don't understand, my value system is radically different from my peers and "normies". I am a very different type of person. Software was a safe place for weird people, including the productive, but sometimes abrasive. It's a shame to make software not a place for a wide range of diverse people. In my difference, I don't want my cultural differences to be targeted by witch hunts, those looking for blood to prop up their own "moral superiority".

And I personally have no cultural issue with people who say things like, "shitty usage of the GPG command line tool by applications." They're welcome in my circles as I sincerely consider myself less judgmental than those calling for a ban, but perhaps in kindness I can urge people who say stuff like that to, over time, to be more kind and sensitive. Kindness is a skill that needs development. Not everyone is in the same place. It's certainly something I'm working on every day. And if not, that's okay too, not all of us are built with the same social skills, and that's okay. To _not_ be self-righteous requires long-suffering. Peter and Drew were worth more effort.

Post reply on HN