Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sourcehut will blacklist the Go module mirror

sourcehut.org

Post reply on HN
Print view View original

351–360 of 365 posts

Re: Sourcehut will blacklist the Go module mirror

#351

Post by politician »

captainmuon wrote: post #328
politician wrote: post #170

Earlier quoted context omitted.

Are you serious? Google Cloud Storage is a service that Google sells to folks using its cloud. If they can't use it for their own project, that would be shocking, no?

They are probably already using something like GCS to store the data at the cache nodes. I was not talking about how the nodes store data, but about a central cache . Purely architecture wise, it doesn't make sense to introduce a central storage that just mirrors Sourcehut (and all other Git repositories). Sourcehut is already that central storage. You would just create a detour. It's also not an easy problem. If the…

Go's Proxy service is already a detour for reasons of trust mentioned in the article. They are in a position to return non-authentic modules if necessary (e.g. court order). That settles all architecture arguments about sources-of-record vs sources-of-truth. The proxy service is a source of truth.

If Google is going to blindly hammer something because they must have their Google scale shared nothing architecture pointed at some unfortunate website, then they should deploy a gitea instance that mirrors sr.ht to Google Cloud Storage, and hammer that.

It's unethical to foist the egress costs onto sr.ht when the solution is so so simple.

Some intern could get this going on GCP in their 20% time and then some manager could hook the billing up to the internal account.

Re: Sourcehut will blacklist the Go module mirror

#352

Post by jdiff »

simon_o wrote: post #348
jdiff wrote: post #340

Earlier quoted context omitted.

No problems were ever mentioned, serious or otherwise. Elevated traffic isn't automatically a problem. Drew's played it up quite a lot elsewhere, but the Go team can only be reasonably expected to follow the one issue filed, not Drew's entire online presence.

You mean the issue they banned him from? ;-)

Yes. He had plenty of opportunity to state problems if they existed. Relaying harm caused would have likely accelerated things, and if harm was being done he would have taken up the still-open offer to solve this problem in the interim while the real solution is pushed out instead of writing misrepresentative and openly salty blog posts for years. Even with him being banned, the Go team is still tracking this issue, still brings it up internally, and has pushed a feature that would fix this ahead by an entire release.

So yes. The issue they banned him from. Because reality's more complicated than flippant one liners.

Re: Sourcehut will blacklist the Go module mirror

#353

Post by psanford »

dpkirchner wrote: post #227
psanford wrote: post #66

Earlier quoted context omitted.

Why? If some open source maintainer goes off the rails and deletes all their packages, why should that break my builds? I still have a valid license to the code, I don't really care that a maintainer rage quit 4 dependencies down from my application. I certainly don't want to have to scramble to deal with that.

This is an argument for maintaining a local cache of necessary build dependencies, not to rely on a third party.

Before the proxy service that is what people did. Now we don't need to because the proxy service handles that automatically.

You can run your own proxy service if you want. There's a large benefit to the go community as a whole for there to be a shared default proxy service.

Re: Sourcehut will blacklist the Go module mirror

#354

Post by MaxBarraclough »

necron99 wrote: post #163

Whether Drew's right or wrong, this is disappointing. At the very least, it's going to make recommending Sourcehut as an alternative to Github harder to justify. This is really unfortunate, especially as sr.ht isn't free (any longer). As far as I know, it's the only remaining managed source hosting/CI option for Mercurial users. What a depressing way to start out 2023.

> especially as sr.ht isn't free (any longer)

This isn't a bad thing, it was always meant to be financially supported by its users. The alternatives are ads or for Drew and co to run it at a loss indefinitely.

Re: Sourcehut will blacklist the Go module mirror

#355

Post by samwyse »

kramerger wrote: post #325
stonogo wrote: post #184

Earlier quoted context omitted.

Seems like a weird default policy, until you remember Google's wifi mapping opt-out process: https://support.google.com/maps/answer/1725632?hl=en#zippy=%... "All this would go away if other people would just do what they're told" is a pretty dystopian policy, but it seems to be a popular choice in Google.

That right there is why i stopped using Google. Anyway, things get a bit kafka-esque when you realise that there is another company doing this WiFi thing and to opt out from that one you need a different SSID suffix. Since you can't have both, you end up with at least one company data mining you. Why GDPR has not put a stop to this is baffling

I’ve got to ask, which other Wi-Fi mapping provider requires a SSID suffix to opt out? Is it one of the big boys or is it someone like wigle.net or openwifimap.net?

Re: Sourcehut will blacklist the Go module mirror

#356

Post by [deleted] »

rsc wrote: post #245
eadmund wrote: post #229

Earlier quoted context omitted.

This reminds me a bit of a disfunctional relationship: clearly Sourcehut wants Google to stop DDoS their servers; clearly Google don’t actually want to DDoS Sourcehut, but Sourcehut also doesn’t want to ask Google to stop, and Google also want to be asked to stop. And so nothing gets done. The question is who will swallow their pride first: Sourcehut or Google.

This isn't true. Sourcehut reported a bug, and since the bug is somewhat involved to fix entirely, we asked what the impact of the bug is to them and offered to make a custom change for the site in the interim. The impact matters: the appropriate response is different for "I saw this in my logs and it looks weird but it's not bothering me" versus "this is causing serious problems for my site". We have been getting mi…

[deleted]

Re: Sourcehut will blacklist the Go module mirror

#357

Post by davidbanham »

The proposed resolutions from Google have been along the lines of changes to the proxy application code. The resistance has been that this is hard because of the way responsibility is divided between the thin layer that provides the proxy web service and the ‘go’ command itself, which actually fetches the packages.

Would it be simple to solve this with an additional layer of the same proxy? Currently, end users request a package from proxy.golang.org as per the default value of the GO_MOD_PROXY env var. Google runs many of these to handle the traffic, let’s say 1000. They all maintain a mirror of all packages that have been recently requested (note that I expect there’s more nuance here around shared lists of required packages, etc, but the structure should hold true)

The result is that every one of the 1000 proxy instances requests the source data from git.sr.ht every day.

Google could set GO_MOD_PROXY on the existing instances to internalproxy.golang.org. They could then run 100, or maybe 10 of these internal instances. This would drop the traffic to hit.sr.ht by one or two orders of magnitude.

I suspect it would require minimal if any change to application code. This might be accomplished entirely within the remit of a sysadmin (SRE?).

Any holes in my reasoning?

Re: Sourcehut will blacklist the Go module mirror

#358

Post by cowl »

maximilianburke wrote: post #223
rsc wrote: post #209

The Go team has been making progress toward a complete fix to this problem. Go 1.19 added "go mod download -reuse", which lets it be told about the previous download result including the Git commit refs involved and their hashes. If the relevant parts of the server's advertised ref list is unchanged since the previous download, then the refresh will do nothing more than the ref list, which is very cheap. The proxy.go…

I realize in the real world most modules are probably hosted by large providers that can absorb the bandwidth, like Github, but it seems incredibly discourteous to not prioritize the hammering of small providers, especially two years on when the response is still "maybe later this year". I think Drew is right in that he shouldn't take a personalized Sourcehut-only exception because this doesn't address the core issue…

The Exception "exclusion solution" definitely it's not personalized and Sourcehut-only but available to anyone that requests it and in the issue track you can see several people that are using this exclusion already. True that an opt-out (what this solution boils down to) is not ideal but it's way better than using your users quality of service to try strong-arming your side. And anyway from the go team it's been made clear that they are working on improving the situation of the refreshes and this opt-out is just the temporary solution until they fix the real issue.

Re: Sourcehut will blacklist the Go module mirror

#359

Post by bunbun69 »

josephcsible wrote: post #221
cratermoon wrote: post #168

Earlier quoted context omitted.

> Google has quite a reputation for wrongfully banning people from things. Extraordinary claims require extraordinary evidence.

Here's some evidence: https://news.ycombinator.com/item?id=3803568 https://news.ycombinator.com/item?id=15066518 https://news.ycombinator.com/item?id=19124324 https://news.ycombinator.com/item?id=20826618 https://news.ycombinator.com/item?id=20841586 https://news.ycombinator.com/item?id=21247759 https://news.ycombinator.com/item?id=24791357 https://news.ycombinator.com/item?id=24965432 https://news.ycombinator.com/it…

link to other articles is NOT evidence

Re: Sourcehut will blacklist the Go module mirror

#360

Post by adityasaky »

lanstin wrote: post #350
adityasaky wrote: post #345

Earlier quoted context omitted.

Even then, you want to revoke those credentials rather than try to wipe it from history, no?

That is what I think but security people want both.

Strange, I hadn't come across that before. Not sure what they're trying to achieve, deny they ever had a leak?
Post reply on HN