Live data from Hacker News

We need a federation of forges

blog.tangled.org

311–320 of 424 posts

Re: We need a federation of forges

#311

Earlier quoted context omitted.

Interoperable identity providers would indeed be useful. Beyond that, maybe resilience when a project's host disappears, changes its policies, or gets blocked by a government?

How does tangled solve that? Repository contents are still hosted by the forges themselves.

I was addressing the question of a use-case for a "federation of forges". Not any specific design or implementation.

Re: We need a federation of forges

#312

Earlier quoted context omitted.

No the problem is that github has to stem exponential usage increase and prepare 30x of their capacity, that's not symptom, that's problem.

It's both and, it's a symptom of exponential usage and a problem with infrastructure. The question you aren't asking is "Why is it a problem with GitHub's infrastructure?" the answer to that lies somewhere in between: Microsoft + Azure + Copilot. Now tell me which of those have anything to do with GitHub as we know it?

Why is it a problem with Githubs infrastructure!? Bcs any website on the planet will struggle when they have to fulfill 30x capacity within 1-2y, no matter which tech stack they're built on, including federated networks. I'm not sure why you're throwikg Copilot in there, you don't like it?

Github as we know it is gone, forever, it will never come back, except for niche hobby clones with .001% capacity that nobody will use. Agents are re-defining what software engineering means, they already have, right now,and are continuikg to do so, it's just that hackernews is lagging 6 months behind for some reason.

Re: We need a federation of forges

#313

Earlier quoted context omitted.

How does tangled solve that? Repository contents are still hosted by the forges themselves.

I was addressing the question of a use-case for a "federation of forges". Not any specific design or implementation.

That sounds more like you want better decentralization, like IPFS or BitTorrent, not necessarily federation between different forge instances. I'm not familiar with any existing federated system that would be resilient to government censorship. Certainly Mastodon and Bluesky aren't.

Re: We need a federation of forges

#314

Earlier quoted context omitted.

ATproto federates in a very different way than Mastodon. There is no concept of "instances" on ATproto. Your account is hosted on a PDS and you sign into the app with your PDS sign-in and records go to your PDS, but everything on the app is from what's called an "AppView" which provides a centralized view of all data in all PDSes so it feels just like you're using a regular centralized app. But there can be multiple…

> There is no concept of "instances" on ATproto. Regardless of name and precise technical details, there are central service components that can ban you. If a proper ecosystem of those ever springs up then the equivalent of fediblock (ie guilt by association) oriented at individual accounts or PDS is the next logical step. At present (last I checked) there's only (approximately) one primary provider plus blacksky mak…

Details matter. Technical details matter.

Most bans today are at the "appview" level: the big indexed view of all the data, that combines the firehoses ("relays") marks accounts as banned & doesn't show their stuff. But the relay and the PDS still work.

Agreed that there aren't many public appviews for Bluesky posting right now, really just the two. Tangled itself though is an appview, of a different sort: one not for posting but for git issues/pr's/rtc. This appview isnt gated on Bluesky or Blacksky's permission. And folks could pretty comfortably host Tangled aplview themselves, subscribing their Tangled instance to any of the dozens of firehose/relay instances, getting all pre-filtered Tangled activity. And that really is quite decentralized a model that is imminently doable. Regarding the technical properly, the concern here about banning feels premature & naive: it assumes Tangled depends on these appviews at all, and it doesn't.

I will note that Phil's constellation project just tackles the key reverse indexing that comprises much of the appview work: taking all the firehose records, and connecting all the threads and likes together. Constellation runs ok as a public service on an rpi. There's a lot of challenges to making new appviews, but it is astounding and comforting seeing the core indexing for a sizable multi-media social network running on an rpi. What seems like a dire situation may actually be opportunity, if folks actually tried.

Whatever problems we want to foresee dooming us, whatever slopes we want to hypothesize sliding down, what we have here sounds way way better than anything else available to me today. Personally I'm much more bright blue sky sunnier about the prospects rather than your dark raincloud doom fall scare-away. The risk imo is immensely more weighted in not trying more than trying.

Re: We need a federation of forges

#315
post #242

Earlier quoted context omitted.

The web is already structured like this. You can poll a URL for updates. You can host your own data. Anyone can build an app that aggregates from everyone's data.

> Anyone can build an app that aggregates from everyone's data. Unless they gut or disable their API, and then ban scrapers, although I understand why after seeing bots crush sites.

Who are “they” here? Hosting providers? Hosting is app-agnostic and you can run it yourself.

Re: We need a federation of forges

#316
post #287

Earlier quoted context omitted.

There’s no such thing as “running a domain” or “atproto provider” in atproto. You’re approaching it with a Mastodon/AP mindset and it doesn’t match that. In atproto, there’s two axes. One is hosting. Bluesky offers hosting but some people host on their own (it’s just a Docker container with sqlite), some on Cloudflare, some on community-hosted nodes like https://npmx.dev and https://selfhosted.social . From app persp…

So does Bluesky app have control over what data it aggregates and can decide (without checking with a user) not to aggregate data from a host? I am trying to understand what are the implications for a user, and a bad scenario where one would disagree with an action of the app. And if the answer is "yes" then at least when someone "makes their own app" can they easily use "Bluesky hosts list" + add special extra hosts…

Yes to both.

An app can choose to ignore/ban some users (or even entire hosting servers if they’re specifically created for network abuse). This is similar to how any web app may choose to ignore POST requests from spammers.

And yes, someone can decide to aggregate data themselves and provide an alternative app over same data with different moderation policies. In fact that’s already the case (Blacksky runs their own application server that mostly piggybacks on Bluesky moderation decisions but overrides some of them. There are also clients that ignore moderation altogether and show you the raw data from hosting.)

Re: We need a federation of forges

#317
post #272

Earlier quoted context omitted.

if I fork a repository to my forge, I expect my forge to have an independent copy of the repo

How does that fix "when one node goes offline you may not have any upstream to pull from"? You'd still have your own local copy—just like git—but you wouldn't be able to access any sense of "upstream"

By hosting a knot.

You may ask, well, that's like hosting forgejo or any other git server, where is the federation?

Tangled uses a protocol. So knots would adhere to that protocol allowing to pull from any upstream.

That's my understanding of federation. not saying tangled will go as far as figuring out discovery across their cloud hosted knots and self hosted infra. But that can be done, and claiming to be able to pulling from any repo with a single identy would imply just that.

Re: We need a federation of forges

#318

Earlier quoted context omitted.

> There is no concept of "instances" on ATproto. Regardless of name and precise technical details, there are central service components that can ban you. If a proper ecosystem of those ever springs up then the equivalent of fediblock (ie guilt by association) oriented at individual accounts or PDS is the next logical step. At present (last I checked) there's only (approximately) one primary provider plus blacksky mak…

Details matter. Technical details matter. Most bans today are at the "appview" level: the big indexed view of all the data, that combines the firehoses ("relays") marks accounts as banned & doesn't show their stuff. But the relay and the PDS still work. Agreed that there aren't many public appviews for Bluesky posting right now, really just the two. Tangled itself though is an appview, of a different sort: one not fo…

I'm not saying people shouldn't build things, merely disputing the idea that the logical equivalent of instances (and the ills they lead to) don't exist on ATProto. Guilt by association currently exists as a fairly common practice on all the community protocols I've made use of - including federated, p2p, and whatever else - so I see no reason to expect it won't also infect ATProto.

The key difference is that ATProto currently only has a small handful of instances, ie it remains largely centralized. Certainly it's a blessing that the operators appear to have generally acted with benevolence to date but that's not really relevant to the point I'm making.

Re: We need a federation of forges

#319

Earlier quoted context omitted.

ATProto simply ignores the need for decentralizing incentives on a human/community level. What we get is a sort of a "top-down" federation rather than a grass-roots one. Whoever invests in the infra ends up running a domain. I mean, practically no one is aware of any other ATPROTO provider other than Bluesky whereas the issue with AP is merely the lack of better implementations, so mastodon.social got the most attent…

There’s no such thing as “running a domain” or “atproto provider” in atproto. You’re approaching it with a Mastodon/AP mindset and it doesn’t match that. In atproto, there’s two axes. One is hosting. Bluesky offers hosting but some people host on their own (it’s just a Docker container with sqlite), some on Cloudflare, some on community-hosted nodes like https://npmx.dev and https://selfhosted.social . From app persp…

You're interpreting my post with the assumption that I don't know what I'm talking about. You don't need to explain the protocol to me.

Domain here referred to the area of influence or control, like what the provider of a relay effectively has. The fact that other groups can run any element of the infra themselves doesn't change the fact that the drift towards centralization is much greater with ATP than with AP.

ATP has its own uses (quick aggregation) but it doesn't even attempt to solve fundamental issues of current ecosystem of social networking,. AP, on the other hand, offers the foundation for further development in the right direction.

Re: We need a federation of forges

#320

Earlier quoted context omitted.

There’s no such thing as “running a domain” or “atproto provider” in atproto. You’re approaching it with a Mastodon/AP mindset and it doesn’t match that. In atproto, there’s two axes. One is hosting. Bluesky offers hosting but some people host on their own (it’s just a Docker container with sqlite), some on Cloudflare, some on community-hosted nodes like https://npmx.dev and https://selfhosted.social . From app persp…

How does a new server discover other servers?

A new hosting provider can preemptively request known relays to crawl it. Or relays (or apps) can lazily discover it when the user hosted there tries to log in for the first time, or their data is linked to by a known user. It’s similar to the relationships between websites and search engines.

Hosting providers don’t need to discover other hosting providers. Data only flows between hosting and apps; not between hosting and hosting or apps and apps.

Post reply on HN