Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bluesky announces data federation for self hosters

bsky.social

Post reply on HN
Print view View original

441–450 of 471 posts

Re: Bluesky announces data federation for self hosters

#441

Post by steveklabnik »

nivenhuh wrote: post #438
jakebsky wrote: post #5

Hey HN, the engineering team at Bluesky is especially excited to get to this point! We're happy to help answer questions and help anyone trying to run their own PDS host. Things should work pretty well for self-hosters right now, but we're standing by to help if there are any problems. Technical details and the installer are in the GitHub repo https://github.com/bluesky-social/pds And we're on Discord available to he…

If I wanted to create a consumer hardware product that packages the PDS host in a user-friendly interface, does the software license permit that? Also, services like Twitter started off with a developer friendly open API, and then it got closed off when the business needed to make money off the platform. What's the difference with Bluesky?

(I don't work at bluesky)

It's MIT/Apache 2.0 licensed, so yes. However, because it's also an open protocol, even if it wasn't, you could write your own under whatever license you want.

> What's the difference with Bluesky?

BlueSky is built off of an open protocol, called AT. https://atproto.com/ BlueSky is a particular app built on the protocol. As such, there's no way to "turn off the API," as BlueSky itself is a participant in the open protocol.

They could like, re-write everything to be a central service, port the user data over to it, and then pull out from the network, but then two things would happen:

1. stuff would break, as it's no longer part of the network.

2. since there is true account portability, users could simply swap to a different PDS and client, and re-route around the damage.

Also given that it's against their entire stated mission and goals, it would be social suicide.

Re: Bluesky announces data federation for self hosters

#442

Post by steveklabnik »

jrm4 wrote: post #440
danpalmer wrote: post #401

Earlier quoted context omitted.

Yeah this is my point. With X users are in a tough position because the platform sucks but they still have a huge amount of sunk cost in it. With Bluesky/AT, that doesn't exist in anywhere near the same way.

I guess the question I'm really trying to get at is "What is the worst that hypothetical Usk can do?" I think I have a handle on it for Mastodon; the answer is "not much" or it's going to be pretty hard. While the deeply decentralized nature of Mastodon makes it harder to find a good "feed," it strongly protects against this; he'd literally have to seek out the individual instances and start trying to control them, o…

Not your original parent, but my take.

> if you have a hand on the control of the relatively centralized recommendation algorithm?

There's nothing centralized about it. Anyone can run an algorithm. If you don't like that algorithm, drop that feed, and find a new one.

Basically, a big difference between BlueSky and Mastodon is that in BlueSky, almost everything is independent parts you can layer together however you like. Mastodon is based around instances, and so ties a lot of these sorts of choices together for everyone on that instance. On BlueSky, "my instance", that is, where my data is hosted, is separate from the posts that I see, because I can choose to follow whatever feeds I like. My instance is also separate from my personal moderation preferences, which I choose independent of my instance.

Re: Bluesky announces data federation for self hosters

#443

Post by PLG88 »

jakebsky wrote: post #427
overstay8930 wrote: post #421

Earlier quoted context omitted.

That’s… not what I said. If you want other people to host services (like to federate), it’s going to be pretty impossible for the vast majority of people to actually do it if you only allow IPv4 connections. You cannot federate behind CGNAT, and most folks can only open ports over IPv6, because they don’t live in America where everyone can get their own public IPv4 address. This is a pretty big miss IMO.

Okay, I get what you're saying. Yes, hosting a PDS server behind NAT is a problem. 1. The expected scenario is that people would run PDS hosts on small VMs (Digital Ocean, Vultr, AWS, etc) where an IPv4 address is the default. 2. People running PDS hosts behind NAT would use a tunnel service of some kind (Cloudflare Tunnel, Tailscale Funnel, Ngrok, etc). But IPv6 is something we do want to support, and it won't be a…

Another option is using zrok - https://zrok.io/. Its open source so you could build it directly into Bluesky and either host the backend yourself or use the zrok free SaaS. zrok also has SDKs so you could embed the capability directly into your binaries without having a separate agent.

Re: Bluesky announces data federation for self hosters

#444

Post by snailmailman »

dudinax wrote: post #384
haberman wrote: post #375

Earlier quoted context omitted.

The element of moderation that you consider essential -- the latitude to apply subjective judgments that rely on knowing the specific identities of the participants -- is precisely the element that I do not trust moderators to perform. That this moderation strategy would prevent the use of all slurs (even reappropriated ones) sounds like a feature to me, not a bug.

"That this moderation strategy would prevent the use of all slurs (even reappropriated ones) sounds like a feature to me, not a bug." You're proposing erring on the side of censorship to avoid some gray areas. While this is a reasonable position, it doesn't satisfy some ideal of neutrality and won't really avoid the gray areas, and so still would require subjective judgement.

And also “what’s a slur” alone is very subjective. For instance on Twitter Elon has decided “cis” and “cisgender” are slurs, but “trans” and “transgender” aren’t. But in gender discussions the terms come up all the time, and they are just terms.

Moderation is full of gray areas, and they are unavoidable.

Re: Bluesky announces data federation for self hosters

#445

Post by stevenicr »

steveklabnik wrote: post #332
stevenicr wrote: post #278

So what data goes and what data is blocked by transitioning away from the main server? I am assuming a person's own posts would come along, but what about conversations with other people, or group discussions or things you have followed? Any other features that would work / things you could see if still connected but no be available if you were federated and banned? I think fbook has a hidden 'export your posts to WP…

> So what data goes and what data is blocked by transitioning away from the main server? Nothing. It is fully transparent to everyone. Your followers won't notice, and you won't notice. Everything Just Works.

So if you port your stuff and blusky bans you, you keep all your posts, all your friends posts, all your groups' info, all your DMS, everything..

.. and the only thing that happens is you can't send data to/through blusky (or receive?) any more and and new data ?

I'd like to test this, any way to get someone from blusky to ban an account to test this?

Re: Bluesky announces data federation for self hosters

#446

Post by miramba »

mehdix wrote: post #402
paulgerhardt wrote: post #243

Earlier quoted context omitted.

Unrelated to engineering but the recent rebrand to a dead butterfly logo[1][2][3] may be off brand for a platform wishing to communicate a more open, social Internet built on first principles and scientific rigor. [1] https://www.emilydamstra.com/please-enough-dead-butterflies/ [2] https://news.ycombinator.com/item?id=14460013 [3] https://bsky.social/about/blog/12-21-2023-butterfly

> If you hadn’t previously noted the difference between a living and a dead butterfly, I’m afraid you will now begin to see dead butterflies EVERYWHERE, as I do. I didn't know this (as most of us I'd guess). It was an interesting read though, thanks.

It's true. I read it some time ago...they're everywhere. You can't unsee it.

Re: Bluesky announces data federation for self hosters

#447

Post by steveklabnik »

stevenicr wrote: post #445
steveklabnik wrote: post #332

Earlier quoted context omitted.

> So what data goes and what data is blocked by transitioning away from the main server? Nothing. It is fully transparent to everyone. Your followers won't notice, and you won't notice. Everything Just Works.

So if you port your stuff and blusky bans you, you keep all your posts, all your friends posts, all your groups' info, all your DMS, everything.. .. and the only thing that happens is you can't send data to/through blusky (or receive?) any more and and new data ? I'd like to test this, any way to get someone from blusky to ban an account to test this?

> you keep all your posts

Yes, but there are some small caveats to how this works exactly, based on the architecture. I'll get to it.

> all your friends posts

Your friends weren't banned, you don't have their posts in the first place.

> all your DMS

AT is 100% public, there are no DMs.

> you can't send data to/through blusky (or receive?) any more and and new data ?

Okay so, we have to talk about the architecture of BlueSky here. There's a few moving parts:

* PDS: personal data server. This holds your posts and such. Before this announcement, BlueSky ran the only PDS. What's happened today is that new PDSes are able to federate with the BlueSky run PDS.

* Relay: this is a server that aggregates data from multiple PDSes. Relays replicate data from the PDSes, and present it as one firehose of information. In some sense, today's announcement is "BlueSky's Relay will now replicate data from PDSes other than theirs."

* App View: this is a service that takes the information from one or more Relays, combines it with whatever other things they care about, and presents the information to an end user. bsky.app is an App View.

I like to think about this like git and GitHub. Your PDS is like a git repository. BlueSky is like GitHub.

So, when BlueSky bans you, aka, when GitHub takes down your git repository on their site, what that means is that they have decided they no longer wish to host your PDS, and they don't let you use their App View. However, if you have a local copy of your PDS, you can simply host it elsewhere, and notify the relay of it, or join a PDS that's already federating with a known relay. Once you do that, you can resume posting. If you don't have a copy of the PDS, I believe you can reconstruct it from the copies that have been federated around the network, but this part is a bit fuzzy to me. Even then, saying "Hey my PDS lives over here now" means your new posts will still get pushed around the network, and the people following you will see them, but you may need to rebuild your following list.

It's also important to note that like, BlueSky's relay could say "never replicate posts from this PDS", but that's why there's multiple relays. If you don't like BlueSky's policies about their relay or app view, use one that implements the policies you care about.

> I'd like to test this, any way to get someone from blusky to ban an account to test this?

This is one step towards realizing this future; for now you have to manually tell BlueSky to federate with your new PDS. The truly federated network is sloooowly coming online. So things will get there.

Re: Bluesky announces data federation for self hosters

#448

Post by overstay8930 »

jakebsky wrote: post #427
overstay8930 wrote: post #421

Earlier quoted context omitted.

That’s… not what I said. If you want other people to host services (like to federate), it’s going to be pretty impossible for the vast majority of people to actually do it if you only allow IPv4 connections. You cannot federate behind CGNAT, and most folks can only open ports over IPv6, because they don’t live in America where everyone can get their own public IPv4 address. This is a pretty big miss IMO.

Okay, I get what you're saying. Yes, hosting a PDS server behind NAT is a problem. 1. The expected scenario is that people would run PDS hosts on small VMs (Digital Ocean, Vultr, AWS, etc) where an IPv4 address is the default. 2. People running PDS hosts behind NAT would use a tunnel service of some kind (Cloudflare Tunnel, Tailscale Funnel, Ngrok, etc). But IPv6 is something we do want to support, and it won't be a…

IPv4 is not the default on any of the platforms you mentioned, it’s a paid add-on you must pay extra for.

It just seems strange that a federated social network platform who wants as wide of an audience as possible, gatekeeps who can participate based on a trivial self imposed limitation.

Hope I don’t sound like a hater because I’m excited for this, but yea it’s a bigger deal than a lot of people think. Even just dealing with bots is 100x easier when you have v6 because you can fine tune rules based on ASNs much easier when you can expect v6 addresses from certain networks and what not. We saw this in India the most where most bot activity would be v4 only, while humans would connect over v6, so we could dynamically tune sensitivity on challenges based on solve rates and what not.

Re: Bluesky announces data federation for self hosters

#449

Post by slily »

zackify wrote: post #417
slily wrote: post #378

Earlier quoted context omitted.

In case you are unaware, it's considered poor etiquette to share links to login-walled social networks, and something like Discord that forces you to join a chatroom with unknown implications (it may trigger a message drawing attention to you, it may require account verification, it may require manual action from a moderator, etc.) just to access some information, all that on top of requiring an account to view anyth…

I only did it because they’re requiring anyone who wants to launch their own self hosted install, to join this exact discord and fill out an application form…. So anyone who wants to run this, has to join that discord anyway. You can find it in the blog post.

I see, I thought that was their support platform for people who can't figure it out from their docs and GitHub. There goes my plan to try out self-hosting.

Re: Bluesky announces data federation for self hosters

#450

Post by mdaniel »

jakebsky wrote: post #169
mariusor wrote: post #91

Earlier quoted context omitted.

Are there any independent projects implementing the AT protocol?

There are a number of independent projects using atproto in various ways. There's an (incomplete) list here: https://docs.bsky.app/showcase And the protocol is documented here: https://atproto.com

I've also had pretty good luck using the GH topics for finding obscure projects https://github.com/topics/atprotocol https://github.com/topics/bluesky https://gitlab.com/explore/projects/topics?search=atprotocol (empty, but I usually check there, too)
Post reply on HN