Live data from Hacker News

Nostr and ATProto (2024)

shreyanjain.net

51–60 of 74 posts

Re: Nostr and ATProto (2024)

#51
post #7
post #4

Earlier quoted context omitted.

I'm not sure I would necessarily draw that conclusion. If the author intentionally deactivated their Bluesky account, does the fact that he can successfully do that on Bluesky lead to the conclusion that it's less resilient?

I think you've nailed a problem with all of these, they would make "deleting your stuff" HARDER. What's stopping the rogue node from saving all your stuff forever? I think "trying to make a thing that can work through rogue or stupid nodes" is just prohibitively harder than "work on making nodes more reliable" (which I absolutely grant is extremely hard.)

The difference is a single point of failure (centralization) vs multiple copies of the same posts all over the internet (decentralization).

Didn't even took a year to see where the texts are still readable today.

Re: Nostr and ATProto (2024)

#52

Earlier quoted context omitted.

>What do you mean by this? RSS is a way to aggregate data from many sites into one place. AT lets you do the same, but with bells and whistles (the data is signed and typed, and there's a realtime stream in addition to pulling on demand). If you're forced to describe AT via existing technologies, AT is basically like RSS for typed JSON in Git over HTTP or WebSockets that scales to millions of users. It is completely…

> The reason you don't see more of these is because an isolated experience is... well, isolated. I don't understand why you become isolated once you've built your own app, it it because the bluesky firehouse has to decide to index posts I make on my server? I guess I'm asking how does an application decide which sources to index from, just anyone advertising that they are serving that lexicon? Why then would I become…

Hmm, no, that’s not what I meant. Let me try to break it down a bit.

There’s really two main kinds of nodes in the system. Hosting servers and app servers. They’re completely unrelated and completely decoupled. It’s like Dropbox vs apps that put data in your Dropbox.

A hosting server stores your personal data. This is similar to having a Git repository with data from all social apps. Or like a Dropbox folder. That’s usually called a “PDS” — a personal data server. Running one is extremely cheap since it’s only your data. It is also optional (eg Bluesky provides AT hosting for free). But this is not an app — it’s literally like Git hosting. Just the data (for all apps).

Then you have app backends. Those are just normal servers. They’re what you’d typically think of web applications. The Bluesky app is one of them. An application server listens to events from all known hosting servers and updates its local database with whatever it’s interested in from the stream. For example, the Bluesky application server updates its local database to put all “post created”, “like created” etc events from all hosting servers into its database that it can query.

So as an app author you have a lot of freedom for what to build:

- You can build a new app that only listens to record of your app’s type. So naturally it would only index your app’s users’ content. Which is presumably not much.

- You can take an app server for existing app (if it’s open source) and run it yourself. But then of course if this app has a million of users, you need to decide which records you want. Do you want to index them all (like the original app)? Do you want to index a subset? Which subset? It could be historical (eg two last weeks of post, one last week of likes etc). Or it could be by proximity (only profiles, posts and likes within one follow from you). Or something else. You decide what to store.

- You can also build something hybrid — an app that remixes data from multiple apps. And you can fetch data from hosting servers without storing it (but this doesn’t give you aggregation) or fetch aggregated data from community indexes (if the aggregation you need already exists and is provided by someone else).

Hope this makes sense.

(As a performance optimization, instead of aggregating from millions of repositories individually, you’d listen to a stream that combines them. That’s called “relay”. Relays are mostly dumb websocket retransmitters and don’t have any app-specific logic. Bluesky runs one, Blacksky runs their own, and it would generally cost $30/mo to run one today. Any hosting server can ask any relay to crawl it. Any relay may also choose to crawl a new hosting server if it encounters links to content on that server. Relays are common infra and you shouldn’t expect there to be a lot of them. App servers choose which relay to listen to, if at all.)

---

Now answering your specific questions:

>I don't understand why you become isolated once you've built your own app

If you've built an app that looks like Bluesky, but only you and your friends' posts/likes show up, is that much better than just using Bluesky? My point is that usually this isn't a differentiator and feels kind of pointless. You might as well just curate your Following feed on Bluesky. So people don't do that often.

>it it because the bluesky firehouse has to decide to index posts I make on my server?

This seems like a misconception; moving your data (to your own hosting) is a completely separate thing from creating an app. See the distinction above. You can move your hosting to a different hosting server, but this wouldn't affect your experience in the Bluesky app at all. The Bluesky application server would simply start ingesting your posts from your new server instead once it gets notified about your account move.

>I guess I'm asking how does an application decide which sources to index from, just anyone advertising that they are serving that lexicon?

Typically an application just listens to a relay (like the one hosted on Bluesky) which already retransmits events from all known repositories. If you operate your own repository, you can send a "request crawl" command to Bluesky's relay, and it will index you. This is kind of similar to a website getting picked up by Google search. Links may also do it but a "request crawl" is the explicit way. See https://pdsls.dev/jetstream?instance=wss%3A%2F%2Fjetstream1.... for a live feed of the relay operated by Bluesky (it's not specific to the Bluesky app).

>Why then would I become isolated by virtue of hosting only data I want to host/indexing only feeds I care to index?

Hosting data !== indexing, again these are separate things.

Hosting your own data doesn't make you isolated — it is pretty much indistinguishable in the apps. You don't see where someone's data is hosted since in the app it all appears seamlessly aggregated.

Creating an app that only shows 0.000001% of the network's content when there's already an app that shows 100% of the same content is what I call isolating. I'm just not sure what it accomplishes since the network is still shared. So this isn't very compelling to most app builders. What's compelling is usually building completely new experiences. Although some people do experiment with more "limited" Bluesky clones.

Re: Nostr and ATProto (2024)

#54
post #46
post #16

Earlier quoted context omitted.

This so much. ATProto just seems so complicated in comparison.

Both of these systems are rebellions against the structure of secure-scuttlebot, but took different paths as they rebelled. Beyond using different cryptography, the biggest difference between the "ATProto System" and the "Nostr System" is that Jay Graber wanted to account for deletes and the re-organization of the message structure of an entire feed. In early ATProto, aka smor-serve, https://github.com/arcalinea/smor…

Correct me if I'm wrong but I think another important aspect of AT is Lexicons, i.e. there's an officially suggested way to do schemas, and application authors are encouraged to create and distribute schemas for their apps. Data is grouped in the repo by its schema too.

Re: Nostr and ATProto (2024)

#55
post #10

from an an average developer perspective, nostr is interesting because it's "just" a digitally signed json data structure sent over a websocket. reading the spec [1] for creating a simple nostr client (aka "nip-1"), my average developer brains thinks: i could do that. i don't get that same feeling when reading atproto or activitypub docs. ultimately, there's a reason why all these protocols get complicated at scale,…

For me, as a veteran of social networks since Fidonet, the question that actually matters is the social construction: who's using it, for what purposes and topics, and - crucially - how is the work of moderation (including spam and abuse prevention) handled.

Technology doesn't make or break these things, the users do.

Re: Nostr and ATProto (2024)

#56
post #30

Earlier quoted context omitted.

nostr can get plenty complicated, too, but nostr successfully tricked me into thinking it was simple enough to get started.

The more nerds that get sniped by a simple-seeming protocol, the more likely it is to catch on. Hitting a 100 page spec doc full of XML and links to other specs is a big de-motivator to start hacking on the protocol.

NGL that's kind of been the appeal of atproto. Once you get into the weeds you start learning about all the many many many moving parts but at its face 99% of it is:

Hey here's some schema files we call lexicons. Every single interaction with the network is a JSON RPC call to the same domain (whatever your PDS is) with the lexicon's path appended to it.

The fact that it's trivial to hack on atproto via the devtools console or a curl prompt makes things so much more fun to play with.

Re: Nostr and ATProto (2024)

#57

Earlier quoted context omitted.

> It’s much closer spiritually to RSS and plain old web What do you mean by this? ATProto requires a giant indexing database that has access to every post in the network. Mastodon is more like a feed reader—you only get notified about the posts you care about. How is needing a giant database that knows about every RSS feed in the world closer to the plain old web?

>What do you mean by this? RSS is a way to aggregate data from many sites into one place. AT lets you do the same, but with bells and whistles (the data is signed and typed, and there's a realtime stream in addition to pulling on demand). If you're forced to describe AT via existing technologies, AT is basically like RSS for typed JSON in Git over HTTP or WebSockets that scales to millions of users. It is completely…

This would make sense if there weren't so many features—like Blocks, DMs, followers-only posts, etc—that were reliant on the AppView enforcing a single global view of the world. I agree that I do think the AT model does have good properties but right now too much of it is reliant on this single shared global app view

But thanks for the link to Konbini! That looks really exciting and promising and I would love to start using it if I can run it completely decoupled from Bluesky infrastructure.

Re: Nostr and ATProto (2024)

#58
post #46

Earlier quoted context omitted.

Both of these systems are rebellions against the structure of secure-scuttlebot, but took different paths as they rebelled. Beyond using different cryptography, the biggest difference between the "ATProto System" and the "Nostr System" is that Jay Graber wanted to account for deletes and the re-organization of the message structure of an entire feed. In early ATProto, aka smor-serve, https://github.com/arcalinea/smor…

Correct me if I'm wrong but I think another important aspect of AT is Lexicons, i.e. there's an officially suggested way to do schemas, and application authors are encouraged to create and distribute schemas for their apps. Data is grouped in the repo by its schema too.

True. Scuttlebot had msg types too.

Re: Nostr and ATProto (2024)

#59

Earlier quoted context omitted.

>What do you mean by this? RSS is a way to aggregate data from many sites into one place. AT lets you do the same, but with bells and whistles (the data is signed and typed, and there's a realtime stream in addition to pulling on demand). If you're forced to describe AT via existing technologies, AT is basically like RSS for typed JSON in Git over HTTP or WebSockets that scales to millions of users. It is completely…

This would make sense if there weren't so many features—like Blocks, DMs, followers-only posts, etc—that were reliant on the AppView enforcing a single global view of the world. I agree that I do think the AT model does have good properties but right now too much of it is reliant on this single shared global app view But thanks for the link to Konbini! That looks really exciting and promising and I would love to star…

I think it's only reliant on them to the extent that you want to build copies of the same exact experience, which I personally don't find very interesting. I think a much more compelling story is not, say, "a clone of Bluesky with a Bluesky DM folder", but, say, "a Spaces-like product that closely integrates with Bluesky (for posting) and is also listed as a stream on on Streamplace".

I agree that some information seems important to know, like blocks. (Although in different apps it's reasonable to expect blocks to be app-specific.) Blocks are public on Bluesky though, for this exact reason. DMs are a disconnected service but the eventual idea is some kind of E2E (https://www.germnetwork.com/ is also building something now). Follower-only things could work through some variation of private state mechanism (see https://pfrazee.leaflet.pub/3lzhmtognls2q, https://pfrazee.leaflet.pub/3lzhui2zbxk2b).

>I would love to start using it if I can run it completely decoupled from Bluesky infrastructure.

You could use Blacksky's relay as the input source (https://atproto.africa/), or run your own relay. The only piece you'd then depend on is PLC registry (since it resolves PLC identity). Bluesky is in the process of separating it into a separate entity in Switzerland, but if that's a hard goal, I guess you could forbid `did:plc` identities in your app (vast majority of users) and only ingest data about `did:web` ones? Or do you feel OK about PLC resolution?

Re: Nostr and ATProto (2024)

#60
post #22

Earlier quoted context omitted.

thanks. also, fwiw, i'm also a very a happy AT user (@hugs.bsky.social) besides also being a happy nostr user. i appreciate bsky's focus on user ux and community building and look forward to seeing more sharing of ideas between nostr and AT. edit to add: to nerd-snipe my brain into wanting to make stuff with AT (or any future protocol) is to focus on a quick-start or tutorial showing the absolute minimal client to se…

Would it be ok to use a library or is the requirement to keep it to raw primitives like curl?

everything we need to authenticate the msg should exist in the msg. sig => timestamp, hash
Post reply on HN