Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Social Filesystem

overreacted.io

Post reply on HN
Print view View original

201–210 of 243 posts

Re: A Social Filesystem

#201

Post by extraduder_ire »

galactus wrote: post #196
voidUpdate wrote: post #189

How does this system determine the amount of likes a post has? Since there is no back reference on a post to people who have liked it, don't you have to iterate over every single person, iterate over their likes, to see if one of them is the post you are viewing, and add all them up?

In the ATProto architecture, this function is handled by the AppView, which monitors the full network and produces the corresponding aggregates.

There's also software called Constellation[0] from microcosm that exclusively collects "backlinks", which are items which link to a certain thing. e.g. follow records for an account, reposts/replies/likes on a post. If you click on the Backlinks tab when examining something on pdsls.dev you can see a list of them.

It can be self hosted and is much lighter than running a full appview. The author has a public endpoint for it that's keeping up with the whole network running on a single raspberry pi on his home network.

0: https://www.microcosm.blue/

Re: A Social Filesystem

#202

Post by skeledrew »

mike_hearn wrote: post #164
skeledrew wrote: post #56

I've been thinking of this for some time, conceptually, but perhaps from a more fundamental angle. I think the idea of "files" is pretty dated and can be thrown out. Treat everything as data blobs (inspired by PerKeep[0]) addressed by their hashes and many of the issues described in the article just aren't even a thing. If it really makes sense, or for compatibility sake, relevant blobs can be exposed through a files…

It's bold to say users don't want apps when the expressed preferences are so strongly for apps that "App Store" is a well known brand. The word app is short for application, which in this context is just a synonym for capability. People have tried breaking apps apart into more fine grained capabilities before and it never worked (OpenDoc and OLE being two well known examples). Brand names matter, they have value to p…

> the expressed preference

Marketing and culture can do wonders to one's preference.

> synonym for capability Yes and no. An "app" is usually a set of capabilities, not a single capability. Many of the core *nix utilities for example are specifically designed to have a single capability each, and these are then composed for complex requirements. But for the most part today's apps contain multiple, usually overlapping capabilities. For example Microsoft Word offers not only text editing and formatting, but also has a dictionary+thesaurus, spelling+grammar check, mail merge and lots more. Of course, you can abstract this all away as "word processor capability", but there are other word processing apps out there with varying capabilities, and so things become problematic fast especially in terms of interoperability.

> OpenDoc and OLE These "never worked" due to people/policy, not due to tech ability. OpenDoc essentially stagnated at Apple due to conflicts and lack of clarity, and then was outright killed by Steve Jobs[0]. OLE(tm) hasn't failed per se, but it's proprietary to Microsoft[1] and thus doesn't have much adoption outside Windows.

> Brand names matter All they really do is put a label on a bundle - or bundles - of capabilities to create handy associations. YouTube is associated primarily with accessing videos that can be made and uploaded by anyone, secondarily allows commenting on and liking said videos, etc. Again, marketing is a significant factor.

> a built-in feature You seem to be missing the forest for a tree here. That's a single example, of infinite permutations. Also, even given that example you're still missing parts of the flow.

[0] https://en.wikipedia.org/wiki/OpenDoc [1] https://en.wikipedia.org/wiki/Object_Linking_and_Embedding

Re: A Social Filesystem

#203

Post by mike_hearn »

danabramov wrote: post #176
mike_hearn wrote: post #165

Earlier quoted context omitted.

There was never any such incentive. MS Office formats were undocumented for years because MS had no incentive to document them. Merely using files did not help at all. Actually the vast majority of all file formats have never been open. Think about all the custom file formats used by video games, for instance.

I actually talk about this in the article. Merely using files is how we got out of the dependency on MS Office. Multiple efforts reverse-engineered them, including Google Docs. Yes they were undocumented, but as long as stuff has to be stored on the disk under user’s control, the overall dynamics are very different from you-can’t-see-the-files systems.

Is it really so different in the service context? You could also reverse engineer the HTTP endpoints and formats used by Word Online to export data from the service. It doesn't feel all that different, except perhaps that the online service can try to detect your custom tooling and block you, whereas with static data that isn't possible.

But other than that admittedly real extra problem, the bulk of the work would still be understanding the undocumented protocols and feature semantics, then matching the feature set.

Re: A Social Filesystem

#204

Post by mike_hearn »

danabramov wrote: post #192
mike_hearn wrote: post #170

Earlier quoted context omitted.

That's true if you define the problem as "does my parser crash" and not whether the app is perceived as working correctly. If some platform adds support for video posts, then the next thing that happens is people start making posts that are only video. Meaning that in every other client, users see what appears to be an entirely empty post. Which will be considered a bug. This is the core argument of Moxie's seminal e…

Well, this doesn't prevent the "flagship" app from shipping things and doesn't slow it down. So it's at least not slowing down development which is the argument the parent post was making. I've actually observed the exact opposite thing. Since Bluesky is open source, it's often visible when developers start working on a feature. And they often check in lexicon changes early on. As a result, there's been a few cases w…

Moxie's argument is that even if something has a flagship app, this doesn't help because if you use a new feature and then your friend complains that they can't see what you posted, the experience is just that the flagship app itself is broken. People don't experience this as, oh well, my friend should just have picked a better client. They experience it as, that's annoying, the video feature doesn't work reliably on Y but it always does on X.

An extreme example of this is WhatsApp and emojis. WhatsApp doesn't use the operating system's text rendering APIs to draw emojis, instead Meta license the Apple emoji font and draw the characters themselves. That's because if you do emoji the standards based, open way you get these problems:

• People use visual puns that only make sense if emojis look a certain way, without realizing they might look very different to other people.

• People use new emoji, without realizing that old operating systems can't draw them.

The experience in both cases is that it's simply broken and buggy. Version skew can kill platforms, which is why the most successful platforms today restrict which clients can connect and forcibly expire them on a regular basis.

BTW I don't think it's worth generalizing from Bluesky. Bluesky is an X clone whose unique competitive advantage is censoring conservatives even more aggressively than Twitter itself once did. It has no technical edge so they can develop the site as open source without it being a concern to them - they don't care if they leak what they're doing because technical innovation isn't a part of their brand to begin with - and the AT protocol clearly doesn't matter to more than a tiny fraction of its users. The point you're making in the essay is general, but ends up feeling overfit to Bluesky.

Re: A Social Filesystem

#205

Post by mike_hearn »

skeledrew wrote: post #202
mike_hearn wrote: post #164

Earlier quoted context omitted.

It's bold to say users don't want apps when the expressed preferences are so strongly for apps that "App Store" is a well known brand. The word app is short for application, which in this context is just a synonym for capability. People have tried breaking apps apart into more fine grained capabilities before and it never worked (OpenDoc and OLE being two well known examples). Brand names matter, they have value to p…

> the expressed preference Marketing and culture can do wonders to one's preference. > synonym for capability Yes and no. An "app" is usually a set of capabilities, not a single capability. Many of the core *nix utilities for example are specifically designed to have a single capability each, and these are then composed for complex requirements. But for the most part today's apps contain multiple, usually overlapping…

FWIW the Look Up feature, whilst built in to macOS, can also be implemented using "services" which are a way to plug in to other apps and system features like text rendering and highlighting. I can also select text in your message, right click, services -> "Add to music as spoken track". That's a plugin offered by an app. It's only one extra click.

My perspective on why OpenDoc/OLE failed are different to yours. The model in which apps host targeted plugins which don't affect data formats proved very successful, the model in which an app is nothing but plugins wasn't. And if you want to talk about marketing and culture, well, OpenDoc and OLE both benefited from corporate marketing. Using OLE was part of Microsoft's "Designed for Windows 95" programme, which came with a badge and marketing dollars attached. This concept was heavily pushed at the time. OLE had more success than OpenDoc but only temporarily and only because it was a less pure implementation of the concept, closer to fancy plugins than truly decomposing apps into components.

I think the reason they didn't take off was a mix of technological problems and user demand problems. It wasn't people/policy related. Quite the opposite: both technologies lived on long after the real problems had become apparent.

Programming for both OpenDoc and OLE was extraordinarily complicated. The developer experience was atrocious. Making your app OLE embeddable required mastering not only COM and Win32 programming but dozens of other poorly documented and flaky APIs, like for compound documents. Code generation was rampant, everything was memory unsafe. I never did OpenDoc apps but I heard it was a very similar experience.

Then there was the user experience: if you embedded an object from an app and sent the resulting document to someone else who didn't have the app, or had an old version of the app, then the document would just have a hole in it. Useless. OLE tried to do a kind of weird mind-meld between totally different app GUIs, meaning that depending on where you clicked in a document the hosting app could look totally different. There were dozens of problems like this.

In the end it was all rendered irrelevant by the web. On the web, if you share a document with someone it always appeared for them correctly (or if it doesn't, it's a minor and quickly fixable bug). Reliable sharing mattered more than anything else and so all the non-web platforms started dying out.

Re: A Social Filesystem

#206

Post by phanimahesh »

sfdlkj3jk342a wrote: post #156
ivell wrote: post #154

Earlier quoted context omitted.

You may want to take a look at India's ONDC ( https://github.com/ONDC-Official ).

Sounds interesting, but I can't find any links to an actual live B2C storefront..

https://ondc.bajajfinservmarkets.in/ https://www.mystore.in/en/

Listing of apps at https://www.ondc.org/ondc-buyer-apps/

Re: A Social Filesystem

#207

Post by bccdee »

isodev wrote: post #186
danabramov wrote: post #179

Earlier quoted context omitted.

I mean, I can literally already self-host my personal data if I want to. And there are also already forks of Bluesky (not just the client, but the server and the database ) that can participate without fragmenting the network. It is not a perfect system but it's so far from where you are when you just rely on a closed app.

That's great, but also Mastodon is just there and has been for quite some time. I see no added value in Bluesky/ATProto beyond the layer of that "social as a service" which looks like a walled garden / app store of sorts in the making. I may be wrong, of course...

I never managed to get into mastodon because mastodon isn't a single space. It's a bunch of separate, linked spaces. Which server you're on matters a lot, and I never found one that really had the vibe I was looking for.

Bluesky, though, is one big pool. There's no seam between servers. The experience is much nicer.

Also, bluesky caught on among my peers in a way that mastodon never did, which was always going to be the deciding factor. I imagine not having to "choose a server" was a big part of that.

Re: A Social Filesystem

#208

Post by danabramov »

mike_hearn wrote: post #203
danabramov wrote: post #176

Earlier quoted context omitted.

I actually talk about this in the article. Merely using files is how we got out of the dependency on MS Office. Multiple efforts reverse-engineered them, including Google Docs. Yes they were undocumented, but as long as stuff has to be stored on the disk under user’s control, the overall dynamics are very different from you-can’t-see-the-files systems.

Is it really so different in the service context? You could also reverse engineer the HTTP endpoints and formats used by Word Online to export data from the service. It doesn't feel all that different, except perhaps that the online service can try to detect your custom tooling and block you, whereas with static data that isn't possible. But other than that admittedly real extra problem, the bulk of the work would st…

Hmm, Word Online isn't what the article is about. I'm not sure if you've read the article, but it is about social apps — like Tumblr, Reddit, HN itself, etc.

I'm using file formats as a metaphor to explain how apps built on AT protocol work (lexicons are like "social file formats"), and what this way of building enables (interoperability between social apps by default).

Re: A Social Filesystem

#209

Post by vitorfblima »

skybrian wrote: post #52
danabramov wrote: post #47

Earlier quoted context omitted.

>The effect of the internet (everything open to everyone) was to create smaller pockets around a specific idea or culture. Just like you have group chats with different people, thats what IG and Snap are. Segmentation all the way down. I actually agree with that. See from the post: >For some use cases, like cross-site syndication, a standard-ish jointly governed lexicon makes sense. For other cases, you really want t…

There's also a risk of adversarial cross-site syndication: your stuff can and probably will show up on websites you don't control. That's just how it works and I accept the risk. People concerned about that probably shouldn't publish on Bluesky. Private chat makes more sense for a lot of things.

Back in the old twitter days you could have screenshots of your tweets being displayed by a number of different 'dubious' accounts on instagram, completely out of your control.

That's just how things work, I guess.

Re: A Social Filesystem

#210

Post by danabramov »

extraduder_ire wrote: post #199

You mention having a "self" record in the app.bsky.actor.profile lexicon to store profile information, is there any reason to have more records of that type in your repository? I've seen a few people make other ones when examining their accounts with pdsls, but they seem to be there for "just because I can" reasons.

Different apps can have different notions of a profile so you'd probably have one per app.
Post reply on HN