Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Personal data storage is an idea whose time has come

blog.muni.town

Post reply on HN
Print view View original

221–230 of 287 posts

Re: Personal data storage is an idea whose time has come

#221

Post by zahlman »

xandrius wrote: post #61
InMice wrote: post #26

Among the first page and 2nd page (top 60) there is always atleast 1 post about how we're gonnna "take back the web" or make it back into some form of our 90s millenial nostalgia memories, self hosting, federated this or that, etc etc. Meanwhile - Nothing changes, everything generally gets worse and younger generations come into the world with no memories of the 90s internet or the world before mobile devices or surv…

If even the people who experience a different time gives up because "nothing changes" then it's truly over. We need to do what we preach: sure, things are worse in certain things but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever. Also I think people forget to realise that the type of people who were online in the 90s are still online, many…

> but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever.

Sometimes HN makes me feel like I'm the literal last remaining person on the planet who just... uses a desktop computer, and stores data on SSDs and HDDs, all physically connected to the machine, and never worries about how to access this data from another device because there are no other devices from which it should be accessed.

I mean, okay, fine, I do things like publishing to GitHub. But I still have a local copy, and I'm in control.

Re: Personal data storage is an idea whose time has come

#222

Post by zahlman »

fooqux wrote: post #166
jasode wrote: post #154

Earlier quoted context omitted.

>but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever. Understand your enthusiasm but to relate the discussion back to Tim Berners-Lee idea for SOLID data storage protocol... Running self-hosted things like email, NextCloud, Plex, sandstorm.io, etc -- are not relevant to the gp's "nothing changes" complaint. Without dissecting the SOLID proto…

> It should be obvious that companies are not incentivized to write transactional data to users' storage pods which explains why the SOLID protocol doesn't seem to gain much traction for the last 9 years. Not simply "not incentived" but actually decentivized. It's not just that companies lose the ability to have a better algorithm to recommend products, but the data itself is worth a fortune. Google, Facebook, etc ar…

> Online advertising is the keystone keeping this pile of shit upright and I can't wait until that bubble finally pops. That is when the narrative will change.

This can't happen until there's another viable revenue stream. Which requires smoothing out everything about microtransactions, creating a culture where people now expect to pay, and building trust that it won't get stuffed with ads anyway.

Re: Personal data storage is an idea whose time has come

#223

Post by Gud »

layer8 wrote: post #152
Gud wrote: post #84

Earlier quoted context omitted.

~the internet~ got easier to consume but self hosting in many ways became harder because of how hostile the internet has become.

Not really that much harder, if it’s only for personal use.

Not really. But sure didn’t get easier. Entropy and all that.

Re: Personal data storage is an idea whose time has come

#224

Post by brendoncarroll »

ianopolous wrote: post #197
brendoncarroll wrote: post #114

I work on a FOSS project in this space, Blobcache. https://github.com/blobcache/blobcache Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make wh…

We should talk. This very similar to how apps use E2EE data in Peergos. Maybe we can join forces. https://peergos.org/posts/a-better-web

I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio).

Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here.

Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics. IPFS provides distributed, available-but-inconsistent, cryptographically signed cells. IPFS chooses availability, and Blobcache chooses consistency. A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node. An IPFS name exists as a distributed entity on the network.

Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way. I blame this required application-specific care for the lack of adoption of CRDTs.

There's a long tail of other differences too. IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes. Blobcache is more like a web browser; it creates transient connections in immediate response to user actions.

That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive. It amounts to a tag for the algorithm used to create a hash, and then the hash output. I'd rather not have that indirection. All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume. In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms.

I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.

Re: Personal data storage is an idea whose time has come

#225

Post by senordevnyc »

jauntywundrkind wrote: post #120
gcanyon wrote: post #45

Both of these proposals (as far as I've read them, YMMV) fail the evolutionary test. At the scale we're talking about, ideas must proceed as evolution does: not with a far-away goal in mind, but with incremental changes, each of which individually must be an improvement over the status quo. We are at (near) a significant local maximum, and (again, as far as I've read, which is not all of it for sure) the people pitch…

99.9% of BlueSky users use only Bluesky services. But BlueSky has a Personal Data Service for each. That means: Those users have credible exit to take their data off BlueSky's hosting to someplace else (and as of a week or two ago to move back to BlueSky if they want). Those users can put whatever kind of data they want in their PDS. They can host their git data via https://tangled.org . They can store their music li…

99.9% of BlueSky users use only Bluesky services.

I highly, highly doubt this, even in the narrowest sense of how many BlueSky users still actively post on X.

Re: Personal data storage is an idea whose time has come

#226

Post by bawolff »

JumpCrisscross wrote: post #204
bawolff wrote: post #203

This is never going to happen. The incentives do not make sense. Any utopian future that requires a party to put in a lot of effort to change something in a way that would be a net negative for them, is just not going to happen. People do not spend money to change the world in a way that would be worse for them but better for other people.

> The incentives do not make sense Commercial incentives, no. If this preference exists, it would need to be pursued civically.

I don't think the average citizen cares enough or even understands the benefits

But lets say you get them on board and pass some law. Unless its a huge market like the EU or USA, probably what immediately happens is everyone pulls out of that market. Not out of malice but because they suddenly have to rewrite their app and that's probably quite expensive.

Re: Personal data storage is an idea whose time has come

#227

Post by ianopolous »

brendoncarroll wrote: post #224
ianopolous wrote: post #197

Earlier quoted context omitted.

We should talk. This very similar to how apps use E2EE data in Peergos. Maybe we can join forces. https://peergos.org/posts/a-better-web

I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio). Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here. Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A…

Thanks! I'll send you an email.

The good news is Peergos also has serializable transactional modifications. This comes from us storing signed roots in a db on your home server (not ipns). We also have our own minimal ipfs implementation that uses 1000x fewer resources than kubo, aka go-ipfs.

Re: Personal data storage is an idea whose time has come

#228

Post by walterbell »

Theodores wrote: post #140
9dev wrote: post #23

Earlier quoted context omitted.

It wasn’t the companies but the users that found it easier. There’s a reason why everyone’s on Facebook, instagram, and gmail instead of running their own hosts—because it’s vastly easier for the majority of people to do so, and because everyone else is there. We have not solved decentralisation in an accessible and useful way yet, and the incentives won’t change until we do. If ever.

It is very easy to sign up to Facebook, Instagram, Gmail and everything else. No manual is needed for doom-scrolling and on-boarding is instant. Personally I would prefer to have my own full-on LAMP stack at home, with Postfix for email and everything accessible via my own subdomain. So, why can't I have that? During my standard install of my favourite distro, I would only need to enter my name, subdomain and email p…

> why can't I buy a glorified router box that does all of this?

Step 0 is to secure that box, as routers are obvious targets, even before they have self-hosted data. There are some products based on RPi, NAS and router form factors.

> suspicious absence of products in this space

Earlier efforts:

  Apache Wave (federated)
  Chandler
  Diaspora   
  FreedomBox
  Microsoft Groove (p2p)
  Urbit.org
  Sandstorm.io
Active OSS projects include Proxmox (https://community-scripts.github.io/ProxmoxVE/), Paperless-NGX (docs), Immich (photos), NextCloud and others, https://github.com/awesome-selfhosted/awesome-selfhosted

Re: Personal data storage is an idea whose time has come

#229

Post by pydry »

dist-epoch wrote: post #8
pydry wrote: post #3

The problem isnt technical feasibility it is market incentives. Most companies have no incentive to let you hold your data when they can just hold it for you. If they do this they can mine it for data to improve their product as well as sell or otherwise indirectly profit from it. And, it's easier. Also, while the market for privacy focused products isnt nothing, the number of people willing to pay a lot extra to com…

You got the market incentives wrong. Most people have no incentive of owning their data. Otherwise the companies which don't give you that would die out because people wouldn't use them if they cared. Same fallacy as believing smartphones are giant and with non-user swappable batteries because somehow smartphone making companies are forcing this on the market, instead of the real reason which is that it's what consum…

I get what you're saying.

People want vendor lock in...otherwise they wouldnt pay for it.

People want bait and switch sales tactics...otherwise they wouldnt work.

People are perfectly fine with high rents...if they didnt, they would not pay them.

People want their smartphones to be deliberately slowed down when they get old...otherwise theyd vote against it with their wallet.

Re: Personal data storage is an idea whose time has come

#230

Post by anon7000 »

Gud wrote: post #84
xandrius wrote: post #61

Earlier quoted context omitted.

If even the people who experience a different time gives up because "nothing changes" then it's truly over. We need to do what we preach: sure, things are worse in certain things but for sure setting up a local network with top-level open source self-hosted alternatives is the easiest it has ever been ever. Also I think people forget to realise that the type of people who were online in the 90s are still online, many…

~the internet~ got easier to consume but self hosting in many ways became harder because of how hostile the internet has become.

Self hosting is so much easier than before, though. Tools like docker and Tailscale make operating servers and using VPNs pretty painless.

Routing to your home address could be hard, but it’s also pretty easy and cheap to set up a reverse proxy from a server you can rent. Routing through a public CDN is also easy and cheap and solves a lot of problems like DDoS.

Post reply on HN