Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keep Your Stuff, for Life

perkeep.org

Post reply on HN
Print view View original

91–100 of 119 posts

Re: Keep Your Stuff, for Life

#91

Post by asdkhadsj »

lukecameron wrote: post #30
natural219 wrote: post #20

Ah, nostalgia :). It seems like with the recent wave of news about social media migrations (reddit, facebook, twitter, twitch, tiktok), people are hopefully starting to get more and more warmed up to the idea of protocolization of their social data. But most of the projects doing it are still just too immature. Solid, Perkeep, Blockstack, etc. just seem like vaporware. Seems like the only serious projects in use are…

I want that protocolization too, although I don't hold out much hope that the monopolies in place can be broken, outside of fairly radical regulation. To add to your list, there is also Secure Scuttlebutt [1] which has had a decent userbase over the past few years, and Planetary [2] which is a funded iOS client for it. I think in general they all suffer from the chicken-egg problem and will need some reason for enoug…

Man, I love the idea of Scuttlebutt but I hate the developer UX. I'm writing some apps that I wanted to put on SSB but have all but given up on the idea. Something about SSB, as a dev, leaves me with a lot of questions and no idea where to even get answers from.

So I'll write my app outside of SSB, hopefully in a way that's mostly compatible, and possibly with future integration.

I may also toy with an SSB-like protocol myself, as the fundamentals of SSB is a work of art imo. I really enjoy what Gossip brings to the table, and how SSB focuses on human->human relationship to bring P2P to the table.

Re: Keep Your Stuff, for Life

#92

Post by [deleted] »

theelous3 wrote: post #86
sukilot wrote: post #84

Earlier quoted context omitted.

It's not for muggles, and that's OK. > Perkeep is under active development. If you're a programmer or fairly technical, you can probably get it up and running and get some utility out of it.

Not for muggles is a silly cop out. It's very nondescriptive. That "set of protocols and file formats" etc. could do literally anything. Is it for a nas? Is it distributed? Is it some backup yoke for a cloud service? Do you run it on your own hardware? Is it all of the above? What does your data look like? Where and how are you searching through it? Is that part not for muggles either or is there a frontend? And so o…

[deleted]

Re: Keep Your Stuff, for Life

#93

Post by lukeschlather »

atq2119 wrote: post #43
BiteCode_dev wrote: post #42

Earlier quoted context omitted.

Why do you think cmd line tools are popular ? Because they are easier to do than a GUI, not because all geeks think the terminal is the only xp that matters.

That seems... offtopic? You can have command line tools that are easy to use, with documentation focused on user-friendliness, and you can have command line tools where figuring out what they even do in the first place takes a lot of effort. And the same is true for GUIs, for what it's worth.

That takes time. Or funding. I have often thought "it would be great if I could suck all my emails and Facebook posts into a local search index." It sounds like this is that. If I wrote it I would have exactly enough time to make sure that all my emails and Facebook posts are in a local index and then I would have no more time for a personal project. Making a nice UI on top of it would easily increase the amount of time fivefold.

Re: Keep Your Stuff, for Life

#94

Post by ebg13 »

taneq wrote: post #76
CGamesPlay wrote: post #9

Wow, it was really hard to find out what this software does. I finally found a demo at the end of an hour-long talk from 2 years ago: https://youtu.be/PlAU_da_U4s?t=2687 So it seems to have a bunch of scripts to import data into its database from a variety of sources (including cloud services), and provides a search interface to navigate that historical data. And it has a lot of under-the-hood stuff about replication…

Does the first sentence not cover that fairly well? > Perkeep (née Camlistore) is a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era. Data may be files or objects, tweets or 5TB videos, and you can access it via a phone, browser or FUSE filesystem. I checked archive.org and that text has been there for a couple of months at least.…

> Does the first sentence not cover that fairly well?

Not for me. Almost all of what you copied there seems like intrinsically meaningless fluff. "a set of formats"? "a set of protocols"? "software"! Of course it's software! "for modeling"? "in the post-PC era"?

I would have preferred GP's ~"imports data/files from cloud services into a local database and provides a search interface for it". Because THAT tells me what it does, not any of the other words you posted.

Someone made a really well done cartoon mascot for it, but the copy isn't _helpful_.

Re: Keep Your Stuff, for Life

#95

Post by pnutjam »

dannyw wrote: post #23
paranoidrobot wrote: post #17

Earlier quoted context omitted.

Even if magnetically they don't have 'bit rot', they use bearings where the lubrication can dry up and wear out when they're not turning for long periods of time. You need to keep them spinning on a regular basis, and replace them as they begin to fail.

HDDs are also prone to silent bitrot, where it will simply return incorrect bytes for a sector, even without any smart errors. (Optical disks also bitrot; but so does HDDs). This is usually a precursor to SMART errors happening in the near future, but unfortunately, it can still result in corrupted replication and corrupted backups; as your backups would be backing up the rotten (corrupt) data. I've witnessed this ha…

I have a home "NAS" (opensuse server) where my main /data partition is xfs, but it mounts a btrfs backup partition, rsyncs, and takes a snapshot.

I should really get around to converting the main drive to btrfs, but this works well.

Re: Keep Your Stuff, for Life

#96

Post by andai »

solarkraft wrote: post #41

Perkeep is super interesting. I have been looking at it as a media database (photos, movies and such, powerful tag-based search, all downloaded on demand), but some things regarding that really hurt: - The data store messes with my files (yes, there's a FUSE mount, but eh, having to adopt a special data store always makes me feel weird since it usually comes with performance and compatiiblity implications, there are…

> ... my time is tempered by 2.5 and 0.5 year old kids. ... I'll pick up my involvement again as kids get a bit older.

> We have no plans to abandon it.

Re: Keep Your Stuff, for Life

#97

Post by andai »

PurpleRamen wrote: post #71
darkwater wrote: post #61

Last released version 0.10 is from May 2018. Is this project still alive? Last commit is from March 11 2020 so maybe they are just "slow" at releasing.

Last commit was just a simple bugfix in documentation. The last juicy commits were in december 2019 and https://github.com/perkeep/perkeep/graphs/code-frequency makes the impression that activity has slowed down in 2019 significantly. Maybe they reached the point of good enough, but the high number of open issues and mergerequests is still problematic.

They reached the point of having kids :)

https://news.ycombinator.com/item?id=22161812

Re: Keep Your Stuff, for Life

#98

Post by shock »

eexan wrote: post #48

I tried perkeep a while ago. While the ideas are cool, the implementation is meh: I added a single 2.7G ubuntu iso - it took 5 minutes to ingest it (on a tmpfs!), and turned it into 45k(!) little chunks, wtf is up with that? At this rate indexing my multiple terabytes of data is going to take it days and I don't even want to think how much seek time it's going to need to store its repo on a spinning HDD.

> I added a single 2.7G ubuntu iso - it took 5 minutes to ingest it (on a tmpfs!), and turned it into 45k(!) little chunks, wtf is up with that?

Ingest times correlate linearly with file sizes because it needs to compute the blobref (which is a configurable hash) for all the blobs (chunks as you call them). Splitting in blobs/chunks is necessary because a stated goal of the project is to have snapshots by default when modifications are done. Doing snapshots/versioning without chunking would be very inefficient.

Re: Keep Your Stuff, for Life

#99

Post by hvs »

taneq wrote: post #76
CGamesPlay wrote: post #9

Wow, it was really hard to find out what this software does. I finally found a demo at the end of an hour-long talk from 2 years ago: https://youtu.be/PlAU_da_U4s?t=2687 So it seems to have a bunch of scripts to import data into its database from a variety of sources (including cloud services), and provides a search interface to navigate that historical data. And it has a lot of under-the-hood stuff about replication…

Does the first sentence not cover that fairly well? > Perkeep (née Camlistore) is a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era. Data may be files or objects, tweets or 5TB videos, and you can access it via a phone, browser or FUSE filesystem. I checked archive.org and that text has been there for a couple of months at least.…

It reads more like a patent description than telling me what it's for.

Re: Keep Your Stuff, for Life

#100

Post by eexan »

shock wrote: post #98
eexan wrote: post #48

I tried perkeep a while ago. While the ideas are cool, the implementation is meh: I added a single 2.7G ubuntu iso - it took 5 minutes to ingest it (on a tmpfs!), and turned it into 45k(!) little chunks, wtf is up with that? At this rate indexing my multiple terabytes of data is going to take it days and I don't even want to think how much seek time it's going to need to store its repo on a spinning HDD.

> I added a single 2.7G ubuntu iso - it took 5 minutes to ingest it (on a tmpfs!), and turned it into 45k(!) little chunks, wtf is up with that? Ingest times correlate linearly with file sizes because it needs to compute the blobref (which is a configurable hash) for all the blobs (chunks as you call them). Splitting in blobs/chunks is necessary because a stated goal of the project is to have snapshots by default whe…

Reading the docs, snapshotting/versioning doesn't strike me as a major feature of perkeep. It's more important and appropriate in the domain of backup software (e.g. restic/attic/borg) and where you'd want it together with delete functionality to reclaim space.

But perkeep's focus, as I understand it, is more on managing an unstructured collection of immutable things (e.g. photo archive), rather than being a tool to back up your mutable filesystem. So I'm not sure they made a good design decision to chunk the sh*t out of my files, which really kills the performance on large files and especially on spinning disks.

Post reply on HN