Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local-first software (2019)

inkandswitch.com

Post reply on HN
Print view View original

261–270 of 308 posts

Re: Local-first software (2019)

#261

Post by strken »

crossroadsguy wrote: post #257
agos wrote: post #233

Earlier quoted context omitted.

I didn’t know they did this change which means it’s time to think about migrating away from bear. Which is a pity because the software itself is rock solid

Yes, I feel the same. A rock solid app dismantled by just one developer choice. Still I won't diss the devs because it's their choice but "a text note" is the last thing I would want "locked" in a sqlite db and bear is a "plain text note taking app" really. So just sad.

From memory the sqlite3 API is something like

    sqlite3 data.db 'select text from notes'
I use sqlite3 to load and query random CSVs all the time. It feels a bit weird to hear data described as "locked" within a SQLite DB, because it's one of the simplest non-text formats for me to read data from. It would surprise me of it took more than five minutes to set up a one-way cron job to dump your notes into plain text files.

Re: Local-first software (2019)

#262

Post by monkeyelite »

necovek wrote: post #220
monkeyelite wrote: post #169

Earlier quoted context omitted.

But this problem isn't going to be solved by every app making its own sync system. Even if there is a magic library you can adopt that does pretty good, then everyone having their own completely independent hosting solution and sync schedule. If files are insufficient, what data-structure would make modular sync possible for multiple applications in an OS? And I’m not suggesting one doesn’t exist, I’m challenging to…

I agree that files solve some rudimentary cases, but they do not even allow simple conflict resolution. Eg. compressed files, including container formats like OpenOffice (text files in a ZIP archive IIRC), might be simple to apply changes from two sides if they are in distant parts, but syncing full files simply barfs. Note that this does not even need two users: I hit this problem with a desktop and laptop and self-…

The log is just a block of data. All the burden to use the log is on the application so the OS is providing very little general functionality.

I’m also suspect of logs as a general form of conflict resolution as you are just hoping the two edits don’t touch the same area. And if they do then you are left in an invalid state.

You brought up zips. Pile of files seems like a way you can divide up data so it can have more pieces that are mergable/diffable.

For example “the folder can contain N files” or “there must be exactly 1 of this file”.

Re: Local-first software (2019)

#263

Post by hodgesrm »

GMoromisato wrote: post #38
hodgesrm wrote: post #35

Earlier quoted context omitted.

> * Data portability guarantees: Vendors must spell out how data gets migrated out, and all formats must be either open or (at minimum) fully documented. This is not practical for data of any size. Prod migrations to a new database take months or even years if you want things to go smoothly. In a crisis you can do it in weeks but it can be really ugly, That applies even when moving between the same version of open so…

I'd love to know more about BYOC. Does that apply to the raw data (e.g., the database lives inside the enterprise) or the entire application stack (e.g., the enterprise is effectively self-hosting the cloud). It seems like you'd need the latter to truly be immune to cloud-vendor problems. [But I may not understand how it works.]

Here's how we do it for analytic systems: data and software. The software services are open source running on Kubernetes. If you don't like the vendor or the vendor goes away, the existing services keep running. You can also maintain them because the stack is open source.

This is different from what the local-first article is describing, which addresses data for individuals. That's a much harder problem to solve at scale.

Re: Local-first software (2019)

#264

Post by fuzzfactor »

DataDaoDe wrote: post #29

Yes a thousand percent! I'm working on this too. I'm sick of everyone trying to come up with a use case to get all my data in everyone's cloud so I have to pay a subscription fee to just make things work. I'm working on a fitness tracking app right now that will use the sublime model - just buy it, get updates for X years, sync with all your devices and use it forever. If you want updates after X years buy the newest…

What if you are an old man and more clouds than ever are appearing which deserve a good fist shaking?

Asking for a friend . . .

Re: Local-first software (2019)

#265

Post by necovek »

monkeyelite wrote: post #262
necovek wrote: post #220

Earlier quoted context omitted.

I agree that files solve some rudimentary cases, but they do not even allow simple conflict resolution. Eg. compressed files, including container formats like OpenOffice (text files in a ZIP archive IIRC), might be simple to apply changes from two sides if they are in distant parts, but syncing full files simply barfs. Note that this does not even need two users: I hit this problem with a desktop and laptop and self-…

The log is just a block of data. All the burden to use the log is on the application so the OS is providing very little general functionality. I’m also suspect of logs as a general form of conflict resolution as you are just hoping the two edits don’t touch the same area. And if they do then you are left in an invalid state. You brought up zips. Pile of files seems like a way you can divide up data so it can have mor…

Log is certainly a blob of data, but the point is that it should be more granular, with clearer delineation of what are and what aren't conflicting changes: there will always be conflicting changes where no automation can really help.

For zip and other container-type files, you'd have log entries to the tune of "changed contained file foo.png: ...".

Operating systems would need to support some basic operations: container file operations like zip files, basic bitmap image editing, basic text document diffing, structured text diffing (XML, JSON, YAML...), etc...

Applications would provide OS-registered services (like MIME handlers are registered today) that can interpret and produce even more semantic events on top of the existing ones.

The environment could offer an interface during "syncing" when it detects a conflict to resolve it using one of the generic (or not) conflict resolution mechanisms (use local or remote version completely; use incremental delta if there is some generic semantic diff; app-provided capability if present on both sides).

Now, you are right that this can be implemented with this log being a regular file next to the file itself and completely user-space, but you will hit issues if you are not able to tie it nicely to things like fwrite/fsync and similar syscalls.

Obviously, for it to make sense, it needs to be widely accepted as the approach, which is what the local-first movement is trying to achieve with CRDTs.

Re: Local-first software (2019)

#266

Post by hollowturtle »

ciju wrote: post #231

We have been building a local-first browser app (PWA) for personal finance, based on double-entry accounting. https://finbodhi.com/ We do use online services like firebase for auth, and some service to fetch commodity prices etc, but rest of the data is stored in browser storage (sqlite) and backed to local disk (and soon dropbox). We also syncs data across devices, always encrypting data in transit. I think it's the…

Try demo doesn't work on my iphone, it keeps spinning forever. Plus please take into consideration of removing the friction via signup, if it's real local first you don't need accounts in the cloud

Re: Local-first software (2019)

#267

Post by throw10920 »

crossroadsguy wrote: post #228
patmorgan23 wrote: post #156

Earlier quoted context omitted.

Obsidian the note taking app is a great model to follow as well. The client is completely free and they sell an optional syncing service. The notes are all on markdown files so the client is completely optional.

This is the reason I have always refused to use Bear note taking app irrespective of how good and snappy that app is. Because they keep their notes in a SQLite db now and even though that file can be backed up and handled locally my notes are not easily accessible to me. I can't easily edit my notes in other editors (which I often like to do on my mac), I can't version controlled backup and sync those files the way I…

> What is sad is that they used to be local files first note app and then they moved to sqlite citing some sync and performance issues.

They're still a local-first note application. It's just slightly harder for you to edit your notes externally, and not even by that much - it's very easy to directly query (read and write) SQLite databases, and if you really cared you could have made a script to grab a note, export it to a temporary text file, allow you to edit it, then update the SQLite database.

> I can't version controlled backup and sync those files

You absolutely can - you can dump SQLite databases to text files that contain SQL queries that will restore the database that you can then backup and sync: https://stackoverflow.com/questions/75675/how-to-dump-the-da...

> then they moved to sqlite citing some sync and performance issues

Yes, that's because "plain text" files are bad for performance and harder to sync correctly. For people who (1) have over a hundred thousand notes they want to keep (like me) and (2) want maximum confidence that they're not going to lose years worth of work, that's incredibly important.

The devs made the right choice. You can always write scripts to interface with a SQLite database with an external editor. You can't take plain text files and magically make them as fast and durable as a database.

Re: Local-first software (2019)

#268

Post by hodgesrm »

> Local-first apps, on the other hand, have better privacy and security built in at the core.

I love this article, but the section on security raised a lot of questions. What's the model for authorizing access to documents for collaboration? How do you managed keys safely for encrypted data? How do users recover "lost" keys?

Cloud computing models have a lot of security mechanisms built-in. You might not like the model (AWS IAM for example) but at least there's a foundation already in place.

Re: Local-first software (2019)

#269

Post by bichiliad »

I feel like local-first or offline-first can be seen as something catered to niche users when it's brought up in front of strategy or planning folks — they imagine most people having good, reliable internet all the time. The truth is always more frustrating to account for. It is extremely frustrating to see the falloff in functionality in apps when internet is spotty or slow, which happens a lot. Try doing anything in most apps on the subway in New York (where there isn't service in most tunnels) and you'll feel the pain. Or, try doing anything in a crowd and the cell towers are saturated. Fastmail's client won't show you emails without internet, Balatro hangs while it looks for a cloud save, the list goes on and on.

Re: Local-first software (2019)

#270

Post by nylonstrung »

crossroadsguy wrote: post #257
agos wrote: post #233

Earlier quoted context omitted.

I didn’t know they did this change which means it’s time to think about migrating away from bear. Which is a pity because the software itself is rock solid

Yes, I feel the same. A rock solid app dismantled by just one developer choice. Still I won't diss the devs because it's their choice but "a text note" is the last thing I would want "locked" in a sqlite db and bear is a "plain text note taking app" really. So just sad.

Is SQLite really "locked in"?

It's possible most ubiquitous open source software ever (far more common than markdown) and your notes exist in fully readable text form inside it

Bear being proprietary is the real threat of lock in

Post reply on HN