Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TinyBase v2.0: reactive data store for local-first apps

tinybase.org

Post reply on HN
Print view View original

21–30 of 65 posts

Re: TinyBase v2.0: reactive data store for local-first apps

#21

Post by jamesgpearce »

pbronez wrote: post #4

Nice work. I love anything that makes local-first easier. I’ve been paying close attention to Dolt for this. In a perfect world I’d have the gut semantics from Dolt and EdgeQL from EdgeDB. This focuses on the reactive feature - the ability to set listeners and get alerts. There seem to be some nice React integration, but haven’t built with React enough to immediately see what that unlocks.

Thank you. I figured that getting/setting/listening are the key primitives, and then the React layer (which is just one of many theoretical UI library bindings) can use hooks and components, or whatever, to simply wrap them. I'm not familiar with Dolt, but seems like I need to become so!

Re: TinyBase v2.0: reactive data store for local-first apps

#22

Post by jamesgpearce »

pbronez wrote: post #5
jamesgpearce wrote: post #2

Author here, if anyone has any thoughts or feedback. Bring it on!

Congratulations on shipping! Can you explain your motive for focusing on the combination of reactive data and local deployment? My first thought is that local-first = one user = I know what’s changing because I’m the only one using it. Perhaps valuable for monitoring external data feeds?

Yeah, I see your point. I've been assuming that local-first doesn't mean local-only. Even if it's just you, you could easily be polling something else to get data updates, as you suggest. (One original use case was a personal GitHub dashboard app that polled for new activity but provided local analytics).

A couple of great essays I've enjoyed in this space are https://www.inkandswitch.com/local-first & https://riffle.systems/essays/prelude/

I feel it should be possible to build certain types of apps that can view, query, and manipulate denormalized data without even being online. (Perhaps I shouldn't over-glamorize those days when one used to jam Access databases into Windows apps, but hopefully you get the point.) Reactivity just serves as a nice binding paradigm.

Re: TinyBase v2.0: reactive data store for local-first apps

#23

Post by jamesgpearce »

mfbx9da4 wrote: post #18
jamesgpearce wrote: post #2

Author here, if anyone has any thoughts or feedback. Bring it on!

The main limitation I see with tiny base is that it has to load all data in one batch and modifying any record results in the whole data store being persisted to disk. Surely this can’t scale well…

The data is in memory, and the persistence strategy is kind of up to you. So it would scale reasonably if you put the sync/persist on a different schedule to the UI - perhaps when the browser is idle.

BUT of course, this is not a library to be used with data sets of billions of rows! I'd recommend a proper RDBMS for that ;)

Re: TinyBase v2.0: reactive data store for local-first apps

#25

Post by mfbx9da4 »

jamesgpearce wrote: post #17
mfbx9da4 wrote: post #13

Earlier quoted context omitted.

Wait so can you actually connect tiny base to SQLite?

No, there's no _actual_ RDBMS involved. The query engine uses a functional style of query that has similar concepts to SQL, but all the evaluation (and reactivity) is first party, in the library itself. Of course it would be pretty easy/fun to persist your TinyBase content into a SQLite database, or pull a dataset from it. The demos load up TSV but you could easily do something similar from an actual database somewhe…

If I understand correctly the issue with storing and retrieving from SQLite is that it won’t be incremental?

Re: TinyBase v2.0: reactive data store for local-first apps

#26

Post by mfbx9da4 »

jamesgpearce wrote: post #23
mfbx9da4 wrote: post #18

Earlier quoted context omitted.

The main limitation I see with tiny base is that it has to load all data in one batch and modifying any record results in the whole data store being persisted to disk. Surely this can’t scale well…

The data is in memory, and the persistence strategy is kind of up to you. So it would scale reasonably if you put the sync/persist on a different schedule to the UI - perhaps when the browser is idle. BUT of course, this is not a library to be used with data sets of billions of rows! I'd recommend a proper RDBMS for that ;)

Yeah but I feel like it could be used for that, if only retrieval and storage could be done incrementally…

Re: TinyBase v2.0: reactive data store for local-first apps

#27

Post by jamesgpearce »

mfbx9da4 wrote: post #25
jamesgpearce wrote: post #17

Earlier quoted context omitted.

No, there's no _actual_ RDBMS involved. The query engine uses a functional style of query that has similar concepts to SQL, but all the evaluation (and reactivity) is first party, in the library itself. Of course it would be pretty easy/fun to persist your TinyBase content into a SQLite database, or pull a dataset from it. The demos load up TSV but you could easily do something similar from an actual database somewhe…

If I understand correctly the issue with storing and retrieving from SQLite is that it won’t be incremental?

It could be made to be. The current persistence implementations are all batch but I believe they could be row, or even cell-level, theoretically. Nice idea.

Re: TinyBase v2.0: reactive data store for local-first apps

#28

Post by mfbx9da4 »

mfbx9da4 wrote: post #26
jamesgpearce wrote: post #23

Earlier quoted context omitted.

The data is in memory, and the persistence strategy is kind of up to you. So it would scale reasonably if you put the sync/persist on a different schedule to the UI - perhaps when the browser is idle. BUT of course, this is not a library to be used with data sets of billions of rows! I'd recommend a proper RDBMS for that ;)

Yeah but I feel like it could be used for that, if only retrieval and storage could be done incrementally…

You’re 80% of the way there now that you have encoded SQL-like syntax now all you have to do is to translate that into raw SQL queries to be executed

Re: TinyBase v2.0: reactive data store for local-first apps

#29

Post by jamesgpearce »

mfbx9da4 wrote: post #26
jamesgpearce wrote: post #23

Earlier quoted context omitted.

The data is in memory, and the persistence strategy is kind of up to you. So it would scale reasonably if you put the sync/persist on a different schedule to the UI - perhaps when the browser is idle. BUT of course, this is not a library to be used with data sets of billions of rows! I'd recommend a proper RDBMS for that ;)

Yeah but I feel like it could be used for that, if only retrieval and storage could be done incrementally…

Correct. I think this could be done, but I would need to think a little about how best to. Thank you for the idea!

Re: TinyBase v2.0: reactive data store for local-first apps

#30

Post by yuchi »

Personally built similar but incomplete solutions in my projects way more times that I’d like to admit.

Could you elaborate on your roadmap on CRDT / real time collaboration? If it’s in plan and will be executed correctly I’m completely sold.

Anyway, congrats, looks awesome.

Post reply on HN