Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An embedded database written in Rust

github.com

Post reply on HN
Print view View original

11–20 of 74 posts

Re: An embedded database written in Rust

#11

Post by krenoten »

dmitrygr wrote: post #7

How well does that handle the storage disappearing halfway through a write? How well does it handle power being cut halfway through an update of some sort? How well does it handle some of the written blocks actually making it to the disc and others not? How about if the ones that made it were not the first or the last it issued to be written? (For predictable answers to these, and many other complex questions, when d…

ALICE showed that's not always true with sqlite. Sled is being built with an extreme bias toward reliability over features, but as the readme says, it has some time to go before reaching maturity. The tests are quite good at finding new issues and deterministically replaying them, so you can help bake it in by mining bugs using the default test suite and help it get there.

Re: An embedded database written in Rust

#12

Post by krenoten »

jacquesm wrote: post #10
tormeh wrote: post #2

No clustering :( I feel like good multi-master asynchronous and synchronous clustering is truly the frontier in DBs.

You can't cluster an embedded database, this comment makes not sense. Compare with SQLite, not with Orcale or Postgres.

Currently it's even more basic. The current usable parts are a pagecache following the llama approach, some great testing utility libraries, and an index (that you can use as a kv) that follows the bwtree approach. Later it will have structured access support, but it needs some more db components to get there. It is a construction kit as well as a kv.

Re: An embedded database written in Rust

#13

Post by krenoten »

tormeh wrote: post #2

No clustering :( I feel like good multi-master asynchronous and synchronous clustering is truly the frontier in DBs.

It's modular, and there is a paxos implementation, but it has been built totally in simulation so far and I haven't plugged it into an io layer yet. But this is trivial. That said, sled will always be a bwtree index, and the other modular crates will stand on their own.

Re: An embedded database written in Rust

#17

Post by krenoten »

dajonker wrote: post #15

I see that MVCC is a planned feature. Why would you need MVCC for an embedded database? It seems like unnecessary overhead that conflicts with the performance goals.

It's not needed for the single-key atomic record store, which is the sled bwtree index that is the current highest level module. MVCC is implemented in most popular embedded DBs because it is an effective way to manage mixed workloads that seek to read snapshots of the entire database at a single point in time as well as not blocking writes as this happens. This functionality is desirable for transactions that support mixed workloads. That's why I'm building it for a higher level module. This is a collection of modules that let you choose the abstraction and associated complexity that you want. There's also a modular pagecache that is totally decoupled and reusable for your own database experiments.

Re: An embedded database written in Rust

#18

Post by catwell »

dajonker wrote: post #15

I see that MVCC is a planned feature. Why would you need MVCC for an embedded database? It seems like unnecessary overhead that conflicts with the performance goals.

I think they mean "embedded" as in "embedded in a program", not "targeting embedded hardware". In particular the README states they intend to use it in https://github.com/spacejam/rasputin. This is a use case similar to LMDB, for instance, which has MVCC.

Re: An embedded database written in Rust

#19

Post by krenoten »

keithnz wrote: post #3
tormeh wrote: post #2

No clustering :( I feel like good multi-master asynchronous and synchronous clustering is truly the frontier in DBs.

what do you think of FoundationDb?

I am a total devotee to their approach to building simulable systems, although I seek to push it even farther and integrate lineage driven fault injection from an early stage. I see a lot of cool things in what they have done, technically. Sled is free from day one.

Re: An embedded database written in Rust

#20

Post by krenoten »

jinqueeny wrote: post #4
tormeh wrote: post #2

No clustering :( I feel like good multi-master asynchronous and synchronous clustering is truly the frontier in DBs.

What about TiKV ( https://github.com/pingcap/tikv ), a distributed transactional Key-Value store?

Use it for large scale HTAP! It's great for its flexible use cases at high scales :)
Post reply on HN