Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

github.com

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#31

Post by whispem »

_s_a_m_ wrote: post #2

I there an official docker image? I am looking for something more light-weighted than MinIO. What are the requirements?

There’s not an “official” image on Docker Hub yet, but the repo ships with a ready-to-use Dockerfile and a Compose cluster example. You can build with docker build . and spin up multi-node clusters trivially. Static Rust binaries make the image compact (typically ≤30MB zipped; nothing compared to MinIO :)), with no heavy runtimes. Requirements are dead simple: a recent Docker engine, any x86_64 (or ARM) host, and a few tens of MB RAM per instance at low load, scaling with data size/traffic.

I plan to push an official image (and perhaps an OCI image with scratch base) as the project matures — open to suggestions on ideal platforms/formats.

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#32

Post by whispem »

kryptiskt wrote: post #21
forgotpwd16 wrote: post #8

Earlier quoted context omitted.

>All the code, architecture, logic, and design in minikv were written by me, 100% by hand. Why people always lie with this? Especially in this case that they uploaded the entire log: Date: Sat Dec 6 16:08:04 2025 +0100 Add hashing utilities and consistent hash ring Date: Sat Dec 6 16:07:24 2025 +0100 Create mod.rs for common utilities in minikv Date: Sat Dec 6 16:07:03 2025 +0100 Add configuration structures for mini…

It looks like that if you want logically separated commits from a chunk of programming you have done. Stage a file or a hunk or two, write commit message, commit, rinse and repeat.

Absolutely: for all meaningful work I prefer small, logical commits using git add -p or similar, both for history clarity and for reviewer sanity. In initial “spike” or hack sessions (see early commits :)), it’s sometimes more monolithic, but as the codebase stabilized I refactored to have tidy, atomic commit granularity. I welcome suggestions on workflow or PR polish!

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#33

Post by whispem »

yes_man wrote: post #10
forgotpwd16 wrote: post #8

Earlier quoted context omitted.

>All the code, architecture, logic, and design in minikv were written by me, 100% by hand. Why people always lie with this? Especially in this case that they uploaded the entire log: Date: Sat Dec 6 16:08:04 2025 +0100 Add hashing utilities and consistent hash ring Date: Sat Dec 6 16:07:24 2025 +0100 Create mod.rs for common utilities in minikv Date: Sat Dec 6 16:07:03 2025 +0100 Add configuration structures for mini…

You have never split your working tree changes into separate commits?

Yes, I do split my working tree into separate commits whenever possible! I use interactive staging (git add -p) to split logical chunks: features, fixes, cleanups, and documentation are committed separately for clarity. Early in the project (lots of exploratory commits), some changes were more monolithic, but as minikv matured, I've prioritized clean commit history to make code review and future changes easier. Always happy to get workflow tips — I want the repo to be easy to follow for contributors!

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#34

Post by whispem »

remram wrote: post #25

Bit of a tangent, but what I'm looking for is a S3-compatible server with transparent storage , ie storing each file (object) as an individual file on disk. Minio used to do that but changed many years ago. Production-grade systems don't do that, for good reason. The only tool I've found is Rclone but it's not really meant to be exposed as a service. Anyone knows of an option?

minikv actually supports a fully S3-compatible API (PUT/GET/BATCH, including TTL extensions and real-time notifications). By default, the storage engine is segmented/append-only with object records in blob files, not “one file per object”. However, you can configure a backend (like the in-memory mode for dev/test, or Sled/RocksDB) and get predictable, transparent storage behavior for objects. Storing each object as an individual file isn’t the default — for durability and atomics, objects are grouped inside segment files to enable fast compaction, consistent snapshots, and better I/O performance.

If you need “one file per object” for a specific workflow, it’s possible to add a custom backend or tweak volume logic — but as you noted, most production systems move away from that model for robustness. That said, minikv’s flexible storage API makes experimentation possible if that’s what your use-case demands and you’re fine with the trade-offs.

Let me know what your usage scenario is, and I can advise on config or feature options!

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#35

Post by whispem »

jabron wrote: post #24
whispem wrote: post #16

Earlier quoted context omitted.

Thanks a lot! I make distinct commits "every 30s" because I'm focused and I test my project. If the CI is green, I don't touch of anything. If not, I work on the project until the CI is fully green.

What does that mean? You got feedback from the CI within 30 seconds and immediately pushed a fix?

Yes, in minikv, I set up GitHub Actions for automated CI. Every push or PR triggers tests, lint, and various integration checks — with a typical runtime of 20–60 seconds for the core suite (thanks to Rust’s speed and caching). This means that after a commit, I get feedback almost instantly: if a job fails, I see the logs and errors within half a minute, and if there’s a fix needed, I can push a change right away.

Rapid CI is essential for catching bugs early, allowing fast iteration and a healthy contribution workflow. I sometimes use small, continuous commits (“commit, push, fix, repeat”) during intense development or when onboarding new features, and the fast CI loop helps maintain momentum and confidence in code quality.

If you’re curious about the setup, it’s all described in LEARNING.md and visible in the repo’s .github/workflows/ scripts!

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#36

Post by unmole »

remram wrote: post #25

Bit of a tangent, but what I'm looking for is a S3-compatible server with transparent storage , ie storing each file (object) as an individual file on disk. Minio used to do that but changed many years ago. Production-grade systems don't do that, for good reason. The only tool I've found is Rclone but it's not really meant to be exposed as a service. Anyone knows of an option?

s3proxy: https://github.com/gaul/s3proxy

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#37

Post by efemeryl »

remram wrote: post #25

Bit of a tangent, but what I'm looking for is a S3-compatible server with transparent storage , ie storing each file (object) as an individual file on disk. Minio used to do that but changed many years ago. Production-grade systems don't do that, for good reason. The only tool I've found is Rclone but it's not really meant to be exposed as a service. Anyone knows of an option?

[dead]

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#38

Post by jabron »

whispem wrote: post #35
jabron wrote: post #24

Earlier quoted context omitted.

What does that mean? You got feedback from the CI within 30 seconds and immediately pushed a fix?

Yes, in minikv, I set up GitHub Actions for automated CI. Every push or PR triggers tests, lint, and various integration checks — with a typical runtime of 20–60 seconds for the core suite (thanks to Rust’s speed and caching). This means that after a commit, I get feedback almost instantly: if a job fails, I see the logs and errors within half a minute, and if there’s a fix needed, I can push a change right away. Rap…

So you read the CI result, implement a fix and stage + commit your changes in ~10 seconds? You might be superhuman.

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#39

Post by kunley »

whispem wrote: post #30
kunley wrote: post #4

Looks nice. What is the memory consumption under a significant load? That seems to be as much important as the throughput & latency.

Very relevant question! The memory profile in minikv depends on usage scenario and storage backend. - With the in-memory backend: Every value lives in RAM (with HashMap index, WAL ring buffer, TTL map, and Bloom filters). For a cluster with a few million objects, you’ll typically see a node use as little as 50–200 MB, scaling up with active dataset size and batch inflight writes; - With RocksDB or Sled: Persistent st…

Thanks for the thorough reply!

Re: Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

#40

Post by ritcgab »

yes_man wrote: post #10
forgotpwd16 wrote: post #8

Earlier quoted context omitted.

>All the code, architecture, logic, and design in minikv were written by me, 100% by hand. Why people always lie with this? Especially in this case that they uploaded the entire log: Date: Sat Dec 6 16:08:04 2025 +0100 Add hashing utilities and consistent hash ring Date: Sat Dec 6 16:07:24 2025 +0100 Create mod.rs for common utilities in minikv Date: Sat Dec 6 16:07:03 2025 +0100 Add configuration structures for mini…

You have never split your working tree changes into separate commits?

But you will never commit them via GitHub's web interface one file at a time :)
Post reply on HN