I recently wrote a KV disk change in Rust that uses the latest syscalls: https://github.com/anacrolix/possum
Build a Database in 3000 Lines with 0 Dependencies
31–40 of 59 posts
Re: Build a Database in 3000 Lines with 0 Dependencies
#32I looked into this at one point, I was typing out entire codebases for didactic purposes: SQLite 3 was 120,000 lines of code, but SQLite 2 was 12,000. So for a bit more effort you get a battle tested real world thing!
one does not simply type out 120 000 lines of code..
Number of lines does not matter anymore.
Re: Build a Database in 3000 Lines with 0 Dependencies
#33If you want 1 dependency rather than 0, there are various low-level libraries designed for building full-featured databases.
For one of these, RocksDB, one reference point is how CockroachDB was built on top of them for many years and many successful Jepsen tests (until they transitioned to an in-house solution).
https://www.cockroachlabs.com/blog/cockroachdb-on-rocksd/
https://www.cockroachlabs.com/blog/pebble-rocksdb-kv-store/
Another possibility is Apple's FoundationDB, with an interesting discussion here: https://news.ycombinator.com/item?id=37552085
Re: Build a Database in 3000 Lines with 0 Dependencies
#34Man I got the first edition of this book and it was so bad. Hopefully this is better…
Currently it's on sales at Leanpub with $19.80 but apparently the ebook with the Golang code is $39:
https://leanpub.com/build_your_own_database_from_scratch/c/L...
To be honest it seems a bit strange to pay for the code since all the source codes to build the database is literally inside the book.
Re: Build a Database in 3000 Lines with 0 Dependencies
#35Earlier quoted context omitted.
Oh no, SQLite3 is a lot more featureful than SQLite2. The proprietary test suite is what makes SQLite3 so solid.
This makes me wonder. Is anyone practicing TDD with genAI/LLMs? If the true value is in the tests, might as well write those and have the AI slop be the codebase itself. TDD is often criticized for being slow. I'd seriously like to compare one vs the other today. I've also heard people find it challenging to get it to write good tests.
TDD suffers from being inflexible when you don't fully understand the problem. Which on software is basically always.
Everytime I've tried it for something I make no progress at all compared to just banginf out the shape that works and then writing tests to interrogate my own design.
Re: Build a Database in 3000 Lines with 0 Dependencies
#36Earlier quoted context omitted.
Really puts the auto- in didact! Very curious to hear how this worked for you; it’s almost directly the opposite of the copilot approach. I learned assembler by typing in listings from magazines and hand dis-assembling and debugging on paper. Your approach seems similar in spirit, but who has the times these days?
I program in neovim with no plugins, no autocomplete and no syntax highlighting. I type everything myself (though I will use copy and paste from time to time). There is a discipline to it that I find very beneficial. As a language designer, it also makes me think very carefully about the syntactic burden of languages that I design. It keeps my languages tight. One of the nice things about typing all of my own code wi…
I program most languages like this, in emacs.
Re: Build a Database in 3000 Lines with 0 Dependencies
#37Earlier quoted context omitted.
Oh no, SQLite3 is a lot more featureful than SQLite2. The proprietary test suite is what makes SQLite3 so solid.
This makes me wonder. Is anyone practicing TDD with genAI/LLMs? If the true value is in the tests, might as well write those and have the AI slop be the codebase itself. TDD is often criticized for being slow. I'd seriously like to compare one vs the other today. I've also heard people find it challenging to get it to write good tests.
I'd sort of invert that and say it's better to use LLMs to just generate tons more test cases for the SQL DBs. Theoretically we could use LLMs to create 100s of Thousands (unlimited really) of test cases for any SQL system, where you could pretty much certify the entire SQL capability. Maybe such a standardized test suite already exists, but it was probably written by humans.
Re: Build a Database in 3000 Lines with 0 Dependencies
#38Here's a different approach in Common Lisp I wrote a while back:
Re: Build a Database in 3000 Lines with 0 Dependencies
#39A similar resource I recently discovered and it’s not that popular: https://github.com/pingcap/talent-plan/tree/master/courses/r... Writing a Bitcask(KV wal) like db in Rust. Really cool and simple ideas. The white paper is like 5 pages.
Bitcask is great, it's such an elegant idea. I've built a few different toy implementations in different languages and learned something new each time. YMMV based on how many deps you do or don't want to use, how complete you want to go, but it's a totally doable small-ish project.
Re: Build a Database in 3000 Lines with 0 Dependencies
#40[dead]