Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do you even need a database?

dbpro.app

Post reply on HN
Print view View original

281–290 of 311 posts

Re: Do you even need a database?

#281

Post by skapadia »

upmostly wrote: post #4
vovanidze wrote: post #3

people wildly underestimate the os page cache and modern nvme drives tbh. disk io today is basically ram speeds from 10 years ago. seeing startups spin up managed postgres + redis clusters + prisma on day 1 just to collect waitlist emails is peak feature vomit. a jsonl file and a single go binary will literally outlive most startup runways. also, the irony of a database gui company writing a post about how you dont a…

The irony isn’t lost on us, trust me. We spent a while debating whether to even publish this one. But yeah, the page cache point is real and massively underappreciated. Modern infrastructure discourse skips past it almost entirely. A warm NVMe-backed file with the OS doing the caching is genuinely fast enough for most early-stage products.

The second paragraph sounds eerily AI-generated.

Re: Do you even need a database?

#282

Post by graemep »

koliber wrote: post #59

I love this article as it shows how fast computers really are. There is one conclusion that I do not agree with. Near the end, the author lists cases where you will outgrow flat files. He then says that "None of these constraints apply to a lot of applications." One of the constraints is "Multiple processes need to write at the same time." It turns out many early stage products need crons and message queues that exec…

If you look at those "When do you actually need a database?" constraints I think its missing consistency which prevents bugs and makes debugging easier.

When you combine all those a database is a better alternative for all but the simplest cases.

Re: Do you even need a database?

#283

Post by Hendrikto »

oliver236 wrote: post #273
koliber wrote: post #162

Earlier quoted context omitted.

Backups are super-simple as well. I'm also a convert.

why sqlite over postgres?

It is simpler and removes failure points. You don’t need a separate database server process or network/socket connections. Everything happens in-process.

Re: Do you even need a database?

#284

Post by Hendrikto »

tnelsond4 wrote: post #239

I love SQLite, I love the idea, I love having something mature and lightweight, but like the author I discovered it's overkill or dare I say insufficient for certain use cases. I was building a client side dictionary app with search functionality and thought that using sqlite's wasm port would be the perfect solution. I used SQLite for a couple years and it started to wear on me. The database files were bigger than t…

> The database files […] didn't compress super well

That’s a good thing. If they were compressible that would indicate inefficiency in the SQLite format.

Re: Do you even need a database?

#285

Post by LtWorf »

tnelsond4 wrote: post #239

I love SQLite, I love the idea, I love having something mature and lightweight, but like the author I discovered it's overkill or dare I say insufficient for certain use cases. I was building a client side dictionary app with search functionality and thought that using sqlite's wasm port would be the perfect solution. I used SQLite for a couple years and it started to wear on me. The database files were bigger than t…

I think for applications, files that already have indexes and can be memory mapped directly are preferred to files that need to be in memory, require index re-creation and de-compression every single time.

Re: Do you even need a database?

#286

Post by tstenner »

koliber wrote: post #167
upmostly wrote: post #102

Earlier quoted context omitted.

Seeing the Rust 1M benches were an amazing reminder as to how fast stuff really is.

The reality is that things will be blazing fast in any language if you save things by PK in HashMaps.

In the benchmark Rust is more than 50% faster than the runner up

Re: Do you even need a database?

#288

Post by dxxvi »

Before reading that post, I would use sqlite/duckdb for everything related to data. After reading that post, I will use sqlite/duckdb + Rust for everything related to data :-)

Re: Do you even need a database?

#289

Post by pdyc »

tnelsond4 wrote: post #239

I love SQLite, I love the idea, I love having something mature and lightweight, but like the author I discovered it's overkill or dare I say insufficient for certain use cases. I was building a client side dictionary app with search functionality and thought that using sqlite's wasm port would be the perfect solution. I used SQLite for a couple years and it started to wear on me. The database files were bigger than t…

what was your data size? i am surprised 800kb made a difference? using stringzilla was smart approach,my guess is it being unusually faster made all the difference.

Re: Do you even need a database?

#290

Post by zaphirplane »

koliber wrote: post #59

I love this article as it shows how fast computers really are. There is one conclusion that I do not agree with. Near the end, the author lists cases where you will outgrow flat files. He then says that "None of these constraints apply to a lot of applications." One of the constraints is "Multiple processes need to write at the same time." It turns out many early stage products need crons and message queues that exec…

SQLite is famous for single writer how does that reconcile with you criticism of the article
Post reply on HN