Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do you even need a database?

dbpro.app

Post reply on HN
Print view View original

151–160 of 311 posts

Re: Do you even need a database?

#151

Post by debo_ »

oliviergg wrote: post #140

Please … Every few years the pendulum swings. First it was “relational databases are too rigid, just use NoSQL.” Then “NoSQL is a mess, just go back to Postgres.” Now: “do you even need a database at all, just use flat files.” Each wave is partially right. But… each wave is about to rediscover, the hard way, exactly why the previous generation made the choices they did. SQLite is the answer to every painful lesson le…

Michael Stonebraker used to write long, scathing critiques of modern data storage/retrieval fads, and how they were forgetting important historical lessons.

They were terrific reads; his writing on object-oriented databases was the most fun technical reading I did in grad school. And I even learned a lot!

Re: Do you even need a database?

#152

Post by leafarlua »

marcosdumay wrote: post #85
leafarlua wrote: post #42

Earlier quoted context omitted.

This always confuses me because we have decades of SQL and all its issues as well. Hundreds of experienced devs talking about all the issues in SQL and the quirks of queries when your data is not trivial. One would think that for a startup of sorts, where things changes fast and are unpredictable, NoSQL is the correct answer. And when things are stable and the shape of entities are known, going for SQL becomes a natu…

No, when things change fast and unpredictably, NoSQL is worse than when they are well-known and stable. NoSQL gains you no speed at all in redesigning your system. Instead, you trade a few hard to do tasks in data migration into an unsurmountable mess of data inconsistency bugs that you'll never actually get into the end of. > is mostly bad design decisions and poor domain knowledge Yes, using NoSQL to avoid data mig…

Makes sense. But in this case, why NoSQL exists? What problems does it resolves and when should it be considered? I'm being naive, but fast changing environment has been one of the main advantages that I was taught from devs when it comes to NoSQL vs SQL (nosql being the choice for flexible schemas). So it is more about BASE vs ACID?

Re: Do you even need a database?

#153

Post by goerch »

oliviergg wrote: post #140

Please … Every few years the pendulum swings. First it was “relational databases are too rigid, just use NoSQL.” Then “NoSQL is a mess, just go back to Postgres.” Now: “do you even need a database at all, just use flat files.” Each wave is partially right. But… each wave is about to rediscover, the hard way, exactly why the previous generation made the choices they did. SQLite is the answer to every painful lesson le…

Yes, but you are probably a bit too polite. And I'm not sure how to do justice to SQLite, Postgres and my new favourite toy DuckDB.

Re: Do you even need a database?

#155

Post by trgn »

z3ugma wrote: post #8

At some point, don't you just end up making a low-quality, poorly-tested reinvention of SQLite by doing this and adding features?

im sure, but honestly, i would love to have a db engine that just writes/reads csv or json. does it exist?

Re: Do you even need a database?

#156

Post by [deleted] »

freedomben wrote: post #35
giva wrote: post #23

Earlier quoted context omitted.

> Dealing with even sqlite is a pain in the ass when you just need to tweak or read something, especially if you're not the dev. How? With SQL is super easy to search, compare, and update data. That's what it’s built for.

Pain in the ass was way too strong, I retract that. Mainly I meant relative. For example `nvim .json` and then /search for what I want, versus tracking down the sqlite file, opening, examining the schema, figuring out where the most likely place is that I care about, writing a SQL statement to query, etc.

[deleted]

Re: Do you even need a database?

#157

Post by hnlmorg »

> Every database you have ever used reads and writes to the filesystem, exactly like your code does when it calls open().

Nope. There are non-persistent in-memory databases too.

In fact, a database can be a plethora of things and the stuff they were building is just a subset of a subset (persistent, local relational database)

Re: Do you even need a database?

#158

Post by herpdyderp »

trgn wrote: post #155
z3ugma wrote: post #8

At some point, don't you just end up making a low-quality, poorly-tested reinvention of SQLite by doing this and adding features?

im sure, but honestly, i would love to have a db engine that just writes/reads csv or json. does it exist?

I wrote a CSV DB engine once! I can't remember why. For fun?

Re: Do you even need a database?

#159

Post by leafarlua »

tracker1 wrote: post #91
leafarlua wrote: post #42

Earlier quoted context omitted.

This always confuses me because we have decades of SQL and all its issues as well. Hundreds of experienced devs talking about all the issues in SQL and the quirks of queries when your data is not trivial. One would think that for a startup of sorts, where things changes fast and are unpredictable, NoSQL is the correct answer. And when things are stable and the shape of entities are known, going for SQL becomes a natu…

I think part of it is the scale in terms of the past decade and a half... The hardware and vertical scale you could get in 2010 is dramatically different than today. A lot of the bespoke no-sql data stores really started to come to the forefront around 2010 or so. At that time, having 8 cpu cores and 10k rpm SAS spinning drives was a high end server. Today, we have well over 100 cores, with TBs of RAM and PCIe Gen 4/…

What about the microservices/serverless functions world? This was another common topic over the years, that using SQL with this type of system was not optimal, I believe the issue being the connections to the SQL database and stuff.

Re: Do you even need a database?

#160

Post by goerch »

winrid wrote: post #89

My recent project - a replacement for CodeMaster's RaceNet, runs on flat files! https://dirtforever.net/ Just have to use locks to be careful with writes. I figured I'd migrate it to a database after maybe 10k users or so.

And crashes you can exclude? Good luck!
Post reply on HN