Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why Databases Instead of Filesystem?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 28 posts

Ask HN: Why Databases Instead of Filesystem?

#1

Post by uticus »

A database could be described crudely as a program for interfacing with data on disk. Most software is built to interface with databases although the software also has (often very capable) ways of interfacing with data on disk via filesystem interaction.

My question is, what historically has driven the industry to focus on database-specific solutions, rather than on filesystem-specific solutions?

This is not a rant against databases, but I do wonder why many major programming languages and frameworks (RoR Active Directory, C# EF, etc) have put much effort into making database interaction smooth from the perspective of the programming language, instead of putting effort into interacting with data on disk. Kind of an alternate reality sort of musing.

Re: Ask HN: Why Databases Instead of Filesystem?

#3

Post by codingdave »

A database is a file system when you get down to it. The reason people use them is to abstract up a layer so you can query the data and get the results you want instead of having to iterate through direct reads of a disk, then having to read, parse, and filter what you want from those reads. You could always write code to help do those things direct from disk, but you know what you have just written if you do so? A database!

Re: Ask HN: Why Databases Instead of Filesystem?

#4

Post by lgeorget »

More of a sidenote than an answer but a database system can be faster than using the disk directly: https://sqlite.org/fasterthanfs.html#approx.

It turns out having a defined abstraction like a database makes things faster than having to rely on a rawer interface like filesystems because you can then reduce the number of system calls and context switches necessary. If you wanted to optimize that in your own code rather than relying on a database, you'd end up reinventing a database system of sorts, when (probably) better solutions already exist.

Re: Ask HN: Why Databases Instead of Filesystem?

#5

Post by PaulHoule »

codingdave wrote: post #3

A database is a file system when you get down to it. The reason people use them is to abstract up a layer so you can query the data and get the results you want instead of having to iterate through direct reads of a disk, then having to read, parse, and filter what you want from those reads. You could always write code to help do those things direct from disk, but you know what you have just written if you do so? A d…

I'd say the filesystem is a database.

It would be straightforward, for instance, to implement a lot of the functionality of a filesystem in a database with BLOBs. Random access might be a hassle, but people are getting used to "filesystem-like" systems which are bad at random access like S3.

Re: Ask HN: Why Databases Instead of Filesystem?

#6

Post by uticus »

codingdave wrote: post #3

A database is a file system when you get down to it. The reason people use them is to abstract up a layer so you can query the data and get the results you want instead of having to iterate through direct reads of a disk, then having to read, parse, and filter what you want from those reads. You could always write code to help do those things direct from disk, but you know what you have just written if you do so? A d…

> You could always write code to help doing those things direct from disk, but you know what you have just written if you do so? A database!

Yes, but that's my point. Why is this not a part of the standard library / typical package with very little friction with the rest of the code, instead of a separate program that the standard library / typical packages provide in an attempt to reduce the friction?

Or are you making the general point that databases already existed prior to the standard libraries etc, and this is just a case of interfacing with an existing technology instead of rebuilding from scratch?

Re: Ask HN: Why Databases Instead of Filesystem?

#7

Post by FrankWilhoit »

It turns out to be quite unreasonably difficult to put something aside for a time -- from milliseconds to decades -- and then go back and find it again. The difficulty is all in the "find", because things that anyone might care about finding have various different aspects that different people may care about. There is no middle ground between the electronic equivalent of a pile of papers on one's desk and the full capabilities of the relational model. Filesystems are somewhere in between; but there is no useful place in between.

Re: Ask HN: Why Databases Instead of Filesystem?

#10

Post by apothegm »

uticus wrote: post #6
codingdave wrote: post #3

A database is a file system when you get down to it. The reason people use them is to abstract up a layer so you can query the data and get the results you want instead of having to iterate through direct reads of a disk, then having to read, parse, and filter what you want from those reads. You could always write code to help do those things direct from disk, but you know what you have just written if you do so? A d…

> You could always write code to help doing those things direct from disk, but you know what you have just written if you do so? A database! Yes, but that's my point. Why is this not a part of the standard library / typical package with very little friction with the rest of the code, instead of a separate program that the standard library / typical packages provide in an attempt to reduce the friction? Or are you mak…

Because a reasonably well optimized database with support for indexes, data integrity enforcement, transactions, and all the other important things we expect from a good (relational) database is complex enough that it takes a rather large codebase to do it reasonably well. It’s not something you slap together out of a handful of function calls.

ETA: look at SQLite for an example — it’s a relatively recent and simple entrant in the field and the closest you’ll find in the mainstream to a purely filesystem based RDBMS. How would you provide a stdlib that would let you implement something like that reasonably simply? What would be the use case for it?

Post reply on HN