A file system can be considered a document database with the key being the file name.
Smaller is better – The rise, fall, and rise of flat file software
51–60 of 152 posts
Re: Smaller is better – The rise, fall, and rise of flat file software
#52I remember when I got started with webdev around 2002, a lot was built on flat files. And so did I, because that’s how my “internet mentor” did it: guestbooks, bulletin boards, mailing lists, shoutboxes, CMSs. All in flat files, except I used php and not Perl like him.
Anyone here remember CuteNews? That was my favourite flatfile CMS back in the day.
Re: Smaller is better – The rise, fall, and rise of flat file software
#53Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
A database can be relational, it can make some guarantees, it can pass Acid. With a file you are responsible for performance, reliability, integrity and security of the data.
Do you believe that you can do a better job than those tens of guys who contributed to a DBMS? Go ahead.
Do you not need reliability, integrity, security, performance? Go ahead.
So use a file to store data instead of database but know the trade-offs.
Re: Smaller is better – The rise, fall, and rise of flat file software
#54I think DokuWiki also deserves a mention for being pretty flat-file with a lot of features and extensions
Re: Smaller is better – The rise, fall, and rise of flat file software
#55Database is basically just ab abstraction to help you ingest and query flat files on disk.
If you skip that and do your own ingestion and querying...you just have a built-in database?
This might have some performance shortcomings, but given maturity of caching these days, you can probably just add caching layer in front of the inefficient query engine when the needs arise.
Re: Smaller is better – The rise, fall, and rise of flat file software
#56One benefit of flat file is you can host it on GitHub pages.
Re: Smaller is better – The rise, fall, and rise of flat file software
#57I feel like you can achieve some pretty awesome feats with a few files and docker. Setting up a simple Mongodb is dead simple. Adding a Fast API is dead simple. Serving a small site via Flask is dead simple. And with a simple docker exec command I can dump the database to JSON. And I can even use Mongo Compass to tinker with the database and try out queries and create indices.
Re: Smaller is better – The rise, fall, and rise of flat file software
#58Does it have to be one or another with SQLite?
Re: Smaller is better – The rise, fall, and rise of flat file software
#59Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
I think that the direction is moving torward SQLite which is a flat file database that has the most installs from apps in general.
Re: Smaller is better – The rise, fall, and rise of flat file software
#60Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…
Perhaps it was deliberate, but your last paragraph (plaintext vs db) really just reinforces your point of standards. There's nothing particularly special about plaintext aside from its total ubiquity.
Everything today uses and understands ASCII/UTF-8 and we've used that, plus some handy human conventions like limited character count between newline characters, to build up tools like git to help manage the changes in smaller chunks.
There's nothing preventing us from doing the same for other formats if they become commonly supported or expected. Much like SQLite and some of the tools cropping up.