uncle roger voice just use SQLite haiyaaaaa
I'd love watch a uncle Roger programming-wise reviews.
Smaller is better – The rise, fall, and rise of flat file software
61–70 of 152 posts
Re: Smaller is better – The rise, fall, and rise of flat file software
#62Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
A database can be relational, it can make some guarantees, it can pass Acid. With a file you are responsible for performance, reliability, integrity and security of the data. Do you believe that you can do a better job than those tens of guys who contributed to a DBMS? Go ahead. Do you not need reliability, integrity, security, performance? Go ahead. So use a file to store data instead of database but know the trade-…
Portability to other systems.
Re: Smaller is better – The rise, fall, and rise of flat file software
#63I like plaintext as I have the option to make edits with various editors in addition to the main application.
This can lead to easier search and manipulation.
I use Logseq and Noteplan, both store data in plaintext.
As a secondary editor to both I use Sublime Text for very fast search and replace, or diagnosing why Logseq might be having issues.
I’ve had issues before in other note taking / task apps and when I run into problems it is much harder to search and manipulate a database file. That’s even if the app has a sensible database format.
Of course depending on how you use and might need edit data you might find databases are better for you. I’ve always found it easier to work with plaintext for note taking/ task apps.
Re: Smaller is better – The rise, fall, and rise of flat file software
#64Sometime in the late 1990's, some programmer in some forum was bragging about the speed of some program of his that was working with a 50,000 record database. I don't remember all the exact details. I replied something along the lines that I regularly load text files having more lines than that into a text editor, that it's almost instant on today's hardware.
Re: Smaller is better – The rise, fall, and rise of flat file software
#65Earlier quoted context omitted.
I also did some Perl and that was super common. I also remember most 90s non-web software also using flat files. Even enterprise stuff, even when they had centralized storage in a file server. Sometimes they had databases but those were also accessed directly in the disk, sqlite-style. I remember seeing Clipper, FoxPro and MS Access being used in this manner. After Perl I graduated to doing web stuff with a Microsoft…
I remember scriptarchive, which I’ve browsed quite a lot, but it wasnt’t Matt. It was a german guy who had a similiar script collection, some of which he sold online. He reviewed and even tested my scripts and helped me along when I got stuck on incorrect cgi-bin file permissions or syntax errors and stuff like that when I was just learning things. Invaluable.
Ha, at that time I ran a website in Germany where I published my own Perl scripts for web forums, guestbooks, counters etc. (all based on flat files) and I helped other people getting started with web programming and "hacking" as well. I know there were a few such sites at the time but mine was quite popular in Germany. We had a really nice community at the time, I still fondly remember all the discussions and the general "small world" feeling the Internet had back then. I guess if you tried to build an online community like that today it would get overrun by trolls and spambots in no time. Those were the good old days.
Re: Smaller is better – The rise, fall, and rise of flat file software
#66Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…
> For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc.
That is not exactly what proprietary means. Though I feel the same - plain text can be versioned and managed by various tools and that is great!
Re: Smaller is better – The rise, fall, and rise of flat file software
#67Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
> There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end.
Curious as to what is not just "all bytes on a disk/memory in the end"? Also can we please stop calling datastores databases?
Re: Smaller is better – The rise, fall, and rise of flat file software
#68I remember when I got started with webdev around 2002, a lot was built on flat files. And so did I, because that’s how my “internet mentor” did it: guestbooks, bulletin boards, mailing lists, shoutboxes, CMSs. All in flat files, except I used php and not Perl like him.
Anyone here remember CuteNews? That was my favourite flatfile CMS back in the day.
Yep! Pretty popular in late 90s to mid 2000s from what I remember?
Re: Smaller is better – The rise, fall, and rise of flat file software
#69I'm trying to get away from a DB-based CMS for some company web sites. Static generators won't do for a number of reasons, so a flat-file CMS seems like a good fit. Currently I'm looking at GravCMS [1] as an alternative. It's free initially, but it can become somewhat expensive with many official plugins. But it's file format is Markdown, and one can combine multiple files into a so-called modular page. It has a back…
I can wholeheartedly vouch for Grav. It’s absurdly fast, easy to deploy and even easier to template for thanks to Twig.
When I was still freelancing and a project was beyond the scope of htmlcssjs, Grav CMS became my tool of choice. Their admin plug-in makes for a easy to use backend GUI and it’s configurable enough to have non-techies use it without losing sleep.
One of the newer features are the so called FlexObjects. It’s an absolutely great idea for a CMS but explaining the possibilities and technical intricacies seems moot as the documentation and Discord community are a better place to start learning. [1]
Websites built with Grav compare to SSG speeds while maintaining a different ease of use and much less time invest to roll out.
And sticking to the topic: being completely flat-file centric, those websites are a breeze to maintain and according to my albeit limited experience also a bit sturdier security wise.
Re: Smaller is better – The rise, fall, and rise of flat file software
#70Earlier quoted context omitted.
A database can be relational, it can make some guarantees, it can pass Acid. With a file you are responsible for performance, reliability, integrity and security of the data. Do you believe that you can do a better job than those tens of guys who contributed to a DBMS? Go ahead. Do you not need reliability, integrity, security, performance? Go ahead. So use a file to store data instead of database but know the trade-…
Portability to other systems.
Portability is more important than guaranteed atomicity, consistency, isolation and durability?