Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Smaller is better – The rise, fall, and rise of flat file software

wilcosky.com

Post reply on HN
Print view View original

81–90 of 152 posts

Re: Smaller is better – The rise, fall, and rise of flat file software

#81

Post by IshKebab »

DeathArrow wrote: post #51

A file system can be considered a document database with the key being the file name.

Some filesystems even have very database-like features e.g. ZFS has copy-on-write snapshotting, sort-of transactions (you can run a Lua script atomically, thought it's meant to be an admin feature). And lets not forget WinFS!

Still, your average extfs/NTFS is a really bad database. There's a reason people created "real" databases, and as far as I can tell the only advantage of flat files is that you can use a text editor instead of an SQL editor to view them, which seems pretty minor given all the downsides.

SQLite is probably better in almost every scenario.

Re: Smaller is better – The rise, fall, and rise of flat file software

#82

Post by OJFord »

tasuki wrote: post #66
OJFord wrote: post #50

Earlier quoted context omitted.

For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…

> For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. That is not exactly what proprietary means. Though I feel the same - plain text can be versioned and managed by various tools and that is great!

I was not aware I gave a definition for it. I didn't mean it as an antonym for 'readable', if that's what you mean.

Plaintext could be in a proprietary format but that's far better than proprietary compressed database that's a job to work out what even made it (if that wasn't proprietary itself!). Proprietary plaintext might even be better than open standard blobby format, depending on what you want to do with it. (Better for git and backup, worse for opening in competitor tools.)

Re: Smaller is better – The rise, fall, and rise of flat file software

#83

Post by irrational »

malux85 wrote: post #70
pixl97 wrote: post #62

Earlier quoted context omitted.

Portability to other systems.

Portability is more important than guaranteed atomicity, consistency, isolation and durability?

Sometimes. It really depends on the use case. Use the right tool for the job.

Re: Smaller is better – The rise, fall, and rise of flat file software

#84

Post by stjohnswarts »

Blackthorn wrote: post #78
OJFord wrote: post #50

Earlier quoted context omitted.

For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…

> I wish Fusion360 for example had a sanely git-able on-disk format like: There are two standards for this common to CAD, called IGES and STEP. They are both plaintext, though their gittability is questionable.

They are structured text though so I don't think you can just do deltas like like with code. You'd have to treat them like binaries would you not? That's the way we always kept our XML files in git forced to be binaries.

Re: Smaller is better – The rise, fall, and rise of flat file software

#85

Post by cseleborg »

hemmert wrote: post #79

I can‘t recommend Kirby enough: https://www.getkirby.com I built many of my (project) websites with it, https://www.humangodinterfaces.com https://www.perspectives-in-play.com https://www.fabianhemmert.com https://www.escape-team.com it‘s blazingly fast (performance-wise and in terms of building your site), super easy to customize and I never missed my database.

Those are some pretty interesting websites of yours! I checked them out to see if they shared some sort of common Kirby look-and-feel, but you seem to put enough emphasis on design to offset any such thing. The projects look very interesting too.

Re: Smaller is better – The rise, fall, and rise of flat file software

#86

Post by ilaksh »

hemmert wrote: post #79

I can‘t recommend Kirby enough: https://www.getkirby.com I built many of my (project) websites with it, https://www.humangodinterfaces.com https://www.perspectives-in-play.com https://www.fabianhemmert.com https://www.escape-team.com it‘s blazingly fast (performance-wise and in terms of building your site), super easy to customize and I never missed my database.

Why do I need a special API to store my data on the file system? Doesn't the file system have an API? And then my interface to that would be a lightweight layer specific to my application.

Re: Smaller is better – The rise, fall, and rise of flat file software

#87

Post by hemmert »

cseleborg wrote: post #85
hemmert wrote: post #79

I can‘t recommend Kirby enough: https://www.getkirby.com I built many of my (project) websites with it, https://www.humangodinterfaces.com https://www.perspectives-in-play.com https://www.fabianhemmert.com https://www.escape-team.com it‘s blazingly fast (performance-wise and in terms of building your site), super easy to customize and I never missed my database.

Those are some pretty interesting websites of yours! I checked them out to see if they shared some sort of common Kirby look-and-feel, but you seem to put enough emphasis on design to offset any such thing. The projects look very interesting too.

Thank you!

Kirby really makes it easy to create unique stuff, there’s also a nice „Made with Kirby“ catalog here:

https://getkirby.com/love

Re: Smaller is better – The rise, fall, and rise of flat file software

#88

Post by chme »

andrewstuart2 wrote: post #60
OJFord wrote: post #50

Earlier quoted context omitted.

For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…

Perhaps it was deliberate, but your last paragraph (plaintext vs db) really just reinforces your point of standards. There's nothing particularly special about plaintext aside from its total ubiquity. Everything today uses and understands ASCII/UTF-8 and we've used that, plus some handy human conventions like limited character count between newline characters, to build up tools like git to help manage the changes in…

> There's nothing preventing us from doing the same for other formats if they become commonly supported or expected. Much like SQLite and some of the tools cropping up.

True. I mean if it would be common to put source code and text into sqlite files, then most tools would probably also deal with it. However there is a case about simplicity. For instance writing a diff algorithm on plain text files is much simpler and easy to understand, than writing diff algorithms on text or binary structures.

Re: Smaller is better – The rise, fall, and rise of flat file software

#89

Post by EGreg »

Why don’t more people store all the uploaded files inside BLOBS in a relational database instead of a file system? What are the downsides?

I see the biggest downside being that the webserver will call your PHP app which will then proxy the data through itself one time. Well, maybe you can use ReactPHP or Swoole and then it’s faster than Node.js even…

Also you then have more programmable options, for example around access control lists or capabilities to access stuff, caching and expiring caches.

How does MySQL compre to ext3 for storing terabytes of data?

As a side note - what do y’all think of FUSE to Amazon S3? Just map a path to it and let it do the rest. GOOFYS I think skimps on the POSIX compliance in favor of speed.

Re: Smaller is better – The rise, fall, and rise of flat file software

#90

Post by hemmert »

ilaksh wrote: post #86
hemmert wrote: post #79

I can‘t recommend Kirby enough: https://www.getkirby.com I built many of my (project) websites with it, https://www.humangodinterfaces.com https://www.perspectives-in-play.com https://www.fabianhemmert.com https://www.escape-team.com it‘s blazingly fast (performance-wise and in terms of building your site), super easy to customize and I never missed my database.

Why do I need a special API to store my data on the file system? Doesn't the file system have an API? And then my interface to that would be a lightweight layer specific to my application.

Yes, exactly! Copying, renaming, backups, hierarchy - it‘s all there already!
Post reply on HN