uncle roger voice just use SQLite haiyaaaaa
lol, that guys a fucking racist sponge
FUCKYY whomever downvoted me
Smaller is better – The rise, fall, and rise of flat file software
141–150 of 152 posts
Re: Smaller is better – The rise, fall, and rise of flat file software
#142Earlier quoted context omitted.
ACID comes with costs, those guys may be smart but they're not magicians. As a rule of thumb, if you never handle a transaction failure or deadlock with something that is not retrying or failing, you shouldn't be using an ACID-based system since you're not getting any benefit from it. And if you e.g. have multiple dataflow paths with different priorities then you also probably shouldn't be using a DBMS, because they'…
> As a rule of thumb, if you never handle a transaction failure or deadlock with something that is not retrying or failing, you shouldn't be using an ACID-based system since you're not getting any benefit from it. Could you expand? Does this mean you wouldn't use an ACID database for CRUD?
Depends how you're interpreting the "U" in CRUD - if you have some complex business logic to resolve conflicting updates to the same object then maybe an ACID database would make sense. But certainly if you're just doing last write wins like 99% of CRUD webapps then using an ACID database is wasteful and pointless. The whole point of ACID is to say that you need these globally isolated transactions and you're willing to pay for them, and while database engines are pretty smart with MVCC and so on there's fundamentally only so much they can do.
99% of the time all you want to do is a) commit input events to a durable store as quickly as possible b) read a state-of-the-world that's consistent in the sense that it represents all the consequences of all the inputs up until some (logical) time t, and IME an event sourcing model is if anything better at doing that than an RDBMS is.
Re: Smaller is better – The rise, fall, and rise of flat file software
#143Earlier quoted context omitted.
Some databases also have the ability to be instantly portable, like a sqlite file. You can tar up your flat files as well, but that is a separate process and can be time consuming for larger directories. Perhaps there is a use case for a file system backed by portable database like sqlite, such that you can just mount the volume and send it around without pain, but can prefer to use normal command line utils (ls, rm,…
If you're running Postgres in Docker, you can just stop the container and tar up the data volume and start it somewhere else (This also works without Docker of course, but Docker makes me feel safer when I do stuff like this)
Does this work even when the Postgres is updated to a newer version? This would make me want to use docker as well.
Re: Smaller is better – The rise, fall, and rise of flat file software
#144HN, are you feeling okay? Many top posts in the past weeks were all about using SQLite for usecases it wasnt originally designed to do, using it as a distributed database, even. Now, here's a post which misses the EXACT usecase sqlite was built for, and its barely even mentioned. SQLite is quite literally the best solution here. Its a file, you open it, and boom you got a database. You can explore it with sqlite3 cli…
> Its a file, you open it, and boom you got a database. Boom! What is the program to open (understand) it? Even my elixir's core lib cannot understand it.
Any SQLite client, which has not only been included on many devices for 20+ years, but you can also just install `sqlite3` and open it manually anyways. You also cannot open an image file without an image viewer, a pdf file without a pdf viewer, and any "hand rolled" raw text format you built yourself (if the file gets large enough) WILL be unreadable just as much.
Or any SQL DB viewer, like DataGrip.
Re: Smaller is better – The rise, fall, and rise of flat file software
#145Re: Smaller is better – The rise, fall, and rise of flat file software
#146Earlier quoted context omitted.
> Its a file, you open it, and boom you got a database. Boom! What is the program to open (understand) it? Even my elixir's core lib cannot understand it.
Any SQLite client, which has not only been included on many devices for 20+ years, but you can also just install `sqlite3` and open it manually anyways. You also cannot open an image file without an image viewer, a pdf file without a pdf viewer, and any "hand rolled" raw text format you built yourself (if the file gets large enough) WILL be unreadable just as much. Or any SQL DB viewer, like DataGrip.
There is tons of photo viewers, tons of pdf viewers, to the point the market is max saturated because it IS SO EASY to create and lots of demands drive the creation. Unlike file format for developers with sophisticated spec. You can also claim that Postgresql is just a bunch of files and folders, it's just viewer, it's just a binary code that read it blah blah.
Re: Smaller is better – The rise, fall, and rise of flat file software
#147Maybe it is just a reflection on the state of filesystems vs databases. There is no fundamental difference between a database and a flat file, it is all bytes on a disk/memory in the end. So it is mostly a question of balancing the roles of the hardware, OS, and application software. For example, if the reason you are using a database is that it does a particularly good job at limiting disk IO, then it may not be nec…
For me it's about proprietary vs. readable - unless it's obviously necessary for performance I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc. Bonus points for some open standard, but some kind of understandable format even if proprietary is miles ahead of proprietary and cryptic. I wish Fusion360 for example h…
> I think I'd still prefer 'flat file' to sqlite, but basically I just want something that isn't an incomprehensible blob, that I can use with other tools etc.
An Sqlite database may be a binary blob, but thanks to the Sqlite software being open source and ubiquitous, it's pretty easily comprehensible. Sure, not quite as accessible as plain text[1], but not far from it.
So when you need something just a little bit more advanced than plain text (i.e. SQL functionality) it seems like a good compromise that doesn't -- pun intended -- too much compromise openness and accessibility.
___
[1]: But, hey, does "plain" text include UTF-8 nowadays or does it still mean ASCII-only?
Re: Smaller is better – The rise, fall, and rise of flat file software
#148Earlier quoted context omitted.
Any SQLite client, which has not only been included on many devices for 20+ years, but you can also just install `sqlite3` and open it manually anyways. You also cannot open an image file without an image viewer, a pdf file without a pdf viewer, and any "hand rolled" raw text format you built yourself (if the file gets large enough) WILL be unreadable just as much. Or any SQL DB viewer, like DataGrip.
There is tons of photo viewers, tons of pdf viewers, to the point the market is max saturated because it IS SO EASY to create and lots of demands drive the creation. Unlike file format for developers with sophisticated spec. You can also claim that Postgresql is just a bunch of files and folders, it's just viewer, it's just a binary code that read it blah blah.
> There is tons of photo viewers, tons of pdf viewers, to the point the market is max saturated because it IS SO EASY to create and lots of demands drive the creation.
And guess what the market situation for database viewers is like...?
Re: Smaller is better – The rise, fall, and rise of flat file software
#149Is sqlite a flat file disqualifier?
no, SQLite was built to be a flat file db. Its quite exactly the best solution if you need a db but want to have it contained in a folder with your other files.
But you can't just put
...tp://my_site/some_dir/any_sqlite_file.db
So in the sense of "put up a website by just transferring static files", an Sqlite database isn't quite "a flat file". It's a single file, yes; but its content is "bumpy" -- not "flat".
(Then again, once you have that translate-content-to-HTML process installed on your Web server, updating the content is reduced to just transferring a new .db file to it.)
Re: Smaller is better – The rise, fall, and rise of flat file software
#150Is sqlite a flat file disqualifier?
I would say yes, that's a disqualification: It's a single file, but not "flat" in the sense of directly human- (or browser-) readable.