41–50 of 95 posts
In other words: could a database have only minimal built-in caching and instead rely on the OS cache?
This is slightly OT, but I always wondered: what is the overhead of fetching cached stuff from the file system vs. having a built-in cache? Is there a way to circumvent going through the kernel and avoid a context switch? In other words: could a database have only minimal built-in caching and instead rely on the OS cache?
Earlier quoted context omitted.
That's a very obtuse point of view. I'm curious sociologically: what field do you work in and what your exposure to data is? Consider an inventory system for a big box retailer. I can't think of anything better than a fat-ass RDBMS as the primary data store. Sharding sounds like a horrific idea. There are myriad workloads like this. Personally, I've seen pgsql handle terabytes of data just fine and it wasn't really n…
Well, yes and no. Everybody knows that relational databases don't scale because they use JOINs and write to disk. Also, relational databases weren't built for web scale. MongoDB handles web scale. You turn it on and it scales right up. And before you knock shards, shards are the secret ingredient in the web scale sauce. They just work. Furthermore, relational databases have impetus mismatch, and Postgresql is slow as…
I love that infinitely nerdy stuff like this can still make it to the HN homepage (there's still hope!). Stuff like this really needs to make it into the PG tuning guide ( https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serv... ). The only place where it will ultimately be seen by a worthwhile audience.
Since when is "in depth" and "technical" equivalent to "infinitely nerdy"? We're professionals using that kind of information for work, not nerds doing infinitely nerdy stuff.
Earlier quoted context omitted.
Well, yes and no. Everybody knows that relational databases don't scale because they use JOINs and write to disk. Also, relational databases weren't built for web scale. MongoDB handles web scale. You turn it on and it scales right up. And before you knock shards, shards are the secret ingredient in the web scale sauce. They just work. Furthermore, relational databases have impetus mismatch, and Postgresql is slow as…
Are you being intentionally sarcastic? Because this reads a lot like http://www.mongodb-is-web-scale.com/ Edit: Whoops, just read your reply :)
http://en.wikipedia.org/wiki/Poe's_law
"without a clear indication of the author's intent, it is difficult or impossible to tell the difference between an expression of sincere extremism and a parody of extremism"
Earlier quoted context omitted.
That's a very obtuse point of view. I'm curious sociologically: what field do you work in and what your exposure to data is? Consider an inventory system for a big box retailer. I can't think of anything better than a fat-ass RDBMS as the primary data store. Sharding sounds like a horrific idea. There are myriad workloads like this. Personally, I've seen pgsql handle terabytes of data just fine and it wasn't really n…
Well, yes and no. Everybody knows that relational databases don't scale because they use JOINs and write to disk. Also, relational databases weren't built for web scale. MongoDB handles web scale. You turn it on and it scales right up. And before you knock shards, shards are the secret ingredient in the web scale sauce. They just work. Furthermore, relational databases have impetus mismatch, and Postgresql is slow as…
2: Not sure, this may be specific to FS, or something that has to do with the behaviour of MMAPed files however I don't know how do you guarantee that what you're writing corresponds to a single block in the FS (unless you're writing directly to /dev/sda and even then)
Earlier quoted context omitted.
FreeBSD has something like SHMMAXPGS, SHMMAX in postgresql.conf in the sysctl.conf kern.ipc.shmall kern.ipc.shmall
I don't think this has any direct influence on performance. It was necessary to increase the default limits for larger shared_buffers, etc on pgsql prior to 9.3 where SysV shared memory was used, but this was common to many *nix operating systems.
Earlier quoted context omitted.
Check this "web scale" out: http://smalldatum.blogspot.com (via http://dom.as/2014/03/31/mongo-io/ )
I was actually referring to this: http://www.mongodb-is-web-scale.com/ - which is what contingencies' comments reminded me of, but I guess people either didn't get it or thought it was a bit stale. C'est la vie. I am a happy Postgres user and always default to it unless I am really sure a project calls for something else.
And same here, I tell people to start their datastore selection with looking for a reason NOT to use Postgres.
Not to detract from the very intelligent and reasoned posting, but what tiny percentage of people honestly still use fat-ass RDBMS as their primary datastore and would be better off performance tuning it at the kernel IO level than actually analyzing their load and subsequently sharding or migrating their data structures to less behemoth-like datastores? Yes , RDBMS are easy to hire developers and DBAs for, are well…
IN DEFENSE OF BEING LAZY AS A PROGRAMMER
The essential mission of a computer programmer is to use computers to solve problems. Being lazy can come in one of two forms:
1) Solving problems badly or not solving them at all, or 2) Relying on someone else's solution instead of coming up with your own.
Using a RDBMS is Type-2 Lazy. Now, I want you to get out a pen and paper and write this next bit down, because it is the most important thing you will ever learn:
EVERYBODY SHOULD BE TYPE-2 LAZY BY DEFAULT, ONLY DEVIATING FROM THIS IF THERE IS A COMPELLING REASON NOT TO.
Why?
1) Other people's solutions have been used, which means they've been tested in real-world use. Things you haven't thought of yet because you don't yet have a working solution have been at least discovered, because people are using it. Sometimes they're even addressed. 2) Other people's solutions may have tools, documentation and communities built around them, making them easier to learn about, use and work with.
There are two decades of work put into Postgres itself, and even longer periods of work put into the general field of relational databases. Corner cases you can't even conceive of have been encountered and patched for. The entire codebase of Postgres contains large amounts of accumulated wisdom on how to store data in a safe and retrievable fashion. And large communities have sprung up, to provide you with tools and wisdom on how to use it to best suit your needs.
NoSQL databases are useful for certain workloads and setups. It would be absolutely wrong to dismiss them out of hand. Having said that, anyone whose DEFAULT PREFERENCE is to eschew traditional RDBMS as a data store in favor of software that has been around for less than a quarter of the time that even the newer of the popular RDBMS systems have been around because using well-tested solutions is LAZY needs to have a restraining order keeping them at least 100 yards away from a keyboard.