Well, intro was promising and full of energy. Like "I am really going to transmit you some good love for databases and explain why it's so stimulating to deal with them!", but then right after this sparkling start it's all just about same old redundancy, consistency, scaling...
Why I love databases
21–30 of 172 posts
Re: Why I love databases
#22What I find both fascinating and scary about databases is how to choose between the wide variety of databases without understanding exactly how they work? And it doesn't help that there are new databases springing up all the time.
Is there a way for application developers to understand these databases quickly without spending weeks working with them?
Re: Why I love databases
#23"The study of databases intersects almost every topic in computer science" - I've heard this before, especially for Compilers. But it has been false for a long time, CS is far more diverse now. For example, how do Databases intersect AI/Machine Learning/Computer Vision, Computer Graphics, Numerics/Simulation, Robotics, Bioinformatics, Computer Architecture or Cryptography?
> how do Databases intersect AI/Machine Learning I am not in any way a computer-science expert, but surely these topics have substantial overlap? (For example, a machine that is learning must store its accumulated knowledge somewhere.)
There is data mining, which is like machine learning + databases. But for many, many machine learning problems, the learning occurs offline and the data is simply stored in files, then the learning algorithm is run. The result is a model, which in many cases has a small constant representation (common exceptions: kNN and SVM). There is no database involved. Now you can of course try to learn from an existing database (data mining) or from huge amounts of data, probably stored in a distributed database (big data). But many systems will not and even then the database part is usually not of much concern. So I still don't see much overlap.
Re: Why I love databases
#24I wonder why RethinkDB hasn't gotten a mention here. Have folks here used it? Thoughts?
Re: Why I love databases
#25What I find both fascinating and scary about databases is how to choose between the wide variety of databases without understanding exactly how they work? And it doesn't help that there are new databases springing up all the time. Is there a way for application developers to understand these databases quickly without spending weeks working with them?
The simple answer is to use PostgreSQL, unless you have a good reason not to.
There are definitely cases where PostgreSQL is not the optimal choice, but if you don't know why you're not a good match for Postgres, you're probably not in that group.
Re: Why I love databases
#26"Designing Data-Intensive Applications" is shaping up to be an excellent treatement of modern databases and their underpinnings. It's at an excellent level of abstraction, deep enough to convey database internals while high level enough (so far at least) to be able to cover a wide variety of database systems. It also has its feet firmly planted in database history, and is NoSQL-koolaid free. Highly recommended. http:…
When you say "NoSQL-Koolaid free", do you mean "focuses on Relational" or "treats the topic objectively/academically"?
Not OP, but the latter. The author talks both about data models (relational, document, graph-based, with some nice historical tidbits about IMS and CODASYL) and storage models (from B-trees to SSTables, passing through bitmap encoding and everything else).
It is an amazing book so far, and I'm pretty excited for the rest of it to be written. It's the best text I've read so far on databases that gives all the references you need if you want to go deeper, but still allows you to get a very good higher-level understanding without it.
Re: Why I love databases
#27What I find both fascinating and scary about databases is how to choose between the wide variety of databases without understanding exactly how they work? And it doesn't help that there are new databases springing up all the time. Is there a way for application developers to understand these databases quickly without spending weeks working with them?
The simple answer is to use PostgreSQL, unless you have a good reason not to. There are definitely cases where PostgreSQL is not the optimal choice, but if you don't know why you're not a good match for Postgres, you're probably not in that group.
Postgres, and MSSQL if you can afford it, are the Swiss Army Knives of database systems. And I mean that in a good way.
Re: Why I love databases
#28I hate databases. People tend to have way too much faith in them (or their surrounding marketing), and thus make poor database choices that don't actually fit the shape of their data. Persistence is fundamentally the programmer's responsibility; a magic box behind a socket can't design it for you.
Most applications I've seen wouldn't even need a database, but apparently a lot of programmers are conditioned into believing that writing anything to disk must involve building a database query string and transmitting it over a socket to another process which parses the string, executes it on an interpreter and stuffs the extracted data into a generic 1970s data model that finally gets written to disk in an opaque format from where it can only be retrieved by sending more strings over sockets. This stuff made sense when 1MB was a huge amount of memory, but today it's just not necessary.
Re: Why I love databases
#29"Designing Data-Intensive Applications" is shaping up to be an excellent treatement of modern databases and their underpinnings. It's at an excellent level of abstraction, deep enough to convey database internals while high level enough (so far at least) to be able to cover a wide variety of database systems. It also has its feet firmly planted in database history, and is NoSQL-koolaid free. Highly recommended. http:…
When you say "NoSQL-Koolaid free", do you mean "focuses on Relational" or "treats the topic objectively/academically"?
It covers modern NoSQL databases but puts them in their proper historical context by comparing them to the (largely rejected) hierarchical database systems. (At least when we're talking about things like 'document stores.')
Re: Why I love databases
#30I've always loved databases, but after having discovered write-only timestamped databases like Datomic, I can't imagine going backwards. It's a real shame that Datomic isn't fully open source. (Aren't BigTable and Spanner also write-only and timestamped?)
Ok, assuming write-only means what I think it does, what is the point in a write only database? /dev/null/ is write only.
Append-only means that every fact is timestamped and nothing is ever thrown away. (Actually, Datomic provides excision, because some regulatory use cases require it, but it's a "black magic" feature.) The idea is that you should never use UPDATE or DELETE, unless it's a hard requirement or a well-studied optimization. The fact that it's 58 degrees today doesn't invalidate the fact that it was 45 degrees yesterday. New facts may replace old ones relative to time-sensitive queries ("what is the most recent temperature?") but you never get rid of the old ones.
This means that queries often have an additional parameter, which is the time. So a type signature for a query type (in a Haskell-like language) would be:
type Query a = ([Fact], Time) -> a