Startup Mistakes: Choice of Datastore

Can't you just idle in a loop waiting for the lock, perhaps waiting a random number of milliseconds (200-800)?

(So that rather than crash, the client just "hangs" waiting for the lock, in a busy loop.)

It just seems like this should not be an issue.

While I'm generally sympathetic to your post, there are some things that are red flags.

>If you have ten services accessing the same database and sharing data between themselves

Whoever access the database schema owns it. If you have ten systems accessing your database then ten teams own it. And if ten teams own it then nobody owns it. Nobody can change it. Seen this at a successful start-up that got big, but then couldn't rev order management, because every team had their finger in the pie, we couldn't do a schema update without breaking everyone, and of course one team was "under tremendous pressure to hit a major milestone and we just can't do that now" for over a year.

Now I would turn this around into a win for RDBMS by suggesting the use of functions or stored procedures: with an RDBMS we can construct an API, and then we can version those APIs. And then the team that owns the database can do what they like. That said, we can do the same with NoSQL databases by not allowing other teams to access them. The team that owns the NoSQL database is required to maintain an API for it.

I've only ever had nightmares with other teams coding against my schema. GraphQL worries me in that respect and I'd love to hear how people here have fared with long lasting GraphQL, in the real world.

>Django, for example, makes migrations trivial, as you just change your application-level classes and the database gets migrated automatically.

I've had automated migration systems grind to a halt and leave the DB fucked too.

>Priscilla used a graph database when her data was relational. Her husband, furious, filed for divorce.

There are no schema that are "relational" but not "a graph". However there are plenty of schema where a graph database is a natural fit but that require either one-table-per-node-type or building a graph model on top of your RDBMS (e.g. an Entity-Attribute schema). Oy.

There are also many schema where there is only one entity type, but every join is against itself, and we're looking to join all the way out to the clique. In this case would you suggest an RDBMS, and then put the iteration in the application? You suggested earlier that making up for the inadequacies of the datastore in the application is a bad idea.

I've got a graph application that uses NoSQL and it was the right call. An RDBMS would have allowed us to write something that worked for simple cases, but that would have brought the system to its knees based on some customer usage. The solution for the RDBMS would be the same as what we had to do for NoSQL. But up to that moment, the NoSQL allowed us to iterate far faster than an RDBMS.

>For example, if you later need to compile a list of all the brands of all the products on your store, an RDBMS can easily do that by reading the “brands” table

Only if you built a brands table. You can't argue that we can't predict the future, so use an RDBS because its easy to change, but then make arguments that require that the builder accurately predicted the future and built the schema with that foresight. Sure, we could go and pull a brands table out of the existing tables but thats work, and it might be work on a live database that brings it down.

A graph database would be just as likely to have 10 brand nodes since the overhead of creating the first such node is far lower than creating an entire table and updating the schema.

>Relational databases excel at easily providing answers to questions that weren’t predicted at the time when the data model was designed.

Or a NoSQL database with spark. "But spark is something new to learn"

And this is the biggest flaw in your argument. You're pro RDBMS because you know SQL and how to run RDBMS, create the schema, and write the queries. It is incredibly easy to get started with MongoDB. That right there is why it is popular. Not because its good. But when you say "Just get something started and use an RDBMS", you're actually saying "I know you know javascript, but I need you to learn Modula-2 for this part of the system". Fundamentally different syntax and strict types (or "schema").

>For all its unparallelizability, ACID is pretty damn nice.

Until someone holds transactions open across network calls and kills throughput. I've seen that issue lose a company a multi-million dollar contract because of contention on a single row. Or until someone chooses the wrong isolation level ("But I used a transaction!") and two transactions happily decrement non-atomically. This shit be hard, and part of the "hard" is not knowing you're doing it wrong.

>You’ll have plenty of time figuring out what to use when you know your exact usage patterns, if the business manages to not die until then.

So start with something quick and easy then. You've already managed to describe two scenarios where an RDBMS blew up in practice (multiple teams hitting a DB causing schema lock; its a network, not a flat hierarchy).

If you have an experienced SQL team, by all means go with RDBMS. But lets not pretend they are a panacea. Honestly, I'd use a graph database pretty much all the time - if I could only trust them. But its the quality, reliability and longevity that I have a problem with, not with the nature of how the database organizes data.

I don't see how that would work reliably. On the client would you source CouchDB or Postgres? Presumably you'd access CouchDB directly, but then why even use Postgres (for that subset of data, anyway).