Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Are An Overapplied Solution (And What To Use Instead)

adamblog.heroku.com

Post reply on HN
Print view View original

11–20 of 67 posts

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#11

Post by marshallp »

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on.

SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far.

The only limitation of relational databases currently is their lack of automatic infinite horizontal scaling on commodity servers, but hopefully someone will solve that soon.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#12

Post by peterwwillis »

marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i think that means basically implementing the equivalent of an 'intelligent' distributed parallel fault-tolerant file system. at that point i figure you could add "lots of glue" (heh, understatement?) to MemcacheDB, MemcacheD, MemcacheQ and some other crap and get something similar. but i could be crazy.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#13

Post by jazzychad »

simonw wrote: post #10

I liked the definition of "transient data" - I've been promoting Redis for that kind of thing but I didn't have the vocabulary to explain what I meant (I've been staying "stuff like stats and counters"). Defining transient data as being frequently written, infrequently read is useful too.

I have been describing this type of data as "high velocity" to my friends, and they seem to get it.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#14

Post by marshallp »

peterwwillis wrote: post #12
marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i thi…

The google appengine datastore is moving in the direction of being a full sql database, and I bet oracle have people working on it (because appengine, salesforce and aws is a threat to their business), so I don't think it's all that far-fetched.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#15

Post by jmm »

I think more specific context on "SQL databases don't scale" might be appropriate here, especially for the noob hackers that are figuring out just which technology to use in building their first or second apps. Is it worth ignoring defaults and conventions on a simple Rails app and not use an SQL based db? Probably not? And I reckon that Heroku itself hosts a good deal of apps where "will this db scale?" isn't really relevant. I could be wrong.

Another question worth knowing the answer to is how much of hassle it is to switch horses midstream (from a well normalized SQL db to something else), after there are some data in the tanks.

Maybe this is Adam's recommendation specifically to the dreaming-big community, which I can certainly appreciate. And maybe everyone should be dreaming big.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#16

Post by wvenable »

There's definitely a lack of imagination when it comes to proponents of NoSQL solutions, and this article shows no exception. While we do want to store, for example, an entire e-commerce order in a single operation we also don't necessarily want to retrieve it that way. That type of storage makes otherwise simple operations considerably more difficult. Do you want to know how much you made in sales today? How many of widget #453 are still in stock? The most popular items for sale? The least popular? Start writing code.

The other example of a user profile seems to be the perfect fit for storage in a single table, so I don't why that's there. Now, do you want to know how many users logged in this week? How about you want to delete every account that hasn't been accessed in a year? Much more slow code.

Every article about NoSQL goes on and on about the supposed advantages, but rarely talks about the considerable disadvantages. And honestly, for most people, the advantages are simply not worth the trade off. I fear that we'll have decades of e-commerce stores written with document stores and mountains of code slowly chugging away to calculate the basic stats that any business needs.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#17

Post by wvenable »

From the article, on where relational databases are appropriate:

> Small records with complex, well-defined, highly normalized relationships.

Why do the records need to be small? And honest, in software development, a large amount of your data is going to be well-defined and easily normalized. The author provided 2 examples that would fit perfectly in a relational database.

> The type of queries you will be running on the data is largely unknown up front.

Or the types of queries you are running are more than just retrieving a single record or simple list of records. I'm afraid a very large number of queries fall into this category.

> The data is long-lived: it will be written once, updated infrequently relative to the number of reads/queries, and deleted either never, or many years in the future.

Yes, a relational databases are for storing long-lived data. For temporary data, you could use an in-memory table or just some other solution entirely. There's no need to mix your permanent data with your temporary data. Databases handle writes and deletes extremely well (in bulk even) so I'm not sure what the author was getting at here.

> The database does not need round-the-clock availability (middle-of-the-night maintenance windows are no problem).

What kind of middle-of-the-night maintenance does a relational database need? I've been running at least one database for several years straight without any downtime or maintenance.

> Queries do not need to be particularly fast, they just need to return correct results eventually.

Relational database queries aren't particularly slow -- in fact, RBMS are heavily optimized to return data very quickly. In the vast majority of cases, this is going to be more than fast enough for nearly every application.

> Data integrity is 100% paramount, trumping all other concerns, such as performance and scalability.

Damn straight. I want the data coming from my data store to be 100% correct always. If I need to trade performance for correctness then I can easily add some caching. But I'm not sure how document stores would solve this any differently.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#18

Post by blasdel »

Consider the source: Heroku needs an alternative to Postgres to be a true competitor to AppEngine.

There's something that never gets brought up in these NoSQL discussions: SQL Databases don't scale down. They aren't very good in multitenant situations where you have a lot of random small-fry users -- you end up just sharding the users across a bunch of different master-slave pairs, and hope that they don't step on each other's toes. Because they take up real resources even if not being used, it's difficult to pull off a freemium model.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#19

Post by cracell »

w3matter wrote: post #6

Yeah, Heroku should put up or shut up until they start offering alternate NoSQL databases (and no, MongoHQ is not an alternative because of network latency). Interesting article though.

As they are hosted on EC2 couldn't you use Amazon Cloudfront with them without too much trouble and with no latency issues? http://aws.amazon.com/cloudfront/

The main issue would be integrating Cloudfront with Rails not getting it to run on Heroku. (A quick search doesn't show much support for using Cloudfront with Rails in general)

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#20

Post by silentbicycle »

blasdel wrote: post #18

Consider the source: Heroku needs an alternative to Postgres to be a true competitor to AppEngine. There's something that never gets brought up in these NoSQL discussions: SQL Databases don't scale down . They aren't very good in multitenant situations where you have a lot of random small-fry users -- you end up just sharding the users across a bunch of different master-slave pairs, and hope that they don't step on e…

SQLite scales down fine, FWIW. It's just not appropriate past a certain amount of concurrent writes, but it's great for prototyping and smaller services.
Post reply on HN