Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Don't Scale

adam.blog.heroku.com

Post reply on HN
Print view View original

11–20 of 112 posts

Re: SQL Databases Don't Scale

#11

Post by pj »

Okay, see... here's more of the no-SQL agenda at hacker news. This isn't a question of "Does SQL Scale or not?" but "How much do SQL Databases Scale?"

I feel like the propaganda here is that because RDBMS doesn't scale to youtube or google scale they suck and that's not true. Like SQL is a waste of time because at some point, you're going to need to shard your database.

Look, at that kind of scale, you're going to have problems with any solution to any problem. Handling that kind of scale is going to be expensive no matter what solution you implement, whether it's map/reduce or flat files or some other solution.

But deciding to build a system from the beginning on something non-relational because someday you may have to accommodate that kind of scale is an example of premature optimization. The vast majority of features you get with SQL are going to outweigh the limitations of noSQL.

I've worked on some pretty high scale systems built on SQL and yes, there are problems, but there's just something irrational going on here and it's off-putting. It's like we are throwing out the baby with the bath water or something.

Re: SQL Databases Don't Scale

#12

Post by alexgartrell »

Crazy idea: Write your data-handling code in it's own tight little module. Use good abstractions so that the rest of your app doesn't give a crap HOW it's happening. When it's time to update to millions of clustered servers running bigTable, you rewrite the module, and you're done.

Software Engineering saves the day!

Re: SQL Databases Don't Scale

#13

Post by qhoxie »

bayareaguy wrote: post #10

SQL databases are fundamentally non-scalable, and there is no magical pixie dust that we, or anyone, can sprinkle on them to suddenly make them scale. I find this claim laughable. If the databases Mr. Wiggins chooses to work with fail to meet his scalability requirements, perhaps he should consider different databases. Scalable SQL databases have been around for over 3 decades.

Your comment would hold more weight if you elaborated some more. Are you referring to scaling by his standards? Please explain.

Re: SQL Databases Don't Scale

#14

Post by ojbyrne »

At a management level there are larger problems. SQL was a solution to the "letting programmers run the show" problem. I.e. a commoditized language that executives could mostly understand and budget for. See pretty well everything Philip Greenspun has written.

Re: SQL Databases Don't Scale

#15

Post by qhoxie »

pj wrote: post #11

Okay, see... here's more of the no-SQL agenda at hacker news. This isn't a question of "Does SQL Scale or not?" but "How much do SQL Databases Scale?" I feel like the propaganda here is that because RDBMS doesn't scale to youtube or google scale they suck and that's not true. Like SQL is a waste of time because at some point, you're going to need to shard your database. Look, at that kind of scale, you're going to ha…

Look, at that kind of scale, you're going to have problems with any solution to any problem. Handling that kind of scale is going to be expensive no matter what solution you implement, whether it's map/reduce or flat files or some other solution.

It is not a point of whether or not it is expensive. Scaling (nearly) always has expense associated with it. The issue is how much expense, and with some applications it is significantly less with one of the "no-SQL" solutions.

But deciding to build a system from the beginning on something non-relational because someday you may have to accommodate that kind of scale is an example of premature optimization. The vast majority of features you get with SQL are going to outweigh the limitations of noSQL.

You are making it sound as though a relational database is the correct choice barring any scaling. Perhaps you have not yet thoroughly evaluated some of the alternatives out there, because in many applications, there would be no step back from *SQL.

Re: SQL Databases Don't Scale

#17

Post by gaius »

pj wrote: post #11

Okay, see... here's more of the no-SQL agenda at hacker news. This isn't a question of "Does SQL Scale or not?" but "How much do SQL Databases Scale?" I feel like the propaganda here is that because RDBMS doesn't scale to youtube or google scale they suck and that's not true. Like SQL is a waste of time because at some point, you're going to need to shard your database. Look, at that kind of scale, you're going to ha…

As others have pointed out, the "no SQL" crowd are invariably MySQL users who have run into the limitations of MySQL but for ideological reasons can't state that the problems they encounter are specific to MySQL.

DB2, Teradata and Oracle users regularly tackle problems 100x larger than MySQL can handle.

Re: SQL Databases Don't Scale

#18

Post by bayareaguy »

qhoxie wrote: post #13
bayareaguy wrote: post #10

SQL databases are fundamentally non-scalable, and there is no magical pixie dust that we, or anyone, can sprinkle on them to suddenly make them scale. I find this claim laughable. If the databases Mr. Wiggins chooses to work with fail to meet his scalability requirements, perhaps he should consider different databases. Scalable SQL databases have been around for over 3 decades.

Your comment would hold more weight if you elaborated some more. Are you referring to scaling by his standards? Please explain.

Mr. Wiggins would have us think in terms of "true scalability" which conflates the following concerns: more servers creates more capacity (the classic definition of scalability), the business logic of the app is separated from concerns of scaling server resources (this is not possible past whatever limit you set and so has no classic definition) and no single point of failure (more classically formalized as availability).

Companies such as Teradata have long offered SQL systems which meet the classic definitions of scalability and availability.

I think TimothyFitz's reply above was accurate. Mr. Wiggins article would be better titled something like "ACID databases have scalability problems, especially cheap ones startups use" but then, like http://news.ycombinator.com/item?id=690653 , it wouldn't get much response.

Re: SQL Databases Don't Scale

#19

Post by Confusion »

TimothyFitz wrote: post #7

RAID doesn't meet his definition of "scalable" because it has a central controller. This whole post could be summed up as "ACID doesn't scale", which has been proven. Consistency, Availability or Partition Tolerance; pick two ( http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1... ). Good introduction to non-ACID databases: http://highscalability.com/drop-acid-and-think-about-data

RAID setups can be outfitted with redundant controllers, in which case it does meet his definition of 'scalable'.

Re: SQL Databases Don't Scale

#20

Post by [deleted] »

bayareaguy wrote: post #18
qhoxie wrote: post #13

Earlier quoted context omitted.

Your comment would hold more weight if you elaborated some more. Are you referring to scaling by his standards? Please explain.

Mr. Wiggins would have us think in terms of "true scalability" which conflates the following concerns: more servers creates more capacity (the classic definition of scalability), the business logic of the app is separated from concerns of scaling server resources (this is not possible past whatever limit you set and so has no classic definition) and no single point of failure (more classically formalized as availabil…

[deleted]
Post reply on HN