Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Don't Scale

adam.blog.heroku.com

Post reply on HN
Print view View original

31–40 of 112 posts

Re: SQL Databases Don't Scale

#31

Post by nettdata »

staunch wrote: post #27
nettdata wrote: post #23

I've been an Oracle database architect for almost 20 years. His whole concept of "SQL doesn't scale" is the typical crap I always hear from people that are either not database experts, are using the wrong database technologies, or don't know what they're doing. More than likely a combination of all three. And just because you can create an object model, and a simplistic data model, does not make you an architect of l…

Yeah, and if Facebook used Oracle instead of MySQL and memcached, they'd be looking at a $50 million/year+ bill no doubt. Those silly kids with their minor technologies should learn to use a real DB. If those 100 million users that login every day only knew...

Firstly, I've never said that you couldn't build big stuff with MySQL or other Open Source tech. But if you for a second think that MySQL has the same kind of performance and feature set as Oracle, you're sadly mistaken.

The example you have given has nothing to do with the technologies, and everything to do with the application requirements.

An Internet Banking site has the need for real-time, centralized, transactional updates. There is no real option to cache a lot of stuff, or to delay the distribution of updates, or to shard/replicate data for reads, etc. It also has the need for real-time transactional replication, sophisticated auditing, global fault-tolerance, etc., etc. It's also a much more transactional site than something like Facebook.

Facebook's application requirements allow for a totally different set of tools to be used in a totally different manner. The dynamics of the site also has a lot to do with how it can be built. For example, Facebook is, for the most part, a read-only, fairly static site. That makes things a HELL of a lot easier to build out with their choice of tech. Same goes for SlashDot.

There are a ton of ways to build out something to the scale of Facebook, Slashdot, or LinkedIn using nothing but open source tools/technologies.

You cannot, however, build a realistic, large scale internet banking site using those same technologies. The only option is something like Oracle.

That's why a truly scalable system is so much more than just bolting on a DB to make it scale. It's about mapping the proper technology and tools to the business requirements, and figuring out how to deal with scaling issues before you even start to write a line of code.

Or, even better, abstract the database layer (Hibernate, etc), so that you can drop in more sophisticated DB technologies as you need to.

Just because something is free (MySQL), doesn't mean it sucks. Likewise, just because something costs money (Oracle), doesn't mean it sucks.

But there are many reasons (some of them even technical) why lots of companies have no problem paying stupid money for Oracle.

Re: SQL Databases Don't Scale

#32

Post by gaius »

jmtulloss wrote: post #22
gaius wrote: post #17

Earlier quoted context omitted.

As others have pointed out, the "no SQL" crowd are invariably MySQL users who have run into the limitations of MySQL but for ideological reasons can't state that the problems they encounter are specific to MySQL. DB2, Teradata and Oracle users regularly tackle problems 100x larger than MySQL can handle.

[citation needed]

OK, try this: http://press.web.cern.ch/press/PressReleases/Releases2003/PR...

Re: SQL Databases Don't Scale

#33

Post by 4buot »

jhammerb wrote: post #30
4buot wrote: post #28

Earlier quoted context omitted.

For what its worth, Facebook does have a big Oracle install. I find it strange that everyone trotted out their preformed opinion for this piece, but no one comments on the cassandra article which actually proposes a solution...although it requires you to learn something, how horrible.

are you sure facebook has a big oracle install? how do they use it?

They use it for OLAP but are building out most of the new stuff in Hadoop,Hive because its cheaper.

Re: SQL Databases Don't Scale

#34

Post by rjurney »

nettdata wrote: post #23

I've been an Oracle database architect for almost 20 years. His whole concept of "SQL doesn't scale" is the typical crap I always hear from people that are either not database experts, are using the wrong database technologies, or don't know what they're doing. More than likely a combination of all three. And just because you can create an object model, and a simplistic data model, does not make you an architect of l…

You can only scale Oracle on Big Iron, which means it doesn't scale well because its too expensive. Cost matters.

Re: SQL Databases Don't Scale

#35

Post by [deleted] »

trezor wrote: post #26
gaius wrote: post #17

Earlier quoted context omitted.

As others have pointed out, the "no SQL" crowd are invariably MySQL users who have run into the limitations of MySQL but for ideological reasons can't state that the problems they encounter are specific to MySQL. DB2, Teradata and Oracle users regularly tackle problems 100x larger than MySQL can handle.

Agreed. I also fund it hard to sympathize with these startups having ambitions to scale to Google-like sizes, yet are unwilling to pony up cash for a proper database system like Oracle or SQL Server. Seeing the amount of ugly hacks people are willing to come up with and employ and features they are willing to cut, just to handle trivial loads, kinda makes me think that MySQL can only be considered free if your time i…

[deleted]

Re: SQL Databases Don't Scale

#36

Post by nettdata »

blasdel wrote: post #29
nettdata wrote: post #23

I've been an Oracle database architect for almost 20 years. His whole concept of "SQL doesn't scale" is the typical crap I always hear from people that are either not database experts, are using the wrong database technologies, or don't know what they're doing. More than likely a combination of all three. And just because you can create an object model, and a simplistic data model, does not make you an architect of l…

You totally ignore the background of the author -- his company Heroku : Ruby/Rack :: Google AppEngine : Python/WSGI But there's a huge problem -- AppEngine succeeds at seamless multi-tenant truly-distributed clustered hosting thanks to BigTable. Heroku needs to support standard Rails apps, so Postgres is the best they can do, and it's a huge hole in their offering. You just can't make Postgres (or Oracle) scale on an…

For what it's worth, you can most definitely do that with Oracle. I can't speak to Postgres scalability, but it is my DB of preference. (I guess it might have something to do with the fact it's as close to Oracle as you're going to get in Open Source).

We design apps to treat the datasource as a network service, and have no problems load balancing DB connections across our database cluster, and adding new DB nodes as required.

The biggest problem I've seen is the relative lack of support for Oracle in something like Rails, or non-Oracle app servers in general. I've had to custom write some connection pool failure detection for a few apps to deal with cluster failure (something that comes out of the box with the Oracle app server, but we wanted to use JBoss and GlassFish), so while it's not perfect, it's most definitely doable.

Just because his particular architecture or offering doesn't fit well with typical database deployment architectures, doesn't mean that "SQL doesn't scale".

Re: SQL Databases Don't Scale

#37

Post by rjurney »

jhammerb wrote: post #30
4buot wrote: post #28

Earlier quoted context omitted.

For what its worth, Facebook does have a big Oracle install. I find it strange that everyone trotted out their preformed opinion for this piece, but no one comments on the cassandra article which actually proposes a solution...although it requires you to learn something, how horrible.

are you sure facebook has a big oracle install? how do they use it?

It was the failure of Oracle to scale cost-effectively that led them to create HIVE.

Re: SQL Databases Don't Scale

#38

Post by rjurney »

nettdata wrote: post #25
TimothyFitz wrote: post #7

RAID doesn't meet his definition of "scalable" because it has a central controller. This whole post could be summed up as "ACID doesn't scale", which has been proven. Consistency, Availability or Partition Tolerance; pick two ( http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1... ). Good introduction to non-ACID databases: http://highscalability.com/drop-acid-and-think-about-data

Oh please. The mere fact that he's talking "RAID" instead of SAN speaks volumes. (No pun intended). Any storage engineer worth their salt would be rolling their eyes right now. The doc you linked to was authored in 2002, about the same time that Oracle's RAC was introduced (late 2001). Some of his citations are from the 80's. THE 80'S! DB technology has come a LONG way in 8 years, and that paper is no longer valid, u…

SANS are too expensive. They don't scale cost-effectively compared to commodity PC hardware.

Re: SQL Databases Don't Scale

#39

Post by nettdata »

rjurney wrote: post #34
nettdata wrote: post #23

I've been an Oracle database architect for almost 20 years. His whole concept of "SQL doesn't scale" is the typical crap I always hear from people that are either not database experts, are using the wrong database technologies, or don't know what they're doing. More than likely a combination of all three. And just because you can create an object model, and a simplistic data model, does not make you an architect of l…

You can only scale Oracle on Big Iron, which means it doesn't scale well because its too expensive. Cost matters.

Absolutely wrong.

We use Dell 2950's for our DB nodes in our Oracle clusters. Biggest one right now is 12 nodes.

And if it scales, but costs too much (a relative concept at best), then it somehow doesn't scale any more?

Of course cost matters, and it comes down to what your application requirements are, and what your revenue model is, and your risk management requirements are, etc., etc.

If you want to talk technology only, it's a no-brainer.

If you want to talk cost and context-appropriate implementation of technology, then provide a detailed context, not generalizations.

Again, you're not going to see an online bank use CouchDB as their back-end, and likewise, you're not going to see a relatively free service use stupidly expensive technology.

Re: SQL Databases Don't Scale

#40

Post by rjurney »

nettdata wrote: post #39
rjurney wrote: post #34

Earlier quoted context omitted.

You can only scale Oracle on Big Iron, which means it doesn't scale well because its too expensive. Cost matters.

Absolutely wrong. We use Dell 2950's for our DB nodes in our Oracle clusters. Biggest one right now is 12 nodes. And if it scales, but costs too much (a relative concept at best), then it somehow doesn't scale any more? Of course cost matters, and it comes down to what your application requirements are, and what your revenue model is, and your risk management requirements are, etc., etc. If you want to talk technolog…

How much data, and with a SAN? How much did the SAN cost? What kind of data?

But yes - if its too expensive to do, that means it doesn't scale well. Since when does money not matter in everything? The entire point of all of this is to use commodity PCs to achieve linear scalability cost-effectively, and to escape relational structure for data ill-suited to relational schemas.

I don't think anyone is suggesting an online bank should not use existing commercial databases and SQL.

Post reply on HN