Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Don't Scale

adam.blog.heroku.com

Post reply on HN
Print view View original

41–50 of 112 posts

Re: SQL Databases Don't Scale

#41

Post by derefr »

nettdata wrote: post #25
TimothyFitz wrote: post #7

RAID doesn't meet his definition of "scalable" because it has a central controller. This whole post could be summed up as "ACID doesn't scale", which has been proven. Consistency, Availability or Partition Tolerance; pick two ( http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1... ). Good introduction to non-ACID databases: http://highscalability.com/drop-acid-and-think-about-data

Oh please. The mere fact that he's talking "RAID" instead of SAN speaks volumes. (No pun intended). Any storage engineer worth their salt would be rolling their eyes right now. The doc you linked to was authored in 2002, about the same time that Oracle's RAC was introduced (late 2001). Some of his citations are from the 80's. THE 80'S! DB technology has come a LONG way in 8 years, and that paper is no longer valid, u…

Why are there no open-source implementations of the "proper DB technologies", then?

Re: SQL Databases Don't Scale

#42

Post by nostrademons »

nettdata wrote: post #31
staunch wrote: post #27

Earlier quoted context omitted.

Yeah, and if Facebook used Oracle instead of MySQL and memcached, they'd be looking at a $50 million/year+ bill no doubt. Those silly kids with their minor technologies should learn to use a real DB. If those 100 million users that login every day only knew...

Firstly, I've never said that you couldn't build big stuff with MySQL or other Open Source tech. But if you for a second think that MySQL has the same kind of performance and feature set as Oracle, you're sadly mistaken. The example you have given has nothing to do with the technologies, and everything to do with the application requirements. An Internet Banking site has the need for real-time, centralized, transacti…

"For example, Facebook is, for the most part, a read-only, fairly static site."

FaceBook's actually one of the more dynamic sites out there. I'd bet that the average FaceBook user makes many more updates to their FaceBook profile than they make financial transactions.

The real difference in requirements is that FaceBook can - and does - drop updates on the floor. If your friend throws a sheep at you and you don't get it on your news feed - oh well, FaceBook never claims the news feed is exhaustive anyway. But if somebody writes you a check for $10,000 and they see that you cashed it and you don't see it in your account, that's a problem.

Re: SQL Databases Don't Scale

#43

Post by nostrademons »

derefr wrote: post #41
nettdata wrote: post #25

Earlier quoted context omitted.

Oh please. The mere fact that he's talking "RAID" instead of SAN speaks volumes. (No pun intended). Any storage engineer worth their salt would be rolling their eyes right now. The doc you linked to was authored in 2002, about the same time that Oracle's RAC was introduced (late 2001). Some of his citations are from the 80's. THE 80'S! DB technology has come a LONG way in 8 years, and that paper is no longer valid, u…

Why are there no open-source implementations of the "proper DB technologies", then?

People can still make money charging for them?

Open-sourcing is the last step in the technology lifecycle, after the technology has become widely understood and commoditized. When people can still make money off something, they will. What's the incentive for them to give it away?

Re: SQL Databases Don't Scale

#44

Post by nettdata »

rjurney wrote: post #38
nettdata wrote: post #25

Earlier quoted context omitted.

Oh please. The mere fact that he's talking "RAID" instead of SAN speaks volumes. (No pun intended). Any storage engineer worth their salt would be rolling their eyes right now. The doc you linked to was authored in 2002, about the same time that Oracle's RAC was introduced (late 2001). Some of his citations are from the 80's. THE 80'S! DB technology has come a LONG way in 8 years, and that paper is no longer valid, u…

SANS are too expensive. They don't scale cost-effectively compared to commodity PC hardware.

I'd beg to differ.

Go take a look at Adam Leventhal's work with the Fishworks stuff.

Specifically, go check out the Sun Storage 7310.

It will scale HUGE, and is nowhere near the stupid cost of NetApp or EMC or the other major vendors.

I'd love to see how a bunch of commodity PC's will scale to 100TB, and still be manageable, and have anywhere near the same feature sets.

Again, if you're railing against something as ubiquitous as a SAN, I'm not sure there's anything I can say to change your mind.

Not that I'm really here to change your mind.

Again, the original focus was about SQL not scaling, and you seem to be fixated on the cost of that scaling.

Re: SQL Databases Don't Scale

#45

Post by nettdata »

nostrademons wrote: post #42
nettdata wrote: post #31

Earlier quoted context omitted.

Firstly, I've never said that you couldn't build big stuff with MySQL or other Open Source tech. But if you for a second think that MySQL has the same kind of performance and feature set as Oracle, you're sadly mistaken. The example you have given has nothing to do with the technologies, and everything to do with the application requirements. An Internet Banking site has the need for real-time, centralized, transacti…

"For example, Facebook is, for the most part, a read-only, fairly static site." FaceBook's actually one of the more dynamic sites out there. I'd bet that the average FaceBook user makes many more updates to their FaceBook profile than they make financial transactions. The real difference in requirements is that FaceBook can - and does - drop updates on the floor. If your friend throws a sheep at you and you don't get…

I agree... never meant to imply that Facebook had fewer hits/etc than a banking site in raw numbers, just that the ratio of DB reads (not considered a transaction, very easy to cache, etc) to DB writes, makes the site much more read-only than write-only.

Facebook has the ability to do a TON of edge caching, with very few (relatively speaking) operations having to go to the database to perform an actual write operation.

It's the DB writes that really kill performance and limit your caching strategies and gains.

Re: SQL Databases Don't Scale

#46

Post by derefr »

nostrademons wrote: post #43
derefr wrote: post #41

Earlier quoted context omitted.

Why are there no open-source implementations of the "proper DB technologies", then?

People can still make money charging for them? Open-sourcing is the last step in the technology lifecycle, after the technology has become widely understood and commoditized. When people can still make money off something, they will. What's the incentive for them to give it away?

Usually, an ideological one. The first group of Linux kernel developers (past the "toy" stage) all hated Microsoft with a passion, and wished to deprive it of as much revenue as possible. They wanted to give people a "free alternative", and lower the total investment people [that is, they] would have to make in owning a computer. I could imagine an analogous situation with some developers and Oracle.

Re: SQL Databases Don't Scale

#47

Post by alexgartrell »

alexgartrell wrote: post #12

Crazy idea: Write your data-handling code in it's own tight little module. Use good abstractions so that the rest of your app doesn't give a crap HOW it's happening. When it's time to update to millions of clustered servers running bigTable, you rewrite the module, and you're done. Software Engineering saves the day!

3 upvotes and 2 downvotes without a single response.

As a junior member of Hacker News, I demand an education when I'm downvoted! :)

Re: SQL Databases Don't Scale

#48

Post by gaius »

trezor wrote: post #26
gaius wrote: post #17

Earlier quoted context omitted.

As others have pointed out, the "no SQL" crowd are invariably MySQL users who have run into the limitations of MySQL but for ideological reasons can't state that the problems they encounter are specific to MySQL. DB2, Teradata and Oracle users regularly tackle problems 100x larger than MySQL can handle.

Agreed. I also fund it hard to sympathize with these startups having ambitions to scale to Google-like sizes, yet are unwilling to pony up cash for a proper database system like Oracle or SQL Server. Seeing the amount of ugly hacks people are willing to come up with and employ and features they are willing to cut, just to handle trivial loads, kinda makes me think that MySQL can only be considered free if your time i…

It's amusing that he thinks master-slave replication is MySQL's killer feature. Didn't IBM have it, umm, 30 years ago?

Also that he says SQL database and not relational database. The whole article reeks of inexperience.

Re: SQL Databases Don't Scale

#50

Post by bayareaguy »

rjurney wrote: post #37
jhammerb wrote: post #30

Earlier quoted context omitted.

are you sure facebook has a big oracle install? how do they use it?

It was the failure of Oracle to scale cost-effectively that led them to create HIVE.

You can't be talking about Hive[1], the SQL system for structuring and quering Hadoop datasets. That would never scale...

1- http://hadoop.apache.org/hive/

Post reply on HN