Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Are An Overapplied Solution (And What To Use Instead)

adamblog.heroku.com

Post reply on HN
Print view View original

61–67 of 67 posts

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#61

Post by Vitaly »

marshallp wrote: post #42
cduan wrote: post #37

Earlier quoted context omitted.

Well, is this a complaint about relational SQL databases? The only issues the author has that actually are tied to the SQL language are #2 (queries are largely unknown up front) and possibly #1 (small records with normalized relationships). Most of the other complaints have to do with ACID data consistency and the overhead it imposes in simple implementations (e.g. no row-level locking).

By leaving sql you're foregoing the ease of development aspects that it has. It's like leaving python to program in c. Memory is cheap these days, for anything but the largest websites it makes more sense to use a relational database with all data in memory (and this will be as performant as any other solution you create).

almost exactly the opposite. its much easier to just get all my data related to some 'entity' in some place then to have to gather it over a half a dozen columns with a weird syntax (I know SQL very well, I just hate it with a passion :)

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#62

Post by viraptor »

wvenable wrote: post #56
viraptor wrote: post #52

Earlier quoted context omitted.

> If you're iterating every order in the database to apply your filter I'll stop you there - filters can use database indexes. Exactly the same way SQL databases do it. Database driver chooses how it fetches those results. Getting the records based on some criteria and indexed attributes does not iterate through every record.

Yes, I did assume that was the case. However, once you've started adding indexes aren't you already losing some of the advantage of NoSQL. An index is a well defined set of columns. In your "Most popular" example, you're still iterating records locally (and sorting locally) to calculate the results rather than having it done on the server. You've also presupposed that you have normalized data -- which in authors exam…

The only thing you lose by adding indexes is insert speed. Indexes in most document stores don't change anything apart from shifting the get/put time balance. They don't define anything and can be often created / dropped / recreated without affecting the whole application.

As mentioned before - I could put the "most popular" script on the server and request only the result if you wanted. TT can do that for sure and I remember reading about another solution that may (redis I think? I may be wrong here).

About the normalised or not data - you have to design your data storage to your solution and needs. If you made something hard to query, maybe it's time to redesign it, or add another denormalised storage, or keep aggregate copies, or use some other solutions... Same applies to rdbmses really.

You know how to run your queries on your DB efficiently (or you should start learning now). Sure - exploratory searching for patterns, custom reports, etc. - that's where SQL is great. If you need it, either you should have additional "archives" in some better data store (sql-capable column-based store?), or simply use rdbms for everything. Over-simplification is as bad as over-generalising ;) just use what you need, instead of what is popular.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#63

Post by wvenable »

viraptor wrote: post #62
wvenable wrote: post #56

Earlier quoted context omitted.

Yes, I did assume that was the case. However, once you've started adding indexes aren't you already losing some of the advantage of NoSQL. An index is a well defined set of columns. In your "Most popular" example, you're still iterating records locally (and sorting locally) to calculate the results rather than having it done on the server. You've also presupposed that you have normalized data -- which in authors exam…

The only thing you lose by adding indexes is insert speed. Indexes in most document stores don't change anything apart from shifting the get/put time balance. They don't define anything and can be often created / dropped / recreated without affecting the whole application. As mentioned before - I could put the "most popular" script on the server and request only the result if you wanted. TT can do that for sure and I…

> About the normalised or not data - you have to design your data storage to your solution and needs.

I understand your point. But once you have normalized data, don't you lose most/all the advantages of a NoSQL solution? RDBMS's are designed to handle the searching, joining, and filtering of normalized data -- NoSQL solutions are not.

My original comment about "lack of imagination" applies -- If you're designed your data to handle as many scenarios as possible (which obviously the author of the article did not) then you will end up with something that is fully normalized.

If you need performance, whether it be in an RDBMS or not, you'll need to sacrifice some flexibility. But it seems like with NoSQL you're prematurely optimizing -- picking a solution with a very specific use-case when you're most likely not going to need any of the supposed advantages.

I'm not suggesting that NoSQL doesn't have it's use, but most of these articles advocate using it by default and in situations where it's clearly not appropriate. The author of this article, using it for e-commerce orders, seems like a very inappropriate use to me.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#64

Post by benwyrosdick »

w3matter wrote: post #6

Yeah, Heroku should put up or shut up until they start offering alternate NoSQL databases (and no, MongoHQ is not an alternative because of network latency). Interesting article though.

If Heroku offers their own NoSQL alternative, they will put it on a different EC2 instance and have the same network latency so MongoHQ seems like an equal solution. They provide private IP addresses to connect to so it is the same as running it on your own separate EC2 instance. How does anyone separate their app and db server without creating latency in your opinion?

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#65

Post by peterwwillis »

dangrossman wrote: post #24
peterwwillis wrote: post #8

There is no explanation of why a filesystem is a bad place to store "binary blobs". If you're collapsing the metrics that you're storing IN SQL there is something really wrong going on. Logs are OK to store in SQL, assuming you're scraping your logs properly and are logging the proper things. Logging every clickthrough in a relational database is somewhat insane. Logging 10 minutes worth of aggregate clickthroughs is…

Meh, I record a couple million page views to a MySQL database (as a new row for every page view) every day, for 6 years. Works fine. I don't expect a million new users to show up any time soon.

six billion five hundred seventy million rows?

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#66

Post by dangrossman »

peterwwillis wrote: post #65
dangrossman wrote: post #24

Earlier quoted context omitted.

Meh, I record a couple million page views to a MySQL database (as a new row for every page view) every day, for 6 years. Works fine. I don't expect a million new users to show up any time soon.

six billion five hundred seventy million rows?

Yes, I do actually have billions of rows. See http://www.w3counter.com

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#67

Post by peterwwillis »

peterwwillis wrote: post #12
marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i thi…

http://ostatic.com/blog/a-bright-future-for-drizzle

I guess this is old news but this is basically a much moreand flexible form of what I was thinking of (I was only considering the infrastructure/fault-tolerant portion of an RDBMS). You could definitely design a well-scaling horizontal cluster using this design; even if the stock components don't support what you need you just modify stuff outside the microkernel.

Post reply on HN