Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Are An Overapplied Solution (And What To Use Instead)

adamblog.heroku.com

Post reply on HN
Print view View original

21–30 of 67 posts

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#21

Post by viraptor »

wvenable wrote: post #16

There's definitely a lack of imagination when it comes to proponents of NoSQL solutions, and this article shows no exception. While we do want to store, for example, an entire e-commerce order in a single operation we also don't necessarily want to retrieve it that way. That type of storage makes otherwise simple operations considerably more difficult. Do you want to know how much you made in sales today? How many of…

Did you use any NoSQL solution seriously? Let's see some examples (imaginary python-like interface, so I don't have to be language/backend-specific). Like you say: Start writing code.

Do you want to know how much you made in sales today?

    sum(amount in db.filter(type='order', date=xxx))
    -vs-
    SELECT SUM(amount) FROM order WHERE date=?
How many of widget #453 are still in stock? (IRL it's never that simple, but...)

    db.filter(type='stock_widget', part_id=453)['amount']
    -vs-
    SELECT amount FROM storage WHERE id = 'widget 453'
Most popular:

    for r in db.filter(type='item', date=xxx): histogram[r['part_id']] += 1
    histogram.sort_value()[0].key()
What I wanted to show is - you're writing the same amount of code for both cases. In some databases (like Tyrant) you can also run the script server-side and just report the result if you prefer. Also depending on the database, you don't need to read the whole record every time - you can just request a list of fields in most of them.

Have some fun with a NoSQL database before rejecting it for reasons like the ones you mentioned... It's also not always about processing speed - I could use either solution, but coding for TT is just simpler than for any SQL in most of what I do (see how my db.filter examples give you the solution in the current language, but queries are just... queries that you have to run and retrieve results (I'm ignoring SQL-LINQ now)).

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#22

Post by blasdel »

silentbicycle wrote: post #20
blasdel wrote: post #18

Consider the source: Heroku needs an alternative to Postgres to be a true competitor to AppEngine. There's something that never gets brought up in these NoSQL discussions: SQL Databases don't scale down . They aren't very good in multitenant situations where you have a lot of random small-fry users -- you end up just sharding the users across a bunch of different master-slave pairs, and hope that they don't step on e…

SQLite scales down fine, FWIW. It's just not appropriate past a certain amount of concurrent writes, but it's great for prototyping and smaller services.

That is a clever idea I'd not thought of, but it isn't really multitenant -- it just pushes the problem down into the filesystem.

It also requires a discontinuous transition to a different SQL database once you graduate from being a small-fry, and from there you're in the same boat as everyone else trying to scale that to multiple machines without application changes.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#23

Post by gnaritas »

marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

And you can easily and incrementally evolve your data model in an object database too with the added benefit that the application is updated to match the new data model at the same time.

> SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model

And horrible and brittle in terms of the application model, which in 99% of cases is not relational.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#24

Post by dangrossman »

peterwwillis wrote: post #8

There is no explanation of why a filesystem is a bad place to store "binary blobs". If you're collapsing the metrics that you're storing IN SQL there is something really wrong going on. Logs are OK to store in SQL, assuming you're scraping your logs properly and are logging the proper things. Logging every clickthrough in a relational database is somewhat insane. Logging 10 minutes worth of aggregate clickthroughs is…

Meh, I record a couple million page views to a MySQL database (as a new row for every page view) every day, for 6 years. Works fine. I don't expect a million new users to show up any time soon.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#25

Post by Retric »

wvenable wrote: post #17

From the article, on where relational databases are appropriate: > Small records with complex, well-defined, highly normalized relationships. Why do the records need to be small? And honest, in software development, a large amount of your data is going to be well-defined and easily normalized. The author provided 2 examples that would fit perfectly in a relational database. > The type of queries you will be running o…

I can only assume the reason this was at negative 1 was most people interested in this topic are already on the (not just SQL) bandwagon. IMO, scaling is a non issue for most well designed websites and as computers get faster this only becomes more apparent. There is a significant advantage to separating complex sites into independent modular components and a only tiny fraction of sites need to scale beyond this point. When you actually need to expand fine, go down that rabbit hole but, for most people it's a complete waste of time.

PS: I suspect the main problem developers actually have with SQL databases is they there ORM is significantly less powerful than SQL. All to often developers focus on row as object and forget the power of more abstract data structures.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#26

Post by marshallp »

gnaritas wrote: post #23
marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

And you can easily and incrementally evolve your data model in an object database too with the added benefit that the application is updated to match the new data model at the same time. > SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model And horrible and brittle in terms of the application model, which in 99% of cases is not relational.

"""And horrible and brittle in terms of the application model, which in 99% of cases is not relational."""

I refer you to Philip Greenspun's explanation of why object databases don't work -

"""After 10 years, the market for object database management systems is about $100 million a year, perhaps 1 percent the size of the relational database market. Why the fizzle? Object databases bring back some of the bad features of 1960s pre-relational database management systems. The programmer has to know a lot about the details of data storage. If you know the identities of the objects you're interested in, then the query is fast and simple. But it turns out that most database users don't care about object identities; they care about object attributes. Relational databases tend to be faster and better at coughing up aggregations based on attributes. The critical difference between RDBMS and ODBMS is the extent to which the programmer is constrained in interacting with the data. With an RDBMS the application program--written in a procedural language such as C, COBOL, Fortran, Perl, or Tcl--can have all kinds of catastrophic bugs. However, these bugs generally won't affect the information in the database because all communication with the RDBMS is constrained through SQL statements. With an ODBMS, the application program is directly writing slots in objects stored in the database. A bug in the application program may translate directly into corruption of the database, one of an organization's most valuable assets."""

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#27

Post by csytan »

marshallp wrote: post #14
peterwwillis wrote: post #12

Earlier quoted context omitted.

if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i thi…

The google appengine datastore is moving in the direction of being a full sql database, and I bet oracle have people working on it (because appengine, salesforce and aws is a threat to their business), so I don't think it's all that far-fetched.

Appengine's GQL is a very limited subset of SQL. The only queries which can be performed are those that can scale well, so it's doubtful that even simple things like JOINs will be supported in the future.

http://code.google.com/appengine/docs/python/datastore/gqlre...

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#28

Post by marshallp »

csytan wrote: post #27
marshallp wrote: post #14

Earlier quoted context omitted.

The google appengine datastore is moving in the direction of being a full sql database, and I bet oracle have people working on it (because appengine, salesforce and aws is a threat to their business), so I don't think it's all that far-fetched.

Appengine's GQL is a very limited subset of SQL. The only queries which can be performed are those that can scale well, so it's doubtful that even simple things like JOINs will be supported in the future. http://code.google.com/appengine/docs/python/datastore/gqlre...

Appengine datastore works by creating indexes for every query. In relational databases you make joins execute fast by creating indexes. I don't see why joins couldn't be added to appengine.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#30

Post by davidnelson »

I love the document record pattern. I use this on google app engine, which is a tad tricky to use as a relational db. It actually ends up being way simpler and faster. For instance, you can model an app with just one "table" (Kind). You can put say a user id primary key, then have blob binary data fields which store your serialized document records. The reason it's so fast is all you're doing is a simple index scan to lookup all the data you want. It also makes it really simple to store items into memcached/memcache.
Post reply on HN