Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQL Databases Are An Overapplied Solution (And What To Use Instead)

adamblog.heroku.com

Post reply on HN
Print view View original

41–50 of 67 posts

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#41

Post by cookiecaper »

I can feel the sincerity of the author, but ultimately I think he's wrong that the solution to this is NoSQL.

It really would be nice to send an e-commerce order as JSON data and have my database know what to do with that. I think we still need the flexibility and power of a relational database behind it, but if someone extended PostgreSQL to take records the way CouchDB or others take records, and taught it how to store into rigid, joinable, relational tables, that would be just great and would help a lot. All of the advanced and relational functionality would still be available when needed, but by default, if one could write and retrieve data in a default format that had been mapped onto tables, etc. previously transparently from the database, that would be awesome.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#42

Post by marshallp »

cduan wrote: post #37
marshallp wrote: post #11

These noSql people are missing the point of relational modeling, that you can easily incrementally evolve your data model. It's why object databases never caught on. SQL databases are absolutely beautiful and elegant when you think of them in terms of the codd relational model, in my opinion the best thing that computer science has produced so far. The only limitation of relational databases currently is their lack o…

Well, is this a complaint about relational SQL databases? The only issues the author has that actually are tied to the SQL language are #2 (queries are largely unknown up front) and possibly #1 (small records with normalized relationships). Most of the other complaints have to do with ACID data consistency and the overhead it imposes in simple implementations (e.g. no row-level locking).

By leaving sql you're foregoing the ease of development aspects that it has. It's like leaving python to program in c. Memory is cheap these days, for anything but the largest websites it makes more sense to use a relational database with all data in memory (and this will be as performant as any other solution you create).

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#43

Post by donaldc »

marshallp wrote: post #42
cduan wrote: post #37

Earlier quoted context omitted.

Well, is this a complaint about relational SQL databases? The only issues the author has that actually are tied to the SQL language are #2 (queries are largely unknown up front) and possibly #1 (small records with normalized relationships). Most of the other complaints have to do with ACID data consistency and the overhead it imposes in simple implementations (e.g. no row-level locking).

By leaving sql you're foregoing the ease of development aspects that it has. It's like leaving python to program in c. Memory is cheap these days, for anything but the largest websites it makes more sense to use a relational database with all data in memory (and this will be as performant as any other solution you create).

I prefer redis to SQL for all-in-memory data. Fast, easy, and simple.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#44

Post by oomkiller »

cookiecaper wrote: post #41

I can feel the sincerity of the author, but ultimately I think he's wrong that the solution to this is NoSQL. It really would be nice to send an e-commerce order as JSON data and have my database know what to do with that. I think we still need the flexibility and power of a relational database behind it, but if someone extended PostgreSQL to take records the way CouchDB or others take records, and taught it how to s…

I think you just described inserting to a SQL view. Never done it myself, but sounds helpful.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#45

Post by jawngee »

viraptor wrote: post #21
wvenable wrote: post #16

There's definitely a lack of imagination when it comes to proponents of NoSQL solutions, and this article shows no exception. While we do want to store, for example, an entire e-commerce order in a single operation we also don't necessarily want to retrieve it that way. That type of storage makes otherwise simple operations considerably more difficult. Do you want to know how much you made in sales today? How many of…

Did you use any NoSQL solution seriously? Let's see some examples (imaginary python-like interface, so I don't have to be language/backend-specific). Like you say: Start writing code. Do you want to know how much you made in sales today? sum(amount in db.filter(type='order', date=xxx)) -vs- SELECT SUM(amount) FROM order WHERE date=? How many of widget #453 are still in stock? (IRL it's never that simple, but...) db.f…

Ok, so write me a query that shows me which orders were placed for a specific widget within a date range for a customer belonging to a particular company where the payment is still being processed.

You say it yourself: (IRL it's never that simple, but...)

What's the but?

And I'm still not certain how different applications access a NoSQL store without clobbering each other, where RDBMS systems enforce regulated access.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#46

Post by regularfry »

marshallp wrote: post #40
gnaritas wrote: post #38

Earlier quoted context omitted.

Which boils down to "relational databases are easier for non programmers to use", but I'm a programmer, I don't care. OODB's allow me to write applications much faster, and bugs affect both kinds of databases equally. Data corruption happens just as much in relational databases. I've used both, various relational db's for more than 10 years. I'm well aware of their strengths and weaknesses. Given the choice, I'll tak…

From your profile i can see that your a smalltalk fan. Well, Alan Kay was onced asked where he thought the future of programming languages lies and he replied something along the lines of knowledge representation/logic programming. guess what - relational databases are based on the first order logic, which is the most common method for representing 'knowledge', look at prolog or opencyc. When you write sql you're act…

Nearly right. Relational databases are based on a solid logical foundation. SQL is not.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#47

Post by CaptainZapp »

It doesn't feel the author has real world experience on relational databases. Let's see :

"Small records with complex, well-defined, highly normalized relationships."

Maybe, but not necessarily. Example? Call Data Records for a phone billing application. While there may be relationships in the inserted rows, even to other databases (i.e. customer information) the data stands pretty much on it's own. The challenge is to get floods of data into a single table, so that it can be later sliced and diced for billing purposes.

"The type of queries you will be running on the data is largely unknown up front."

Excuse me? This is so very wrong. I don't even no where to start. EVERY good RDB application is designed in a way where (ideally) all queries are known up-front. If you have to guarantee response times (i.e. think of a cash withdrawal at an ATM) you absolutely MUST control the queries that run on the db. Ad-hoc analysing and reporting MUST be relegated to dedicated, possibly replicated databases.

"The data is long-lived: it will be written once, updated infrequently relative to the number of reads/queries, and deleted either never, or many years in the future."

Mostly so, but absolutely not necessarily the case. I work on an application where the entire data is toasted after a couple weeks and in fact: today and yesterday would suffice.

"The database does not need round-the-clock availability (middle-of-the-night maintenance windows are no problem). Queries do not need to be particularly fast, they just need to return correct results eventually."

This is so full of crap, I won't even get into it

"Data integrity is 100% paramount, trumping all other concerns, such as performance and scalability"

Yes, 100% integrity is paramount, but most certainly not at the cost of scalability, let alone performance.

Recently, methinks, there are a lot of proponents of new and improved data management capabilities, who see their little walled environment, but seem to have no whatsoever experience running databases in a real business. A normal (even big, huge or multinational business) does not have those "cloud-data-management-requirments" that very, very few companies really have.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#48

Post by blasdel »

silentbicycle wrote: post #39
blasdel wrote: post #22

Earlier quoted context omitted.

That is a clever idea I'd not thought of, but it isn't really multitenant -- it just pushes the problem down into the filesystem. It also requires a discontinuous transition to a different SQL database once you graduate from being a small-fry, and from there you're in the same boat as everyone else trying to scale that to multiple machines without application changes.

I'm not sure what you mean by, "it just pushes the problem down into the filesystem". Could you explain? On the upside, SQLite will take disk space, memory, and CPU proportional to actual usage, and compared to (say) Ruby or Python, it's a drop in the bucket. As to the discontinuous transition between it and a bigger system, sure. It's a trade-off. So is worrying about scalability plans during early prototyping, thou…

If you're building a multitenant distributed system like GAE, the SQLite blob needs to be stored in a distributed system too, even if you don't need to handle concurrent access.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#49

Post by smcq »

wvenable wrote: post #36
viraptor wrote: post #21

Earlier quoted context omitted.

Did you use any NoSQL solution seriously? Let's see some examples (imaginary python-like interface, so I don't have to be language/backend-specific). Like you say: Start writing code. Do you want to know how much you made in sales today? sum(amount in db.filter(type='order', date=xxx)) -vs- SELECT SUM(amount) FROM order WHERE date=? How many of widget #453 are still in stock? (IRL it's never that simple, but...) db.f…

The question is what is this doing: sum(amount in db.filter(type='order', date=xxx)) If you're iterating every order in the database to apply your filter or to calculate the most popular items then you wasting a huge amount of processing power and RAM for something an RDBMS can do very efficiently. It's not an advantage that your incrementing values in your programming language of choice either, it's a disaster. As f…

You don't seem to understand how stored views work. This computation cost is felt at insertion time and is not theoretically worse than the exact same computation cost at insertion time in a rdbms to build almost identical index structures.

What current implementations of rdbms's gain you is the ability to write completely ad-hoc queries and get reasonable performance most of the time. This is an implementation advantage, not a theoretical advantage.

Re: SQL Databases Are An Overapplied Solution (And What To Use Instead)

#50

Post by smcq »

jawngee wrote: post #45
viraptor wrote: post #21

Earlier quoted context omitted.

Did you use any NoSQL solution seriously? Let's see some examples (imaginary python-like interface, so I don't have to be language/backend-specific). Like you say: Start writing code. Do you want to know how much you made in sales today? sum(amount in db.filter(type='order', date=xxx)) -vs- SELECT SUM(amount) FROM order WHERE date=? How many of widget #453 are still in stock? (IRL it's never that simple, but...) db.f…

Ok, so write me a query that shows me which orders were placed for a specific widget within a date range for a customer belonging to a particular company where the payment is still being processed. You say it yourself: (IRL it's never that simple, but...) What's the but? And I'm still not certain how different applications access a NoSQL store without clobbering each other, where RDBMS systems enforce regulated acces…

Do the same thing you would do in SQL, run the efficient query (you're almost guaranteed have a view for date limited for a specific customer) then filter it based on product. This is exactly what your RDBMS does, and I don't see how you're gaining anything except a different language.
Post reply on HN