Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does a relational database work?

coding-geek.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: How does a relational database work?

#51

Post by parasubvert »

angrybits wrote: post #32
threeseed wrote: post #26

Earlier quoted context omitted.

Not really sure what you are talking about. Teradata, Oracle, PostgreSQL for example are reasonably complex databases to cluster and manage yourself. Just as easy/hard as setting up HDFS and installing Hive. In all cases people who are at big data scale are buying OTS solutions e.g. Cloudera appliance. They aren't rolling their own. And if you are using Hive then I can understand why you are not feeling the buzz. But…

You are either being dishonest or just patently out of your mind if you think that you can query 100MB and 100PB in the same way. That's not even reasonable by HN standards of hyperbole. Do you have any idea how many orders of magnitude that is?

He's talking about Apache Spark.

While it can handle 100 MB easily there probably are faster ways to handle that small amount of data. But yes, Spark can handle many PB and doesn't require a ton of changes in the code as you scale up from say 10 TB to 100 PB. The underlying cluster would change, and the performance profile would change a lot (10 TB can be done in-memory ... many PB, not so much)

Re: How does a relational database work?

#52

Post by cmrdporcupine »

parasubvert wrote: post #48
mkehrt wrote: post #11

Earlier quoted context omitted.

As someone who spent several years studying programming languages, the thing that drives me crazy about traditional relational databases is the assumption that all data is tuple-structured. Much data is structured as unions of alternates or more complex things like maps. Shoehorning your data model into a tuple-based system is always possible, but often unnatural. The place NoSQL shines is the acknowledgement that mo…

The relational model is extremely general, its' been argued (fairly successfully) by Date and Codd to be THE most general model (Graph being a close second). It's a rigorous approach to managing data with integrity. I used to be a programming language oriented person, was big into data structures and objects, but then I read Date and my mind was blown at how beautiful and expressive the relational model is -- for its…

Agree, 100%. I too early in my career was very enamored by object and graph databases (this is pre the 'nosql' buzz), but once I started reading Date and Codd (and the inflammatory Fabian Pascal) some lights starting turning on in my brain.

Firstly, it should be made clear to people that SQL is not truly relational, and a lot of the things people dislike about it are nothing to do with the relational model and more to do with its late 70s, early 80s heritage. It was thrown together at a time when business systems were still very focused around COBOL.

The second thing that people are not picking up on is that the industry _already_ went through a pre-sql "nosql" phase in the 60s and 70s when network and hierarchical databases were the norm, and the relational model was developed to deal with the perceived faults those systems had: an enforced topology which could not easily rearranged at query time, lack of rigor in modeling, lack of standardized modeling concepts and notation...

Finally, I did find in previous jobs certain uses for nosql systems -- very low latency high throughput quasi-realtime systems that deal with very small bits of simply structured data and need to distribute it widely across a cluster. For that I used Cassandra (tho I understand now that there are successors that are better).

What I don't get is the point of systems like Redis or MongoDB which don't offer a compelling distribution implementation and simply replace the fairly well understood quasi-relational model of SQL with their own ultimately inferior graph/network/hierarchical models.

Re: How does a relational database work?

#53

Post by mgachka »

njharman wrote: post #19

I believe that to be the best technical document I've ever read. Surely biased as I learned so much.

Hi, I'm glad to read this comment.

If you liked this article, maybe you'll like my article on Shazam. I used the same pattern: I start from the basics of sound processing and computer science and finish with an in-depth explanation of Shazam.

Re: How does a relational database work?

#54

Post by otis_inf »

jeffdavis wrote: post #18
otis_inf wrote: post #15

> When it comes to relational databases, I can’t help thinking that something is missing. They’re used everywhere. There are many different databases: from the small and useful SQLite to the powerful Teradata. But, there are only a few articles that explain how a database works. That's because the inner workings are really old, as in: emerged before blogging etc. was popular, hell before the internet was invented. In…

"An Introduction to Database Systems" is more about the relational model, whereas this article is more about the implementations.

Only chapter 5 and 6, the rest isn't. Are we talking about the same book?

Re: How does a relational database work?

#56

Post by parasubvert »

cmrdporcupine wrote: post #52
parasubvert wrote: post #48

Earlier quoted context omitted.

The relational model is extremely general, its' been argued (fairly successfully) by Date and Codd to be THE most general model (Graph being a close second). It's a rigorous approach to managing data with integrity. I used to be a programming language oriented person, was big into data structures and objects, but then I read Date and my mind was blown at how beautiful and expressive the relational model is -- for its…

Agree, 100%. I too early in my career was very enamored by object and graph databases (this is pre the 'nosql' buzz), but once I started reading Date and Codd (and the inflammatory Fabian Pascal) some lights starting turning on in my brain. Firstly, it should be made clear to people that SQL is not truly relational, and a lot of the things people dislike about it are nothing to do with the relational model and more t…

As a fan of Cassandra (though not for its relational model ;), what successors do you believe are better? Riak is the only one that comes to mind.

Btw, to me the point of Redis and Mongo is a very fast distributed dictionary. They're data structure servers, for when you want to persist and share data structures across processes, not "manage information". It depends on your goal.

Re: How does a relational database work?

#57

Post by mkehrt »

parasubvert wrote: post #48
mkehrt wrote: post #11

Earlier quoted context omitted.

As someone who spent several years studying programming languages, the thing that drives me crazy about traditional relational databases is the assumption that all data is tuple-structured. Much data is structured as unions of alternates or more complex things like maps. Shoehorning your data model into a tuple-based system is always possible, but often unnatural. The place NoSQL shines is the acknowledgement that mo…

The relational model is extremely general, its' been argued (fairly successfully) by Date and Codd to be THE most general model (Graph being a close second). It's a rigorous approach to managing data with integrity. I used to be a programming language oriented person, was big into data structures and objects, but then I read Date and my mind was blown at how beautiful and expressive the relational model is -- for its…

Oh, this is true, and I think I came off too harshly. The relational model is general enough to support (almost?) any data model, it certainly has a lot of advantages in terms of efficient implementation, and the math is elegant.

I just don't think that it naturally reflects the data structures people use, and we should be willing to make the computer do the work, rather than humans.

Re: How does a relational database work?

#58

Post by why-el »

mikethedyke wrote: post #49
why-el wrote: post #2

Good write up. Another excellent resource straight out of the UC Berkeley Database Group that I keep close by is "Architecture of a Database System"[1] by three researchers in the field. It is very readable. [1] http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf

how do you find these articles?

Honestly for this one I do not remember since it was two years ago, but I'd start over at http://arxiv.org/, which is a treasure. Granted most are research papers, but even those will link to overview papers, of which the one I link to is an example.

Re: How does a relational database work?

#59

Post by dang »

angrybits wrote: post #32
threeseed wrote: post #26

Earlier quoted context omitted.

Not really sure what you are talking about. Teradata, Oracle, PostgreSQL for example are reasonably complex databases to cluster and manage yourself. Just as easy/hard as setting up HDFS and installing Hive. In all cases people who are at big data scale are buying OTS solutions e.g. Cloudera appliance. They aren't rolling their own. And if you are using Hive then I can understand why you are not feeling the buzz. But…

You are either being dishonest or just patently out of your mind if you think that you can query 100MB and 100PB in the same way. That's not even reasonable by HN standards of hyperbole. Do you have any idea how many orders of magnitude that is?

Please don't make technical comments (or any comments) in this inflammatory, nasty way.

We're lucky that you didn't spark a horrible flamewar, but instead got patient, factual replies.

Re: How does a relational database work?

#60

Post by angrybits »

dang wrote: post #59
angrybits wrote: post #32

Earlier quoted context omitted.

You are either being dishonest or just patently out of your mind if you think that you can query 100MB and 100PB in the same way. That's not even reasonable by HN standards of hyperbole. Do you have any idea how many orders of magnitude that is?

Please don't make technical comments (or any comments) in this inflammatory, nasty way. We're lucky that you didn't spark a horrible flamewar, but instead got patient, factual replies.

I know. And I'm sorry, bitterly sorry, but I know that... no apologies I can make can alter the fact that in our thread you have been given a dirty, filthy, smelly piece of technical argument.
Post reply on HN