Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does a relational database work?

coding-geek.com

Post reply on HN
Print view View original

11–20 of 62 posts

Re: How does a relational database work?

#11

Post by mkehrt »

jchrisa wrote: post #8
mgachka wrote: post #6

Earlier quoted context omitted.

(I'm the author of the article) I'm 28 and I’m currently a Big Data developer (I use Hadoop, HBase, Hive …) and I don’t understand the buzz surrounding Big Data and NoSQL. With a relational database the complexity is hidden (more or less…) whereas with Big Data and NoSQL the developer needs to deal with this complexity himself/herself. As a result, most of the Big Data applications I’ve seen don’t work well. A really…

At Couchbase we did a survey of developers (this was ages ago) and the biggest motivator for NoSQL was schema flexibility. Not having to coordinate migrations is seen as a productivity boost. [1] The other thing document databases can offer that relational databases struggle with is taking subsets (which we use for offline sync.) [2] [1] http://blog.couchbase.com/nosql-adoption-survey-surprises [2] http://developer.c…

As someone who spent several years studying programming languages, the thing that drives me crazy about traditional relational databases is the assumption that all data is tuple-structured. Much data is structured as unions of alternates or more complex things like maps. Shoehorning your data model into a tuple-based system is always possible, but often unnatural.

The place NoSQL shines is the acknowledgement that most data is complex. Of course it often also does away with ACIDity, which is a huge disaster (EDIT: the doing-away-with is a disaster, I mean).

Re: How does a relational database work?

#13

Post by mgachka »

kodablah wrote: post #9
mgachka wrote: post #6

Earlier quoted context omitted.

(I'm the author of the article) I'm 28 and I’m currently a Big Data developer (I use Hadoop, HBase, Hive …) and I don’t understand the buzz surrounding Big Data and NoSQL. With a relational database the complexity is hidden (more or less…) whereas with Big Data and NoSQL the developer needs to deal with this complexity himself/herself. As a result, most of the Big Data applications I’ve seen don’t work well. A really…

"With a relational database the complexity is hidden" That is my main issue. I use Cassandra over relational firstly for its linear scalability and multi-master-esque HA. But even ignoring those, I understand exactly what is being scanned and what is not, I don't have to fight with an optimizer at runtime based on several parameters.

I understand your point (and it’s a good one) and here is mine: Unless you're working in a team with a lot of good IT guys, you're likely to end up with worse performances and problems.

For example, when I started in Big Data, in less than 3 weeks I was able to optimize some batches just because I read the documentation of the framework used (PIG in this case) and read a small part of the source code to dig deeper. And it was not some touchy optimizations: I used in-memory joins and reduced the number relations in the scripts to reduce the generation of Hadoop jobs (which led to batchs 4 times faster).

There are often problems with our HBase database because it’s often overloaded (I’m not an IT operator so I can’t give more details) and no one really masters this database whereas it’s in production since 2014.

I do understand that in some cases a NoSQL database is mandatory and like you I like to understand what I’m doing. But:

- I’m not working in Silicon Valley

- Most of my co-worker are not geeks (and I respect that)

- It's VERY hard to find guys with real Big Data or NoSQL skills (this comes from a French technical recruiter)

So, if the geek part of me loves Big Data and NoSQL, the rational part prefers using well known technologies. If NoSQL and Big Data becomes mainstream and more known then the rational part will love them too.

Re: How does a relational database work?

#14

Post by kodablah »

mgachka wrote: post #13
kodablah wrote: post #9

Earlier quoted context omitted.

"With a relational database the complexity is hidden" That is my main issue. I use Cassandra over relational firstly for its linear scalability and multi-master-esque HA. But even ignoring those, I understand exactly what is being scanned and what is not, I don't have to fight with an optimizer at runtime based on several parameters.

I understand your point (and it’s a good one) and here is mine: Unless you're working in a team with a lot of good IT guys, you're likely to end up with worse performances and problems. For example, when I started in Big Data, in less than 3 weeks I was able to optimize some batches just because I read the documentation of the framework used (PIG in this case) and read a small part of the source code to dig deeper. A…

While I agree with a couple of your premises, I don't think they all apply to Cassandra as much and is too broad of a brush to use.

I don't believe NoSQL means no validation. In fact, I've found things like Cassandra CQL actively prevent me from running expensive queries unless I opt in (e.g. ALLOW FILTERING). Validation is DB specific, but I don't think it's fair to say it's a footgun in NoSQL any more than in SQL databases.

As for choosing what is known by the employee market, I personally don't choose technologies that way (but I do choose based on maturity of course). I rarely look for skills as much as the ability to learn new ones, but I understand it can be a pipe dream when in the market for juniors.

Re: How does a relational database work?

#15

Post by otis_inf »

> When it comes to relational databases, I can’t help thinking that something is missing. They’re used everywhere. There are many different databases: from the small and useful SQLite to the powerful Teradata. But, there are only a few articles that explain how a database works.

That's because the inner workings are really old, as in: emerged before blogging etc. was popular, hell before the internet was invented.

In the 'before/early internet days', we read books like 'An introduction to Database Systems' by C.J. Date. (I had to blow the dust off my copy to read the exact title ;)), which are more in depth than this article, but I like the article better, because it's more to the point and easier to understand. Well done!

Re: How does a relational database work?

#16

Post by brudgers »

Because I am interested in databases, I found the Se-radio's 2013 interview with Michael Stonebreaker [1] interesting, particularly in regard to traditional database design and more recent ideas:

http://www.se-radio.net/2013/12/episode-199-michael-stonebra...

[1]: http://www.theregister.co.uk/2015/03/25/mike_stonebraker_win...

Re: How does a relational database work?

#18

Post by jeffdavis »

otis_inf wrote: post #15

> When it comes to relational databases, I can’t help thinking that something is missing. They’re used everywhere. There are many different databases: from the small and useful SQLite to the powerful Teradata. But, there are only a few articles that explain how a database works. That's because the inner workings are really old, as in: emerged before blogging etc. was popular, hell before the internet was invented. In…

"An Introduction to Database Systems" is more about the relational model, whereas this article is more about the implementations.
Post reply on HN