Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evernote blog: WhySQL?

blog.evernote.com

Post reply on HN
Print view View original

21–30 of 77 posts

Re: Evernote blog: WhySQL?

#21

Post by robinwarren »

true_religion wrote: post #20
shingen wrote: post #19

It's amazing how when you focus on proven (but supposedly boring or old) technology that just works, and works very well, you can devote a lot of other resources to the actual product and usability. Maybe it's the 30 year old in me showing, but I'm sticking with the 'it just works' crowd. Until some other approach provides a staggeringly overwhelming reason to switch. I find scaling up with MySQL to be ridiculously e…

Er... you seem to be implying that people are using NoSQL solutions like Redis, Toyko Tyrant, Elastic Search, etc. for fun but that's not the case. Sometimes you just pick the simplest thing that works and that can be a key value store, data structures in memory, or a lucrene interface. It's a better solution and less fragile than trying to force everything into a relational model.

I must admit that I picked mongodb (+ node) for my latest project for fun. But I then discovered that being able to dump json straight into a datastore without having to create tables etc. first was a great benefit in quickly building something out.

So something I did for fun turned out to be the simplest thing which worked. Only I didn't know that when I did it. I think developers who don't occasionally pick something new for fun on a side project are doomed not to know when that new thing could actually be applicable on a project that matters.

Re: Evernote blog: WhySQL?

#22

Post by true_religion »

robinwarren wrote: post #21
true_religion wrote: post #20

Earlier quoted context omitted.

Er... you seem to be implying that people are using NoSQL solutions like Redis, Toyko Tyrant, Elastic Search, etc. for fun but that's not the case. Sometimes you just pick the simplest thing that works and that can be a key value store, data structures in memory, or a lucrene interface. It's a better solution and less fragile than trying to force everything into a relational model.

I must admit that I picked mongodb (+ node) for my latest project for fun. But I then discovered that being able to dump json straight into a datastore without having to create tables etc. first was a great benefit in quickly building something out. So something I did for fun turned out to be the simplest thing which worked. Only I didn't know that when I did it. I think developers who don't occasionally pick somethi…

MongoDB isn't the best example to use as a genralization of NoSQL.

Many NoSQL databases have a structure you have to conform to---its just not a relational structure.

Re: Evernote blog: WhySQL?

#24

Post by rmc »

dabeeeenster wrote: post #17

It helps that they have a perfectly shardable product I guess.

Yes, if you're Google (where any page can link to any other) or Facebook (where a person can friend any other) you wouldn't have this. But lots of businesses that provide software solutions do usually have something that is very localised.

Re: Evernote blog: WhySQL?

#25

Post by huggyface »

opendomain wrote: post #18

There are other reasons to choose NoSQL For example, when Craigslist was using mySql and they had to change their schema, it took MONTHS to facilitate the change across all their slaves. You can also have a mixed strategy of using both RDBMS and NoSQL to achieve consistency while being able to be flexible to architecture changes. Lastly- have you looked at total overal cost? Setting up a large cluster with mySql will…

There are other reasons to choose NoSQL For example, when Craigslist was using mySql and they had to change their schema, it took MONTHS to facilitate the change across all their slaves.

That isn't a reason to choose NoSQL. That's a reason to-

-doubt MySQL -- many of the purported downsides of SQL solutions (for instance the ridiculous "avoid joins" meme that has zero bearing on any good database product) are actually MySQL problems. Or rather, they were -- the product has made some pretty incredible strides.

-understand and embrace normalization. Most of the "we keep trying to change our schema and SQL is just so restrictive" stories could often be described as "the problems with denormalization".

Re: Evernote blog: WhySQL?

#26

Post by luigi »

shingen wrote: post #19

It's amazing how when you focus on proven (but supposedly boring or old) technology that just works, and works very well, you can devote a lot of other resources to the actual product and usability. Maybe it's the 30 year old in me showing, but I'm sticking with the 'it just works' crowd. Until some other approach provides a staggeringly overwhelming reason to switch. I find scaling up with MySQL to be ridiculously e…

It's amazing how when you use recent technology that has been designed and engineered from the ground up for the demands of modern web apps, you can devote a lot of other resources to the actual product and usability.

Re: Evernote blog: WhySQL?

#27

Post by jcromartie »

opendomain wrote: post #18

There are other reasons to choose NoSQL For example, when Craigslist was using mySql and they had to change their schema, it took MONTHS to facilitate the change across all their slaves. You can also have a mixed strategy of using both RDBMS and NoSQL to achieve consistency while being able to be flexible to architecture changes. Lastly- have you looked at total overal cost? Setting up a large cluster with mySql will…

I don't want to presume that I'm smarter than Craigslist's engineers, but why is this so hard? Schema changes should be captured in migrations, which are programs, and could be coordinated to run in parallel on thousands of machines in a moment with a shell script over SSH.

Re: Evernote blog: WhySQL?

#28

Post by zzzeek »

opendomain wrote: post #18

There are other reasons to choose NoSQL For example, when Craigslist was using mySql and they had to change their schema, it took MONTHS to facilitate the change across all their slaves. You can also have a mixed strategy of using both RDBMS and NoSQL to achieve consistency while being able to be flexible to architecture changes. Lastly- have you looked at total overal cost? Setting up a large cluster with mySql will…

Doing data migrations up front in a NoSQL system means you've changed the fields around in your document types, then you're done. Now 95% of your documents are wrong, missing those changes. Maybe you have linkages between different types of document (akin to a foreign key) - now a lot of those might be pointing to nothing.

In this model, it's the application's job to anticipate and work around these inconsistencies. Assertions that check for data integrity, if you have them, have to be modified to work around this. For a lot of the web applications we talk about these days, who cares - it's people's lists of friends and TODO notes. It's simple data and some dangling records aren't going to hurt anyone.

In the SQL world, we instead write migration scripts that migrate the structure and data all at once. This is a little more work up front, but as long as you stuck to a mostly normalized form and use great tools (which we now have) this is not a big deal ("MONTHS" to migrate across slaves sounds like they had some very wrong decisions made earlier on). The application can move straight to supporting the new structure and entirely forget that the old one existed. In this world, we can also have really complex relationships between fields, like when we're storing accounting or medical data linked to temporal records of changes. The application can consume this structure without worrying about consistency.

Re: Evernote blog: WhySQL?

#29

Post by huggyface »

jcromartie wrote: post #27
opendomain wrote: post #18

There are other reasons to choose NoSQL For example, when Craigslist was using mySql and they had to change their schema, it took MONTHS to facilitate the change across all their slaves. You can also have a mixed strategy of using both RDBMS and NoSQL to achieve consistency while being able to be flexible to architecture changes. Lastly- have you looked at total overal cost? Setting up a large cluster with mySql will…

I don't want to presume that I'm smarter than Craigslist's engineers, but why is this so hard? Schema changes should be captured in migrations, which are programs, and could be coordinated to run in parallel on thousands of machines in a moment with a shell script over SSH.

MySQL had some historic issues with schema changes -- adding a column to a table with a few million records could take hours. While it does have a lot of work to do -- it is restructuring every page of the database -- in competitive products such a change takes a minute fraction of the time (limited only by IO performance).

Re: Evernote blog: WhySQL?

#30

Post by route66 »

luigi wrote: post #26
shingen wrote: post #19

It's amazing how when you focus on proven (but supposedly boring or old) technology that just works, and works very well, you can devote a lot of other resources to the actual product and usability. Maybe it's the 30 year old in me showing, but I'm sticking with the 'it just works' crowd. Until some other approach provides a staggeringly overwhelming reason to switch. I find scaling up with MySQL to be ridiculously e…

It's amazing how when you use recent technology that has been designed and engineered from the ground up for the demands of modern web apps, you can devote a lot of other resources to the actual product and usability.

Is it possible that

a) not all code written is for web apps

b) that said recent technology also only fits a niche for modern web apps?

I think you view on what constitutes modern web apps or software in general is biased in favor of some examples who benefit by using mongo/couch/redis whatever. (And are very vocal about it)

[EDIT: spelling, and to add that I have tremendous joy in playing with couch, redis, node lately. But at the same time I try to stay critical: what are the real trade offs when I employ these tools?]

Post reply on HN