How NoSQL forced the evolution of a scalable relational database • Hacker News Board

Re: How NoSQL forced the evolution of a scalable relational database

#161

Post by njharman » Fri, Jul 13, 2018, 6:19 PM UTC

richpimp wrote: post #149

Someone please help me understand. I use relational databases everyday for my projects (db, api, frontend). I'm well-versed in SQL and general best practices, and my projects scale well (admittedly I'm not in the petabytes level of scale). Does the desire to use NoSQL databases come from a disdain for having to think in terms of declarative programming instead of imperative? Is it desirable to have dynamically typed…

> what it's like not to have the ability to manipulate the database.

In general "not wanting to understand, depend on, or work with other systems/teams" (often cause of dysfunctional eng/pm organizations) has, over 20years experience, been the number one reason devs (esp front end devs cause they tend to move the fastest (faster than traditional backend teams are prepared for) and have least experience in back end systems) go for something new, something they can control, something they can spin up fast and quick. NoSQL, cloud, anything they can do themselves.

Re: How NoSQL forced the evolution of a scalable relational database

#162

Post by kodablah » Fri, Jul 13, 2018, 6:23 PM UTC

mylons wrote: post #160

kodablah wrote: post #158

Earlier quoted context omitted.

> allow a one- or two-programmer team to achieve massive scale without hiring devops Know what isn't massive scale? Subjecting yourself to the off switch of a single company. So in that light, AWS-specific or Google-specific options are the worst scale you could ever have.

a meteor could also hit your datacenter, neutrinos can fly through your RAM, a person with malicious intent could detonate an EMP next to your servers, etc etc etc.

Probability is not binary. The probability that you will want to move your app, cross-cloud distribute your app, or get it turned off is much higher than those things you mentioned. This is also true about the contents of the data instead of just the ifaces since using NoSQL databases often means distributed in lieu of backups.

It's not that hard to run a few Cassandra machines, or if you can accept relational, RDS or other managed Postgres options are fine. It's just this vendor lock in stuff gets really annoying to refactor/migrate IME and in many cases if your time to market is that important, use a cloud-provided RDBMS that will let you be portable. I'd rather go from PG to Cassandra than Google data store to AWS data store to whatever other provider data store du jour.

Re: How NoSQL forced the evolution of a scalable relational database

#163

Post by eikenberry » Fri, Jul 13, 2018, 6:28 PM UTC

stickfigure wrote: post #123

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully hosted, infinitely scalable, zero-maintenance, transactional, reliable, and - at least with Google's offering - scales down to a free tier. They aren't perfect or applicable in every situation, but they're cheap and easy enough to allow a one- or two-programmer…

Just as an FYI DynamoDB has a free tier as well; per-month you get 25GB storage, 200 Million requests and 2.5 million stream requests.

https://aws.amazon.com/dynamodb/pricing/

Re: How NoSQL forced the evolution of a scalable relational database

#164

Post by stickfigure » Fri, Jul 13, 2018, 6:59 PM UTC

kodablah wrote: post #162

mylons wrote: post #160

Earlier quoted context omitted.

a meteor could also hit your datacenter, neutrinos can fly through your RAM, a person with malicious intent could detonate an EMP next to your servers, etc etc etc.

Probability is not binary. The probability that you will want to move your app, cross-cloud distribute your app, or get it turned off is much higher than those things you mentioned. This is also true about the contents of the data instead of just the ifaces since using NoSQL databases often means distributed in lieu of backups. It's not that hard to run a few Cassandra machines, or if you can accept relational, RDS o…

This is terrible advice for startups.

While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features.

The probability that you will want to cross-cloud distribute your app is infinitesimally small compared to the probability that you will never get the right set of features to make your product take off. Platform portability is fake work that makes developers feel productive about doing something that their customers couldn't give a rat's ass about.

There's a reason Pivotal doesn't let you point chores - your velocity is determined only by how fast you make the customer happy. Focus on developing features, and use whatever tools get you to product-market fit fastest.

Re: How NoSQL forced the evolution of a scalable relational database

#165

Post by stickfigure » Fri, Jul 13, 2018, 7:04 PM UTC

qaq wrote: post #144

stickfigure wrote: post #140

Earlier quoted context omitted.

Yes. Genuinely, why is that hard to understand?

Because for avg. developer reasoning about eventual consistency and handling all the edge cases is fairly hard. If it's easy for you I guess you are far above avg. dev.

I hired a couple junior devs and they did fine.

There really aren't that many edge cases. If you're changing data, you're in a transaction and that's always strongly consistent. If you're querying data, nobody really cares if it's 10s behind (and usually it was 1-2s).

On the other hand, counting a few thousand things is a pretty major PITA. And doing bulk updates requires map/reduce. So there's a cost... but it's not what you seem to think it is.

Re: How NoSQL forced the evolution of a scalable relational database

#166

Post by kodablah » Fri, Jul 13, 2018, 7:27 PM UTC

stickfigure wrote: post #164

kodablah wrote: post #162

Earlier quoted context omitted.

Probability is not binary. The probability that you will want to move your app, cross-cloud distribute your app, or get it turned off is much higher than those things you mentioned. This is also true about the contents of the data instead of just the ifaces since using NoSQL databases often means distributed in lieu of backups. It's not that hard to run a few Cassandra machines, or if you can accept relational, RDS o…

This is terrible advice for startups. While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features. The probability that you will want to cross-cloud distribute your app is infinitesimally small compared to the probability that you will…

> While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features.

I'm not sure if you're responding to a different comment or missed where I suggested just use the cloud Postgres. I specifically mentioned time to market and acknowledge it. I am saying you don't have to sacrifice portability for time to market, and what is terrible advice is suggesting that you do.

Re: How NoSQL forced the evolution of a scalable relational database

#167

Post by qaq » Fri, Jul 13, 2018, 7:52 PM UTC

stickfigure wrote: post #165

qaq wrote: post #144

Earlier quoted context omitted.

Because for avg. developer reasoning about eventual consistency and handling all the edge cases is fairly hard. If it's easy for you I guess you are far above avg. dev.

I hired a couple junior devs and they did fine. There really aren't that many edge cases. If you're changing data, you're in a transaction and that's always strongly consistent. If you're querying data, nobody really cares if it's 10s behind (and usually it was 1-2s). On the other hand, counting a few thousand things is a pretty major PITA. And doing bulk updates requires map/reduce. So there's a cost... but it's not…

Some of your projects looks like are in e-commerce space so say if I look at my cart and it is missing products I added say 5 sec ago that would be an issue.

Re: How NoSQL forced the evolution of a scalable relational database

#168

Post by nightski » Fri, Jul 13, 2018, 7:56 PM UTC

bunderbunder wrote: post #130

soundwave106 wrote: post #128

Earlier quoted context omitted.

Dapper is great IMHO. However, the Microsoft-recommended solution after Linq to SQL ended up being Entity Framework... which was an even heavier ORM. My personal opinion is akin to the parent: the more you abstract away from thinking like your data store, the easier it is to write poorly performing code, because some of the fundamentals get lost in the process. For instance, in EF and other ORMs, one classic pitfall…

From a performance perspective, N+1 queries are spectacular, but the much more pervasive problem is simply that they encourage poor discipline about hitting the database. Most ORM code bases I've looked at do a pretty decent job with avoiding N+1 queries. But they've all been guilty of doing things like incurring orders of magnitude more network transfer than is necessary by pulling down a collection of objects just…

This just comes down to caring about the work you do and putting in the effort to ensure you are delivering quality/performant code. If your devs can't handle that then you have a lot more problems than a heavyweight ORM. No developer worth their salt would be "encouraged" by a library to write bad code.

Re: How NoSQL forced the evolution of a scalable relational database

#169

Post by ecnahc515 » Fri, Jul 13, 2018, 8:01 PM UTC

threeseed wrote: post #32

dragonwriter wrote: post #21

Earlier quoted context omitted.

> Of course if your data is not relational, you can't do SQL. The relational model is, as I understand it, fully general, so this literally cannot be the case. There is a problem of “if you do not know the shape of the data in advance” and “if the RDBMSs available to you do not efficiently handle data of the particular shape and access pattern you are using”, but “relational” isn't an inherent property of data, it's…

There are plenty of models that don't work well modelled relationally. Few examples here: (1) Wide table with 100K analytical features keyed against a customer. Now if you need to combine a few dozen of these features for predictive purposes then how do you do that in SQL ? You can't. Since every SQL database has a tiny column limit. In Cassandra that is a trivial O(1) fetch. (2) Time series data can be many orders o…

You can do all of that with a relational database. Whether or not your specific relational database makes it easy, or efficient is a different question from "can it be modeled relationally".

Re: How NoSQL forced the evolution of a scalable relational database

#170

Post by DenisM » Fri, Jul 13, 2018, 8:13 PM UTC

manigandham wrote: post #141

nojvek wrote: post #126

Earlier quoted context omitted.

I don’t understand the “call to get a quote” sales cycle. I bet memsql has a sales time the size of their engineering team trying to get those fat Enterprise deals but in the grand scheme of things why should I care? It doesn’t have a free tier, I can’t use it easily and just seems like a lot of hype without any 3rd party to back up their claims. Its the kind of things mongodb used to say to gain mindshare. Most popu…

They're focusing on customers that can afford it, so it's not about you caring but you not being their primary customer. You can download the developer edition and it does everything except cross-region replication if you want to test it. It's a great product and the real deal, we used it before (as a startup too) and it did the job and is one of the most polished databases out there.

Nick, your account is shadow-banned. Email hn admins to revert the ban.