Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How NoSQL forced the evolution of a scalable relational database

blog.memsql.com

Post reply on HN
Print view View original

151–160 of 211 posts

Re: How NoSQL forced the evolution of a scalable relational database

#151

Post by wild_preference »

ChicagoDave wrote: post #110

NoSQL is an excellent technology for rapid R&D, but once a domain “settles”, data should be modeled and data stores should be switched to relational or graph backends. If a system already has a well-defined domain, NoSQL adds little value. Of course you could leverage AWS DynamoDB and reduce cost, but you still have downstream implications for things like reporting, which requires a known schema with relational parad…

I don’t see how NoSQL could be better for rapid dev when something like Postgres can migrate its schema and data at once while enforcing constraints. That’s clearly better precisely when your schema is changing.

Re: How NoSQL forced the evolution of a scalable relational database

#152

Post by nick_kline »

manigandham wrote: post #141
nojvek wrote: post #126

Earlier quoted context omitted.

I don’t understand the “call to get a quote” sales cycle. I bet memsql has a sales time the size of their engineering team trying to get those fat Enterprise deals but in the grand scheme of things why should I care? It doesn’t have a free tier, I can’t use it easily and just seems like a lot of hype without any 3rd party to back up their claims. Its the kind of things mongodb used to say to gain mindshare. Most popu…

They're focusing on customers that can afford it, so it's not about you caring but you not being their primary customer. You can download the developer edition and it does everything except cross-region replication if you want to test it. It's a great product and the real deal, we used it before (as a startup too) and it did the job and is one of the most polished databases out there.

You can also play with it quick and easy by getting a premade docker image, see https://docs.memsql.com/quickstarts/v6.0/quick-start-with-do.... Disclaimer: I work at Memsql.

Re: How NoSQL forced the evolution of a scalable relational database

#153

Post by wild_preference »

nostalgeek wrote: post #101
amirouche wrote: post #95

Earlier quoted context omitted.

Why EAV is plague? Isn't JSON type a wrapper of EAV?

JSON type is JSON type, EAV isn't a data type, it's a data modelling pattern.

You can see what they were referring to here: https://en.m.wikipedia.org/wiki/Entity–attribute–value_model

Re: How NoSQL forced the evolution of a scalable relational database

#154

Post by bunderbunder »

adrianpike wrote: post #131
nojvek wrote: post #126

Earlier quoted context omitted.

I don’t understand the “call to get a quote” sales cycle. I bet memsql has a sales time the size of their engineering team trying to get those fat Enterprise deals but in the grand scheme of things why should I care? It doesn’t have a free tier, I can’t use it easily and just seems like a lot of hype without any 3rd party to back up their claims. Its the kind of things mongodb used to say to gain mindshare. Most popu…

> I don’t understand the “call to get a quote” sales cycle. It's about maximizing revenue from enterprise clients - when the customer LTV is huge but the volume is low, it's smart to focus your resources on those dollar amounts. When the volume is high but the dollar amounts are low, it's smart to focus on optimizing the volume funnel.

More specifically, it's about price discrimination.

If you publish a price, you need to publish a price that's going to be palatable to your smallest clients. But the moment you do so, your most deep-pocketed clients will demand the same price. This could be disastrous, because those same clients also tend to be more demanding, so a profitable price for one client could be a disastrous money-loser for another.

If you're the kind of person who needs to know the price up-front, it's probably safe to interpret this as a way of saying, "If you have to ask, this isn't for you." At the very least, it's probably the case that you have a limited appetite for products where success requires significant assistance from support engineers to help you deal with all the critical details that typically aren't in the documentation for these kinds of products.

Re: How NoSQL forced the evolution of a scalable relational database

#155

Post by nick_kline »

189523954 wrote: post #96
Amezarak wrote: post #93

Earlier quoted context omitted.

> Nobody is using joins at scale and every large company that started their scaling journey in a SQL database started by performing multiple queries on a distributed database. I've used joins "at scale" in multiple jobs in (conventional relational) databases up to a few TB with tens of thousands of transactions per second and never experienced any performance problem that was the result of a join, unless you count cr…

>I've never understood the "joins are slow" meme or where it came from. Well SQL databases generally don't support joining across a sharded database, which is usually necessary to scale unless you try to scale vertically with high powered machines and your data fits into memory and so on. They are also obviously slow compared to denormalizing and querying without a join. Then there is the other fact I mentioned that…

I think a big point of the article was that these more recent relational dbs (like memsql) figured out how to make distributed joins across multiple shards scale really well - that's one of their core value adds. So you can shard your for example customer and order data stored across multiple nodes and partitions and do distributed joins, aggregations etc. Scaling to use hardware resources on multiple machines is a crucial aspect of these systems.

Disclaimer, I work at Memsql, speaking for myself.

Re: How NoSQL forced the evolution of a scalable relational database

#156

Post by dragonwriter »

threeseed wrote: post #32
dragonwriter wrote: post #21

Earlier quoted context omitted.

> Of course if your data is not relational, you can't do SQL. The relational model is, as I understand it, fully general, so this literally cannot be the case. There is a problem of “if you do not know the shape of the data in advance” and “if the RDBMSs available to you do not efficiently handle data of the particular shape and access pattern you are using”, but “relational” isn't an inherent property of data, it's…

There are plenty of models that don't work well modelled relationally. Few examples here: (1) Wide table with 100K analytical features keyed against a customer. Now if you need to combine a few dozen of these features for predictive purposes then how do you do that in SQL ? You can't. Since every SQL database has a tiny column limit. In Cassandra that is a trivial O(1) fetch. (2) Time series data can be many orders o…

> There are plenty of models that don't work well modelled relationally.

There are plenty of data shapes and access patterns for which existing RDBMS implementations are inefficient, as I acknowledged. That's not a feature of whether the data is or is not relational—“relational” is a feature of a data model, and any data can be modelled relationally—but rather of the available RDBMS implementations.

> Wide table with 100K analytical features keyed against a customer. Now if you need to combine a few dozen of these features for predictive purposes then how do you do that in SQL ?

In terms of both the relational model and the SQL language, this is trivial. In most concrete SQL implementations, it is both inefficient and requires a concrete schema that diverges from the abstract relational model of the data, because you need multiple tables with a common key rather than a single table, even if abstractly it is a single relation in a normalized model. There's nothing fundamental about either SQL or the relational model that requires this, its just not the usage pattern common RDBMS storage implementations are optimized for.

> Time series data can be many orders of magnitude faster in JSON document stores where you can have embedded/nested records.

Yes, that's an efficiency of access patterns issue, not an “is the data relational” issue.

Of course, outside the strict relational model of data, SQL based object-relational databases like Oracle and Postgres have supported embedded and nested records (via the combination of composite and collection data types) since before “JSON object stores” existed.

Re: How NoSQL forced the evolution of a scalable relational database

#157

Post by scarface74 »

stickfigure wrote: post #123

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully hosted, infinitely scalable, zero-maintenance, transactional, reliable, and - at least with Google's offering - scales down to a free tier. They aren't perfect or applicable in every situation, but they're cheap and easy enough to allow a one- or two-programmer…

I am an AWS true believer but I hate DynamoDB. It is far more limiting than Mongo DB.

MongoDB works great with C#. Using a JSON based database with a statically typed object oriented language goes hand to hand. You don’t have the object relational mismatch and the languages type system helps enforce a “schema” on the database.

With C# you work with a MongoCollection And all of your inserts, Linq queries, etc are statically checked by the compiler.

Re: How NoSQL forced the evolution of a scalable relational database

#158

Post by kodablah »

stickfigure wrote: post #123

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully hosted, infinitely scalable, zero-maintenance, transactional, reliable, and - at least with Google's offering - scales down to a free tier. They aren't perfect or applicable in every situation, but they're cheap and easy enough to allow a one- or two-programmer…

> allow a one- or two-programmer team to achieve massive scale without hiring devops

Know what isn't massive scale? Subjecting yourself to the off switch of a single company. So in that light, AWS-specific or Google-specific options are the worst scale you could ever have.

Re: How NoSQL forced the evolution of a scalable relational database

#159

Post by kodablah »

richpimp wrote: post #149

Someone please help me understand. I use relational databases everyday for my projects (db, api, frontend). I'm well-versed in SQL and general best practices, and my projects scale well (admittedly I'm not in the petabytes level of scale). Does the desire to use NoSQL databases come from a disdain for having to think in terms of declarative programming instead of imperative? Is it desirable to have dynamically typed…

For me, it's two main reasons: 1) Generating and storing massive amounts of pre-computed permutations of the same data based on how I'll read it with easy linear growth by adding servers and 2) Transparent multi-master reads/writes providing easy HA failover. Note, not all NoSQL options offer this. If existing free relational DBs had these options easy and out of the box, I would use them instead.

I just need a ton of data, often redundant, that I never delete, that supports distributed writes and reads (ACID not required). The things I use Cassandra for would require huge Postgres servers and custom sharding/partitioning setups and Citus HA and all sorts of other things.

Re: How NoSQL forced the evolution of a scalable relational database

#160

Post by mylons »

kodablah wrote: post #158
stickfigure wrote: post #123

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully hosted, infinitely scalable, zero-maintenance, transactional, reliable, and - at least with Google's offering - scales down to a free tier. They aren't perfect or applicable in every situation, but they're cheap and easy enough to allow a one- or two-programmer…

> allow a one- or two-programmer team to achieve massive scale without hiring devops Know what isn't massive scale? Subjecting yourself to the off switch of a single company. So in that light, AWS-specific or Google-specific options are the worst scale you could ever have.

a meteor could also hit your datacenter, neutrinos can fly through your RAM, a person with malicious intent could detonate an EMP next to your servers, etc etc etc.
Post reply on HN