Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How NoSQL forced the evolution of a scalable relational database

blog.memsql.com

Post reply on HN
Print view View original

71–80 of 211 posts

Re: How NoSQL forced the evolution of a scalable relational database

#71

Post by threeseed »

davidgould wrote: post #62
threeseed wrote: post #32

Earlier quoted context omitted.

There are plenty of models that don't work well modelled relationally. Few examples here: (1) Wide table with 100K analytical features keyed against a customer. Now if you need to combine a few dozen of these features for predictive purposes then how do you do that in SQL ? You can't. Since every SQL database has a tiny column limit. In Cassandra that is a trivial O(1) fetch. (2) Time series data can be many orders o…

I think you misunderstood the parents use of the term "model". All the things you describe here can be described in the relational model. The parent was arguing that the model is general enough to cover almost all collections of data. You appear to be arguing that for specific use cases specific implementations perform badly. These arguments pass each other unseen like ships in the night.

Actually wide tables for example can't be done at all.

And technically we can use Microsoft Excel as a photo editor. But that's just ridiculous. Just like it's ridiculous to say you can use relational models for any data structure even if the queries will never return.

Re: How NoSQL forced the evolution of a scalable relational database

#72

Post by lotsofcows »

Sjenk wrote: post #39
keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

Not only easy in Javascript, it is easy in any language that has easy serliazation to JSON. I always use it for my own hobby projects because I dont have that much SQL knowledge. Which makes Nosql easy choice for me, just serialize your object into json and push it to your DB.

That's not a choice. To make a choice you need to be aware of alternatives. You use NOSQL because it was easy to achieve your goal.

Re: How NoSQL forced the evolution of a scalable relational database

#73

Post by logophobia »

qaq wrote: post #67
logophobia wrote: post #35

Earlier quoted context omitted.

Cassandra is AP (available and partition tolerant), with tunable consistency. HBase is CP (consistent and partition tolerant, but not always available in case of network partitions). Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization that google achieved on their hardware and network, an innovation I hope will spread. So yes, that's new. I don't think I claimed…

Spanner is CP I think the A part comes from reliability of google network infrastructure which is not part of DB system's design per se.

I think I misunderstood that yes.

https://storage.googleapis.com/pub-tools-public-publication-....

TLDR: It's technically CP, but google claims partitions are so rare people can assume it's CA as well (99.999% available).

I assumed TrueTime bypassed the CAP theorem, but apparently that's marketing bs. It's to ensure something called external consistency, which is important if you want to take consistent snapshots over a distributed system.

Re: How NoSQL forced the evolution of a scalable relational database

#74

Post by threeseed »

stephenr wrote: post #56
threeseed wrote: post #53

Earlier quoted context omitted.

Facebook, Yahoo, Google etc built these databases to use not as coding exercises. So are they lazy and drinking the kool aid for not relying on SQL databases instead of building their own ? Or maybe SQL isn't perfect for every use case and other developers should stop arrogantly belittling those that choose to architect their own systems a particular way.

I'd say that 99.9% of organisations using any database technology, don't have the scaling problems of Google, Yahoo, Facebook or Apple. They objectively can't be "drinking the koolaid" or cargoculting something, because they've specifically created their own solutions, to solve their own problems. I specifically didn't say SQL is perfect - I said that they're very popular for the wrong reasons, and often are used in…

So it's okay for developers to build their own database for their specific use case. But it's not okay for developers to use a database for their specific use case.

Makes sense.

Re: How NoSQL forced the evolution of a scalable relational database

#75

Post by CuriousSkeptic »

threeseed wrote: post #27
logophobia wrote: post #6

Earlier quoted context omitted.

I would say the the rise of nosql options is driven by the need for scalability by a few large companies that desperately need it, and cargo-culting by developers that don't actually need it, but want to be like google and don't want to take the time to properly learn and optimize sql. There's a genuine need for good scalable options, and with stuff like google spanner, those don't necessarily need to be "eventually…

Not sure what you are talking about here. There have been scalable and strongly consistent databases since the invention of the concept of NoSQL i.e. HBase and Cassandra (CL=ALL). And the idea that "learning and optimize SQL" would instantly change people's rationale for using these databases shows you don't understand them much at all. There are many factors that come into play. For example you can't use SQL databas…

To me “large scale feature engineering” is precisely the kind of problem SQL excel at.

Is it something that makesme grab for SQL it is the need to do adhoc querying exploring data.

Care to elaborate?

Re: How NoSQL forced the evolution of a scalable relational database

#76

Post by ddorian43 »

siscia wrote: post #30
ddorian43 wrote: post #23

Earlier quoted context omitted.

The same answer applies to your product.

Honestly, I believe that for small workload you can definitely use RediSQL in production, it will happily contain your cache or it will be a great SQL database. However, I need a way to cut it between people just using the free product and people actually supporting the project, so provide as paying feature something that the big company will require it seemed to me the only way to go. Unfortunately, I don't have the…

To be honest, I fail to see what I could use your product for so I'm out of the target audience.

Assuming nosql is for something very efficient or very scalable, I need some space to use it before I have to shell $$. There are many products where I have to pay before going on production.

Re: How NoSQL forced the evolution of a scalable relational database

#77

Post by orwin »

threeseed wrote: post #74
stephenr wrote: post #56

Earlier quoted context omitted.

I'd say that 99.9% of organisations using any database technology, don't have the scaling problems of Google, Yahoo, Facebook or Apple. They objectively can't be "drinking the koolaid" or cargoculting something, because they've specifically created their own solutions, to solve their own problems. I specifically didn't say SQL is perfect - I said that they're very popular for the wrong reasons, and often are used in…

So it's okay for developers to build their own database for their specific use case. But it's not okay for developers to use a database for their specific use case. Makes sense.

This is not what he said. He basically said that in many cases, some developpers pushed something like mongo, ES or cassandra in their project or even company when a small sqlite or an effective postgressql were much more appropriate.

Specific databases solve specific problems. In my current company, for log storage and analysis, InfluxDB was our solution after testing both ES and postgress, but we did compare all the products, and we still do as we expand. In my previous company, the main database in the product was Zope. I agree that mongoDB may be better in some use case, but i strongly doubt that most company are running it because it's the most appropriate product.

Re: How NoSQL forced the evolution of a scalable relational database

#78

Post by mmt »

threeseed wrote: post #53
mmt wrote: post #37

Earlier quoted context omitted.

I think the accusation was against the developers who caused the rise in popularity (i.e. the users), not the developers who created the technologies (i.e. the authors). In that light, it might be best to exclude BigTable, since it's only available publically as a service.

Facebook, Yahoo, Google etc built these databases to use not as coding exercises. So are they lazy and drinking the kool aid for not relying on SQL databases instead of building their own ? Or maybe SQL isn't perfect for every use case and other developers should stop arrogantly belittling those that choose to architect their own systems a particular way.

Again, the accusation isn't against the builders, but against the popularizers (subsquent users).

To stretch the kool aid metaphor, they're the ones mixing the kool aid, not drinking it.

Re: How NoSQL forced the evolution of a scalable relational database

#79

Post by matwood »

keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

> Microsoft at one stage had Linq to SQL which was quite good..... but they killed it :)

They also had a rest API directly into the db server at one point, but killed that also.

Re: How NoSQL forced the evolution of a scalable relational database

#80

Post by matwood »

189723954 wrote: post #38
keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a b…

> Everybody uses an ORM with a SQL database.

I'm ripping ORMs out of any backend code and using things like jOOQ.

> I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more".... > Joins are slow and complicated

You know what's really slow? Creating joins in code.

> This is not true at all. Programming languages are vastly superior at querying a database. Just go write a complex query and compare it to the one in linq or whatever you use.

It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job.

Post reply on HN