How NoSQL forced the evolution of a scalable relational database • Hacker News Board

Re: How NoSQL forced the evolution of a scalable relational database

#61

Post by hnaccy » Fri, Jul 13, 2018, 9:04 AM UTC

threeseed wrote: post #24

stephenr wrote: post #3

> NoSQL came into existence because the databases at the time couldn’t handle the scale required. Arguably the first "NoSQL" database was a DIT, generally accessed via the LDAP protocol. OpenLDAP is one of the better known open source instances, but Novell eDirectory was there in the early part of this century, as was MS AD. In the case of Directory Services, it's a completely different approach to both SQL and the c…

So the developers at Google (BigTable), Facebook (Cassandra), Yahoo (HBase), Apple (FoundationDB), Redis etc are "lazy" and have drunk "too much Kool-aid". Because these technologies every day power the most demanding applications on the planet.

I believe Yahoo is Sherpa not HBase.

Re: How NoSQL forced the evolution of a scalable relational database

#62

Post by davidgould » Fri, Jul 13, 2018, 9:06 AM UTC

threeseed wrote: post #32

dragonwriter wrote: post #21

Earlier quoted context omitted.

> Of course if your data is not relational, you can't do SQL. The relational model is, as I understand it, fully general, so this literally cannot be the case. There is a problem of “if you do not know the shape of the data in advance” and “if the RDBMSs available to you do not efficiently handle data of the particular shape and access pattern you are using”, but “relational” isn't an inherent property of data, it's…

There are plenty of models that don't work well modelled relationally. Few examples here: (1) Wide table with 100K analytical features keyed against a customer. Now if you need to combine a few dozen of these features for predictive purposes then how do you do that in SQL ? You can't. Since every SQL database has a tiny column limit. In Cassandra that is a trivial O(1) fetch. (2) Time series data can be many orders o…

I think you misunderstood the parents use of the term "model". All the things you describe here can be described in the relational model. The parent was arguing that the model is general enough to cover almost all collections of data. You appear to be arguing that for specific use cases specific implementations perform badly. These arguments pass each other unseen like ships in the night.

Re: How NoSQL forced the evolution of a scalable relational database

#63

Post by davidgould » Fri, Jul 13, 2018, 9:10 AM UTC

pull_my_finger wrote: post #54

I keep looking at Tarantool longingly, but I can't figure out when I would need it.

I have the same feeling. If you ever figure this out, let me know. It looks very interesting, but I don't think my clients will let me remake their production with it and it is difficult to evaluate databases without data or a workload.

Re: How NoSQL forced the evolution of a scalable relational database

#64

Post by janemanos » Fri, Jul 13, 2018, 9:13 AM UTC

What concernes me is that the article is strongly biased in the sense, that modern developments of non-relational databases just get ignored while ranting on their state from a few years ago. Schemaless databases do support full ACID transactions (MongoDB, ArangoDB), with some you can also enforce a schema and not everybody loves SQL. So having competition even on the query language-side, can only improve the status quo.

Yes, thinking about your data model will for sure increase your understanding of what you are actually doing... and every professional developer is doing this as well when choosing a non-relational database. Anythings else would be stupid.

For me the article rather shows that the modern developments in non-relational databases do affect vendors on the relational side of the spectrum. Otherwise, it would not make sense to invest so much time in writing such a long article.

Re: How NoSQL forced the evolution of a scalable relational database

#65

Post by janemanos » Fri, Jul 13, 2018, 9:22 AM UTC

threeseed wrote: post #36

antirez wrote: post #31

Redis is not mentioned here at all so maybe the author is thinking mostly of other NoSQL software here, but well, in the case of Redis the whole point was not just the in-memory performance part, but the data model as well. My claim is that you can't really exploit the advantage of using memory if you perpetuate in using the memory to represent the same data model that you were using with relational databases. For in…

This is the problem with people that use the term NoSQL. Do you mean Cassandra (BigTable), MongoDB (Document), Riak (Key/Value), Redis (Mix), Kafka (Log Structured) ? There are dozens of fundamentally different systems many of which are closer to an RDBMS than their NoSQL peers. And many of them have rigid schemas so it definitely isn't that either.

Don't forget native multi-model databases like ArangoDB

Re: How NoSQL forced the evolution of a scalable relational database

#66

Post by CuriousSkeptic » Fri, Jul 13, 2018, 9:22 AM UTC

pbz wrote: post #48

sidstling wrote: post #34

Earlier quoted context omitted.

Linq to SQL is still a thing, you may need to click it’s package when you install visual studio 2017+, but it’s still there and it’s still the best SQL interface they have build in my opinion. We rarely use anything else, but I can see the value of entity if you’re changing your DB a lot. We typically don’t do that though, and to be honest, working with changes with linq to sql, typically goes smoother than entity mi…

LINQ to SQL is a dead end. Look into LINQ to DB https://github.com/linq2db/linq2db which is way faster than LINQ to SQL ever was (faster than even Dapper). If you must you could give EF a try, but stay away from LINQ to SQL.

The value of Dapper IMHO is not the speed. It is precisely the interface. With dapper I can copy paste code between C# and SQL contexts freely. Exploring the database directly in SQL and then use the result directly in the program without a translation step.

Re: How NoSQL forced the evolution of a scalable relational database

#67

Post by qaq » Fri, Jul 13, 2018, 9:29 AM UTC

logophobia wrote: post #35

threeseed wrote: post #27

Earlier quoted context omitted.

Not sure what you are talking about here. There have been scalable and strongly consistent databases since the invention of the concept of NoSQL i.e. HBase and Cassandra (CL=ALL). And the idea that "learning and optimize SQL" would instantly change people's rationale for using these databases shows you don't understand them much at all. There are many factors that come into play. For example you can't use SQL databas…

Cassandra is AP (available and partition tolerant), with tunable consistency. HBase is CP (consistent and partition tolerant, but not always available in case of network partitions). Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization that google achieved on their hardware and network, an innovation I hope will spread. So yes, that's new. I don't think I claimed…

Spanner is CP I think the A part comes from reliability of google network infrastructure which is not part of DB system's design per se.

Re: How NoSQL forced the evolution of a scalable relational database

#68

Post by CuriousSkeptic » Fri, Jul 13, 2018, 9:33 AM UTC

189723954 wrote: post #38

keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a b…

Linq works for trivial left join project style of queries. But for anything actually using the features of modern sql-databases it comes short quickly.

As a example try to express something like “sum(x) over (partition by y order by z)”

I do agree that SQL the syntax leaves a lot to be desired, and a proper relational language with a syntax optimized for actual development, and even better, optimized for 6NF style databases, would be awesome, but linq is not by any stretch sufficient to be that

Re: How NoSQL forced the evolution of a scalable relational database

#69

Post by saurik » Fri, Jul 13, 2018, 9:33 AM UTC

logophobia wrote: post #35

threeseed wrote: post #27

Earlier quoted context omitted.

Not sure what you are talking about here. There have been scalable and strongly consistent databases since the invention of the concept of NoSQL i.e. HBase and Cassandra (CL=ALL). And the idea that "learning and optimize SQL" would instantly change people's rationale for using these databases shows you don't understand them much at all. There are many factors that come into play. For example you can't use SQL databas…

Cassandra is AP (available and partition tolerant), with tunable consistency. HBase is CP (consistent and partition tolerant, but not always available in case of network partitions). Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization that google achieved on their hardware and network, an innovation I hope will spread. So yes, that's new. I don't think I claimed…

> Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization...

This is nonsensical; even in a universe without relativity and with instantaneous communication--an unrealistic world where being able to talk about synchronized clocks can even make sense (as otherwise you fundamentally must deal with an inability to decide the order of two events due to latency)--you still cause indeterminate simultaneity by introducing a partition: your perfectly accurate and infinitely precise timestamps on one side of the partition would no longer be comparable with timestamps on the other side of the partition for purposes of establishing causality between events, as the recent changes being made on one side of the partition "should have" taken place between any older changes on the other side and any transactions being performed right now, and yet they couldn't because they were unknown. Spammer either gives up availability (waiting to verify what happened) or consistency (accepting that it will have to fix the result later) when a partition occurs: they can't do both, fundamentally; the only extent to which there is an engineering challenge here is "maybe you can minimize the number of partitions you have", which I totally believe Google to have done; but, given a partition (which cannot be fully prevented), the tradeoff is clear and unavoidable.

Re: How NoSQL forced the evolution of a scalable relational database

#70

Post by qaq » Fri, Jul 13, 2018, 9:34 AM UTC

189723954 wrote: post #42

stephenr wrote: post #3

> NoSQL came into existence because the databases at the time couldn’t handle the scale required. Arguably the first "NoSQL" database was a DIT, generally accessed via the LDAP protocol. OpenLDAP is one of the better known open source instances, but Novell eDirectory was there in the early part of this century, as was MS AD. In the case of Directory Services, it's a completely different approach to both SQL and the c…

>lazy developers and too much Kool-aid consumption. When your argument is based on personal attacks of the people using that thing, it sounds more like you have an emotional investment in the alternative rather than a rational argument.

It is a very rational argument people want to copy what Big boys are doing often fairly rationally as they want that stuff on the resume. I've seen countless projects using "BIG data" tools for dealing with 100GB or less of data that has very modest grows rate.