How NoSQL forced the evolution of a scalable relational database • Hacker News Board

Re: How NoSQL forced the evolution of a scalable relational database

#171

Post by ben509 » Fri, Jul 13, 2018, 8:29 PM UTC

d4l3k wrote: post #132

ben509 wrote: post #118

Earlier quoted context omitted.

> I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. NoSQL caught on because of what it doesn't do: attempt to implement the relational model. A lot of people were using MySQL in infamous LAMP stacks to set up websites with virtually no business logic. So you had: 1. An application that didn't need the relational model 2. A DBMS that had a very poor implement…

MongoDB certainly was missing a lot of things when they first launched and were very hyped, but now pretty much everything you could want in a database is now in MongoDB. Multi-document transactions + strong consistency: https://docs.mongodb.com/manual/core/write-operations-atomic... Passes Jepsen test: https://www.mongodb.com/mongodb-3.4-passes-jepsen-test Schema: https://docs.mongodb.com/manual/core/schema-validati…

Thanks for posting that; I haven't had a good reason to look at it in quite a while. It makes sense that they're adding those features as they all exist in SQL DBMS's because they're a need.

Re: How NoSQL forced the evolution of a scalable relational database

#172

Post by 52-6F-62 » Fri, Jul 13, 2018, 8:36 PM UTC

scarface74 wrote: post #157

stickfigure wrote: post #123

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully hosted, infinitely scalable, zero-maintenance, transactional, reliable, and - at least with Google's offering - scales down to a free tier. They aren't perfect or applicable in every situation, but they're cheap and easy enough to allow a one- or two-programmer…

I am an AWS true believer but I hate DynamoDB. It is far more limiting than Mongo DB. MongoDB works great with C#. Using a JSON based database with a statically typed object oriented language goes hand to hand. You don’t have the object relational mismatch and the languages type system helps enforce a “schema” on the database. With C# you work with a MongoCollection And all of your inserts, Linq queries, etc are stat…

It's been a pleasure working with MongoDB in TypeScript as well

Re: How NoSQL forced the evolution of a scalable relational database

#173

Post by ben509 » Fri, Jul 13, 2018, 8:47 PM UTC

Sjenk wrote: post #39

keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

Not only easy in Javascript, it is easy in any language that has easy serliazation to JSON. I always use it for my own hobby projects because I dont have that much SQL knowledge. Which makes Nosql easy choice for me, just serialize your object into json and push it to your DB.

Pushing a serialized object into a distributed hash definitely solves some problems.

And you have finite time, so if the choice is between that solution that you can do now and searching for another solution that may or may not be better, it's often reasonable to go with the known solution. In economics, this is known as "rational ignorance."

The value proposition of learning will change, though. You will find it's worth it, especially if you move beyond hobby projects and have to deal with any kind of business logic, to spend some time learning how databases work, and maybe even the math behind them, the relational model.

There are a lot of mediocre or awful references out there. If you need a good text on the subject, I recommend CJ Date's "Introduction to Database Systems." Note, though, I said good, not easy.

Re: How NoSQL forced the evolution of a scalable relational database

#174

Post by astine » Fri, Jul 13, 2018, 9:06 PM UTC

189523954 wrote: post #102

Amezarak wrote: post #98

Earlier quoted context omitted.

> Yes. A programming language, combined with a database like mongo or an ORM, allows you to create complex queries much more quickly compared to SQL. You can maybe go into stored procedures and start doing loops and recombining multiple queries in there, but programming languages like javascript etc are typically much nicer than those used in stored procedures. I guess this is where we differ. I've written many SQL q…

Yes well it would be ideal if I had code to show you and compare it to the SQL, but all of that is at my old workplace and I will get PTSD if I ever look at it again. I agree that loops in SQL are not great. Loops in programming languages are fine obviously. There are just many more and nicer constructs in a programming language to manipulate data. Maybe you have a root table that is anchoring your query, say a Staff…

I use both raw SQL and the entity framework on a regular basis. That Linq to entities query gets translated pretty directly to SQL. Each of those includes translates directly to a left join on whatever column is specified as the key. You would need a group by, but no self joins. Assuming that you are familiar with the database schema and are proficient in SQL, it shouldn't be any slower to write the SQL version than the C# version.

It depends a little on the exact columns you need, but it would look something like this, which is a supper common form for a SQL query:

    select 
    m.id Manager,
    count(sh.id) as ShiftsCompleted,
    sum(iif(sh.coworker = m.id,1,0) as ShiftsWithManager
    from staff s
    left join manager m
    on m.id = s.manager_id
    left join shifts sh
    on sh.id = s.shifts_id
    group by s.id, m.id

The SQL version has the advantage that it's more intuitive to specify the columns that you need so if your query is running slow because you're pulling too much data (something that's happened to me a bunch,) you can omit unneeded columns pretty easily.

Re: How NoSQL forced the evolution of a scalable relational database

#175

Post by rbyrne » Fri, Jul 13, 2018, 9:18 PM UTC

ben509 wrote: post #118

keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

> I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. NoSQL caught on because of what it doesn't do: attempt to implement the relational model. A lot of people were using MySQL in infamous LAMP stacks to set up websites with virtually no business logic. So you had: 1. An application that didn't need the relational model 2. A DBMS that had a very poor implement…

This is exactly how I feel.

I am currently working on a project for a client where they expect to have about 3,000 users each with some basic profile information, and the ability to upload some documents etc - nothing too crazy. I don't think they ever have to worry about scaling to 50,000 users and performing any data-intensive database queries. Why not use something like Mongo in this situation?

Re: How NoSQL forced the evolution of a scalable relational database

#176

Post by stickfigure » Fri, Jul 13, 2018, 9:23 PM UTC

kodablah wrote: post #166

stickfigure wrote: post #164

Earlier quoted context omitted.

This is terrible advice for startups. While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features. The probability that you will want to cross-cloud distribute your app is infinitesimally small compared to the probability that you will…

> While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features. I'm not sure if you're responding to a different comment or missed where I suggested just use the cloud Postgres. I specifically mentioned time to market and acknowledge it.…

I love Postgres, I really do. And Google's Cloud Postgres is my "default" axe.

It will not easily give you scale. It will not give you the ability to "not think about your database". You will not be able to go fishing (or wherever) with confidence that your app will just keep running without you.

I find there are two sweet spots for GCD (and probably DynamoDB, although I haven't used it in anger):

1) Persistence for really small and/or simple systems. Services that need a tiny bit of persistent state and really could do without the fuss of setting up databases and running migrations.

2) Persistence for big systems that need absolute rock solid availability, and can relax (or shunt off) analytical needs. Postgres is always one poorly-thought-out migration script away from downtime, and how many organizations test their migration scripts against full copies of the production database? Even Stripe has database-driven downtime, with their armies of developers and devops. Your two-person startup hasn't got a chance.

For everything else (and I'll be the first person to say that includes the majority of web applications!) there's Postgres.

Re: How NoSQL forced the evolution of a scalable relational database

#177

Post by collyw » Fri, Jul 13, 2018, 9:28 PM UTC

ben509 wrote: post #118

keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

> I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. NoSQL caught on because of what it doesn't do: attempt to implement the relational model. A lot of people were using MySQL in infamous LAMP stacks to set up websites with virtually no business logic. So you had: 1. An application that didn't need the relational model 2. A DBMS that had a very poor implement…

NoSQL caught on because our industry is like the fashion industry and we need to follow the fads year after year. Plus a lot of FUD about relational databases not scaling (it's true that NoSQL will scale certain problems way better than relational, but the vast majority of people were/are not writing applications at that scale).

Re: How NoSQL forced the evolution of a scalable relational database

#178

Post by collyw » Fri, Jul 13, 2018, 9:30 PM UTC

rbyrne wrote: post #175

ben509 wrote: post #118

Earlier quoted context omitted.

> I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. NoSQL caught on because of what it doesn't do: attempt to implement the relational model. A lot of people were using MySQL in infamous LAMP stacks to set up websites with virtually no business logic. So you had: 1. An application that didn't need the relational model 2. A DBMS that had a very poor implement…

This is exactly how I feel. I am currently working on a project for a client where they expect to have about 3,000 users each with some basic profile information, and the ability to upload some documents etc - nothing too crazy. I don't think they ever have to worry about scaling to 50,000 users and performing any data-intensive database queries. Why not use something like Mongo in this situation?

Because your data is probably relational and data being consistent is probably beneficial?

Re: How NoSQL forced the evolution of a scalable relational database

#179

Post by gwbas1c » Fri, Jul 13, 2018, 9:36 PM UTC

ChicagoDave wrote: post #110

NoSQL is an excellent technology for rapid R&D, but once a domain “settles”, data should be modeled and data stores should be switched to relational or graph backends. If a system already has a well-defined domain, NoSQL adds little value. Of course you could leverage AWS DynamoDB and reduce cost, but you still have downstream implications for things like reporting, which requires a known schema with relational parad…

I used to tell people that MongoDB was the "visual basic of databases." Easy and useful for very simple applications.

Re: How NoSQL forced the evolution of a scalable relational database

#180

Post by jokoon » Fri, Jul 13, 2018, 9:45 PM UTC

Can't you have ACID with write on ram first, write on disk second, delay the commit by committing after writing on disk, and have some tolerance for the time between write on ram and write on disk? How probable is it that a crash happens in that window? The data loss would seem very minimal and rare.

And is ACID always so important for most databases? I don't think most databases in the world are related to banks or require that much guarantees and safety. Better hardware often achieves that level of safety.

Other side note, I wish there would be alternative to the SQL "english sentence" flavor of the language, so that one could give more precise parameters for querying data, instead of building a sentence query which doesn't always make a lot of sense when compared to a non-sentence programming language.