Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How NoSQL forced the evolution of a scalable relational database

blog.memsql.com

Post reply on HN
Print view View original

181–190 of 211 posts

Re: How NoSQL forced the evolution of a scalable relational database

#181

Post by collyw »

sytse wrote: post #115
_Codemonkeyism wrote: post #109

Earlier quoted context omitted.

Same here, using Mongo helped us a lot to iterate fast, JSON is much more agile than SQL. We now migrate to PG using 'core' attributes in tables and 'flexible' attributes in JSONB to get the best of two worlds. I agree to the API problem, JDBC and PG/SQL is not as nice as a interace compared to Mongo libs.

When you say JSON is more agile is that because you don't need to write migrations? Do you use a framework that supports migrations like active record or south?

Writing a load of consistency checks in the application layer sounds the opposite of agile to me. Unless you just want sloppy data.

Re: How NoSQL forced the evolution of a scalable relational database

#182

Post by collyw »

189723954 wrote: post #38
keithnz wrote: post #22

I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. While scaling is one of the advantages, I'd be super surprised if many people actually need scaling capabilities ( other than because their design is super inefficient ). Recently I've been inspired to play around with kicking out as many layers between a relational db and a REST Api, largely because I've bee…

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a b…

> Joins are slow and complicated, and ORMs are slow and complicated, so you can never actually use any of these things in a high-traffic environment. As you say, not many people will reach these levels of traffic, but it is still a factor.

I have to clean up crap written by people like you. Go learn how to use a relational database properly instead of writing tosh like this and crap buggy code in the application layer.

Re: How NoSQL forced the evolution of a scalable relational database

#183

Post by lrem »

qaq wrote: post #167
stickfigure wrote: post #165

Earlier quoted context omitted.

I hired a couple junior devs and they did fine. There really aren't that many edge cases. If you're changing data, you're in a transaction and that's always strongly consistent. If you're querying data, nobody really cares if it's 10s behind (and usually it was 1-2s). On the other hand, counting a few thousand things is a pretty major PITA. And doing bulk updates requires map/reduce. So there's a cost... but it's not…

Some of your projects looks like are in e-commerce space so say if I look at my cart and it is missing products I added say 5 sec ago that would be an issue.

That's what session affinity is for. I'm pretty ok with 5 seconds delay between adding something to the cart on my laptop and it appearing on my phone. Or, you may do user affinity and I'll have both the same second, but my latency will go bad the first day I travel overseas.

Re: How NoSQL forced the evolution of a scalable relational database

#184

Post by collyw »

lmm wrote: post #119
Amezarak wrote: post #93

Earlier quoted context omitted.

> Nobody is using joins at scale and every large company that started their scaling journey in a SQL database started by performing multiple queries on a distributed database. I've used joins "at scale" in multiple jobs in (conventional relational) databases up to a few TB with tens of thousands of transactions per second and never experienced any performance problem that was the result of a join, unless you count cr…

> never experienced any performance problem that was the result of a join, unless you count cross joins with no filters. An unindexed join will have major performance problems, and look exactly like an indexed join. > I have never seen an instance where doing a join was faster in code and I don't see how it could be except perhaps in some unusual edge cases. Faster to do the same thing? No. Easier to see which things…

>An unindexed join will have major performance problems, and look exactly like an indexed join.

Then add an index.

Its not exactly difficult in a database. Using a database without indexes is kind of stupid. Why would you do that?

And personally I find SQL to be one of the easiest languages to read. I agree that it's kind of back to front in many ways, but its way easier than trying to work out what happening in some nested loops that someone else has written.

Re: How NoSQL forced the evolution of a scalable relational database

#185

Post by tracker1 »

d4l3k wrote: post #132
ben509 wrote: post #118

Earlier quoted context omitted.

> I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. NoSQL caught on because of what it doesn't do: attempt to implement the relational model. A lot of people were using MySQL in infamous LAMP stacks to set up websites with virtually no business logic. So you had: 1. An application that didn't need the relational model 2. A DBMS that had a very poor implement…

MongoDB certainly was missing a lot of things when they first launched and were very hyped, but now pretty much everything you could want in a database is now in MongoDB. Multi-document transactions + strong consistency: https://docs.mongodb.com/manual/core/write-operations-atomic... Passes Jepsen test: https://www.mongodb.com/mongodb-3.4-passes-jepsen-test Schema: https://docs.mongodb.com/manual/core/schema-validati…

Do they have a scale/redundancy option for write scaling beyond shard + mirror set? I think my bigger issue with mongo, is you really need 6+ servers for a production deployment in many scenarios. It's great for getting a single developer instance up and running. But if I'm sticking to one instance or a couple mirrors for production, I may just reach for ms-sql or postgresql.

Aside: I'm sad that RethinkDB didn't get more of a foothold.

Re: How NoSQL forced the evolution of a scalable relational database

#186

Post by scarface74 »

52-6F-62 wrote: post #172
scarface74 wrote: post #157

Earlier quoted context omitted.

I am an AWS true believer but I hate DynamoDB. It is far more limiting than Mongo DB. MongoDB works great with C#. Using a JSON based database with a statically typed object oriented language goes hand to hand. You don’t have the object relational mismatch and the languages type system helps enforce a “schema” on the database. With C# you work with a MongoCollection And all of your inserts, Linq queries, etc are stat…

It's been a pleasure working with MongoDB in TypeScript as well

I could see that. But I can’t tell from briefly perusing the docs, does Typescript give you transpile time checking of objects that it will allow you to put in Mongo?

Using a dynamically type language with a schemaless database isn’t something I would want to do for a large project.

Re: How NoSQL forced the evolution of a scalable relational database

#187

Post by DenisM »

DenisM wrote: post #170
manigandham wrote: post #141

Earlier quoted context omitted.

They're focusing on customers that can afford it, so it's not about you caring but you not being their primary customer. You can download the developer edition and it does everything except cross-region replication if you want to test it. It's a great product and the real deal, we used it before (as a startup too) and it did the job and is one of the most polished databases out there.

Nick, your account is shadow-banned. Email hn admins to revert the ban.

Aaand it's back to normal. Welcome to the club.

Re: How NoSQL forced the evolution of a scalable relational database

#188

Post by lmm »

collyw wrote: post #184
lmm wrote: post #119

Earlier quoted context omitted.

> never experienced any performance problem that was the result of a join, unless you count cross joins with no filters. An unindexed join will have major performance problems, and look exactly like an indexed join. > I have never seen an instance where doing a join was faster in code and I don't see how it could be except perhaps in some unusual edge cases. Faster to do the same thing? No. Easier to see which things…

>An unindexed join will have major performance problems, and look exactly like an indexed join. Then add an index. Its not exactly difficult in a database. Using a database without indexes is kind of stupid. Why would you do that? And personally I find SQL to be one of the easiest languages to read. I agree that it's kind of back to front in many ways, but its way easier than trying to work out what happening in some…

> Then add an index.

Then you're adding work to your writes, and your database will block your live write transactions until the corresponding index updates are done.

> Its not exactly difficult in a database. Using a database without indexes is kind of stupid. Why would you do that?

You wouldn't intend to, but you might do it by accident. The failure modes can be pretty bad, since an SQL database will take whatever nonsense query you give it and try to run it, even if doing so impacts your live operations. Whereas in many NoSQL systems if you try to use an index that doesn't exist it'll fail fast.

Re: How NoSQL forced the evolution of a scalable relational database

#189

Post by bcheung »

How are people running migrations in these distributed SQL environments? Especially with something like Kubernetes that has canary / rollback functionality. Do people run their migrations as a manual step, is integrated into the CI/CD system?

I can't find anybody talking about how they handle it in a scalable way. Not having a good DB migration story for dev/prod environments with automatic rollout / rollback makes using a SQL DB a non-starter for me.

Most Docker / Kubernetes documentation and their ecosystem assumes the app is fully developed and development doesn't exist. Real world companies develop products and have to perform DB migrations on the production database on a regular basis with each release.

What am I missing?

Re: How NoSQL forced the evolution of a scalable relational database

#190

Post by d4l3k »

tracker1 wrote: post #185
d4l3k wrote: post #132

Earlier quoted context omitted.

MongoDB certainly was missing a lot of things when they first launched and were very hyped, but now pretty much everything you could want in a database is now in MongoDB. Multi-document transactions + strong consistency: https://docs.mongodb.com/manual/core/write-operations-atomic... Passes Jepsen test: https://www.mongodb.com/mongodb-3.4-passes-jepsen-test Schema: https://docs.mongodb.com/manual/core/schema-validati…

Do they have a scale/redundancy option for write scaling beyond shard + mirror set? I think my bigger issue with mongo, is you really need 6+ servers for a production deployment in many scenarios. It's great for getting a single developer instance up and running. But if I'm sticking to one instance or a couple mirrors for production, I may just reach for ms-sql or postgresql. Aside: I'm sad that RethinkDB didn't get…

MongoDB supports a combination of shards and replica sets. Each replica set uses a protocol similar to raft for strong consistency. The replicas are a lot better than just active-secondary mirroring

https://docs.mongodb.com/manual/replication/

There's also Atlas which is the hosted version and makes it very easy to scale up https://www.mongodb.com/cloud/atlas

They also just added a free tier if you want to play with that https://docs.mongodb.com/manual/tutorial/atlas-free-tier-set...

Post reply on HN