61–70 of 172 posts
I used both NoSQL and SQL databases. Proper replication topologies (e.g. multi-master configurations) and JSON columns killed the NoSQL movement. I'm glad the NoSQL movement existed and forced the SQL camp into out-of-the-box thinking, but I wouldn't go for any NoSQL solution today on a greenfield project.
I use MongoDB as a front-end that does full-stack prototypes from time to time in Node, and it's great using javascript from start to finish, you don't have to switch mental gears, so creating an MVP it's fast... but. Sometimes I feel like I'm just avoiding SQL for no reason, not using any of the advantages of NoSQL, and only using Mongo as a SQL-wannabe with a shoehorn like mongoose. I wonder how many MongoDB projec…
I was using Mongo and NodeJS a lot but then switched to Python. Now I find the world of ORMs to be onerous. I just want to define tables in MySQL and just run raw SQLs. The best compromise I have found so far. I mean it does involve boiler plate code but ultimately it's an MVP.
And you are right, most of MongoDBs uptick was because of the reasons you state. E2E Javascript and in addition the ability to query nested JSON natively.
MongoDB is like snapchat for database
There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…
This thread is like a piñata that gets pulled out every 6 months or so take a wack at MongoDB and talk about how everyone should just use PostgreSQL for every database problem no matter whether it's relational or not.
No-one cares about whether MongoDB is useful or not.
I used both NoSQL and SQL databases. Proper replication topologies (e.g. multi-master configurations) and JSON columns killed the NoSQL movement. I'm glad the NoSQL movement existed and forced the SQL camp into out-of-the-box thinking, but I wouldn't go for any NoSQL solution today on a greenfield project.
Earlier quoted context omitted.
What if you didn't need relational queries and was dealing with a large volume of data that needed to be sharded?
In what scenarios does one work on data without any relations? Honest question - perhaps I'm too used to think in SQL but even with e.g. a completely flat bunch of documents one usually needs at least access control (i.e. document owners, which is a relation).
It's why Data Lakes become so commonplace because it was an easy way to just get all of the data out of the silos into one place so that the business could attempt to join between them.
And it's why MongoDB (and other unstructured friendly storage systems) have remained so popular because the reality is that those datasets are not well governed, no-one knows the schemas and no-one knows how to join between them.
Earlier quoted context omitted.
That sounds like a great use case for OLAP and fuzzy search solutions. Eg. Dump all of the data to S3 and search it in parallel via AWS Athena. If you want fast fuzzy search, then elasticsearch is built for this. If the data is small enough that you want B-tree indexes (with the accompanying slower writes and lower query latencies), then any SQL database with a JSON column will work.
> Eg. Dump all of the data to S3 and search it in parallel via AWS Athena. Too proprietary. Can't be run locally, on robots, etc. Can't be transferred to other cloud providers. Also S3 is horribly slow to write or delete thousands of records at once.
Every major cloud provider has a blob store. Most are S3-interface compatible, or can be fronted by something which is, like Minio.
> Can't be run locally, on robots, etc.
If the contents of your S3 keys are just line-delineated JSON files, you can easily download those files and run scripts or process them locally.
> Can't be transferred to other cloud providers.
Again, not true — a tool like rclone handles this case in a single command. Depends on the amount of data, but if anything it's easier to move a bunch of flat files between providers than copying a database backup around. You have to pay for egress bandwidth, though, of course.
> Also S3 is horribly slow to write or delete thousands of records at once.
If you're keeping one JSON record per S3 key, that is a blatant misuse of the tool and performance will be terrible. On the other hand, if you batch records into files of appropriate sizes, it's very cost-effective to store and query via Athena, Spark on EMR, etc.
If you need individual record-level access as identifiable by a key, then you'd probably be better off with Redis or Memcached. (though, those will not be as good for bulk offline processing) It's all about your access patterns.
Earlier quoted context omitted.
There are plenty of modern, distributed RDBMSes that make sharding transparent to the user (E.g. cockroachDB, yugabyte, vitess, many cloud offerings, etc.). Most NoSQL databases end up adding transactions because they are important, and thus the scale advantages for NoSQL systems over relational databases are diminishing, if remaining at all.
Latency at p99 is substantially better at scale with a key value lookup in nosql over postgresql. Thing is most people don’t have scale these days. You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd. Until you exceed that you don’t necessarily have scale.
And then that box falls over because the entire region fails like just happened yesterday with OVH. Or it just randomly fails like has happened to me with AWS dozens of times.
Vertically scaling a database on your own cloud instances is amateur hour. Either use a cloud-managed database or one that is highly available.
Earlier quoted context omitted.
In what scenarios does one work on data without any relations? Honest question - perhaps I'm too used to think in SQL but even with e.g. a completely flat bunch of documents one usually needs at least access control (i.e. document owners, which is a relation).
Search engines
There were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have fa…
How so? All the documentation on upgrading say you need to shutdown each instance to then step up to the next minor level.
For instance 3.4 -> 3.6 -> 4.0 etc.
https://docs.mongodb.com/manual/release-notes/4.0-upgrade-st...