Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you still use MongoDB?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 243 posts

Re: Ask HN: Do you still use MongoDB?

#191

Post by wlll »

arnejenssen wrote: post #187
wlll wrote: post #175

Earlier quoted context omitted.

How do you handle issues around data integrity? From http://jepsen.io/analyses/mongodb-4.2.6 > Jepsen evaluated MongoDB version 4.2.6, and found that even at the strongest levels of read and write concern, it failed to preserve snapshot isolation. Instead, Jepsen observed read skew, cyclic information flow, duplicate writes, and internal consistency violations. Weak defaults meant that transactions could lose writes…

Interesting. I haven't had a usecase where data integrity is critial yet. A project i'm working on now will have credits and accounts. To accomplish that in mongo. I create a transaction with with "pending" status. Then i try to debit the source account, and credit the destination account and I add the pending transaction to the accounts. If that works I set the transaction to "committed" and I remove the pending tra…

If you haven't already I would absolutely read the Jepsen articles on MongoDB, at least so you are aware of the risks and failure states. There's some stuff about transactions that may be relevant too.

Re: Ask HN: Do you still use MongoDB?

#192

Post by smoe »

subhobroto wrote: post #95
smoe wrote: post #82

Earlier quoted context omitted.

I never understood the appeal of the JSON to SQL columns workflow. At least with the ORM(-ish) tools I worked with, it always felt much more straightforward to just change classes within the application code and automatically generate the respective migrations files to be run on the relational database, than having to interact as a human with json (for the app as well as for business intelligence and reporting/monito…

> I never understood the appeal of the JSON to SQL columns workflow. The appeal is non-technical. Writing good migrations, having tests around them to ensure they didn't leave the DB in an inconsistent state if a subset of them failed requires good understanding of a RDBMS and the specific product. You'll be surprised how many engineers don't meet that criteria. The JSON to SQL columns workflow allows any developer t…

Thing is, for the early stage development or prototyping, where JSON to SQL is most praised for, to be completely honest, I don't write good migrations either.

I change the application code, trigger the cli to generate the sql migration and be done with it. In 99% of cases I don't even look at the generate file, I don't write specific migration tests, etc. If migration runs without error and the application tests pass in development and staging there is a good chance it will as well in production or auto rollback saving my ass.

This works until it doesn't and you have to become much more vigilant. But at that point I much rather build upon the relative robustness and consistency of the sql I got so far, than JSON that is all over the place.

Maybe it's my tooling, but almost every time I made a JSON column in Postgres for the sake of saving time in the past, it slowed me down almost immediately after adding it.

Re: Ask HN: Do you still use MongoDB?

#193

Post by doublesCs »

takeda wrote: post #190
scarface74 wrote: post #67

Earlier quoted context omitted.

You make technical decisions based on what happened a decade ago? Do you make all decisions based on ten year old information?

> You make technical decisions based on what happened a decade ago? Do you make all decisions based on ten year old information? Yes? The problem was not that MongoDB was young, the problem was that authors of it had no idea about databases when they started their work. They stored data in RAM and get fastest benchmarks, because they didn't worry about persisting the data on disk. They outright lied in their document…

> They outright lied in their documentation about the guarantees.

> It wasn't the being early product part, it was the misrepresenting the truth part that hurt their credibility.

That's why I replied to the person you're responding to that his view of software development is cartoonish, but I don't think he got it.

> the problem was that authors of it had no idea about databases when they started their work

Speaking of which, have you heard of influxdb? It's going down the exact same route.

Re: Ask HN: Do you still use MongoDB?

#194

Post by danenania »

garethmcc wrote: post #118
tmlee wrote: post #68

Earlier quoted context omitted.

While there are many people here who are sharing their bad experience with MongoDB, just curious if you all find the experience with DynamoDB similar? Since they are both of the NoSQL family.

DynamoDB is a completely different beast and would use no other data store unless I had to. It can pretty much handle any transactional workload I need. It's cheap, its fast and it scales super high. Don't need much more

This came up in a thread a few days back, but people considering it should note that Dynamo’s transactions are limited to 25 rows. That may be enough for most operations, but I definitely wouldn’t say it can handle “any transactional workload”. I ran into this limit pretty quickly when trying it out.

Re: Ask HN: Do you still use MongoDB?

#195

Post by thomascgalvin »

rubyfan wrote: post #115
thomascgalvin wrote: post #102

Earlier quoted context omitted.

This is (probably) an artifact of Mongo's schema-less nature; when you don't have tables with structure, every document you store has to detail its own schema inline. In a relational database, you have columns with names and types, and that info is shared by all of the rows. In Mongo, every cell has to specify its name and type, even if that layout is shared by every other cell in the document. Mongo's way is more fl…

/caveat i know nothing about how mongo is storing data and haven’t used it since 2010 it doesn’t really have to approach the schemas that way. one would think it would be optimized for repeat schema in the same way one might create the schema definition and then reference it in packing and unpacking the data. seems like if there was schema overhead taking up storage unnecessarily that could be optimized relatively ea…

> one would think it would be optimized for repeat schema

I don't think that's the problem Mongo is designed to solve. Mongo's promise was the ability to work with un- and semi- structured data, and it would make sense if its optimizations were focused on that problem, not on reducing overhead when someone tries to strongarm it into being MySQL.

Generally speaking, if your data is structured well enough that you can define a schema ahead of time, you're better off with a traditional RDBMS, because that's the problem an RDBMS is designed to solve.

Re: Ask HN: Do you still use MongoDB?

#196

Post by bulldoa »

codr7 wrote: post #103

I've promised myself to never touch MongoDB again. Worked for a valley startup back in 2013 that picked Mongo as primary data store because the CEO liked the simplicity and was too busy with the future to learn anything more complicated. I only implemented a couple of features before I got out of that mess. But from my experience, compared to dozens of SQL and NoSQL databases I've worked with; it was definitely the w…

Whats your opinion on postgres vs nosql like cassandra and aerospike, fundamentally is there any reason that postgres can't scale as well as nosql? If I store key value in postgres and add read replica to scale read and partition to scale write will I not be able to keep up with other nosql solutions? If so what are the reasons?

Re: Ask HN: Do you still use MongoDB?

#197

Post by franciscop »

takeda wrote: post #180
franciscop wrote: post #30

I haven't used it for a while and I've been planning on learning Postgres after much praise from a friend (and the internet at large). I find DB design and migrations quite difficult though, any book/course recommendation?

I don't have a specific book for you, I myself learned about relational databases at school, not sure why is not being taught everywhere. The things that you have problems with aren't really specific to PostgreSQL so if you are looking for a book while one based on PostgreSQL might be easier, there might be better books that talk about relational databases and what you learn will still apply. About migrations, from P…

I studied a different Engineering, not CS, so never really had any course (besides basic C). I've read quite a bit and understand the normalization forms, but it's more from a practical sense. What data should we include, how to structure it so that it's not too complex, but doesn't come back biting me, etc.

A good example (but not only!) is login. You can have just user session, or session+token in DB, or session + devices (1 active token/device), etc. So maybe seeing business problems and how the DB was structured to help would be the best here.

> The problem comes that developers want to have it version controlled.

Oh maybe that's the reason I found those so tricky. I don't explicitly care about VCS, but I definitely care about migrations going wrong, and hopefully being able to revert if it goes wrong. Maybe if I get myself comfortable enough with manual migrations this is not such a huge topic though.

Re: Ask HN: Do you still use MongoDB?

#198

Post by bmm6o »

dreur wrote: post #62

There is a very recent Jepsen report on MongoDB. http://jepsen.io/analyses/mongodb-4.2.6 > Jepsen evaluated MongoDB version 4.2.6, and found that even at the strongest levels of read and write concern, it failed to preserve snapshot isolation. Instead, Jepsen observed read skew, cyclic information flow, duplicate writes, and internal consistency violations. Weak defaults meant that transactions could lose writes and…

That's a great read, Kyle seems to really enjoy taking apart their claims.

Re: Ask HN: Do you still use MongoDB?

#199

Post by subhobroto »

smoe wrote: post #192
subhobroto wrote: post #95

Earlier quoted context omitted.

> I never understood the appeal of the JSON to SQL columns workflow. The appeal is non-technical. Writing good migrations, having tests around them to ensure they didn't leave the DB in an inconsistent state if a subset of them failed requires good understanding of a RDBMS and the specific product. You'll be surprised how many engineers don't meet that criteria. The JSON to SQL columns workflow allows any developer t…

Thing is, for the early stage development or prototyping, where JSON to SQL is most praised for, to be completely honest, I don't write good migrations either. I change the application code, trigger the cli to generate the sql migration and be done with it. In 99% of cases I don't even look at the generate file, I don't write specific migration tests, etc. If migration runs without error and the application tests pas…

> I much rather build upon the relative robustness and consistency of the sql I got so far, than JSON that is all over the place.

Yup.

> almost every time I made a JSON column in Postgres for the sake of saving time in the past, it slowed me down almost immediately after adding it

Backs my experience fixing software from companies that had terrible data loss issues after these JSON hacks reared their ugly head.

I am not saying JSON does not belong in a DB - I have many well tested and audited code that does do that. What I am saying is I have seen things go wrong far too much, for me to trust that someone who proposes that approach to actually know what they are getting themselves into.

Plus, it's way cheaper to use a custom built KeyValDB that actually supports object semantics if one does not mean to use RDBMS semantics.

Anyways, I am being severely downvoted and cant respond until a few hours between posts so I wont be responding any longer on this thread as I have a lot of work to do the next few days and cant wait around.

See my other comments. Contact info in my profile, would love to be in touch!

Re: Ask HN: Do you still use MongoDB?

#200

Post by aabbcc1241 »

heliodor wrote: post #63
josephg wrote: post #27

No. Every time I've used mongodb we've ended up regretting it for one reason or another. And migrating to a different database after launch is a huge hassle. I've done a couple projects where we kicked off with postgres using JSONB columns for early iteration. Then we gradually migrated to normal SQL columns as the product matured and our design decisions crystallized. That gave us basically all the benefits of mongo…

Yep, just regret and misery. Back in 2010 when the MongoDB hype was high, the well-known company where I was working at the time decided to build the next version of the product using MongoDB. I was on the analytics team and had to code a whole bunch of intricate map-reduce jobs to extract summary data out of Mongo. I'd repeatedly head to the product team and ask them to explain the edge cases I was seeing in the dat…

Would that be better if they had use typesceipt or jsflow for the data type ?
Post reply on HN