Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

241–250 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#241
post #124

Earlier quoted context omitted.

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

This. My company used mongo for years before we got our shit together. Schmemas were always implicit (until we got our shit together and started defining and enforcing them with Python Schematics). Migrations were crazy scripts you run in prod or hacks you stick into your code to "transition". And yes, surprise constrains left and right causing awful anti-patterns. One-character key names to save disk. Hashed values…

Basically reinventing many of the features of a relational db at the application level.

Re: MongoDB queries don’t always return all matching documents

#244
post #43

The article is interesting, but title is fud. Besides, all this is not unexpected: > How does MongoDB ensure consistency? > Applications can optionally read from secondary replicas, where data is eventually consistent by default. Reads from secondaries can be useful in scenarios where it is acceptable for data to be slightly out of date, such as some reporting applications. https://www.mongodb.com/faq

This is not related to reading from secondaries. This issue can occur in single node systems.

Yes, as I said, all this is NOT unexpected from them.

Re: MongoDB queries don’t always return all matching documents

#245
This is exactly what I thought. You are a DBA. You are not a programmer. Whenever I read these threads I think "either these people are all DBAs or they have an extreme emotional attachment to a particular database".

There it is - this is a thread full of DBAs rebelling against a database that takes away their power and responsibility.

Re: MongoDB queries don’t always return all matching documents

#246
post #124

Earlier quoted context omitted.

> Because some people can't stand having to work with SQL,migrations,schema and constraints The thing is, if you actually try and write an app using MongoDB, you will rapidly find that you: 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). 2) Have schemas (except they'll be implicit and undocumented) 3) Constraints (exce…

Exactly. While the process of designing the structure of your data can make you feel like “you're not getting real work done”, in the long run, it actually prevents headaches caused by inconsistent data. Data always has a structure, it's just that some people are too lazy or mentally feeble to figure out what it is.

For me that is the most important aspect of starting / designing an application. If the data model is accurate, then the code falls into place easily. If its not quite right, more and more code ends up in the application trying to make up for the poor data model.

Re: MongoDB queries don’t always return all matching documents

#247
MongoDB reminds me of an old saying that if you have a problem and you use a regex to solve it, you end up with two problems.

I have personally used MongoDB in production two times for fairly busy and loaded projects, and both times I ended up to be the person that encouraged migrating away from MongoDB to a SQL based storage solution. Even at my current job there's still evidence that MongoDB was used for our product, but eventually got migrated to PostgreSQL.

Most of the times I've thought that I chose the wrong tool for the right job, which may be true, but still leaves a lot of thought about the correct application. Right now I have a MongoDB anxiety - as soon as I start thinking about maybe using it(with an emphasis on maybe), I remember all the troubles I went through and just forget it.

It is certainly not a bad product, but it's a niche product in my opinion. Maybe I just haven't found the niche.

Re: MongoDB queries don’t always return all matching documents

#248

Strongly biased comment here, but hope its useful. Have you tried ToroDB ( https://github.com/torodb/torodb )? It still has a lot of room for improvement, but it basically gives you what MongoDB does (even the same API at the wire level) while transforming data into a relational form. Completely automatically, no need to design the schema. It uses Postgres, but it is far better than JSONB alone, as it maps data to re…

How does ToroDB handles sharding across multiple instances?

Re: MongoDB queries don’t always return all matching documents

#249

Earlier quoted context omitted.

How does it compare to Couchbase? That seems to be lighting the world on fire in that space lately.

I'm not sure if lack of overbearing marketing speak counts for something, but RethinkDB definitely has that going for it. I'm not an expert on couchbase (and neither on RethinkDB, to be frank, though I am a huge fan), but here's what RethinkDB has going for it: - Changefeeds - easily open a persistent connection to the server and get updates when the results of an almost arbitrary query changes. - Joins - Expressive…

I'm ex-Couchbase, so I can probably give a reasonably informed but independent view on this.

Firstly, regarding the marketing, it may not have been to many people's tastes - but it definitely worked, and achieved a lot of what was set out in terms of raising the awareness of what was a decent product that wasn't as well known as its competitors. There may be cases where people avoid it because they don't like the marketing, but the reality, having seen its effect, is they are in the minority, and would probably serve themselves better by assessing products based on technology rather than spiel.

Now, on the actual technology!

What Couchbase has historically been good at is highly scalable Key-Value access, at very high performance and low latency. Performance is comparable to Redis, but CB much more mature sharding, clustering and HA. e.g. fully online growing/shrinking of cluster, protection from node failures, rack/zone failures and data center failures. Redis may be a good fit for single-machine caching situations, and also has its own advantages in terms its datastructures support, etc.

Quality of SDK's is pretty subjective, but I'd say the 2.x re-write of Couchbase SDK's makes them very solid. The Java SDK in particular is extremely good both in performance and by providing native RxJava interfaces.

In terms of query interface, there's geospatial and a new freetext capability on the way.

Couchbase chose down to go down the route of a SQL based interface as their main query language. This seems to be a bit love/hate with developers with some delighted and some perplexed. Maybe for devs it should really be about higher level interfaces like Spring are increasingly important anyway?

The native interface being SQL based is usually very popular with the BI / Reporting side of things.

Changefeeds (continuous query?) is a feature not in Couchbase which I would very much like to see in the future. One thing I would say is that it's something you have to be very careful in the design of to ensure scalability and performance. Consistency is something which would obviously need thought as well.

Re: MongoDB queries don’t always return all matching documents

#250
post #223

> 1) Have migrations (except they're going to be some scary ad hoc nodejs script that loop through your document store and modify fields on the fly). Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql. What is even worse is that, despite nobody here acknowledging it, must SQL databases are used with an ORM. Now in your migration script you have to hac…

> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" f…

> I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice. The difference between, eg, merging your "first_name" and "last_name" fields into a single "name" field with Mongo or a SQL DB is basically that the SQL DB has better tooling so it'll be cleaner, faster, and more concise.

No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

> Also false. Nothing requires you to use the ORM commands if they don't make your life easier; literally last week I wrote a migration for some ORM based code using pure vanilla SQL because it was more appropriate. If you know what your ORM is doing, it's trivial. And if you don't, well, you have bigger problems.

Err no it isn't false. It is based on real-world experience, rather than "theory" about "what you can do". In the real world, people use ORMs and the ORM migration library, and then they find that the migration library doesn't do what they need it to do, so they have to use raw SQL. But then they run into the problem that because the ORM handles all of the generation of indexes and column names and so on, they have to figure out the correct naming of everything so that the ORM understands what you have done. This often results in failure because the ORM does not want you to tinker with the schema yourself.

> With a relational DB: Check the table schema.

Which requires you to context switch from the code to the schema and piece together all of the relations based on foreign keys. It is much easier to do this in the programming language, but as I have figured out, most of the people in this thread are actually DBAs and have no idea about programming.

> No. Mongo still lists flexible schemas as a feature, ie, they do not enforce any specific schema.

err yeah I was thinking of constraints. I heard some news about it the other day.

> Yeah, but what relational DBs are very good at is enforcing foreign key constraints. If your data is relational at all (and let's be real, very, very little world data isn't at least somewhat relational), you'll need it, and Mongo doesn't have it. (Well, Mongo doesn't even have foreign keys, but it lacks the equivalent feature for dealing with denormalized data too.) Managing to set some non-numeric characters in your phone number field when some of your app code contains a hard assumption it will be numeric is bad, yeah. But when you start screwing around with relations, foreign keys, failing to properly propagate changes to every copy of a denormalized data structure...oh man, you can spend days trying to untangle that mess. And it's a class of error that relational DBs don't have unless you misuse them badly.

This isn't exactly correct. Foreign key restraints are required when you denormalise the data. Many of mongos relationships are going to be in the same document and therefore a foreign key isn't necessary. You do have to manage consistency within the code though, which is something you don't have to do with a SQL database. Having said that, there are many consistency issues that can occur in a SQL database as the constraints are limited in what they can enforce. You have to start to get into triggers and stored procedures to enforce more complex business rules.

Post reply on HN