Live data from Hacker News

MongoDB queries don’t always return all matching documents

engineering.meteor.com

301–310 of 419 posts

Re: MongoDB queries don’t always return all matching documents

#301

Earlier quoted context omitted.

> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language. SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously. > Err no it…

> SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously. You can argue how great SQL is as a language all you like. Hands up everyone in here who likes writing raw SQL.. lets point out how unrepresentative t…

You have a house with an array of rooms, and each room has a house with an array of rooms? Etc.

Unless that's a reference to a house in the room class, that is not in any way a good design. And if it is a reference, then you implement it like this:

House is a table

Room is a table that has a foreign key that references the House primary key.

Can't get much simpler than that!

Re: MongoDB queries don’t always return all matching documents

#303
post #261

Earlier quoted context omitted.

Tl:dr - you used a crappy ORM and now think all ORM libraries sick. So... use MongoDB... which is always accesses through an ORM... All your comments come across as confused mate. Enforcing arbitrary constraints belongs in the business logic of your app, but dealing with consistency issues does not. Postgres handles row constraints just fine, you can enforce any arbitrary logic (i.e colX != colY and its not the first…

Mongodb isn't always accessed through an ORM. I have no idea what you are talking about. > Enforcing arbitrary constraints belongs in the business logic of your app Oh so you have some constraints enforced in your database, like the length of a strength can't be greater than 5, yet others you consider arbitrary just because they are not in SQLs list of constraints? How on earth did you define arbitrary otherwise? Any…

What do you mean what do I mean? I just gave you an example. You don't seem to understand that "consistency" in a database, is simply defined within the realms of the database itself. It isn't a universal concept of "consistency". It refers to the fact that the database will enforce certain constraints and checks. You can even denormalise your data and make it so that a persons name is different in different places. My whole point, which none of you are able to grasp because you aren't programmers, is that there are MORE CONSTRAINTS then there are options available for constraints in SQL. There are business constraints that lead to "consistency" in the real-world usage of the term, issues.

Consistency in database terms is limited by the types checks the database performs. Someone earlier said you can write any types of checks you want because pgsql supports writing constraints in any programming language you want. This does not counteract my point which was to refute the original comments proposition that IN MONGODB YOU HAVE TO WRITE YOUR CONSTRAINTS IN THE CODE AND THAT IS NOT THE CASE WITH SQL.

Jesus guys, you are slow learners.

Re: MongoDB queries don’t always return all matching documents

#304

Earlier quoted context omitted.

> Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? Because some people can't stand having to work with SQL,migrations,schema and constraints, it's as simple as that ( That's not my opinion,that's just the rational behind MongoDB). Even if you use Postgres with the Json column type, you still need to write SQL queries and schemas. In the context of…

So get them to use ToroDB.

Never heard of ToroDB. Just checked out the website and it looks interesting, however the tagline "The first NoSQL and SQL database" is untrue.

At least OrientDB has had both schema+schema-free and SQL + NoSQL querying interfaces.

That is, you can optionally supply a schema for your documents. IIRC you could choose either schemaless, schema or mixed (where mixed allows fields not in the schema to exist as schemaless fields).

The default query language was SQL with "enhancements" (to allow for graph traversal), but you could also query with Gremlin. Not sure if this is still the case or not as I don't use OrientDB.

The above was true in 2012 and possibly a lot earlier. I see ToroDB's first Github commit was in 2014.

Re: MongoDB queries don’t always return all matching documents

#305
post #41

Said it before, will say it again... "MongoDB is the core piece of architectural rot in every single teetering and broken data platform I've worked with." The fundamental problem is that MongoDB provides almost no stable semantics to build something deterministic and reliable on top of it. That said. It is really, really easy to use.

As a guy who works on ACID database internals, I'm appalled that people use MongoDB. You want a document store? Use Postgres. Why on earth would you use a database that makes so little in the way of guarantees about what results you get from it? I think most people have really low load and concurrency, so things seem to work. When things get busier you're in for a world of pain. Look I get that's it's easy to use and…

Here's one:

You're dealing with a torrent of incoming semi-unstructured data, where losing a good chunk of it is minor nuisance because you only need a decent sample, from which you extract data.

In those kind of scenarios, making it easy to work on the code can often be far more important than reliability.

I have a project like that now. I'd love to use Postgres, and probably will eventually once things "settle down" and we know what data we need to store. But or now MongoDB is the "quick and dirty" solution. We define a schema client side for everything we nail down, so as we nail down more aspects of what data to process, it gets easier to transition to a proper database.

As ORMs get better support for Postgres' JSON capabilities, it will likely get less and less appealing to use MongoDB for stuff like this too.

Re: MongoDB queries don’t always return all matching documents

#306

Earlier quoted context omitted.

> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language. SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously. > Err no it…

> SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously. You can argue how great SQL is as a language all you like. Hands up everyone in here who likes writing raw SQL.. lets point out how unrepresentative t…

Err yeah you are right. Ignore the whole room class.

class House { Array[] rooms; }

A house has many rooms. That is all.

Re: MongoDB queries don’t always return all matching documents

#307

Earlier quoted context omitted.

Thank you. Not everything is easy. This is the difference between engineering and 'hacking'. Hacking is not something to aspire to; it's something you do because of crushing, external pressures.

Hacking is trying to get the SQL representation into something programmers use via the way of an ORM. At the end of the day, a SQL database doesn't represent the data in a way the programmer uses the data. It has to be transformed and that is a bigger hack than loosening some constraints.

ORMs are a really bad attempt to force a square peg into a round hole. The mismatch between the relational model and object-oriented design principles is simply too big.

In the relational model:

(0) A relation is a collection of tuples of primitive values. Every relation has a relation schema, which determines the arity of its tuples and the type of each tuple component. In other words, the relational model is first-order.

(1) There are a few basic operators for computing relations from other relations (relational algebra).

(2) There is a mathematical theory (database normalization) of how to design primitive relation schemas to avoid storing duplicate information, and running into insertion, update and deletion anomalies.

On the other hand, in a pure object-oriented program:

(0) An object is a collection of data and operations on it. The data is hidden from the rest of the program, so the only way to operate on it is to use the object's operations. The operations may take objects as arguments and return objects as results, so objects are intrinsically higher-order.

(1) In general, there are no limits on how one can define a single object's operations. However, it's impossible to define operations which require knowledge of the internal representation of two or more objects at a time.

(2) There are heuristic guidelines (e.g., SOLID principles) for designing flexible object-oriented systems. However, they lack any sort of rigorous foundation beyond “it seems to work in practice”, so object-oriented designers may deviate from these guidelines at their own discretion.

---

For data-oriented applications, it's pretty clear to me that the relational model has important advantages over object-orientation:

(0) The decoupling between data and operations allows the database designer to focus exclusively on data integrity constraints, instead of anticipating whatever queries users will want to make.

(1) The limited expressiveness of relational algebra (with no recursively defined relations) is also a blessing, because it makes automated query optimization tractable in practice.

While objects present problem after problem:

(0) Object graphs are intrinsically directed, and must be traversed in the direction of its links. This makes queries less declarative.

(1) Objects have a notion of identity, which destroys many opportunities for using equational reasoning to build large queries. This also makes queries less declarative.

Of course, the relational model says nothing about general-purpose programming, whereas object-orientation does. But there exist other paradigms for general-purpose programming that are less badly in conflict with the relational model. For instance, functional and logic programming:

(0) Don't reject the use of first-order data, decoupled from operations.

(1) Prefer the notion of mathematical variable, whose meaning is given by substitution (a first-order operation), to imperative assignment, whose meaning is given by certain predicate transformers (intrinsically higher-order gadgets).

Re: MongoDB queries don’t always return all matching documents

#308

My general feeling is that MongoDb was designed by people who hadn't designed a database before, and marketed to people who didn't know how to use one. Its marketing was pretty silly about all the various things it would do, when it didn't even have a reliable storage engine. Its defaults at launch would consider a write stored when it was buffered for send on the client, which is nuts. There's lots of ways to solve…

I really agree with your sentiments, that first paragraph is a great quote. I grew quite an adverse to MongoDB after researching it. While I never found this specific caveat, I found other very worrying decisions.

> reliable storage engine

By "reliable" I assume you mean "consistent?" While MongoDB claims that it's CP (which it's not, as per the article) there's nothing wrong with inconsistent databases (AP, e.g. CouchDB). Mathematically there is no reason for MongoDB to behave like this. It's fundamentally broken; it's neither AP nor CP.

Re: MongoDB queries don’t always return all matching documents

#309

Earlier quoted context omitted.

Mongodb isn't always accessed through an ORM. I have no idea what you are talking about. > Enforcing arbitrary constraints belongs in the business logic of your app Oh so you have some constraints enforced in your database, like the length of a strength can't be greater than 5, yet others you consider arbitrary just because they are not in SQLs list of constraints? How on earth did you define arbitrary otherwise? Any…

What do you mean what do I mean? I just gave you an example. You don't seem to understand that "consistency" in a database, is simply defined within the realms of the database itself. It isn't a universal concept of "consistency". It refers to the fact that the database will enforce certain constraints and checks. You can even denormalise your data and make it so that a persons name is different in different places.…

Your code enforces consistency and detects when a transaction fails the consistency rule rolls back the changes to the state when the transaction started? That's some great code I'm sure, but a lot of effort.

The only limitation on an SQL check constraint is that you cannot use columns in other tables.

As has been pointed about by many of us slow learners, it's far easier to understand constraints in SQL than it often is to dive into a code base.

Re: MongoDB queries don’t always return all matching documents

#310

Earlier quoted context omitted.

This is key and often overlooked - MongoDB is so popular not because it's the best database but because it's so easy to get started with. Download/unzip/run to have a database engine ready. It also helps that you can also immediately store anything without any prior setup steps. Postgres/mysql/sqlserver/etc are nowhere near as easy to install, as fast to get started with or as portable to move around.

Postgres members should listen this and have a simple getting started guide for osx, Windows, Linux. I tried brew install postgresql. There was no single place which tells me how to start server, access command line, create db etc.

On OSX there is the fantastic http://postgresapp.com/ . It installs into /Applications so it is easy to remove, and comes with a start/stop GUI and taskbar icon. Great for local development.

But installing and configuring Postgres "properly" on a server is still something of a challenge. Do I need to modify random_page_cost on a SSD or not? What are good memory limits on modern big servers? What exactly needs to go into pg_hba.conf?

None of these seem too difficult after reading a few tutorials and wikis, but it would be nice if the server set itself up with reasonable defaults based on the machine its running on.

Post reply on HN