Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB History

quickprogrammingtips.com

Post reply on HN
Print view View original

61–70 of 74 posts

Re: MongoDB History

#61

Post by trashcan »

jinqueeny wrote: post #60
trashcan wrote: post #59

Earlier quoted context omitted.

I apologize, because I literally don't know, but have you used any of these solutions at the scale where there might be billions or trillions of rows in a table/collection? I'm currently using Mongo at that scale and would love to evaluate some alternatives. If it helps for context, we have accepted that ad-hoc queries are not possible, and we have our own solution for searching.

TiDB has a similar case study: Queries over 1.3 Trillion Rows of Data Within Milliseconds of Response Time at Zhihu.com https://pingcap.com/success-stories/lesson-learned-from-quer... The latest stats in the same case scenario (already-read posts) Zhihu is: - 2.6 Trillion Rows - 560TB data - 200 TiKV instances

Very cool! Thanks, I will look into it.

Re: MongoDB History

#62

Post by jd_mongodb »

manigandham wrote: post #15
numlock86 wrote: post #10

Earlier quoted context omitted.

I never got a definite answer: What problem does MongoDB even try to solve?

Schemaless data (or at least schema-on-read rather than on-write) is the primary feature. Store JSON documents and index on any field. Also it was great at sharding and scaling horizontally when first released, and one of the few options available at that time. It's since been eclipsed by much better systems that don't have such a convoluted and fragile setup. These days there's not much benefit over a JSON field in…

Most JSON field databases treat the JSON field as text and index it as free text. with MongoDB you can index at any level into the document. Since 4.2 you can use wildcard indexes to index a document and any new fields that are subsequently added automatically.

https://docs.mongodb.com/manual/core/index-wildcard/

Re: MongoDB History

#63

Post by manigandham »

jd_mongodb wrote: post #62
manigandham wrote: post #15

Earlier quoted context omitted.

Schemaless data (or at least schema-on-read rather than on-write) is the primary feature. Store JSON documents and index on any field. Also it was great at sharding and scaling horizontally when first released, and one of the few options available at that time. It's since been eclipsed by much better systems that don't have such a convoluted and fragile setup. These days there's not much benefit over a JSON field in…

Most JSON field databases treat the JSON field as text and index it as free text. with MongoDB you can index at any level into the document. Since 4.2 you can use wildcard indexes to index a document and any new fields that are subsequently added automatically. https://docs.mongodb.com/manual/core/index-wildcard/

The whole point of having a JSON field is so that it has more structured than just text, otherwise you can just use a text field (like SQL Server does). Also they all support various JSON querying and indexing functions that including subfield access with optional computed properties.

Sure MongoDB has some extra ergonomics for dealing with JSON/BSON data, but how much benefit this really adds is still up for debate. As horizontal scalability becomes more natively supported, MongoDB will lose even more of its benefits.

Re: MongoDB History

#64

Post by manigandham »

trashcan wrote: post #59
manigandham wrote: post #58

Earlier quoted context omitted.

I'd recommend sticking with relational databases since they all support JSON columns now. If you need horizontal scalability then there are many choices like CockroachDB, Yugabyte, TiDB, Vitesse, MemSQL, and others. If still want a document-store then RavenDB is a great choice with proper clustering, full-text search, SQL-like querying, graph queries, etc. ArangoDB is also good choice.

I apologize, because I literally don't know, but have you used any of these solutions at the scale where there might be billions or trillions of rows in a table/collection? I'm currently using Mongo at that scale and would love to evaluate some alternatives. If it helps for context, we have accepted that ad-hoc queries are not possible, and we have our own solution for searching.

We used MemSQL with 100s of billions of rows for years in production.

If you're working at that scale, it sounds like that's more of an OLAP use-case where MemSQL and other column-oriented databases would suit better than an OLTP document-store. Maybe you can share more details for better recommendations.

Re: MongoDB History

#65

Post by jd_mongodb »

manigandham wrote: post #63
jd_mongodb wrote: post #62

Earlier quoted context omitted.

Most JSON field databases treat the JSON field as text and index it as free text. with MongoDB you can index at any level into the document. Since 4.2 you can use wildcard indexes to index a document and any new fields that are subsequently added automatically. https://docs.mongodb.com/manual/core/index-wildcard/

The whole point of having a JSON field is so that it has more structured than just text, otherwise you can just use a text field (like SQL Server does). Also they all support various JSON querying and indexing functions that including subfield access with optional computed properties. Sure MongoDB has some extra ergonomics for dealing with JSON/BSON data, but how much benefit this really adds is still up for debate.…

Making JSON the coin of the realm and putting it at the core of your database design and query language is a little bit more than extra ergonomics :-)

Re: MongoDB History

#66

Post by adoxyz »

fiedzia wrote: post #28
threeseed wrote: post #21

Earlier quoted context omitted.

There is this myth that schemas must be enforced at the database level. But the majority of databases are only accessed by one web app. And in that web app you can enforce that schema in code. In fact in code you have much safer and powerful options e.g. enforcing business rules such as this string field must start with aaa.

>There is this myth that schemas must be enforced at the database level. You must have single point to enforce anything. This is very rarely the case with the app, where a) there will be 20 places that access database and b) often some tasks are done by operating on a database directly Some rules cannot be enforced by database, sure, but "a field must exists and be a string" is infinitely better than noting.

The thing about MongoDB is that it does support $jsonSchema for enforcing a schema in a very flexible way. So rather than having to have a strict schema for every piece of data, you can use $jsonSchema for as little or as much of your data as you see fit so really you can have the best of both worlds.

For reference: https://docs.mongodb.com/manual/reference/operator/query/jso...

Re: MongoDB History

#67

Post by NegativeLatency »

jd_mongodb wrote: post #54
blibble wrote: post #45

Earlier quoted context omitted.

at large firms the clueless technology managment often buy based on the marketing crap, then force it down the throats of the engineers at my firm we're being all but forced to use Azure even though it costs more and makes absolutely no sense whatsoever for the domain, but since it's the "technology strategy" fighting it becomes exceptionally difficult

In seven years working at MongoDB the one thing I have learned is technology is never "forced down the throat of the engineers". Developers have choices and would not stay in an organisation that made decisions that way. The reason developer relations exists is the acknowledgement that developers ARE the decision makers in most technology selection.

I recently worked for a company where a different popular JSON document database was in fact forced down my throat.

I proved that a regular database would be orders of magnitude faster with a real example on real data and while the project/product was canceled for marketing/sales reasons. Until it was canceled we were told by multiple levels of "technical" managers to use the worse database because of unknown reasons.

Re: MongoDB History

#68

Post by jd_mongodb »

NegativeLatency wrote: post #67
jd_mongodb wrote: post #54

Earlier quoted context omitted.

In seven years working at MongoDB the one thing I have learned is technology is never "forced down the throat of the engineers". Developers have choices and would not stay in an organisation that made decisions that way. The reason developer relations exists is the acknowledgement that developers ARE the decision makers in most technology selection.

I recently worked for a company where a different popular JSON document database was in fact forced down my throat. I proved that a regular database would be orders of magnitude faster with a real example on real data and while the project/product was canceled for marketing/sales reasons. Until it was canceled we were told by multiple levels of "technical" managers to use the worse database because of unknown reasons…

Technology selection is a black art in many companies. Sounds like people were open to listening to the opinion of a developer in your case. Often-times the selection heuristic is habit or history. The right companies will pay more attention to reasoned arguments of their own developers.

Re: MongoDB History

#69

Post by manigandham »

hodgesrm wrote: post #48
manigandham wrote: post #16

Earlier quoted context omitted.

Sounds like a bad architect or someone who only knows surface features and doesn't have experience with the actual databases. Adding a column is not hard in any relational database, and pretty much all modern ones support no-downtime transactional schema updates with backfills, concurrent index builds, etc. Also the schema always exists somewhere, and it's usually to put it in the database so it's next to (and valida…

Adding columns can be hard at scale. MongoDB allows you to add columns incrementally without stopping the world. You just add the value and it's materialized in the DBMS. The point is you don't have to manually define schema but instead the database works with the structure of the data. I'm a dyed-in-the-wool RDBMS user but I can see the value of this feature. The trade-off, of course, is that your application has to…

Isn't adding a column even easier in column-oriented OLAP databases?

I agree that more complex row formats are needed though. Bigquery has done well with nested/repeating structures and Snowflake uses the PAX data format for JSON which has been very useful (however their JSON/VARIANT column doesn't support structured types).

Re: MongoDB History

#70

Post by hodgesrm »

manigandham wrote: post #69
hodgesrm wrote: post #48

Earlier quoted context omitted.

Adding columns can be hard at scale. MongoDB allows you to add columns incrementally without stopping the world. You just add the value and it's materialized in the DBMS. The point is you don't have to manually define schema but instead the database works with the structure of the data. I'm a dyed-in-the-wool RDBMS user but I can see the value of this feature. The trade-off, of course, is that your application has to…

Isn't adding a column even easier in column-oriented OLAP databases? I agree that more complex row formats are needed though. Bigquery has done well with nested/repeating structures and Snowflake uses the PAX data format for JSON which has been very useful (however their JSON/VARIANT column doesn't support structured types).

It's definitely easier than conventional RDBMS. Adding a column in ClickHouse is just a metadata operation. BiqQuery has great nested structure support.

Still, it's hard to beat MongoDB in this respect. In my first app I was amazed that I could just insert a BSON object and MongoDB created a queryable table automatically. You pay for it of course in other ways but the ease of use is quite extraordinary.

Post reply on HN