Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

381–390 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#381

Post by orf »

aeonsky wrote: post #378
orf wrote: post #362

Earlier quoted context omitted.

> If you have pid,{name:’val’, others...}. And an index of name with a million John and one Jane. Good luck getting fast results. Fast results: https://dbfiddle.uk/?rdbms=postgres_10&fiddle=dd9370966b1528... . Postgres does actually keep statistics on json columns, but if you've got a functional index on the table and the query uses it then it doesn't matter if there is one "jane" and a million "johns". You're lookin…

No it doesn’t. But what do I know running several clusters in production for a $100 million business? Please read up on the subject before arguing.

> SELECT most_common_vals

> FROM pg_stats

> WHERE tablename = 'json_test'

> AND attname = 'json_column';

> {"{\"name\": \"john\"}"}

Hmm. Looks like it does though. Not that it makes a damn bit of difference because if you haven't got a functional index (i.e the stats are next to useless) then you're doing a full table scan, and in that case it sounds like you “expect full table scans to always be fast” :)

And sure, the statistics don't help with the query planner, unless you've got a computed column, but again see "I expect full table scans to always be fast" and re-consider the statement "postgres doesn't keep statistics on json columns" given the fact that it actually does, just like any other column.

Read up indeed!

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#382

Post by chx »

tasuki wrote: post #297
chx wrote: post #64

Earlier quoted context omitted.

There was a time when I advocated for MongoDB with the usual caveats. The ability to easily store and index complex data was of great value. And then in 2015 October, within a week of each other, SQLite and MySQL both learned how to index on expressions and store JSON (SQLite 3.9 2015-10-14, MySQL 5.7 2015-10-21). PostgreSQL added jsonb the year prior in 9.4. At that moment the value of MongoDB for me diminished grea…

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

> Why is storing json in a database important to you?

I don't need JSON per se, I want to store data with a predefined, recursive format which most of the time will be just serialized / deserialized but occasionally also queried without having any idea ahead of the time what the query will be. For all I care, it could be Python pickle, PHP serialize, XML or whatever ungodly format you want as long as the above stands. (XML actually works in MySQL thanks to ExtractValue and UpdateXML but alas, I break out in rashes when I touch XML :P)

Let's say you want to display a page of text and images. (Doh.) But you want to give your authors great flexibility and yet nice styling so you give them components: a rolodex, a two-by-two grid, tab groups and so forth (we have 44 such components). This stuff is recursive, obviously. Are you going to normalize this? It's doable but a world of pain is an accurate description for the results. The data you get back from deserializing JSON should map pretty well to the templating engine.

Rarely you want to query and update things, some analytics, some restructuring etc. If it were just a key-value store with page id and serialized data the only way to do maintenance would be to deserialize each page and manually dig in. Sure, it's doable but having the database do that is simply convenient. That's the reason we use database engines, right? At the end of the day, you don't need SQL, you can just iterate a K-V store and manually filter in the application in whatever ways you want -- it's just more convenient to have the engine do so. Same here. The nicest thing here is that if someone wants ongoing analytics you actually can add an index on a particular piece in the blob and go to town.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#383

Post by tjalfi »

jtdev wrote: post #361
m0zg wrote: post #126

Earlier quoted context omitted.

You laugh, but I bet Excel produces orders of magnitude more real "business intelligence" than all other "BI" tools combined.

...And orders of magnitude more wasted time and capital due to inaccurate and isolated data.

People use what they know to solve the problems they have.

You can complain about their solution or see it as an opportunity.

I posted elsewhere[0] in this thread about my employer's practice of replacing shared spreadsheets with web applications.

This approach works quite well for us and I would encourage you to consider it as an option.

[0] https://news.ycombinator.com/item?id=23292374

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#384

Post by mateuszf »

wdb wrote: post #85
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Why not use PostgreSQL instead? It supports a JSON document data type natively. It also has exceptional stewardship as an open source project. Mongo should never be a first choice, but a last choice for edge cases.

I really enjoy using PostgreSQL only I just don't know how to make it scale easily. Running it on large VM in the cloud works fine until you have lots of data or need it easily accessible. How can you have the data in three different regions (e.g. Europe, US, Asia) when you using something like Google Cloud? Seems to be a hard problem to crack.

Try with Aurora Postgres Global Database, it's on AWS, but should be accessible from other clouds.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#385

Post by darkarmani »

macintux wrote: post #36
madhadron wrote: post #27

In the circles I run in, MongoDB is regarded as a joke and the company behind it as basically duplicitous. For example, they still list Facebook as their first user of MongoDB on their website, for example, but there is no MongoDB use in Facebook hasn't been for years (it came in only via a startup acquisition). I had the misfortune to use MongoDB at a previous job. The replication protocol wasn't atomic. You would f…

The joke I learned early on: "Migrating away from Mongo is trivial: wait long enough, and all your data will be gone anyway." I imagine things are better now.

I just call it a probablistic datastore.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#386

Post by Andrew_nenakhov »

calcifer wrote: post #97
gigatexal wrote: post #86

Typical HN posts of late hating on Javascript and MongoDB from database elitists -- the thing is there's a tool for a job and as engineers we need to figure out what tool best suits our use cases. It could very well be a NoSQL database such as Mongo or a relational one like Postgres or MySQL.

> the thing is there's a tool for a job Really? Which job do you belive needs a "maybe store some of this data, sometimes" kind of database?

I'm not defending mongodb in and sense and had stern talks with some of my junior developers who were too eager to try out this new hot mongo thingy on a new website, but there are plenty such jobs.

For example, climate data gathered from hundreds of thousands of devices every minute can very much survive some data to be lost. Or some astronomical observations data.

I wouldn't choose mongoDB for it, though.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#387

Post by SSLy »

zaphar wrote: post #372
dannyw wrote: post #242

Earlier quoted context omitted.

Fair enough, but just because you call a bug a feature doesn’t mean it’s not a bug. I think most people have realised weak typing is not a good idea in programming languages. It’s especially not a good idea in databases.

Most people have not in fact realized weak typing is not a good idea. I myself vastly prefer strongly typed languages and think they are superior. However there are a huge number of people I work with and know professionally who prefer dynamically typed languages. Weak versus strong typing is a highly subjective opinion. Each one has different costs and benefits and which camp you land in depends in large part on wha…

Most "scripting" plangs are both strong and dynamic typed at the same time.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#388

Post by kqr »

GuB-42 wrote: post #185
m0zg wrote: post #126

Earlier quoted context omitted.

You laugh, but I bet Excel produces orders of magnitude more real "business intelligence" than all other "BI" tools combined.

Here is an anecdote. I had to work on a tool that shows what's wrong with an assembly line: missing parts, delays, etc... So that management can take corrective action. Typical "BI" stuff but in a more industrial setting. The company went all out on new technologies. Web front-end, responsive design, "big data", distributed computing, etc... My job was to use PySpark to extract indicators from a variety of data sourc…

I think you drew the right conclusion from your experience, but I also want to point out that building the first prototype is always anywhere from one to three orders of magnitude easier than building the actual product.

The devil is in the details, and software is nothing but details. The product owner at the company I work for likens it (somewhat illogically, but it works) with constructing walls. You can either pick whatever stones you have lying around, and then you'll spend a lot of time trying to fit them together and you'll have a hell of a time trying to repair the wall when a section breaks. Or you can build it from perfectly rectangular bricks, and it will be easy to make it taller one layer at a time.

Using whatever rocks you have lying around is like building a prototype in Excel. Carefully crafting layers of abstraction using proper software engineering procedures means taking the time to make those rectangular bricks before building the wall. End result more predictable when life happens to the wall.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#389

Post by [deleted] »

gav wrote: post #101
1123581321 wrote: post #70

Earlier quoted context omitted.

As someone who chose MySQL and provides direction to developers who really like Postgres, and who also uses Postgres for fun, I do find myself having to both defend MySQL as a prudent option and convince them that I know anything at all about Postgres or computer science. :)

I tend to find people who argue with me against MySQL bring up things that haven't been true in a long time such as Unicode or NULL handling. I'd probably choose Postgres over MySQL for a new project just to have the improved JSON support, but there's upsides to MySQL too: - Per-thread vs per-process connection handling - Ease of getting replication running - Ability to use alternate engines such as MyRocks

[deleted]

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#390

Post by philwelch »

thomascgalvin wrote: post #53

You can tell a lot about a developer by their preferred database. * Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively * MySQL: I don't like to rock the boat, and MySQL is available everywhere * PostgreSQL: I'm not afraid of the command line * H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this) * SQLite: I'm…

Neo4j?
Post reply on HN