Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

401–410 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#401

Post by golergka »

thomascgalvin wrote: post #53

You can tell a lot about a developer by their preferred database. * Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively * MySQL: I don't like to rock the boat, and MySQL is available everywhere * PostgreSQL: I'm not afraid of the command line * H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this) * SQLite: I'm…

I love postgresql, but I don't remember when did I last interact with it with command line instead of pgadmin.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#402

Post by jatone »

lmm wrote: post #312
balfirevic wrote: post #230

Earlier quoted context omitted.

This question sounded familiar - turns out I replied to it in another thread: https://news.ycombinator.com/item?id=23286054 To repeat my (non)answer: There is no way to recommend NoSQL database without knowing what you need it for because NoSQL databases are highly specialized systems. If you need general-purpose database use an SQL one. It's kind of a weird question, now that I think about it. Why would anyone seek…

I'd actually say the reverse. SQL databases are highly specialised datastores: they make sense if you need one particular transaction model and one particular query language and are prepared to coerce your data into one particular model to do so. If you're starting from just "I need to store some data" I'd look to e.g. Riak or Cassandra before looking to an SQL database.

SQL DBs are not specialized.... they're incredibly general...

You are never starting from "I need to store some data" you're always going to start from "I need to store and read some data" otherwise /dev/null would work if you are not going to read the data back.

the problem with cassandra and riak is precisely the read aspect of the problem which quickly degrades the performance of those systems.

I've used both cassandra and postgresql at scales most companies never reach. cassandra I'd only touch for immutable time series data and only if that information was large enough to not fix on a single server and i didn't care about consistency. everything else is a SQL rdbms.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#403

Post by jatone »

wdb wrote: post #85
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Why not use PostgreSQL instead? It supports a JSON document data type natively. It also has exceptional stewardship as an open source project. Mongo should never be a first choice, but a last choice for edge cases.

I really enjoy using PostgreSQL only I just don't know how to make it scale easily. Running it on large VM in the cloud works fine until you have lots of data or need it easily accessible. How can you have the data in three different regions (e.g. Europe, US, Asia) when you using something like Google Cloud? Seems to be a hard problem to crack.

generally speaking the only reason that would be difficult would be due to legal constraints on the data locality.

if it was just performance than read only replica's in each region would solve most of the issues.

for the legal case generally I just end up with a separate postgresql DB in each region and during login the region is determined by user/company.

usually ends up being like 1k LOCs total.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#404

Post by jatone »

Andrew_nenakhov wrote: post #386
calcifer wrote: post #97

Earlier quoted context omitted.

> the thing is there's a tool for a job Really? Which job do you belive needs a "maybe store some of this data, sometimes" kind of database?

I'm not defending mongodb in and sense and had stern talks with some of my junior developers who were too eager to try out this new hot mongo thingy on a new website, but there are plenty such jobs. For example, climate data gathered from hundreds of thousands of devices every minute can very much survive some data to be lost. Or some astronomical observations data. I wouldn't choose mongoDB for it, though.

your example is a perfect use case for postgresql via the timescaledb extension.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#405

Post by databrecht »

jatone wrote: post #402
lmm wrote: post #312

Earlier quoted context omitted.

I'd actually say the reverse. SQL databases are highly specialised datastores: they make sense if you need one particular transaction model and one particular query language and are prepared to coerce your data into one particular model to do so. If you're starting from just "I need to store some data" I'd look to e.g. Riak or Cassandra before looking to an SQL database.

SQL DBs are not specialized.... they're incredibly general... You are never starting from "I need to store some data" you're always going to start from "I need to store and read some data" otherwise /dev/null would work if you are not going to read the data back. the problem with cassandra and riak is precisely the read aspect of the problem which quickly degrades the performance of those systems. I've used both cass…

I wonder, given your experience, did you ever try FaunaDB? It grew from the not optimal experience when scaling databases like Cassandra etc @ Twitter. Consistent + relational + multi-region.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#406

Post by Andrew_nenakhov »

jatone wrote: post #404
Andrew_nenakhov wrote: post #386

Earlier quoted context omitted.

I'm not defending mongodb in and sense and had stern talks with some of my junior developers who were too eager to try out this new hot mongo thingy on a new website, but there are plenty such jobs. For example, climate data gathered from hundreds of thousands of devices every minute can very much survive some data to be lost. Or some astronomical observations data. I wouldn't choose mongoDB for it, though.

your example is a perfect use case for postgresql via the timescaledb extension.

I actually agree. I love postgresql and we've been using it for all our projects since our company was founded (well, except mobile apps, obviously), and it never failed us.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#407

Post by lmm »

jatone wrote: post #402
lmm wrote: post #312

Earlier quoted context omitted.

I'd actually say the reverse. SQL databases are highly specialised datastores: they make sense if you need one particular transaction model and one particular query language and are prepared to coerce your data into one particular model to do so. If you're starting from just "I need to store some data" I'd look to e.g. Riak or Cassandra before looking to an SQL database.

SQL DBs are not specialized.... they're incredibly general... You are never starting from "I need to store some data" you're always going to start from "I need to store and read some data" otherwise /dev/null would work if you are not going to read the data back. the problem with cassandra and riak is precisely the read aspect of the problem which quickly degrades the performance of those systems. I've used both cass…

For simple reads, the SQL model forces significantly worse performance: MySQL benchmarks found that 75% of the time for a pkey lookup was spent on parsing the SQL. For more complex querying, SQL databases can be fast... and they can also be extremely slow, and you can't tell for any given query just by looking at it.

The much-vaunted consistency comes at a significant cost: index updates block writes, and more insidiously, it's very easy to be surprised by a deadlock or a stale transaction with a long-running query. I've seen an SQL database stop committing any new writes because someone ran a seemingly innocuous query 23 days ago. And a lot of the time - including every web use case I've seen - you can't actually make any real use of those consistency guarantees.

Writing either a transformation pipeline that serves the same function as a secondary index, or a deliberate map-reduce style aggregation, takes more up-front effort. But it means you understand what's actually going on a lot more clearly and are much less likely to hit that kind of unpleasant surprise.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#408

Post by tasuki »

dkersten wrote: post #345
tasuki wrote: post #297

Earlier quoted context omitted.

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

I use json (in postgres) for user generated config data (that is, both the schema and the values of the config are user generated). I will never query it other than to read the entire data to send elsewhere for processing and write it when the user sends new data. I don’t know or care about the content, since I can’t know what’s going to be there, but if I ever did want to query inside it, postgres allows that. I cou…

Good use case indeed!

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#409

Post by tasuki »

jeltz wrote: post #349
tasuki wrote: post #297

Earlier quoted context omitted.

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

I use it for when I get extra feilds from a third party API which our system does not need at the moment but which may or may not become useful in the future. Sometimes these extra fields are undocumented so creating columns for them would require investigation, just throwing them in there as JSON is a much better use of my time.

Also good use case! :)

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#410

Post by tasuki »

coolgeek wrote: post #373
tasuki wrote: post #297

Earlier quoted context omitted.

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

Inconsistent json data is exactly why you use a json data type or database. I know that you , dear developer, would never produce inconsistent data. But lots of other developers do. It is often the case that you will need to query that inconsistent json data, but either the pain is too great, or the value too low, to normalize that data. Thus, you dump it as is into a json field or database.

Ah, well, I guess I prefer having consistent data. The only way to have consistent data is to forbid the database from representing inconsistent states. If inconsistent states are allowed, they will appear.
Post reply on HN