Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

351–360 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#351

Post by skywhopper »

thaumasiotes wrote: post #290
jen20 wrote: post #177

Earlier quoted context omitted.

Firstly let me point out that this response is neither intended as a defence of MongoDB defaults which are atrocious, or of the company, who are arguably duplicitous. However I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Wow! Where are you from that you consider it a "fact" that "that's nice" is a "serious insult"? (I'm genuinely curious... I assume not the US based on the time this was posted?)

To my ear (American who grew up in the South, lives in the Midwest, and works with people largely from the Mountain West), "that's nice" could definitely be used dismissively or sarcastically, but there's any number of ways to say it that are actually sincere and genuine, and I can't imagine a situation in which it would be a "serious insult".

I mean, I can imagine that in a professional setting, if the coworker was saying it with a sarcastic tone or being showily bored or dismissive when saying it, that their behavior might be insulting. But anything can be insulting if it's delivered in an insulting way. "That's nice" has no particular edge to it to my ear.

So I have to assume you aren't American, or that this is a regional thing that I don't have experience with. In any case, your reaction to "that's nice" reminds me of an American friend of mine who moved to London and when her coworkers would ask her if she had a preference about where to get lunch, she would reply "I don't care", which would be totally normal (to me) in the US. But to her British colleagues, that word choice made it a very rude thing to say (the appropriate reply being "I don't mind" to indicate that she didn't have a preference and was willing to go anywhere).

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#352

Post by marcan_42 »

tannhaeuser wrote: post #330
marcan_42 wrote: post #301

Earlier quoted context omitted.

Why is storing all data normalized in a database important to you? Whenever I see an exceedingly complicated database schema, I know I'm most likely in for unfixable legacy DB warts and incessant schema updates for every little feature. Hyperbole aside, the best option often is somewhere in between. I find that a relational database with columns for primary keys/relations/anything used in WHERE statements in the norm…

Are you saying fully normalized RDBs end up reimplementing key/value stores, and then suggest to just use a JSON blob instead? The reason why RDBs are normalized is to avoid update anomalies, and it isn't really difficult to understand. Technically, adding a column to a DB table is a matter of "ALTER TABLE T ADD (mycol NUMBER)", but it sounds like the real problem to be worked on is the organizational overhead going…

I'm not sidestepping the DBA, because I'm the DBA :-)

Fully normalized databases are a nice academic idea, but the supposed benefits of going all the way don't materialize in the real world. That kind of approach is just old school cargo cult - just like full NoSQL is new school nonsense. Good developers know that the answer isn't to use whatever was the fad was when they were in school, be it fully relational databases or NoSQL or anything else, but rather to look at the available technologies and take the most appropriate bits and pieces in order to make a project successful.

After all, if JSON were nonsense, why would a serious relational database like PostgreSQL be adding full support for it? They know it has good use cases.

I know it has full use cases, so I use it, along with proper primary and foreign keys and a properly relational base data model. Yes, all my primary objects are tables with proper relations and foreign keys (and a few constraints for critical parts; not for everything because 100% database side consistency is also an impossible pipe dream, as not every business rule can be encoded sanely in SQL). Just don't expect me to add a user_attribute table to build a poor man's KVS just because people in the 90s thought that was the way to go and databases didn't support anything better. I'll have an attributes JSONB column instead.

And yes, JSON is just a trivial data serialization format that happens to have become de facto. It has an intuitive model and the concept isn't new. It just happens to have been what became popular and there is no reason not to use it. Keep in mind that PostgreSQL internally stores it in a more compact binary form anyway, and if clients for some programming languages don't yet support skipping the ASCII representation step that is an obvious feature that could be added later. At that point it ceases to be JSON and just becomes a generic data structure format following the JSON rules.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#353

Post by danpalmer »

ceocoder wrote: post #91
thomascgalvin wrote: post #53

You can tell a lot about a developer by their preferred database. * Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively * MySQL: I don't like to rock the boat, and MySQL is available everywhere * PostgreSQL: I'm not afraid of the command line * H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this) * SQLite: I'm…

This might be a stupid question, but surely no one thinks of RabbigMQ as a database right? I’ve used it from 2012 to 2018 extensively, including using things like shovels to build hub spoke topologies, however not once did I think of it as anything but a message broker. Did I miss something huge?

It's both. It's best used when it's being used as a message broker, but any sufficiently advanced message broker will need many of the features of a database – durability of messages, querying in various ways, etc. I think it's reasonable to think of it as a very specialised database.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#354

Post by ghaff »

thaumasiotes wrote: post #333
pastacacioepepe wrote: post #294

Earlier quoted context omitted.

Did the joke just went over my head or "that's nice" is a bad thing for real?

"That's nice" is a way of explicitly stating that you don't care to any degree about whatever the other person just said.

It's very context sensitive. "That's nice" (with at least an implied exclamation point) can be a literal reaction to something you like. "Yeah, that's nice" as a reaction to a statement/argument can also be the equivalent of "Whatever." i.e. it's stupid but I'm not even going to dignify it with an explanation of why it's stupid.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#355

Post by danpalmer »

mping wrote: post #328
tgsovlerkhgsel wrote: post #325

Earlier quoted context omitted.

How far does SQLite scale? Obviously not good for anything public facing with thousands of concurrent users, obviously good enough for something you only use yourself, but what about internal tools with a couple hundred users total (few of them concurrent) - where's the limit when it starts slowing down?

Curiously, I just read this: https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...

Expensify aren't really scaling SQLite in the way that people would expect. To say it's scaling SQLite is not exactly wrong, but probably gives the wrong impression. The users of their database likely wouldn't see it as SQLite, and they don't use the stock SQLite code.

They have their own layer on top that happens to use SQLite as the storage format on disk[1]. This layer means they aren't using full SQLite at the application level, but rather using their custom database in the application, and SQLite within their custom database.

Further, they've customised the SQLite codebase as far as I can tell to remove much of the functionality that SQLite uses to ensure that multiple instances can safely edit the same file on disk together, then they memory map the file and just have many threads all sharing the same data.

[1]: FoundationDB also does this, and scales to thousands of nodes. The trick is that it's essentially _many_ separate, very simple SQLite databases, each being run independently.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#356

Post by ghaff »

skywhopper wrote: post #351
thaumasiotes wrote: post #290

Earlier quoted context omitted.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Wow! Where are you from that you consider it a "fact" that "that's nice" is a "serious insult"? (I'm genuinely curious... I assume not the US based on the time this was posted?) To my ear (American who grew up in the South, lives in the Midwest, and works with people largely from the Mountain West), "that's nice" could definitely be used dismissively or sarcastically, but there's any number of ways to say it that are…

Language is complicated. Even to your "I don't care" example in the US, it indeed often means "I don't have a real personal preference, so if others do please go ahead and make a reasonable decision and I'll be fine with it." However, in other circumstances, it can mean something more dismissive in the vein of "I don't give an f what you do. Just go away."

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#357

Post by ghaff »

cosmodisk wrote: post #298
pastacacioepepe wrote: post #294

Earlier quoted context omitted.

Did the joke just went over my head or "that's nice" is a bad thing for real?

I think it sits on the same level as ' it's interesting',when 9/10 times you could see that the person who said so is not even remotely interested.

I'm not sure I agree with that. A lot of the time "interesting" serves as a filler word for "I'm not totally sure what to think about it" or "I'm not ready to articulate in 1000 words why X may or may not be a good approach right now."

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#358

Post by crazybit »

MongoDB is horrible, I get it.

What do I use in this situation:

1) I need to store 100,000,000+ json files in a database

2) query the data in these json files

3) json files come from thousands upon thousands of different sources, each with their own drastically different "schema"

4) constantly adding more json files from constantly new sources

5) no time to figure out the schema prior to adding into the database

6) don't care if a json file is lost once in awhile

7) only 1 table, no relational tables needed

8) easy replication and sharding across servers sought after

9) don't actually require json, so long as data can be easily mapped from json to database format and back

10) can self host, no cloud only lock-in

Recommendations?

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#360

Post by kreetx »

crazybit wrote: post #358

MongoDB is horrible, I get it. What do I use in this situation: 1) I need to store 100,000,000+ json files in a database 2) query the data in these json files 3) json files come from thousands upon thousands of different sources, each with their own drastically different "schema" 4) constantly adding more json files from constantly new sources 5) no time to figure out the schema prior to adding into the database 6) d…

I don't think it's that simple (being horrible). MongoDB can be great for some specific situations, perhaps yours. It's just that it's not for many others, and you'd need to be an expert to find this out from the docs.
Post reply on HN