Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just use Postgres

mccue.dev

Post reply on HN
Print view View original

221–230 of 238 posts

Re: Just use Postgres

#221

Post by afiori »

afiori wrote: post #216
regularfry wrote: post #158

Earlier quoted context omitted.

MongoDB salesdroids rely heavily on you having a low familiarity with other database tech to spin themselves as the only game in town. "Being led astray" isn't a passive, ambient occurence, and it makes sense to push back against it.

On the other hand I know of multiple databases where all tables had attribute_1, attribute_2,..., attribute_5 columns Just in Case™ But more seriously the one feature I like in MongoDB is the pipeline API, where you can express a complex query with multiple filters/aggregations/transformations/joins as a list of simple steps. There are some use cases where it is very ergonomic (even if I suspect that mongo can easily…

I forgot about what I hate about mongo, so I will rant here about it: its data model is close enough but different enough from json to be both annoying and dangerous.

The empty string is a valid json key but not a mongo document key.

Mongo uses $operator keys to serialize its datatypes to json but does not sanitise the result: which means that {"foo":100000000000000000} and {"foo":{"$longInteger":"100000000000000000"}} will have a collision with the json export format. (Even if you choose the fully explicit Canonical format as there is no $document operator to wrap ambiguous documents)

So if your plan is to dump json to mongo you should plan for that (also sometimes $operators are evaluated sometimes they are not, it depends on each method and the documentation does not tell you )

The official client (both csv and json) is unable to export a collection if a field is both a value field both an atomic value and an object, so a collection with two documents: {a:1} and {a:{b:1}} will cause problems of you try to export it.

My colleagues have other issues with the json DSL and how most operators exist in 2-3 different forms with different syntax or how the syntax {$operator:{arg1:..., arg2:...}} is unintuitive but I actually sort of like it.

Re: Just use Postgres

#222

Post by afiori »

asah wrote: post #214
orf wrote: post #198

Earlier quoted context omitted.

> This is exactly a great point. When data size goes to a billion rows, Postgres is tough. You’ve been led astray. You can handle a billion rows on a developer laptop, let alone a production grade instance.

Depends on row width. 10KB JSON fields are all too common.

I have more mongodb experience than postgres but my impression is that a lot of the json handling I ended up doing in mongo would have been easier/reliable in postgres

Re: Just use Postgres

#224

Post by roenxi »

re-thc wrote: post #136
roenxi wrote: post #55

Earlier quoted context omitted.

> Especially as a fresh user. It's how we learn. From both good and bad experiences. I'd do that, but a superior strategy is letting other people make mistakes and then learning from them. It is best to always be making choices that seem like they could be optimal, with very rare exceptions.

> but a superior strategy is letting other people make mistakes and then learning from them If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work? Not everything is strictly better. And by that logic... >> the superior strategy is letting other people comment (and make mistakes) and then learn from them i.e. don't read the post until years later to ensure you have all…

> If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work?

I agree with duckmysick, and also please take note that having a strategy of not making mistakes will not avoid all mistakes. Outcomes and intent never match up perfectly. But that is why it is important to learn from others right from the start.

Re: Just use Postgres

#225

Post by orf »

asah wrote: post #214
orf wrote: post #198

Earlier quoted context omitted.

> This is exactly a great point. When data size goes to a billion rows, Postgres is tough. You’ve been led astray. You can handle a billion rows on a developer laptop, let alone a production grade instance.

Depends on row width. 10KB JSON fields are all too common.

Yes, you obviously can’t fit 10 TB of data onto a developer laptop. However you’ll run out of disk before you run into Postgres issues.

Re: Just use Postgres

#226

Post by lostmsu »

orf wrote: post #225
asah wrote: post #214

Earlier quoted context omitted.

Depends on row width. 10KB JSON fields are all too common.

Yes, you obviously can’t fit 10 TB of data onto a developer laptop. However you’ll run out of disk before you run into Postgres issues.

With an SSD cache + a few external 20TB hard drives I think you can easily make it to 40TB with redundancy.

Re: Just use Postgres

#227

Post by orf »

lostmsu wrote: post #226
orf wrote: post #225

Earlier quoted context omitted.

Yes, you obviously can’t fit 10 TB of data onto a developer laptop. However you’ll run out of disk before you run into Postgres issues.

With an SSD cache + a few external 20TB hard drives I think you can easily make it to 40TB with redundancy.

Yes, you obviously can fit 10 TB of data onto a developer laptop with 10tb of external storage. However you’ll run out of disk before you run into Postgres issues.

Re: Just use Postgres

#228

Post by re-thc »

duckmysick wrote: post #217
re-thc wrote: post #136

Earlier quoted context omitted.

> but a superior strategy is letting other people make mistakes and then learning from them If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work? Not everything is strictly better. And by that logic... >> the superior strategy is letting other people comment (and make mistakes) and then learn from them i.e. don't read the post until years later to ensure you have all…

> If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work? Either people aren't aware of an optimal strategy (if one exists) or they ignore it for various reason. The latter is surprisingly common. People know they should exercise, get enough sleep, eat healthy, stay hydrated, tackle high priority tasks instead of procrastinating, etc. - and yet they still aren't doing…

> Either people aren't aware of an optimal strategy (if one exists) or they ignore it for various reason.

And my point was precisely that it often doesn't exist. What is an optimal strategy?

> People know they should exercise, get enough sleep, eat healthy, stay hydrated, tackle high priority tasks instead of procrastinating, etc. - and yet they still aren't doing those things (or as much as they should). > Knowing something is not enough.

Your example doesn't even support this. People know they have to get enough sleep but they also know they have to . It depends on what they are optimizing for. i.e. there is no singular optimal strategy.

You've just proved my point rather than yours.

I don't know if knowing is not enough, but clearly the people in your example don't know what the optimal strategy is. E.g. eating healthy is NOT the optimal strategy as it could make them unhappy (e.g. don't like the taste). It's not optimal unless it's strictly better.

Re: Just use Postgres

#229

Post by re-thc »

roenxi wrote: post #224
re-thc wrote: post #136

Earlier quoted context omitted.

> but a superior strategy is letting other people make mistakes and then learning from them If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work? Not everything is strictly better. And by that logic... >> the superior strategy is letting other people comment (and make mistakes) and then learn from them i.e. don't read the post until years later to ensure you have all…

> If this was a superior strategy that was so obvious no 1 would be making mistakes so how does this work? I agree with duckmysick, and also please take note that having a strategy of not making mistakes will not avoid all mistakes. Outcomes and intent never match up perfectly. But that is why it is important to learn from others right from the start.

> I agree with duckmysick > But that is why it is important to learn from others right from the start.

It's important if you know what you're learning but...

What do you even agree with? i.e. what's the learning?

>> duckmysick claims people ignore the optimal strategy but could not even give an example of 1.

This proves it's better to try it yourself than to assume you're learning and make even worse mistakes.

> and also please take note that having a strategy of not making mistakes will not avoid all mistakes

Where is the strategy to begin with? 0 + 0 was 0 to begin with.

Re: Just use Postgres

#230

Post by lostmsu »

orf wrote: post #227
lostmsu wrote: post #226

Earlier quoted context omitted.

With an SSD cache + a few external 20TB hard drives I think you can easily make it to 40TB with redundancy.

Yes, you obviously can fit 10 TB of data onto a developer laptop with 10tb of external storage. However you’ll run out of disk before you run into Postgres issues.

I was kinda agreeing with you :-D
Post reply on HN