Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

221–230 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#221

Post by scrollaway »

mrsuprawsm wrote: post #208
scrollaway wrote: post #90

Earlier quoted context omitted.

2.3m content items is tiny. So is 30m. You're at least 1-2 magnitudes away from something that will start to bother postgres. Anything before that is likely to be an index or IOPS issue.

Even with HSTORE/JSONB columns? I would imagine (gut feeling, no evidence) that the performance of these is worse than more traditional column types.

Aye even so. They're definitely heavier to store, and their indexes can get quite big, though, so disk space / IOPS can be more of an issue. But still 2.3M is a drop in the ocean. (I don't know about HSTORE though…)

At my previous startup I was ingesting Hearthstone games at a rate of 1-2M / day. Before being handed off to permanent storage (s3, redshift etc) a bunch of data would get stored in a JSONB, with 14-day retention. This all ran on a 200GB t2.large instance on RDS, was our smallest instance and never really caused an issue.

Re: Bye Bye Mongo, Hello Postgres

#222

Post by mattbreeden »

quietbritishjim wrote: post #210
csytan wrote: post #68

Earlier quoted context omitted.

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

> genuinely better than stitching together fragments of SQL in the form of text strings. Injection attacks and inserting commas (but not trailing commas) come to mind as obvious difficulties.

You're using the Pymongo library as an example. Someone can just as easily use SQLAlchemy and not have to worry about those things.

Re: Bye Bye Mongo, Hello Postgres

#223

Post by brown9-2 »

jack6e wrote: post #35

Specific technology choices aside, this was an incredible write-up of their migration process: thorough, organized, readable prose about a technical topic. It is helpful to read how other teams handle these types of processes in real production systems. Perhaps most refreshing is the description of choices made for the various infrastructure pieces, because it is reasonable and real-world. Blog posts so often describ…

I imagine this is a nice benefit of working at an organization that is primarily about writing.

Re: Bye Bye Mongo, Hello Postgres

#224

Post by funkaster »

quietbritishjim wrote: post #210
csytan wrote: post #68

Earlier quoted context omitted.

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

Well... you can also use a modern ORM. I think "stitching ... text strings" is definitively not the way to go when interfacing a SQL database. My go-to ORM is Sequel[1]. I think their API is one of the best I've seen: you can choose to use models, but you can also work directly with "datasets" (tables or views, or queries) and compose them as you like. It's really powerful and simple.

[1]: http://sequel.jeremyevans.net/

Re: Bye Bye Mongo, Hello Postgres

#225

Post by Thaxll »

It's not a great article tbh, it's well written but it shows the clear lack of knowledge running a backend. The title should be "we didn't know what we were doing so we switched to a managed DB"

I mean yeah who knew that blocking NTP therefore time drifting would break everything...

For those criticizing MongoDB, Fortnite generates $3B/year and runs on MongoDB, you should tell them it's a mistake and that they should use PG instead.

Re: Bye Bye Mongo, Hello Postgres

#226

Post by aidos »

quietbritishjim wrote: post #210
csytan wrote: post #68

Earlier quoted context omitted.

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

I’ll try to avoid a flame war, but since you’re using python, SQLAlchemy allows for composing sql strings.

Re: Bye Bye Mongo, Hello Postgres

#227

Post by manigandham »

Why did they actually migrate databases? The real issue is database operations, not the database itself. The data model is simple so anything would've worked fine, and they already have a working system.

The proper solution is to onboard the proper talent and tools or outsource it all. MongoDB does have fully-managed offerings that will automatically migrate and run your database, across cloud-providers if you need, and would've cost less than all the time and effort spent on this. This is just another example of poor technical competency at media publishers.

Re: Bye Bye Mongo, Hello Postgres

#228

Post by user5994461 »

anarazel wrote: post #94
InGodsName wrote: post #86

While we are here. I know HN hates mongo. But lemme ask this question, what are the things in which mongo is better than postgres?

Marketing / Evangelism / Outreach. I'm not kidding, and I don't predominantly mean that in a bad way. I think it's something that the postgres community / companies really aren't good at.

+1 on that. It takes 80% of the work to build the product and 80% to sale it. MongoDB was really good at the later, creating and riding the hype wave.

Re: Bye Bye Mongo, Hello Postgres

#229

Post by raverbashing »

aidos wrote: post #226
quietbritishjim wrote: post #210

Earlier quoted context omitted.

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

I’ll try to avoid a flame war, but since you’re using python, SQLAlchemy allows for composing sql strings.

Yes, an ORM, not SQL itself.

Re: Bye Bye Mongo, Hello Postgres

#230

Post by anarazel »

ddebernardy wrote: post #195
mmsimanga wrote: post #92

Earlier quoted context omitted.

I don't have any links to prove my point but I suspect we need to thank Mongo for pushing the traditional SQL databases to support JSON.

FWIW and insofar as I can recollect, JSON was supported by Postgres before Mongo was a thing, and Mongo was built based on Postgres.

No, that's not correct: Initial json support in PG was added in early 2012 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... whereas mongodb was made public in 2009, according to wikipedia: https://en.wikipedia.org/wiki/MongoDB

I don't see how Mongo was built based on Postgres, either?

Post reply on HN