Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

291–300 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#291

Post by nexuist »

AlexCoventry wrote: post #288
dmix wrote: post #271

Earlier quoted context omitted.

It’s probably better if they didn’t answer this question...

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work. > Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

>Since all our other services are running in AWS, the obvious choice was DynamoDB – Amazon’s NoSQL database offering. Unfortunately at the time Dynamo didn’t support encryption at rest. After waiting around nine months for this feature to be added, we ended up giving up and looking for something else, ultimately choosing to use Postgres on AWS RDS.

Re: Bye Bye Mongo, Hello Postgres

#292

Post by xkcd-sucks »

quietbritishjim wrote: post #278
QuinnWilton wrote: post #237

Earlier quoted context omitted.

Elixir's primary database wrapper, Ecto [0], lets you dynamically build queries at runtime, and also isn't an ORM. Here's two examples directly from the docs: # Query all rows in the "users" table, filtering for users whose age is > 18, and selecting their name "users" |> where([u], u.age > 18) |> select([u], u.name) # Build a dynamic query fragment based on some parameters dynamic = false dynamic = if params["is_pub…

As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline unless it can do analogous things to Postgres's JSONB fields. For example, can it unwind an array field, match those subrecords where one field (like a "key") matches a value and another field (like a "value") exceeds an overall value, and then apply this condition to filter the overall rows in the table? Also, one of the benef…

>For example, can it...

Yes. There will be a subquery and jsonb indexes need to be thought out in order to make it fast

Re: Bye Bye Mongo, Hello Postgres

#293

Post by romeisendcoming »

SllX wrote: post #70
perfunctory wrote: post #57

Earlier quoted context omitted.

Reminds me of “The computer industry is the only industry that is more fashion-driven than women’s fashion.” — Larry Ellison

This actually resonates with me, maybe not in the way Ellison intended as I’m not familiar with the context he said it in. A bit off the main topic but the more I revert to just using emacs for some task I previously used a .app bundle or web page for, the more I question how much we the computer industry has just been spinning its wheels for the last 30+ years. I honestly can’t really tell what value WIMP-centric GU…

vi, cvs|svn, C, bash, awk,sed, python and tcl. Linux (rpm based) or FreeBSD. Firefox or lynx. Haven't changed much in 20 years.

Re: Bye Bye Mongo, Hello Postgres

#294

Post by ak4g »

anarazel wrote: post #230
ddebernardy wrote: post #195

Earlier quoted context omitted.

FWIW and insofar as I can recollect, JSON was supported by Postgres before Mongo was a thing, and Mongo was built based on Postgres.

No, that's not correct: Initial json support in PG was added in early 2012 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... whereas mongodb was made public in 2009, according to wikipedia: https://en.wikipedia.org/wiki/MongoDB I don't see how Mongo was built based on Postgres, either?

My guess is the grandparent is (mis-)remembering the kerfuffle[0] around mongo shipping a copy of PG as a "BI Connector". But yeah, the timeline is off, that was in 2015.

0: https://news.ycombinator.com/item?id=10697692

Re: Bye Bye Mongo, Hello Postgres

#295

Post by quietbritishjim »

inferiorhuman wrote: post #289
quietbritishjim wrote: post #278

Earlier quoted context omitted.

As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline unless it can do analogous things to Postgres's JSONB fields. For example, can it unwind an array field, match those subrecords where one field (like a "key") matches a value and another field (like a "value") exceeds an overall value, and then apply this condition to filter the overall rows in the table? Also, one of the benef…

> As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline Huh? The aggregation framework is a solution to a mongo-only problem. Most other databases are performant, but Mongo suffers wildly from coarse locking and slow performance putting things into and retrieving things from the javascript VM. > For example, can it unwind an array field, match those subrecords where one field (lik…

The "this" was referring to dynamically building queries (the GP comment by me) in Ecto (the parent comment by QuinnWilton). What you've said is a non-sequitur in the context of this little discussion. My whole original point is that raw SQL isn't right in all situations, and you appear to be arguing that I just use SQL instead.

Re: Bye Bye Mongo, Hello Postgres

#296

Post by quietbritishjim »

luhn wrote: post #279
quietbritishjim wrote: post #262

Earlier quoted context omitted.

For it to replace MongoDB's aggregation pipeline, it would need to play nicely with JSONB. Does it do that? This is the thing I'm really missing. For example, if documents in the JSONB column all look roughly like this: { "someArrayField": [ { "key": "steve", "value": 7 }, { "key": "bob", "value": 15 }, ], "someOtherField": [ "whatever" ] } * Can I count the number of entries in someArrayField, summed across all reco…

Yes, you can! The jsonb_array_elements function is roughly similar to Mongo’s $unwind pipeline op. It explodes a JSON array into a set of rows. From there it’s pretty simple aggregates to achieve what you’re looking for. I was evaluating Mongo a couple months back to solve roughly the same problems. Eventually discovered Postgres already had what I was looking for.

Allow me to restate this question:

> Does it do that?

It was supposed to be clear from the context that this meant:

> Does building queries programmatically with SQLAlchemy do that?

Maybe I'm misreading your comment, but you seem to just be talking about writing queries directly in SQL.

If not, could you give an example/link of how to programmically build a query in SQLAlchemy that dynamically makes use of jsonb_array_elements? It would be hugely useful if I could do that.

Re: Bye Bye Mongo, Hello Postgres

#297

Post by vips7L »

mopierotti wrote: post #251
quietbritishjim wrote: post #210

Earlier quoted context omitted.

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

To add to the pile of responses: in Scala, Slick is great library that lets you compose sql queries and fragments of queries quite effectively. ( http://slick.lightbend.com/ ) At my company we built a UI on top of Slick that lets users of our web app define complex triggers based on dynamic fields and conditions which are translated to type-safe SQL queries.

QueryDSL and jOOq as well for java.

Re: Bye Bye Mongo, Hello Postgres

#298

Post by inferiorhuman »

quietbritishjim wrote: post #295
inferiorhuman wrote: post #289

Earlier quoted context omitted.

> As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline Huh? The aggregation framework is a solution to a mongo-only problem. Most other databases are performant, but Mongo suffers wildly from coarse locking and slow performance putting things into and retrieving things from the javascript VM. > For example, can it unwind an array field, match those subrecords where one field (lik…

The "this" was referring to dynamically building queries (the GP comment by me) in Ecto (the parent comment by QuinnWilton). What you've said is a non-sequitur in the context of this little discussion. My whole original point is that raw SQL isn't right in all situations, and you appear to be arguing that I just use SQL instead.

I can't speak to every ORM or database interface in existence but ActiveRecord will happily handle Postgres arrays and let you use the built-in array functions just handily without having to write queries by hand. Ecto is less elegant, but you can still finangle some arrays with it.

As far as views are concerned, I don't know what to tell you. Sure, you'll probably have to craft the view itself by hand. The result is that you can then use most abstractions of your choosing on top of it though.

Re: Bye Bye Mongo, Hello Postgres

#299

Post by chrisoverzero »

SllX wrote: post #70
perfunctory wrote: post #57

Earlier quoted context omitted.

Reminds me of “The computer industry is the only industry that is more fashion-driven than women’s fashion.” — Larry Ellison

This actually resonates with me, maybe not in the way Ellison intended as I’m not familiar with the context he said it in. A bit off the main topic but the more I revert to just using emacs for some task I previously used a .app bundle or web page for, the more I question how much we the computer industry has just been spinning its wheels for the last 30+ years. I honestly can’t really tell what value WIMP-centric GU…

> I honestly can’t really tell what value WIMP-centric GUIs have brought to the table besides fashion […]

Literal billions of users.

Re: Bye Bye Mongo, Hello Postgres

#300

Post by 6t6t6t6 »

kstrauser wrote: post #176
astine wrote: post #167

Earlier quoted context omitted.

For a serious note, there are occasionally situations where I have to take in a lot of data that I know nothing about. Sooner of the time this data is either in json or readily convertible to json so losing it into a mongodb database and poking around in it is a reasonable preliminary step to whatever the more permanent solution should be.

Would PostgreSQL + JSONB work for you there?

Allegedly, Mongo is more efficient if you need to ingest billions of records.
Post reply on HN