Live data from Hacker News

PostgreSQL 14

postgresql.org

111–120 of 293 posts

Re: PostgreSQL 14

#111
post #52
post #19

PostgreSQL is one of those tools I know I can always rely on for a new use-case. There are very few cases where it can't do exactly what I need (large scale vector search/retrieval). Congrats on the 14.0 release. The pace of open source has me wondering what we'll be seeing 50 years from now.

I recall seeing some library that adds vector search to Postgres. Maybe https://github.com/ankane/pgvector ? Also there's Pinecone ( https://www.pinecone.io ) which can sit alongside Postgres or any other data warehouse and ingest vector embeddings + metadata for vector search/retrieval.

I need a self managed solution, so I'm not sure Pinecone is feasible and I don't think pgvector scales well enough for my use-case (hundreds of millions of vectors).

So far I think I'm going to go with Milvus[1], ideally I'd just have a foreign data wrapper for Milvus or FAISS.

[1] https://github.com/milvus-io/milvus

Re: PostgreSQL 14

#112

Somewhat related, but does anybody have suggestions for a quality PostgreSQL desktop GUI tool, akin to pgAdmin3? Not pgAdmin 4, whose usability is vastly inferior. DBeaver is adequate, but not really built with Postgres in mind.

I use Valentina Studio (free license). Does everything I need.

Re: PostgreSQL 14

#113

Somewhat related, but does anybody have suggestions for a quality PostgreSQL desktop GUI tool, akin to pgAdmin3? Not pgAdmin 4, whose usability is vastly inferior. DBeaver is adequate, but not really built with Postgres in mind.

another vote for TablePlus

Re: PostgreSQL 14

#114

Somewhat related, but does anybody have suggestions for a quality PostgreSQL desktop GUI tool, akin to pgAdmin3? Not pgAdmin 4, whose usability is vastly inferior. DBeaver is adequate, but not really built with Postgres in mind.

When did you last use pgadmin4? It recently went through a big changes, in my opinion it's the best client for PostgreSQL.

Re: PostgreSQL 14

#115

Earlier quoted context omitted.

An orthogonal migration issue which I'm hitting right now: we need to migrate from heroku postgres to aws rds postgres, and I'm stressed about the risk and potential downtime in doing so. If there was a way to make a replica in rds based on heroku, promote the rds replica to be the primary, hard switch our apps over to rds, that'd be a lifesaver. I'm working through this blog post [1] now, but there is still a bit to…

I did that exact migration. Unfortunately, to my knowledge, there's no way to do it with zero downtime. You need to make your app read only until the RDS instance has ingested your data, then you can cut over. For me, that was roughly one gigabyte of data and took about forty seconds. My best advice is to automate the whole thing. You can automate it with the Heroku and AWS CLIs. Test on your staging site until you c…

Thank you for this - extremely helpful in validating the current approach and de-risking the developer time.

Re: PostgreSQL 14

#116

Earlier quoted context omitted.

An orthogonal migration issue which I'm hitting right now: we need to migrate from heroku postgres to aws rds postgres, and I'm stressed about the risk and potential downtime in doing so. If there was a way to make a replica in rds based on heroku, promote the rds replica to be the primary, hard switch our apps over to rds, that'd be a lifesaver. I'm working through this blog post [1] now, but there is still a bit to…

logical replication, but this is one of the walls that heroku creates.

Coming from the outside, with zero understanding of the internal details, my hunch is the same: lack of support for logical replication is more of a business decision than a technical decision. (But again, this a hunch -- partially based on how good heroku is from a technical perspective)

Re: PostgreSQL 14

#117

Any suggestions to learn and go deep in PostgreSQL for someone who worked mostly on NoSQL (MongoDB)? From the few days I have explored it, it is absolutely incredible, so congratulations for the work done and good luck on keeping the quality so high!

As a young developer I stayed away from traditional relational databases because I thought they were boring and restrictive. I quickly fell in love with them. I realized they painlessly provided all the things I was already doing with data and they were doing it much faster and more reliably.

I hope you have the same kind of experience and enjoy Postgres!

As far as actual advice...

1. The key concept of an RDBMS is that each table typically specifies relationships to other tables. So in order to query data you will typically be joining multiple tables. Sometimes new DB programmers get a lot of enlightenment from the simple Venn diagrams that illustrate the basic types of JOINs: https://www.google.com/search?q=inner+join+diagram ...IMHO, once you get this (fairly simple) paradigm the rest is easy.

2. Database constraints are your friends. At a minimum you specify the types of the columns, obviously, but you can go much farther - specifying NOT NULL constraints, foreign keys, and more complex check conditions. Do as much of this as feasible at the database level. Otherwise it is something you need to code and enforce at the application level, where it will generally be less performant and more prone to coder error. Additionally, any constraints not enforced at the application level will need to be implemented across multiple applications if more than one app uses the database.

The second one above is an example of something that sounds boring and restrictive but really frees you up to do other things as a developer. About a decade ago, the "trend" was to treat RDBMSs as "dumb" storage and do all that stuff at the application level, in the name of being database-agnostic. Opinions remain divided, but I think that was an objectively bad trend. For one thing, folks noticed they hardly ever needed to suddenly switch databases, but there are other reasons as well.

Re: PostgreSQL 14

#118

I'm trying to understand if with v14 I will be able to connect Debezium to a "slave" node and not to the "master" in order to read the WAL but can't figure it out. Can someone help me with this?

I was just yesterday talking to someone about this; they mentioned that Patroni leverages some way for setting up replication slots on replicas [1]. Haven't tried my self yet, but seems worth exploring.

Something I'd like to dive into within Debezium is usage of the pg_tm_aux extension, which supposedly allows to set up replication slots "in the past", so you could use this to have seamless failover to replicas without missing any events. In any case, this entire area is of high importance for us and we're keeping an eye on any improvements closely, so I hope it will be sorted out sooner or later.

[1] https://twitter.com/cyberdemn/status/1443130986116624388 [2] https://github.com/x4m/pg_tm_aux

Re: PostgreSQL 14

#119
post #69

Earlier quoted context omitted.

FWIW Mysql 8 has gotten a lot better in standards compliance and ironing out legacy quirks, with some config tweaks. While my heart still belongs to PostgreSQL things like no query hints, dead tuple bloat (maybe zheap will help?), less robust replication (though getting better!), and costly connections dampens my enthusiasm.

> maybe zheap will help? According to Robert Haas, the zheap project is dead.

Where did Robert Haas say this? Quick search didn't surface much, except for a blog post from July by Hans-Jürgen Schönig: https://www.cybertec-postgresql.com/en/postgresql-zheap-curr... Doesn't sound like they discontinued the project.

Re: PostgreSQL 14

#120

Any suggestions to learn and go deep in PostgreSQL for someone who worked mostly on NoSQL (MongoDB)? From the few days I have explored it, it is absolutely incredible, so congratulations for the work done and good luck on keeping the quality so high!

Pg docs are so good I reference them whenever I want to check the SQL standards, even if I'm working on another DB. (I prefer standard syntax to minimize effort moving DBs.) Otherwise maybe try it with a toy project.

I stick to standard SQL syntax/features whenever possible as well, but...

Honest question: how often do you switch databases?

I've never really found myself wanting or needing to do this.

Only time I could really see myself wanting to do this is if I was writing some kind of commercial software (eg, a database IDE like DataGrip) that needed to simultaneously support various multiple databases.

    > MySQL
It feels particularly limiting to stick to "standard" SQL for MySQL's sake, since they frequently lag behind on huge chunks of "standard" SQL functionality anyway. For example, window functions (SQL2003 standard) took them about a decade and a half to implement.
Post reply on HN