How to use Postgres for everything • Hacker News Board

Re: How to use Postgres for everything

#41

Post by Cthulhu_ » Sat, Dec 07, 2024, 9:19 AM UTC

anonzzzies wrote: post #19

whateveracct wrote: post #7

Just don't use a single Postgres DB for everything as you scale up to 100+ engineers. You'll inevitably get database-as-the-API. Now if you have the actual technical leadership [1] to scale your systems by drawing logical and physical boundaries so that each unit has its own Postgres? Yeah Postgres for everything is solid. [1] Surprisingly rare I've found. Lots of "successful" CTOs who don't do this hard part.

You don't have to plan this very early; most companies won't get to 100+ engineers. Let's ship first and worry about this, much much much much later. Overarchitecting stuff makes life hard; coming into companies that have 40 servers running with everything architected for 1000000 engineers and billions of visitors while in reality there aren't even 2 users and there is 1 overworked engineer. Stop doing that and stop…

This (don't overarchitecture stuff) is actually an argument in favor of using e.g. Postgres for everything, as adding more and more tools adds complexity and (architectural) overhead.

Re: How to use Postgres for everything

#42

Post by Cthulhu_ » Sat, Dec 07, 2024, 9:23 AM UTC

RedShift1 wrote: post #22

jensneuse wrote: post #16

I absolutely love Postgres, but please allow me to say that you absolutely don't want to expose an API generated from a database to people outside of your team. This limits you a lot in changing the way you store your data. I wrote about this topic before and haven't changed my opinion much. You don't want to have that tight coupling: https://wundergraph.com/blog/six-year-graphql-recap#generate...

What exactly is the problem with tight coupling? You're going to insert an entire layer that basically translates format A to format B, just so you can later change a column name in the database and not have to change it in the API or something?

There's a few issues; one is that if you have the DB do everything, all of your business logic lives there too, instead of just the data. This is still fine if you have a single use case, but what if in addition to your main application, you also need to use it for things like BI, customer service, analytics / predictions, etc? It then quickly becomes better to use it as a datastore and have another layer decide what to do with it.

And in 30 odd years, everything will be different again, but your company's main data store will not have moved as fast.

Re: How to use Postgres for everything

#43

Post by benrutter » Sat, Dec 07, 2024, 9:33 AM UTC

whateveracct wrote: post #7

Just don't use a single Postgres DB for everything as you scale up to 100+ engineers. You'll inevitably get database-as-the-API. Now if you have the actual technical leadership [1] to scale your systems by drawing logical and physical boundaries so that each unit has its own Postgres? Yeah Postgres for everything is solid. [1] Surprisingly rare I've found. Lots of "successful" CTOs who don't do this hard part.

I'm so glad to see somebody made this comment!!

Postgres for everything is pretty neat in that you can take expertise from one place, and use it somewhere else (or just not have to learn a zillion tools)

The same database for everything is a really good way to have a tangled mess (are people still using the word complected?) where nobody knows which parts are depended on by what.

Re: How to use Postgres for everything

#44

Post by [deleted] » Sat, Dec 07, 2024, 9:36 AM UTC

[deleted]

Re: How to use Postgres for everything

#45

Post by boomskats » Sat, Dec 07, 2024, 9:39 AM UTC

jjallen wrote: post #14

CalRobert wrote: post #12

Earlier quoted context omitted.

You could go whole hog with postgrest I suppose

I always wonder if postgrest works well after growing quite a bit. I really should google the largest companies using it.

Just check out who the key project maintainers are, and then look at who they're employed by. You'll probably recognize the companies.

If you don't have schema cache reloading enabled (i.e. you're running it in prod), you can linearly scale postgrest instances pretty much infinitely without impact to the db. Each new instance just costs you a handful of connections and a small amount of memory.

Re: How to use Postgres for everything

#46

Post by boomskats » Sat, Dec 07, 2024, 9:41 AM UTC

feverzsj wrote: post #17

jjallen wrote: post #14

Earlier quoted context omitted.

I always wonder if postgrest works well after growing quite a bit. I really should google the largest companies using it.

In my experience, without careful tunning, postgres performance starts dropping significantly around 10 million rows.

Could you tell us a bit more about your experience?

Re: How to use Postgres for everything

#47

Post by cvalka » Sat, Dec 07, 2024, 9:43 AM UTC

[flagged]

Re: How to use Postgres for everything

#48

Post by anonzzzies » Sat, Dec 07, 2024, 9:44 AM UTC

malux85 wrote: post #35

anonzzzies wrote: post #20

Is there a local-first postgres solution?

What do you mean local first? Postgres is an executable that you can run on your machine if you wish

I mean storing data in your https://localfirstweb.dev . Sqlite allows that so you can work fully in our client (browser/mobile) without internet potentially and then later sync (rqlite, pouchdb, etc for instance). If I get to do 'everything' with postgres, I need that too, so I was wondering if it exists.

Re: How to use Postgres for everything

#49

Post by boomskats » Sat, Dec 07, 2024, 9:45 AM UTC

dunwaldo wrote: post #18

jensneuse wrote: post #16

I absolutely love Postgres, but please allow me to say that you absolutely don't want to expose an API generated from a database to people outside of your team. This limits you a lot in changing the way you store your data. I wrote about this topic before and haven't changed my opinion much. You don't want to have that tight coupling: https://wundergraph.com/blog/six-year-graphql-recap#generate...

you could use views as a layer in between?

You should use views as the layer in between. They'll let you version your API and make changes without breaking things.

Re: How to use Postgres for everything

#50

Post by paradite » Sat, Dec 07, 2024, 9:47 AM UTC

Anyone experienced with postgres full text search?

I want to get something simple setup but couldn't get it to work.

I want to match substrings like "/r/chatgpt" (sub reddits) in url links, but couldn't get it to match.

Tried a few types of queries like phrase, plain, default, simple, english. All have some weird issues, either not matching special characters, or not matching substrings (partial match). Also I'm somewhat limited on the syntax side by what can be done with drizzle ORM.