Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The startup's Postgres survival guide

hatchet.run

Post reply on HN
Print view View original

241–250 of 255 posts

Re: The startup's Postgres survival guide

#241

Post by brianhama »

traceroute66 wrote: post #21

I did a search in that post for "function", zero results. Unimpressive. Not even the most cursory of discussion of stored functions ? Given that many startup's Postgres instances will no doubt be backing some web-ui or app that takes untrusted input, surely they could have at least had a brief discussion about how stored functions can help against SQL injection attacks ? Not only that but it means you have to think,…

Stored procedures can be such a performance hack. Removing the need for multiple round trips to the database can make a huge difference.

Re: The startup's Postgres survival guide

#242

Post by drdexebtjl »

skullone wrote: post #240
drdexebtjl wrote: post #232

Earlier quoted context omitted.

Are you saying DBAs would be doing _continuous_ recovery testing? That’s not even a thing that exists. Managed databases don’t lose customer data. They would be sued and lose, and if word got out, they would lose their customers. They are optimized for the machines they run on. The proprietary tooling from AWS for managing HA clusters, doing blue-green upgrades and monitoring for slow queries is all miles ahead of th…

DBAs in orgs who resource their data teams indeed do constant validation of recovery. I know many DBAs who are in more demand than ever due to the rise of managed databases. If you don't need a DBA, you aren't at a scale or maturity just yet. That you're so dismissive of this role, I have to dismiss your entire opinion because you simply don't grasp the scale of large databases

>constant validation of recovery

So the same thing I said we do, a periodic validation of recovery, but with an acceptably short period.

SWEs can do that too. They can even write software to do most of it automatically.

They went through systems design interviews to ensure they know how to scale without picking technology choices that require a dedicated human caretaker.

It’s the opposite. Needing a DBA to operate a completely managed database is a sign of not enough maturity.

Re: The startup's Postgres survival guide

#243

Post by skullone »

drdexebtjl wrote: post #242
skullone wrote: post #240

Earlier quoted context omitted.

DBAs in orgs who resource their data teams indeed do constant validation of recovery. I know many DBAs who are in more demand than ever due to the rise of managed databases. If you don't need a DBA, you aren't at a scale or maturity just yet. That you're so dismissive of this role, I have to dismiss your entire opinion because you simply don't grasp the scale of large databases

>constant validation of recovery So the same thing I said we do, a periodic validation of recovery, but with an acceptably short period. SWEs can do that too. They can even write software to do most of it automatically. They went through systems design interviews to ensure they know how to scale without picking technology choices that require a dedicated human caretaker. It’s the opposite. Needing a DBA to operate a…

DBAs write software to do this too. You think DBAs are just clicking around in some gui? Hate to break it to you, but every field has done leaps and bounds over the decades too. SWEs can certainly "learn the trade" as a DBA can "learn to code" their own tooling as well, but you realize people specialize, there's fields of practice and domain experts. One of the greatest inventions is the division of labor and specialization.

Re: The startup's Postgres survival guide

#244

Post by ComputerGuru »

christophilus wrote: post #177
busymom0 wrote: post #11

Earlier quoted context omitted.

I went with the self host route by putting it on a few years old computer with much better specs than cheap vps. Cloudflare tunnels to make the web server accessible on the internet.

Nice. How fast is your home internet connection? I’ve thought about putting an old laptop to this use. But I don’t want my day to day internet to suffer if my site gets traffic spikes. And also, I’m nervous about non-ecc ram.

Bandwidth is almost never a concern for databases. The right number to worry about instead is latency.

Re: The startup's Postgres survival guide

#245

Post by frollogaston »

arkh wrote: post #189
frollogaston wrote: post #88

This advice is good, but every startup I've worked with has run into lower hanging fruit than this even. Less scaling problems and more just organizational. Usually what fixes that is: 1. Don't use an ORM. 2. Use serial PKs, not meaningful fields (article mentions this). 3. Use jsonb if needed, but sparingly. 4. Make your source of truth append-only, meaning you only insert, never update or delete. You can have secon…

> Don't reinvent a type system I'd like to add: stay away from using arrays and user defined types. They feel like a good idea at first, but will become a problem when using fetched data in your code. And another miss, if you're managing your servers: pgtune. The default postgres configuration is very conservative regarding resources so you can get good performance gains just by adjusting them to your server specific…

Agreed. Jsonb is the only thing I'll use, and that's only if I'm getting/setting the whole col at once, not doing SQL that goes into the payload.

Re: The startup's Postgres survival guide

#246

Post by throwaway894345 »

dwedge wrote: post #238
throwaway894345 wrote: post #226

Earlier quoted context omitted.

More likely it’s just not worth going out of their way to support niche deployments like that. I’ve been using various clouds for years, I’ve done half a dozen cloud migrations, currently working at a multicloud org—the vendors aren’t doing much to lock us in. They very much enable us to move platforms by offering things like bulk data transfer tools, standard application runtimes like Kubernetes, workload identity f…

> Like I have no doubt that they’re all greedy bastards, but they aren’t doing much to lock people in. A few years ago we were trying to do a live migration from on-prem into GCP (mostly to have GCP as a failover option but move everything there if on-prem went down). We configured an async GCP slave from the on-prem cluster and once running it was easy. Multi-TB migration straight into CloudSQL with seconds of downt…

A company not going out of their way to make it expedient for you to transition off of their cloud isn’t “lock in”.

Re: The startup's Postgres survival guide

#248

Post by [deleted] »

hmokiguess wrote: post #8

Postgres is my favourite thing, but I find it's prohibitively costly when bootstrapping something that is lean and frugal. I end up with a mixture of serverless storage like DynamoDB, S3, DuckDB on S3, and SQLite. Am I crazy? How can one have a decent Postgres and not pay at least $100/mo (yes, when I say frugal I mean really frugal ... think solo founder that likes to stay on free tiers haha) -- I am aware of Neon/S…

[deleted]

Re: The startup's Postgres survival guide

#249

Post by dynotable »

hmokiguess wrote: post #8

Postgres is my favourite thing, but I find it's prohibitively costly when bootstrapping something that is lean and frugal. I end up with a mixture of serverless storage like DynamoDB, S3, DuckDB on S3, and SQLite. Am I crazy? How can one have a decent Postgres and not pay at least $100/mo (yes, when I say frugal I mean really frugal ... think solo founder that likes to stay on free tiers haha) -- I am aware of Neon/S…

[flagged]

Re: The startup's Postgres survival guide

#250

Post by dynotable »

hmokiguess wrote: post #8

Postgres is my favourite thing, but I find it's prohibitively costly when bootstrapping something that is lean and frugal. I end up with a mixture of serverless storage like DynamoDB, S3, DuckDB on S3, and SQLite. Am I crazy? How can one have a decent Postgres and not pay at least $100/mo (yes, when I say frugal I mean really frugal ... think solo founder that likes to stay on free tiers haha) -- I am aware of Neon/S…

[dead]
Post reply on HN