Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

github.com

Post reply on HN
Print view View original

91–100 of 133 posts

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#91

Post by efxhoy »

blue_pants wrote: post #12

Can someone briefly explain how ZDD works in general? I guess both versions of the app must be running simultaneously, with new traffic being routed to the new version of the app. But what about DB migrations? Assuming the app uses a single database, and the new version of the app introduces changes to the DB schema, the new app version would modify the schema during startup via a migration script. However, the previ…

Strong migrations helps writing migrations that are safe for ZDD deploys. We use it in our rails app, catches quite a few potential footguns. https://github.com/ankane/strong_migrations

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#92

Post by francislavoie »

simonw wrote: post #70

Does this implement the “traffic pausing” pattern? That’s where you have a proxy which effectively pauses traffic for a few seconds - incoming requests appear to take a couple of seconds longer than usual, but are still completed after that short delay. During those couple of seconds you can run a blocking infrastructure change - could be a small database migration, or could be something a little more complex as long…

Caddy does this! (As you know, I think. I feel I remember we discussed this some time ago)

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#93

Post by francislavoie »

ignoramous wrote: post #77
simonw wrote: post #70

Does this implement the “traffic pausing” pattern? That’s where you have a proxy which effectively pauses traffic for a few seconds - incoming requests appear to take a couple of seconds longer than usual, but are still completed after that short delay. During those couple of seconds you can run a blocking infrastructure change - could be a small database migration, or could be something a little more complex as long…

tbh, sounds like "living dangerously" pattern to me.

Not really, works quite well as long as your proxy/server have enough memory to hold the requests for a little while. As long as you're not serving near your max load all the time, it's a breeze.

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#94

Post by francislavoie »

moondev wrote: post #28

Does this handle a host reboot?

In theory it should, because they do health checking to track status of the upstreams. The upstream server being down would be a failed TCP connection which would fail the health check.

Obviously, rebooting the machine the proxy is running on is trickier though. I don't feel confident they've done enough to properly support having multiple proxy instances running side by side (no shared storage mechanism for TLS certs at least), which would allow upgrading one at a time and using a router/firewall/DNS in front of it to route to both normally, then switch it to one at a time while doing maintenance to reboot them, and back to both during normal operations.

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#95

Post by mrkurt »

igortg wrote: post #61
martinald wrote: post #9

Earlier quoted context omitted.

I don't think that's the real point. The real point is that 'big 3' cloud providers are so overpriced that you could run hugely over provisioned infra 24/7 for your load (to cope with any spikes) and still save a fortune. The other thing is that cloud hardware is generally very very slow and many engineers don't seem to appreciate how bad it is. Slow single thread performance because of using the most parallel CPUs p…

I'm using a managed Postgres instance in a well known provider and holy shit, I couldn't believe how slow it is. For small datasets I couldn't notice, but when one of the tables reached 100K rows, queries started to take 5-10 seconds (the same query takes 0.5-0.6 in my standard i5 Dell laptop). I wasn't expecting blasting speed on the lowest tear, but 10x slower is bonkers.

Laptop SSDs are _shockingly_ fast, and getting equivalent speed from something in a datacenter (where you'll want at least two disks) is pretty expensive. It's so annoying.

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#96

Post by noisy_boy »

whartung wrote: post #67
wejick wrote: post #31

Earlier quoted context omitted.

This is very good explanation, no judgment and simply educational. Appreciated Though I'm still surprised that some people run DB alteration on application start up. Never saw one in real life.

We do this. It has worked very well for us. There's a couple of fundamental rules to follow. First, don't put something that will have insane impact into the application deploy changes. 99% of the DB changes are very cheap, and very minor. If the deploy is going to be very expensive, then just don't do it, we'll do it out of band. This has not been a problem in practice with our 20ish person team. Second, it was kind…

The main challenge I have noticed with that approach is maintaining the sequencing across different branches being worked upon by different developers - solvable by allocating/locking the numbers from a common place. The other is rolling back multiple changes for a given view/stored proc where, say, each change added a separate column - if only one is rolled back, how do you automate that? Easily done manually though.

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#98

Post by whartung »

noisy_boy wrote: post #96
whartung wrote: post #67

Earlier quoted context omitted.

We do this. It has worked very well for us. There's a couple of fundamental rules to follow. First, don't put something that will have insane impact into the application deploy changes. 99% of the DB changes are very cheap, and very minor. If the deploy is going to be very expensive, then just don't do it, we'll do it out of band. This has not been a problem in practice with our 20ish person team. Second, it was kind…

The main challenge I have noticed with that approach is maintaining the sequencing across different branches being worked upon by different developers - solvable by allocating/locking the numbers from a common place. The other is rolling back multiple changes for a given view/stored proc where, say, each change added a separate column - if only one is rolled back, how do you automate that? Easily done manually though…

I will say that stored procs are specifically messy, and we did not have many of those. They had a tendency to really explode the change file. With DDL, you can fix a table column in isolation. Fixing a typo in a 100 line stored proc is another 100 lines. And we certainly didn't have multiple people working on the same proc at the same time.

We had no real need to address that aspect, and I would do something more formal with those if I had to, such as having a separate file with the store proc, and simply a note that it has changed in the change file. I mean, that's a bit of a trick.

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#99

Post by mwcampbell »

mrkurt wrote: post #95
igortg wrote: post #61

Earlier quoted context omitted.

I'm using a managed Postgres instance in a well known provider and holy shit, I couldn't believe how slow it is. For small datasets I couldn't notice, but when one of the tables reached 100K rows, queries started to take 5-10 seconds (the same query takes 0.5-0.6 in my standard i5 Dell laptop). I wasn't expecting blasting speed on the lowest tear, but 10x slower is bonkers.

Laptop SSDs are _shockingly_ fast, and getting equivalent speed from something in a datacenter (where you'll want at least two disks) is pretty expensive. It's so annoying.

To clarify, are you talking about when you buy your own servers, or when you rent from an IaaS provider?

Re: Kamal Proxy – A minimal HTTP proxy for zero-downtime deployments

#100

Post by chipdart »

dewey wrote: post #76
viraptor wrote: post #51

It's an interesting choice to make this a whole app, when the zero-downtime deployments can be achieved with other servers trivially these days. For example any app+web proxy which supports Unix sockets can do zero-downtime by moving the file. It's atomic and you can send the warm-up requests with curl. Building a whole system with registration feels like an overkill.

That's just a small part of Kamal ( https://kamal-deploy.org ), their deployment tool they build and used to move from the cloud to their own hardware, saving millions ( https://basecamp.com/cloud-exit ).

> That's just a small part of Kamal (...)

That does not address OP's point. Any plain old reverse proxy handles this usecase, particularly those already used extensively as ingress controllers for container orchestration systems. Some even support specialized routing strategies.

Other than NIH, what exactly does kamal-proxy offers that any other reverse proxy doesn't already provide?

Post reply on HN