const openfuse = new OpenfuseCloud(...); what happens when your service goes down
It is answered in the FAQ at the bottom of the page "The SDK is fail-open by design. If our service is unreachable, it falls back to the last known breaker state. If no state has ever been cached (e.g., a cold start with no connectivity), it defaults to closed, meaning your protected calls keep executing normally. Your app is never blocked by Openfuse unavailability."
Show HN: I built a fuse box for microservices
11–20 of 25 posts
Re: Show HN: I built a fuse box for microservices
#12Now I have seen it all... a SaaS solution for making local outages global.
Re: Show HN: I built a fuse box for microservices
#13Earlier quoted context omitted.
It is answered in the FAQ at the bottom of the page "The SDK is fail-open by design. If our service is unreachable, it falls back to the last known breaker state. If no state has ever been cached (e.g., a cold start with no connectivity), it defaults to closed, meaning your protected calls keep executing normally. Your app is never blocked by Openfuse unavailability."
Yup, that is true for both Cloud and Self-hosted, it never blocks any executions by any external factors other than the breaker is KNOWN as open. The state sync and the hot path are 2 completely separated flows.
Re: Show HN: I built a fuse box for microservices
#14Making your circuit breaker state global seems like it would just exacerbate the problem. Failures are often partial in the real world.
Re: Show HN: I built a fuse box for microservices
#15How does it deal with partial failures like the upstream being unreachable from one datacenter but not the other, or from one region but not another? Or when the upstream uses anycast or some other way to route to different origins depending on where the caller is? Making your circuit breaker state global seems like it would just exacerbate the problem. Failures are often partial in the real world.
Re: Show HN: I built a fuse box for microservices
#16But this is a feature, not a bug. You seems to be assuming that people use circuit-breaks only on external requests, in this situation your approach seems reasonable.
If you have cbs between every service call your model doesn't seem a good idea. Where I work every network call is behind a cb (external services, downstream services, database, redis, s3, ...) and it's pretty common to see failures isolated in a single k8s node. In this situation we want to have independent cbs, they can open independently.
Your take on observability/operation seems interesting but it is pretty close to feature flags. And that is exactly how we handle these scenarios, we have a couple of feature flags we can enable to switch traffic around during outages. Switching to fallback is easy most of the time, but switching back to normal operation is harder to do.
Re: Show HN: I built a fuse box for microservices
#17> they make per-instance decisions with per-instance state But this is a feature, not a bug. You seems to be assuming that people use circuit-breaks only on external requests, in this situation your approach seems reasonable. If you have cbs between every service call your model doesn't seem a good idea. Where I work every network call is behind a cb (external services, downstream services, database, redis, s3, ...)…
Openfuse is aimed at the other case: shared external dependencies where 15 services all call the same dependency and each one is independently discovering the same outage at different times. Different failure modes, different coordination needs, and you have no way to manually intervene or even just see what's open. Think of your house: every appliance has its own protection system, but that doesn't exempt you from having the distribution board.
You can also put it between your service/monolith and your own other services, e.g. if a recommendations engine, or a loyalty system in an E-Commerce or POS softwares go down, all hotpath flows from all other services will just bypass their calls to it. So with "external" I mean another service, whether it's yours or from a vendor.
On the feature flag point: that's interesting because you're essentially describing the pain of building circuit breaker behavior on top of feature flag infrastructure. The "switching back" problem you mention is exactly what half-open state solves: controlled probe requests that test recovery automatically and restore traffic gradually, without someone manually flipping a flag and hoping. That's the gap between "we can turn things off" and "the system recovers on its own." But yeah, we can all call Openfuse just feature flags for resilience, as I said: it's a fusebox for your microservices.
Curious how you handle the recovery side, is it a feature flag provider itself? or have you built something around it and store in your own database?
Re: Show HN: I built a fuse box for microservices
#18> they make per-instance decisions with per-instance state But this is a feature, not a bug. You seems to be assuming that people use circuit-breaks only on external requests, in this situation your approach seems reasonable. If you have cbs between every service call your model doesn't seem a good idea. Where I work every network call is behind a cb (external services, downstream services, database, redis, s3, ...)…
You're right, for intra-cluster calls where failures are scoped between the node itself and the infra around it, per-instance breakers are what you want. I wouldn't suggest centralizing those, and I might be wrong, but in most of these scenarios there is no fallback anyways (maybe except Redis?) Openfuse is aimed at the other case: shared external dependencies where 15 services all call the same dependency and each o…
Re: Show HN: I built a fuse box for microservices
#19Instead of paying for a SaaS, a team can autoprogram an on-prem clone for less.
Totally possible, and some teams do. You need a state store, a evaluator job, a propagation layer to push state changes to every instance, a SDK, a dashboard, alerting, audit logging, RBAC, and a fallback strategy for when the coordination layer itself goes down. It's not complex individually, but it takes time, and it's the ongoing maintenance that gets you. Openfuse is a bet that most teams would rather pay $99/mo…
Re: Show HN: I built a fuse box for microservices
#20> they make per-instance decisions with per-instance state But this is a feature, not a bug. You seems to be assuming that people use circuit-breaks only on external requests, in this situation your approach seems reasonable. If you have cbs between every service call your model doesn't seem a good idea. Where I work every network call is behind a cb (external services, downstream services, database, redis, s3, ...)…
You're right, for intra-cluster calls where failures are scoped between the node itself and the infra around it, per-instance breakers are what you want. I wouldn't suggest centralizing those, and I might be wrong, but in most of these scenarios there is no fallback anyways (maybe except Redis?) Openfuse is aimed at the other case: shared external dependencies where 15 services all call the same dependency and each o…
I don't really see what problem this solves. If you have proper timeouts and circuit breakers in your service this shouldn't really matter. This solution will save a few hundred requests, but I don't think this really matters. If this is a pain point its easier to adjust the circuit-breaker settings (reduce the error rate, increase the window, ...) than introduce a whole new level of complexity.
> Curious how you handle the recovery side
We have a feature flag provider built in-house. But it doesn't support this use-case, so what we done is to create flag where we put the % value we want to bring back and handle the logic inside the service. Example: if you want to bring back 6,25% (1/16) of our users this means we should switch back every user that has an account-id ending in 'a'. For 12.5% (2/16) we want users with account-id ending either in 'a' or 'b'. This is a pretty hacky solution, but it solves our problem when we need to transition from our fallback to our main flow.