Live data from Hacker News

Cell-Based Architecture Explained, with Zombies

bencane.com

1–10 of 11 posts

Re: Cell-Based Architecture Explained, with Zombies

#2
“ A cell router is responsible for detecting failure within a cell (Zombie outbreak) and rerouting requests (Citizens) to other neighborhoods.”

Misleading. The difference between a pure horizontally-scaled cluster and cell-based architecture is that traffic destined for a compromised cell does not get routed to a healthy cell. Fault isolation (preventing noisy neighbors and poison pills from spilling over to progressively more cells) is a primary tenet of cell-based architecture.

Re: Cell-Based Architecture Explained, with Zombies

#3
I'm sure this will come in handy for me at some point. I've led a team through moving parts of their terraform/aws monolith to cells. Some people are able to grasp what cell-based architecture is and when it's useful pretty easily, but for a few it just doesn't click.

Re: Cell-Based Architecture Explained, with Zombies

#4

“ A cell router is responsible for detecting failure within a cell (Zombie outbreak) and rerouting requests (Citizens) to other neighborhoods.” Misleading. The difference between a pure horizontally-scaled cluster and cell-based architecture is that traffic destined for a compromised cell does not get routed to a healthy cell. Fault isolation (preventing noisy neighbors and poison pills from spilling over to progress…

How would you maintain high availability? I assume something like what AWS proposes[1] where you have an independent routing layer and each cell still has a gateway, but cells do not re-route requests to other cells.

[1]: https://aws.amazon.com/solutions/guidance/cell-based-archite...

Re: Cell-Based Architecture Explained, with Zombies

#5

“ A cell router is responsible for detecting failure within a cell (Zombie outbreak) and rerouting requests (Citizens) to other neighborhoods.” Misleading. The difference between a pure horizontally-scaled cluster and cell-based architecture is that traffic destined for a compromised cell does not get routed to a healthy cell. Fault isolation (preventing noisy neighbors and poison pills from spilling over to progress…

How would you maintain high availability? I assume something like what AWS proposes[1] where you have an independent routing layer and each cell still has a gateway, but cells do not re-route requests to other cells. [1]: https://aws.amazon.com/solutions/guidance/cell-based-archite...

If a cell dies, oh well. You then run a "rebalance" job to move users off the dead cell. If the cell has downtime, that doesn't apply to your overall downtime. In other words, maybe all central-us customers cannot use your service; but it eventually comes back up. If it is dead-dead (ie, nuke or catastrophic failure), then you would migrate those users to a new cell -- at the expense of latency for those users.

Re: Cell-Based Architecture Explained, with Zombies

#6

“ A cell router is responsible for detecting failure within a cell (Zombie outbreak) and rerouting requests (Citizens) to other neighborhoods.” Misleading. The difference between a pure horizontally-scaled cluster and cell-based architecture is that traffic destined for a compromised cell does not get routed to a healthy cell. Fault isolation (preventing noisy neighbors and poison pills from spilling over to progress…

How would you maintain high availability? I assume something like what AWS proposes[1] where you have an independent routing layer and each cell still has a gateway, but cells do not re-route requests to other cells. [1]: https://aws.amazon.com/solutions/guidance/cell-based-archite...

If failed cells cause immediate re-routing to other cells, you get a lot of fun new cascading failure scenarios to think about.

Re: Cell-Based Architecture Explained, with Zombies

#7

“ A cell router is responsible for detecting failure within a cell (Zombie outbreak) and rerouting requests (Citizens) to other neighborhoods.” Misleading. The difference between a pure horizontally-scaled cluster and cell-based architecture is that traffic destined for a compromised cell does not get routed to a healthy cell. Fault isolation (preventing noisy neighbors and poison pills from spilling over to progress…

How would you maintain high availability? I assume something like what AWS proposes[1] where you have an independent routing layer and each cell still has a gateway, but cells do not re-route requests to other cells. [1]: https://aws.amazon.com/solutions/guidance/cell-based-archite...

It would be part of your architecture design trade off. Disregarding whatever AWS peddles, you can have a cellular infrastructure (like Google), but have non-cellular systems architecture built on top of it (still deployed across different cells but the dependent services in the critical path residing in different cells or regions depending on resource availability and/or use case). The reroutes can occur at the global load balancer level, or it can happen at your own FE service. Or you could just drain the cell from the load balancer and the client retries go to a separate cell. Or you could have the load balancer not failover at all in case, which is generally the recommendation to prevent cascading failure.

It's down to your own design and acceptable business decisions, and what your business deems as availability.

Post reply on HN