Live data from Hacker News

Toxiproxy is a framework for simulating network conditions

github.com

21–30 of 31 posts

Re: Toxiproxy is a framework for simulating network conditions

#21
This is one of my favorite tools out there. So simple but yet so powerful.

You get the real deal when you pair it with other tools to analyze and monitor traffic.

Worth the note, I sometimes use this front end in case I want to quickly adjust stuff https://github.com/buckle/toxiproxy-frontend

Re: Toxiproxy is a framework for simulating network conditions

#23

I wrote the initial version of Toxiproxy back in 2014, but Jacob Wirth took it way beyond during his internship at Shopify. It came out of a need for writing integration tests for resiliency work we did at Shopify back then. [1] We didn't want someone to suddenly re-introduce a hard dependency on e.g. Redis on Shopify's storefronts. The initial prototype was a shell script that used lsof(1) and gdb(1) to close the fi…

What a useful tool for resilience engineering.

https://github.com/dastergon/awesome-chaos-engineering#notab... does list toxiproxy.

Any general pointers for handling network connectivity issues (from any OSI layer) in client and server apps?

Many apps lack 'pending in outbox' functionality that we expect from e.g. email clients.

- [ ] Who could develop a set of reference toxiproxy 'test case mutators' (?) for simulating typical #DisasterRelief connectivity issues?

(In Python, Pytest + Hypothesis + Toxiproxy-python would be useful.)

Re: Toxiproxy is a framework for simulating network conditions

#24

Anyone know of anything like this but at the TCP level? I would love to have a way of simulating network partitions and different message delays for distributed algorithms implemented in Elixir. In an ideal world I'd be able to hook the elixir send/receive primitives to intercept messages between processes even in a single node.

May or may not be what you're looking for, but you can do lots of this in linux by using tc. https://man7.org/linux/man-pages/man8/tc.8.html Requires care to use and understand, so may not be applicable, but allows introducing latency, jitter, packet loss, etc.

+1 for tc. Used to run a network testing lab of about 12 racks of network gear and tc was my go-to for simulating network level performance impacts for both small and large test scenarios.

Re: Toxiproxy is a framework for simulating network conditions

#25
I have recently used it to simulate a network breakdown between application and the database server to study what the application does. There are various clients you can use with toxiproxy via which you can set "toxics" (i.e. a delay in the network traffic, etc), but I found the cli more suited to my needs. The other clients, (for e.g. nodejs) can help you write unit tests which are meant to test the resiliency.

The other fact that caught my interest was the concept of "gamedays". It really about introducing "problems" in the production system randomly and keeping the support staff that manage application incidents on it toes. (More about in this talk: https://www.youtube.com/watch?v=TTfWpHuCJXk)

Re: Toxiproxy is a framework for simulating network conditions

#26

Anyone know of anything like this but at the TCP level? I would love to have a way of simulating network partitions and different message delays for distributed algorithms implemented in Elixir. In an ideal world I'd be able to hook the elixir send/receive primitives to intercept messages between processes even in a single node.

Isn't this already at a TCP level? HTTP is only used for management of the proxy?

Yeah, all the other replies confuse me. Toxiproxy _is_ a TCP proxy. You can use it for Redis, MySQL, or anything else that runs on TCP.

Re: Toxiproxy is a framework for simulating network conditions

#27

Anyone know of anything like this but at the TCP level? I would love to have a way of simulating network partitions and different message delays for distributed algorithms implemented in Elixir. In an ideal world I'd be able to hook the elixir send/receive primitives to intercept messages between processes even in a single node.

You might be interested in Partisan's fault injector. I don't know if @cmeiklejohn is still working on it, but it's really cool nonetheless.

https://www.youtube.com/watch?v=KrwhOkiifQ8

https://github.com/lasp-lang/partisan

Re: Toxiproxy is a framework for simulating network conditions

#29

Anyone know of anything like this but at the TCP level? I would love to have a way of simulating network partitions and different message delays for distributed algorithms implemented in Elixir. In an ideal world I'd be able to hook the elixir send/receive primitives to intercept messages between processes even in a single node.

Isn't this already at a TCP level? HTTP is only used for management of the proxy?

Ah, I see, my bad. If I understand correctly though it's mostly for hooking client/server architectures? I'm interested more in things like hooking comms between replicas of a replicated database or an implementation of paxos. Would it be useable for that? For example, can I take a list of processes running on different ports and set up a proxy per connection (assuming they are connected all-to-all)? Maybe I should just RTFM :)

Re: Toxiproxy is a framework for simulating network conditions

#30

Anyone know of anything like this but at the TCP level? I would love to have a way of simulating network partitions and different message delays for distributed algorithms implemented in Elixir. In an ideal world I'd be able to hook the elixir send/receive primitives to intercept messages between processes even in a single node.

Iptables random module
Post reply on HN