Launch HN: Speedscale (YC S20) – Automatically create tests from actual traffic

We’re Ken, Nate and Matt, co-founders of Speedscale (https://speedscale.com), a tool that automatically generates continuous integration (CI) tests from past traffic. Carefully scaling rollouts to ever larger groups of customers is the safest deployment strategy, but can take weeks. Even for elite DevOps organizations up to 15% of changes to production can result in degraded service [1] [2].

We met as undergrads at Georgia Tech and come from a DevOps and operations background so we’ve seen this first hand. Each of us has over 15 years of experience building high-reliability systems, starting in the early days with satellite earth station monitoring. As interns we once wrote a bug that caused a 32 meter antenna to try to point down through the earth, almost flattening the building we were in. It was a great environment to learn about engineering reliability. We leveraged this experience to tackle monitoring Java app servers, SOA, SaaS observability and cloud data warehouses. What if we could use a form of observability data to automatically test the reliability of new deployments before they hit production? That’s the idea that got us started on Speedscale.

Most test automation tools record browser interactions or use AI to generate a set of UI tests. Speedscale works differently in that it captures API calls at the source using a Kubernetes sidecar [3] or a reverse proxy. We can see all the traffic going in and out of each service, not just the UI. We feed the traffic through an analyzer process that detects calls to external services and emulates a realistic request and response -- even authentication systems like OAUTH =). Unlike guessing how users call your service, Speedscale automation reflects reality because we collected data from your live system. We call each interaction model a Scenario and Speedscale generates them without human effort leading to an easily maintained full-coverage CI test suite.

Scenarios can run on demand or in your build pipeline because Speedscale inserts your container into an ephemeral environment where we stress it with different performance, regression, and chaos scenarios. If it breaks, you can decide the alerting threshold. Speedscale is especially effective in ensuring compliance with subtle Service Level Objective (SLO) conditions like performance regression [4].

We're not public yet but would be happy to give you a demo if you contact us at hello@speedscale.com. Also, we are doing alpha customer deployments to refine our feature set and protocol support - if you have this problem or have tried to solve it in the past we would love to get your feedback. Eventually we’ll end up selling the service via a subscription model but the details are still TBD. For the moment we’re mainly focused on making the product more useful and collecting feedback. Thanks!

[1] https://services.google.com/fh/files/misc/state-of-devops-20...

[2] https://aws.amazon.com/builders-library/automating-safe-hand...

[3] https://kubernetes.io/blog/2015/06/the-distributed-system-to...

[4] https://landing.google.com/sre/sre-book/chapters/service-lev...

For sure we see a lot of synergy with the CI systems. One of our alpha customers is using CircleCI (no surprise there). They have an issue where devs deploy services on top of each other in staging and accidentally take it down for their internal users. So Speedscale lets them detect their new build is not a good candidate to deploy to staging.

Thanks for sharing the info about your project, I am checking it out on GitHub right now. :)

Terrific stuff and glad this is becoming industrialized at scale.will keep you guys in mind as we expand

For a little background story on the satellite antenna. I was building a monitoring device driver for an Antenna Control Unit (ACU). It's like a 4 rack-unit computer with special hardware for talking to the antenna motors (azimuth, elevation and polarization). After sending it a command, the device froze up, so we rebooted it. The CMOS battery was dead so when it came back up the date was wrong, but I did not notice. I sent it a command to reposition and it began moving to point below the horizon... The bad date meant that it had the wrong geo location position for itself. Well it turns out "below horizon" is really important because the building was just a structure to hold up the antenna and it was going to crash into the ground. Fortunately someone ran in and hit the STOP button while I was staring at the monitor. That day I learned that monitoring and alerting is important stuff.

You can run it in your own environment. If you're running Kubernetes we provide an operator that orchestrates the test runs. If you are using docker we give you containers that you control with ENV VARs.

Do you have more background on the multi geo use case?