Our Rails test suite runs in 1 minute on Buildkite

Would be nice if they shared how many tests they have :)

Indeed. The only hint is "Our whole test suite locally takes around 12 minutes running serially on a MacBook Pro" so it should be a real world case. They used 64 cores on Buildkite.

One minute? That's terrible. What kinds of tests and how many are you running?

> We never run all of our application’s tests in local development. It’s not a good use of time and will never be as fast as running them on CI.

WTF this is bad practice. Developers should always get in the habit of running tests locally first. No test should require running in the cloud. And, even more, it's actually rare that CI is more performant than your developers' machines. You should invest in your developers.

> Our whole test suite locally takes around 12 minutes running serially on a MacBook Pro.

That's really really really awful. Most projects' tests, even when they're poorly written, still run in 3 or 4 minutes. My own tests run on the order of single-digit seconds.

> We haven’t put much effort here because it’s not something our engineers ever run.

Well of course not. They can't iterate fast if they're slowed down by tests. That's why tests have to be fast!

> This gave us some speed gains, but we wanted it really fast. Our infrastructure team set us up with some 64 core machines

Holy schnapple if I had 64 core machines, a complete test suite would finish on the order of tens of milliseconds!

I'm glad ya'll have gotten your tests down from 12 minutes to 1 minutes. There's a ton more improving you can do though.

In the grand scheme of things it doesn't matter that much.

Once you starts parallelizing, CI time is: setup_time + (test_run_time / workers), so assuming money isn't a problem you can add more workers as you keep adding more tests.

What really matter is how fast you can setup your test workers and how slow individual tests are.

> Most projects' tests, even when they're poorly written, still run in 3 or 4 minutes. My own tests run on the order of single-digit seconds.

That is an absurd statement to make without context.

I work on a large project with a few thousand very well-written tests, which run in 24 minutes serially on a recent-era laptop, or under a minute in well-parallelized CI environment.