Live data from Hacker News

Ruby on Rails load testing habits

rorvswild.com

11–20 of 45 posts

Re: Ruby on Rails load testing habits

#11

> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address. Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.

If your application can’t saturate the CPU, Rails probably isn’t your bottleneck.

Re: Ruby on Rails load testing habits

#12

> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address. Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.

The author means efficiency in terms of the software that is running, not efficiency in terms of all possible optimizations such as switching to a different lang.

Re: Ruby on Rails load testing habits

#13
post #11

> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address. Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.

If your application can’t saturate the CPU, Rails probably isn’t your bottleneck.

[flagged]

Re: Ruby on Rails load testing habits

#15

> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address. Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.

While you missed the point of the saturation comment, this is why we love AWS Lambda over ECS+Fargate.

Rails has really poor startup time due to loading all codepaths. We switched to Django and it runs beautifuly on AWS Lambda where our CI is more expensive than actual server costs. We're a b2b application so traffic is quite low so we REALLY don't saturate the CPU in a normal Fargate setup.

Re: Ruby on Rails load testing habits

#16
post #4

> My initial requirement was to send requests with unique parameters. To the best of my knowledge, no tool could do this. wrk does this with lua. https://github.com/wg/wrk/blob/master/src/wrk.lua Also even things like the venerable jmeter supported pulling parameters from a csv file.

I've never done load testing before, but would it be hard to write a script in pure ruby (maybe with a few libraries) that makes a lot of concurrent requests to whatever endpoints and using whatever params you like?

It is simple enough to do a script doing that in Ruby, but there also enough off the shelf tools specifically for load testing that encodes a lot of experience and avoids a lot of obvious and not so obvious mistakes, so it's usually not worth writing your own.

Re: Ruby on Rails load testing habits

#17
post #4

> My initial requirement was to send requests with unique parameters. To the best of my knowledge, no tool could do this. wrk does this with lua. https://github.com/wg/wrk/blob/master/src/wrk.lua Also even things like the venerable jmeter supported pulling parameters from a csv file.

I've never done load testing before, but would it be hard to write a script in pure ruby (maybe with a few libraries) that makes a lot of concurrent requests to whatever endpoints and using whatever params you like?

Don't write your own load testing tool other than as a fun little exercise. At least not without understanding coordinated omission and thinking about workload modeling (open? closed? hybrid? all of the above?) [1]. Get this wrong and the results produced by your tool will be worthless.

Once you've got that out of the way, don't forget that you'll want a distribution story. It does not matter how efficient your tool might be on a single machine - you'll want to distribute your tests across multiple clients for real-world testing.

"Sure it's easy" you might say, "I know UNIX. Give me pssh and a few VMs on EC2". Well, now you've got 2 problems: aggregating metrics from multiple hosts and merging them accurately (especially those pesky percentiles. Your tool IS reporting percentiles rather than averages already, right?!), and a developer experience problem - no one wants to wrangle infra just to run a load test, how are you going to make it easier?

And, this developer experience problem is much bigger than just sorting out infra... you'll probably want to send the metrics produced by your tool to external observability systems. So now you've got some plugins to write (along with a plugin API). The list goes on.

I'm very biased, but it's 2024. Don't waste your time and just use https://www.artillery.io/

1. https://www.artillery.io/blog/load-testing-workload-models

Re: Ruby on Rails load testing habits

#18

> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address. Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.

While you missed the point of the saturation comment, this is why we love AWS Lambda over ECS+Fargate. Rails has really poor startup time due to loading all codepaths. We switched to Django and it runs beautifuly on AWS Lambda where our CI is more expensive than actual server costs. We're a b2b application so traffic is quite low so we REALLY don't saturate the CPU in a normal Fargate setup.

I'm surprised to see a mention of Django when talking about fast startup times! This is one of my main issues with Django at the moment. How big is your project?

Our ~500k lines app takes multiple seconds to start, which is why I'm not really investigating a lambda-style setup... Do you have specific strategies to make startup fast?

Re: Ruby on Rails load testing habits

#19
If you'd like to write your load tests in Ruby (plain ruby or browser based), and use your own libraries (internal libraries, specs, ets) browserup does that:

Ruby with browser:

https://browserup.com/docs/en/load/ruby-load-test.html

Command-line installation:

https://www.npmjs.com/package/browserup

Post reply on HN