K6: Like unit testing, for performance • Hacker News Board

Re: K6: Like unit testing, for performance

#81

Post by JohnL4 » Sun, Nov 29, 2020, 2:30 PM UTC

na-- wrote: post #29

skybrian wrote: post #6

It seems like the hard part is deciding when a change is suspicious enough to be investigated? Simple pass/fail tends not to work at scale because complicated systems tend to be noisy.

That's one of the main uses, yes. Although k6 gives you a lot of flexibility to decide what your pass/fail criteria are with its custom metrics ( https://k6.io/docs/using-k6/metrics#custom-metrics ), checks ( https://k6.io/docs/using-k6/checks ) and thresholds ( https://k6.io/docs/using-k6/thresholds ). See the example in https://github.com/loadimpact/k6/#checks-and-thresholds . If, at the end of the test run, some o…

Yup. I put a couple of metrics for failures of a couple of types in the load script and ship them out to InfluxDb/Grafana for plotting in the very first graph. That way, if I see a lot of "failure" color, flags go up. Otherwise the usual amount of background noise is ok.

Re: K6: Like unit testing, for performance

#82

Post by randomswede » Sun, Nov 29, 2020, 10:45 PM UTC

rlonn wrote: post #53

na-- wrote: post #47

Earlier quoted context omitted.

I was skimming your "extensive" article again, and I saw this little gem I'd forgotten about: https://github.com/ragnarlonn/curl-basher ^^

Oh yeah, that's my own load testing tool, written in the high-performance language Bash. It is a bit feature sparse but produces about the same RPS numbers as Drill ( https://github.com/fcsonline/drill ) and not too far behind Artillery. And it is about 0.0025 times as fast as Wrk!

Hm, rlonn as in "Algonet"?

Re: K6: Like unit testing, for performance

#83

Post by Seb-C » Mon, Nov 30, 2020, 1:57 AM UTC

That seems interesting, but the animated gif video that cannot get paused does not allow me to see what this is about and what it does exactly.

Re: K6: Like unit testing, for performance

#84

Post by na-- » Mon, Nov 30, 2020, 9:03 AM UTC

nullsense wrote: post #80

na-- wrote: post #36

I am one of the k6 developers. I've commented in a bunch of threads here, but if you have any questions, feel free to AMA :) I'll be around for the next ~1h and then probably later in the day as well.

Is there some way you can do whatever it is AWS CDK does under the hood to provide language bindings to other languages for writing tests? That'd be killer. I want to be able to write the tests in my backend language.

Unfortunately not... At some point in the far future, we might release support for Go "scripting" [1] or we might extend xk6 [2] so it allows different script types, but those things are far from certain.

[1] https://github.com/loadimpact/k6/issues/751

[2] https://github.com/k6io/xk6

Re: K6: Like unit testing, for performance

#85

Post by rlonn » Mon, Nov 30, 2020, 9:47 AM UTC

randomswede wrote: post #82

rlonn wrote: post #53

Earlier quoted context omitted.

Oh yeah, that's my own load testing tool, written in the high-performance language Bash. It is a bit feature sparse but produces about the same RPS numbers as Drill ( https://github.com/fcsonline/drill ) and not too far behind Artillery. And it is about 0.0025 times as fast as Wrk!

Hm, rlonn as in "Algonet"?

Possibly :)

Re: K6: Like unit testing, for performance

#86

Post by Cort3z » Tue, Dec 01, 2020, 11:16 AM UTC

rlonn wrote: post #38

Cort3z wrote: post #34

Earlier quoted context omitted.

Gatling is super slow. Suspicious slow in fact. You will likely not be able to really hit your endpoints hard enough by using one machine only. Personally I use autocannon which in my testing is faster than k6 and easier to use with a node.js environment. It is only a npm package, not a full software you need to install on your machine. Autocannon in my testing is 2 orders of magnitude faster than gatling, but you sh…

2 orders of magnitude faster than Gatling? Would be interesting to see how you ran your tests, and the results. In my testing, Gatling isn't catastrophically slow by any means. Wrk, which is faster than any other tool I've seen, is about one order of magnitude faster than Gatling, in terms of raw RPS generation. I find it very hard to believe a NodeJS-based tool executing JS would be faster than the fastest tool writ…

I am doing login and custom data per request