We are using Soasta at your company. We use it to do record, parametrize, playback to simulate multi user workflows. Cant see any such support here. One of the problems with Soasta is it requires lot of servers to trigger load. Looking for moving away from it. k6 seems interesting, but couldn't find any record/playback functionality. Any addon/extension which can be used to achieve similar purpose with K6?
K6: Like unit testing, for performance
41–50 of 86 posts
Re: K6: Like unit testing, for performance
#42I also discovered K6 a few days ago, it looked quite nice and could be a good replacement for our Gatling tests. We were initially looking for something slightly different though: we were interested to have perhaps less tests, but tests that would run much much more often (like every seconds or couple of seconds), in a continuous manner. Tue goal was to have something at the same time like a healtcheck (is it still w…
That's what k6 does. It's called soak testing: https://k6.io/docs/test-types/soak-testing
I use it with influxdb and grafana.
Set it up with one virtual user running for 6 hours, requesting different endpoints, you should get the 1req/sec.
Re: K6: Like unit testing, for performance
#43We are using Soasta at your company. We use it to do record, parametrize, playback to simulate multi user workflows. Cant see any such support here. One of the problems with Soasta is it requires lot of servers to trigger load. Looking for moving away from it. k6 seems interesting, but couldn't find any record/playback functionality. Any addon/extension which can be used to achieve similar purpose with K6?
The canonical way to record web traffic are HAR files: https://en.wikipedia.org/wiki/HAR_(file_format)
You can export HAR files from all web browsers and a lot of proxies and other tools. Then you can convert that recording to a k6 script with our HAR converter: https://k6.io/docs/test-authoring/recording-a-session/har-co...
We also have browser plugins to directly record browser traffic and import it as a k6 script in our cloud service: https://k6.io/docs/test-authoring/recording-a-session/browse...
However, you don't need a cloud subscription to run the tests, you can copy-paste the generated script code and `k6 run` it locally, without paying us anything.
Re: K6: Like unit testing, for performance
#44what is the max number of VUS you guys have run through it?
The most I started on a single machine was about 50.000 VUs. Back then, the limit was in available sockets. k6 now has support for multiple source IPs, so that limit has been effectively removed, and more VUs should be possible.
On the `k6 cloud` side, we have executed 500k+ VUs. The most RPS I achieved with k6 was 4 791 928 (~4.8 million requests per second). That test lasted for 6 min and generated 1.5 billion requests in total.
More is possible, but we didn't push further.
Source: I'm one of the guys behind k6.
Re: K6: Like unit testing, for performance
#45We are using Soasta at your company. We use it to do record, parametrize, playback to simulate multi user workflows. Cant see any such support here. One of the problems with Soasta is it requires lot of servers to trigger load. Looking for moving away from it. k6 seems interesting, but couldn't find any record/playback functionality. Any addon/extension which can be used to achieve similar purpose with K6?
The canonical way to record web traffic are HAR files: https://en.wikipedia.org/wiki/HAR_(file_format) You can export HAR files from all web browsers and a lot of proxies and other tools. Then you can convert that recording to a k6 script with our HAR converter: https://k6.io/docs/test-authoring/recording-a-session/har-co... We also have browser plugins to directly record browser traffic and import it as a k6 script…
Thanks. Will check it out.
Re: K6: Like unit testing, for performance
#46How does it compare to Tsung?
Tsung was made in another era, before devops. I think it shows in the UX and everywhere when using the tool that it wasn't really meant for developers.
Specifically, and with the disclaimer that I haven't used Tsung so much, I'd say the biggest difference is that k6 tests are scripted in a real language (JS) as compared to Tsung's XML.
But k6 also has a modern CLI UX that beats Tsung (and most other tools) by a mile, more integrations now and possibly support for more protocols, or at least more relevant ones (not sure if Tsung supports HTTP/2 for instance). Plus k6 is being very actively developed.
From what I've seen of Tsung I really like it though. Seems like a very performant and solid piece of software. But like mentioned earlier, it was made in another era. Just like Jmeter. If I ranked tools in terms of ascending "DevOps:ness" and UX for developers it'd probably be something like:
Lowest: Jmeter, Tsung
Low: Gatling
Medium: Apachebench, Siege, Hey, Wrk (simple testing)
High: Artillery, Locust
Highest: k6, Vegeta (though Vegeta is only for simple/static tests)
I wrote an article about this topic also: how most load testing tools we have are just not geared for developers and DevOps: https://k6.io/blog/another-load-testing-tool
Re: K6: Like unit testing, for performance
#47Earlier quoted context omitted.
I am not completely sure why the Go stdlib's HTTP client (which k6 uses) is faster than the NodeJS one. I think part of it is the fact that k6 spins up a separate JS runtime for each VU. goja is a much, _much_, slower JS interpreter than V8, but load tests are IO-bound, so that's usually not an issue. And you can spin up thousands of VUs in k6 (especially with --compatibility-mode=base), making full use of the load g…
What do you mean "very long article"? You could have used "extensive" or something ;) Anyway, what I've seen when comparing the performance of tools, is that Artillery, which is running on NodeJS, is perhaps the wors performer of all the tools I've tested. I don't know if it's because of NodeJS or that Artillery in itself isn't a very performant piece of software (It also consumes a lot of memory, btw). If you want t…
I was skimming your "extensive" article again, and I saw this little gem I'd forgotten about: https://github.com/ragnarlonn/curl-basher ^^
Re: K6: Like unit testing, for performance
#48Earlier quoted context omitted.
The CEOs response doesn't inspire any confidence: If you statically or dynamically link any part of the k6 codebase with some other code (a derivative work), the license’s copyleft virality is triggered and that other code also needs to be made available under AGPLv3. When it comes to using the k6 binary and interacting with it from another process or over a network it is not certain exactly how the copyleft virality…
Happy to answer any questions you have regarding the license and why we chose it. Is your concern the copyleftness of the license?
Thanks. I must say, I prefer weak copylefts for my projects (MPLv2 with the incompatibility clause), so my concern isn't copyleft per se, but the virility of it.
That said, I do have a couple questions:
- Would a non-AGPLv3 open-source or source-available project using K6 be incompatible with AGPLv3 given they might "distribute" CI/CD along with their open-source or source-available code?
- If a closed-source project publishes results from K6, are they now in violation of AGPLv3?
Re: K6: Like unit testing, for performance
#49I'm glad this was posted. I tried it this evening and it worked great. It took me like 20 seconds to get started, and I was hooked. (A few more minutes were required to setup a tsconfig.json so that my editor could type-check the k6 imports. I don't think the documentation really says "add a tsconfig with compilerOptions.types = [k6]" anywhere, but that worked for me. Writing the tests in Typescript instead of Javascript is an exercise for another day.)
I like the design of running each virtual user in its own Javascript VM inside of a Go process. I can just install a single binary, but still write tests in a high level language (and not rebuild the load testing framework for every change).
An idea that's been kicking around in my head for a while is extending Go applications by embedding a WebAssembly VM, exposing relevant hooks, and letting users add whatever they want at runtime. That way, you don't have to bundle a bunch of crap into a statically linked binary, users can go add that later :) K6 seems to validate that this is a good idea; while not WebAssembly, it sure works well. (I would want to write my plugins in Go, not Javascript... but the idea is good.)
Re: K6: Like unit testing, for performance
#50i wish it had support for MQTT.