Live data from Hacker News

K6: Like unit testing, for performance

github.com

51–60 of 86 posts

Re: K6: Like unit testing, for performance

#51
post #50

i wish it had support for MQTT.

The soon to be released "xk6 - k6 extensions"[1] will bring support for many new protocols. There are already proof of concept extensions for Redis, ZeroMQ, kafka, SQL, and others.

Adding MQTT should be rather straightforward.

[1] https://github.com/k6io/xk6

Re: K6: Like unit testing, for performance

#52
I would gladly recommend this to anyone needing to load test their endpoints. We were using Gatling at work to do load testing and it was very slow to develop in it as we were lacking experience in scala. We found this and never looked back. The impact it had on benchmarking, profiling and tuning the servers was massive. I'm pretty sure the company saves a few thousand dollars every month till today because of those optimizations.

Re: K6: Like unit testing, for performance

#53
post #47
post #37

Earlier quoted context omitted.

What do you mean "very long article"? You could have used "extensive" or something ;) Anyway, what I've seen when comparing the performance of tools, is that Artillery, which is running on NodeJS, is perhaps the wors performer of all the tools I've tested. I don't know if it's because of NodeJS or that Artillery in itself isn't a very performant piece of software (It also consumes a lot of memory, btw). If you want t…

I was skimming your "extensive" article again, and I saw this little gem I'd forgotten about: https://github.com/ragnarlonn/curl-basher ^^

Oh yeah, that's my own load testing tool, written in the high-performance language Bash. It is a bit feature sparse but produces about the same RPS numbers as Drill (https://github.com/fcsonline/drill) and not too far behind Artillery. And it is about 0.0025 times as fast as Wrk!

Re: K6: Like unit testing, for performance

#54
post #13

I also discovered K6 a few days ago, it looked quite nice and could be a good replacement for our Gatling tests. We were initially looking for something slightly different though: we were interested to have perhaps less tests, but tests that would run much much more often (like every seconds or couple of seconds), in a continuous manner. Tue goal was to have something at the same time like a healtcheck (is it still w…

[deleted]

Re: K6: Like unit testing, for performance

#55
I've been using this for nearly a year and it's by far the best thing I've found so far for testing realistic websockets load where servers are put under strain not just by connections but interactions between each other. Many thanks for the dev team behind this. It's a great piece of work.

Re: K6: Like unit testing, for performance

#56

Reminds me of http://artillery.io - a modern, powerful & easy-to-use solution for load testing and synthetic monitoring

Used artillery at work but it's rather pricy, I ended up using locust to load test.

Artillery team member here. "Rather pricey" really depends on what you compare it to. Artillery Core is free. Artillery Pro costs money, but it runs directly in your cloud environment, so it's extremely cost effective compared to hosted SaaS solutions. We designed it for large-volume use, especially in CI/CD pipelines. There are no limits on test minutes, vuser concurrency, number of tests you can run etc.

Can you DIY a solution that will let you instantly scale up from running tests locally to running them on 500 workers in any of 13 different geographical regions? With no servers to manage or maintain whatsoever. Sure you can, but it's not the best use of time for a lot of teams.

Re: K6: Like unit testing, for performance

#57
post #16

In their documentation ( https://k6.io/docs/ ) they claim that > JavaScript is not generally well suited for high performance. To achieve maximum performance, the tool itself is written in Go, embedding a JavaScript runtime allowing for easy test scripting. How is it possible that pure go JavaScript interpreter (goja) with bindings for net/http and some reports would be faster than the same tool written in nodejs usi…

Many benchmarks say JS-VMs are blazingly fast, at the same time others say they are battery-hogs. And Apple is even optimizing their CPU for it to run faster.

IMHO that is about JS-memory usage and GC, which never really get's proper attention.

You can have really fast algorithms, if you keep creating and forgetting millions of objects, which most JS-frameworks do, you will have lags and GC-pauses.

Re: K6: Like unit testing, for performance

#58

K6 is licensed AGPL and so anyone who uses it needs to AGPL their code?

Member of team artillery.io here, my 2c on the subject. We chose MPLv2 specifically to address licensing concerns. With MPLv2, you can build on top of Artillery, you can build plugins and extensions for it, and integrate it into your systems without worrying about licensing. It's a well-understood license with very clear boundaries between your code and MPLv2-licensed dependencies (e.g. all Hashicorp tools use MPLv2).

That's unfortunately not the case for AGPL. There's a reason a lot of companies have policies banning any AGPL dependencies outright, Google probably being the most prominent example: https://opensource.google/docs/using/agpl-policy

AGPL is designed to be extremely viral and has not been tested in court. The definitions of boundaries between your code and AGPL-licensed code are not well understood. Consider that MongoDB, probably the most popular AGPL-licensed project in use before 2018 when they switched to SSPL, had to explicitly publish their drivers under Apache because communicating with an AGPL dependency over a network was not sufficiently distant enough to prevent infection. https://www.mongodb.com/blog/post/the-agpl

As engineers we need to be aware of licensing implications of code we depend on. I am obviously not a lawyer, and obviously your employer's legal team are the people you should be talking to if you have an absolutely critical dependency on an AGPL-licensed project.

Re: K6: Like unit testing, for performance

#59

This looks really nice; I think I'm likely to try it out. What I'm really excited to have discovered is the `HAR` format. https://en.wikipedia.org/wiki/HAR_(file_format) Over the past 10 years I have glued together project-specific variants of "parse some proxy or WAF log and spit out test or automation data" at least 4 or 5 times. Including as recently as a couple of weeks ago: https://git.sr.ht/~tuxpup/parse-charle…

HAR is great for a number of things!

You can use it to archive websites and recreate them offline, Internet Archive supports HAR files for archiving.

You can use them to recreate user scenarios for acceptance testing and performance tests, via tools like K6 (K6 which also supports InfluxDB, so you can gather the metrics yourself and have your own UI [via Grafana or anything else that supports InfluxDB])

You can use them for troubleshooting all sorts of user/client issues if you have a easy way of gathering the HAR files, or if your users are developers.

They're simply a invaluable tool in web development these days, if you deal with web frontends/backends.

Re: K6: Like unit testing, for performance

#60
post #8

As a C++ developer who writes a lot of unit tests and cares a lot about performance, the tag line sounds appealing, but I can't tell whether this is something meant for me. Is this a general tool, or is it specific to web development? e.g. Would this be an appropriate tool for Blender to detect performance regressions in their rendering?

We (Insightful) are developing a tool for you! :) It's called Insightful PerfLab - it will work for native code (CPU and/or GPU) and is designed to help things like Blender detect performance regressions in rendering (we built it to help detect performance regressions in our game engine, and then realized it would likely be generally useful to other developers).

Sounds interesting. Do you have newsletter I can sign up for to get notified when it's released?
Post reply on HN