Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hyperfine: A command-line benchmarking tool

github.com

Post reply on HN
Print view View original

21–30 of 56 posts

Re: Hyperfine: A command-line benchmarking tool

#21

Post by PhilipRoman »

oguz-ismail wrote: post #20
sharkdp wrote: post #17

Earlier quoted context omitted.

Yes. If you don't make use of shell builtins/syntax, you can use hyperfine's `--shell=none`/`-N` option to disable the intermediate shell.

You still need to quote the command though. `hyperfine -N ls "$dir"' won't work, you need `hyperfine -N "ls ${dir@Q}"' or something. It'd be better if you could specify commands like with `find -exec'.

Oh that sucks, I really hate when programs impose useless shell parsing instead of letting the user give an argument vector natively.

Re: Hyperfine: A command-line benchmarking tool

#22

Post by CathalMullan »

usrme wrote: post #13

I've also had a good experience using the 'perf'[^1] tools for when I don't want to install 'hyperfine'. Shameless plug for a small blog post about it as I don't think it is that well known: https://usrme.xyz/tils/perf-is-more-robust-for-repeated-timi... . --- [^1]: https://www.mankier.com/1/perf

There's also 'poop', which is a nice middle-ground between 'hyperfine' and 'perf'. https://github.com/andrewrk/poop

Re: Hyperfine: A command-line benchmarking tool

#24

Post by maccard »

mmastrac wrote: post #2

Hyperfine is a great tool but when I was using it at Deno to benchmark startup time there was a lot of weirdness around the operating system apparently caching inodes of executables. If you are looking at shaving sub 20ms numbers, be aware you may need to pull tricks on macos especially to get real numbers.

Not being able to rely on numbers to 20ms is pretty poor. That’s longer than a frame in a video game.

Windows has microsecond precision counters (see QueryPerformanceCounter and friends)

Re: Hyperfine: A command-line benchmarking tool

#26

Post by sharkdp »

PhilipRoman wrote: post #21
oguz-ismail wrote: post #20

Earlier quoted context omitted.

You still need to quote the command though. `hyperfine -N ls "$dir"' won't work, you need `hyperfine -N "ls ${dir@Q}"' or something. It'd be better if you could specify commands like with `find -exec'.

Oh that sucks, I really hate when programs impose useless shell parsing instead of letting the user give an argument vector natively.

I don't think it's useless. You can use hyperfine to run multiple benchmarks at the same time, to get a comparison between multiple tools. So if you want it to work without quotes, you need to (1) come up with a way to separate commands and (2) come up with a way to distinguish hyperfine arguments from command arguments. It's doable, but it's also not a great UX if you have to write something like

    hyperfine -N -- ls "$dir" \; my_ls "$dir"

Re: Hyperfine: A command-line benchmarking tool

#27

Post by llimllib »

CathalMullan wrote: post #22
usrme wrote: post #13

I've also had a good experience using the 'perf'[^1] tools for when I don't want to install 'hyperfine'. Shameless plug for a small blog post about it as I don't think it is that well known: https://usrme.xyz/tils/perf-is-more-robust-for-repeated-timi... . --- [^1]: https://www.mankier.com/1/perf

There's also 'poop', which is a nice middle-ground between 'hyperfine' and 'perf'. https://github.com/andrewrk/poop

worth mentioning that it's linux-only

Re: Hyperfine: A command-line benchmarking tool

#28

Post by llimllib »

mosselman wrote: post #15

Hyperfine is great. I use it sometimes for some quick web page benchmarks: https://abuisman.com/posts/developer-tools/quick-page-benchm... As mentioned here in the thread, when you want to go into the single ms optimisations it is not the best approach since there is a lot of overhead especially the way I demonstrate here, but it works very well for some sanity checks.

I find k6 a lot nicer for HTTP benching, and no slower to set up than hyperfine (which I love for CLI benching): https://k6.io/

Re: Hyperfine: A command-line benchmarking tool

#29

Post by edwardzcn »

Hyperfine is great! I remember I learned about it when comparing functions with/without tail recursion (not sure if it was from the Go reference or the Rust reference). It provides simple configurations for unit test. But I have not tried it on DBMS (e.g. like sysbench). Does anyone have a try?

Re: Hyperfine: A command-line benchmarking tool

#30

Post by oguz-ismail »

sharkdp wrote: post #26
PhilipRoman wrote: post #21

Earlier quoted context omitted.

Oh that sucks, I really hate when programs impose useless shell parsing instead of letting the user give an argument vector natively.

I don't think it's useless. You can use hyperfine to run multiple benchmarks at the same time, to get a comparison between multiple tools. So if you want it to work without quotes, you need to (1) come up with a way to separate commands and (2) come up with a way to distinguish hyperfine arguments from command arguments. It's doable, but it's also not a great UX if you have to write something like hyperfine -N -- ls…

> not a great UX

Looks fine to me. Obviously it's too late to undo that mistake, but a new flag to enable new behavior wouldn't hurt anyone.

Post reply on HN