Hey, I wrote about this once! I use it a ton. Thanks for your work. I can’t wait to dig into 1.0.
Show HN: Mitata – Benchmarking tooling for JavaScript
21–27 of 27 posts
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#22Definitely going to try this out!!
I’ve been using the `vitest bench` command; being able to slap a `.bench.ts` file next to a module and go to town is convenient: https://vitest.dev/guide/features.html#benchmarking
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#23Wow, I was just looking how to benchmark a streaming JSON parser that I'm working on! I'm creating it specifically for performance-intensive situations with JSON strings sizes up to gigabytes, and I thought that I had to implement about half of the features you mention there, like parametrisation and automatic GC after every test.
When you say streaming JSON parser, do you mean that it outputs a live “observable” object as it is steaming, or that it just doesn’t keep the entire source data in memory? I’ve done some work for the former for displaying rich LLM outputs as they are delivered - it’s a surprisingly underexplored area from what I’ve seen.
It means that prior to parsing JSON, parser is given exact path (or paths, or wildcards) it must retrieve, and then it will scan the string in one forward path with minimum possible allocations. It's for cases where you, for some reason, have to process enormous amount of serialised objects as strings, and need to get just a few small things out of them occasionally, and do it in JS.
As it processes input in batches, you can also use it in cases where you don't even need to load the whole input data in memory, if you chose so.
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#24The lack of miata is always the answer comments in this thread and the readme are troubling.
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#25Definitely going to try this out!! I’ve been using the `vitest bench` command; being able to slap a `.bench.ts` file next to a module and go to town is convenient: https://vitest.dev/guide/features.html#benchmarking
vitest is nice but it’s completely unsuited for micro-benchmarks as it ends up oom crashing after just 2 optimized out benchmarks
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#26Definitely going to try this out!! I’ve been using the `vitest bench` command; being able to slap a `.bench.ts` file next to a module and go to town is convenient: https://vitest.dev/guide/features.html#benchmarking
vitest is nice but it’s completely unsuited for micro-benchmarks as it ends up oom crashing after just 2 optimized out benchmarks
Yeah, I’ve hit those OOM issues with vitest before too. Mitata’s time budget + sample approach sounds like a solid way to keep things simple while avoiding those long GC pauses. Excited to give it a try on my own benchmarks!
Re: Show HN: Mitata – Benchmarking tooling for JavaScript
#27Definitely going to try this out!! I’ve been using the `vitest bench` command; being able to slap a `.bench.ts` file next to a module and go to town is convenient: https://vitest.dev/guide/features.html#benchmarking
That's very interesting. I saw some screenshots of benchmarks used by Bun and Deno on Twitter at that time and it inspired me to suggest to Vitest to add a benchmark command. Later I learned that they were all using Mitata internally. https://github.com/vitest-dev/vitest/issues/917#issuecomment...
Very good work and design, glad to see a stable 1.0 is released!