Beware of Bun.js

Looks like the sqlite bench complaint was fixed 5 days ago and the react-hello benchmark was fixed earlier today (before the creation time of this gist).

I don't know if the suggested FFI benchmark is fair, as the change would no longer be simulating doing many ffi_hashes on many byteArrays, instead it simulates doing many ffi_hashes on the same byteArray over and over and benefits from caching the type casting necessary by deno.

Replace the const byte array with a randomly generated list (or real bytes you'd want to hash) of many passed to the benchmark as a parameter and the benchmark and the deno one loses the advantage from the cached type cast.

The benchmark, while it seems (without knowing the function specifics) that it is overtesting (testing hash(toptr(n)) instead of just hash(m)), it is the same on both tests so it does feel like a fair comparison? Sure it could be done differently to make a more atomic comparison, but the composite comparison still feels fair if both operations tend to be performed together. Def not an "expose".

Overall though, Bun is an amazing accomplishment for many reasons and I'd love if the project focused a bit less on the supposedly benchmark/performance and more in some other things it brings to the table, like it being based on `JavaScriptCore`, being an alternative to Node but still mostly compatible, the speed of development (vs Node which we've been complaining that has stagnated for years), etc. Not saying not to mention impressive benchmarks, just that it'd be nice to highlight the other points as well!

I personally also don't like the direction Deno was taking, and def welcome Bun as a modern alternative to Node.

This seems to not be entirely reasonable.

[edit: apparently the author is a deno employee but did not choose to disclose such? If so then that throws any moral position they have out the window. The below however is still accurate to my reading of the complaints.]

First up is the FFI test. On the one hand I think that this is a legitimate complaint, changing an existing benchmark for your comparisons is always suspect. That said I can see technically legitimate reasons for the change - for example if bun's ffi interface doesn't actually handle byte arrays as well as other runtimes. But in such a case that should be explicitly called out as a caveat.

The sqlite complaint is not reasonable, if sqlite is being used as a demonstration of wasm performance there is nothing at all wrong with this test - if anything using wasm makes it more valuable as a comparison of runtime perf, otherwise you're very quickly going to just be benchmarking one native build of sqlite to another native build of sqlite.

The React complaint - this just looks like bog standard formatted minified code, which is an absolutely reasonable thing to do - in fact I would argue it is a much better reflection of real world performance than using the non-minified code. Production websites only ever send minified content (at the react scale) and minification absolutely impacts page load performance, and historically could result in catastrophic runtime impact (though they're better at not doing this now).

Mind you, despite being a possible conflict of interest this also provides credentials for likely relevant expertise, with skill and interest in correcting errors.

(Also, disclosure, not disclaimer.)