WASM compression benchmarks and the cost of missing compression APIs
1–10 of 24 posts
Re: WASM compression benchmarks and the cost of missing compression APIs
#2just for scale it would have been fun to see zstd/lz4 native (=non-wasm) performance
Re: WASM compression benchmarks and the cost of missing compression APIs
#3Note that both the x and the y axis for Chrome vs Firefox are significantly different.
Re: WASM compression benchmarks and the cost of missing compression APIs
#4Related to compressing data before storing on SSD:
Blosc - faster than memcpy()
https://github.com/Blosc/c-blosc
On right circumstances Blosc is so fast that even speed ups reading data from RAM (read less, decompress in L1 and L2 caches)
Re: WASM compression benchmarks and the cost of missing compression APIs
#5The web platform could really use zstd everywhere. As a content encoding for HTTP, and as an API available to JS/wasm. It's really clumsy to use a wasm version of zstd in a JS application because it's hard to get data in and out of wasm efficiently.
Re: WASM compression benchmarks and the cost of missing compression APIs
#6The web platform could really use zstd everywhere. As a content encoding for HTTP, and as an API available to JS/wasm. It's really clumsy to use a wasm version of zstd in a JS application because it's hard to get data in and out of wasm efficiently.
I'd be happy if just CompressionStream could be available as an API in all browsers already, so I could at least use gzip... Once available, I'm guessing zstd could be easier to add in later, or lz4.
Re: WASM compression benchmarks and the cost of missing compression APIs
#7I wonder how this compares to the OS built-in NTFS compression?
Windows shell has "Compress contents to save disk space" checkbox in folder properties. Usually, that compressed flag is inherited by new files created in a folder with that checkbox. OP can probably set the flag on Default\IndexedDB or Default\Service Worker folder and see whether this changes the results of that IO benchmark.
Re: WASM compression benchmarks and the cost of missing compression APIs
#8I wonder how this compares to the OS built-in NTFS compression? Windows shell has "Compress contents to save disk space" checkbox in folder properties. Usually, that compressed flag is inherited by new files created in a folder with that checkbox. OP can probably set the flag on Default\IndexedDB or Default\Service Worker folder and see whether this changes the results of that IO benchmark.
The built-in NTFS filesystem compression is fairly limited in that it optimizes for performance over compression ratio; the more optimal compression scheme that's built-in is WOF:
https://devblogs.microsoft.com/oldnewthing/20190618-00/ https://learn.microsoft.com/en-us/windows/win32/api/wofapi/n... https://learn.microsoft.com/en-us/windows-hardware/manufactu...
It's not transparent like the filesystem compression, but it offers far more potentially beneficial compression algorithms such as LZX.
Re: WASM compression benchmarks and the cost of missing compression APIs
#9I wonder how this compares to the OS built-in NTFS compression? Windows shell has "Compress contents to save disk space" checkbox in folder properties. Usually, that compressed flag is inherited by new files created in a folder with that checkbox. OP can probably set the flag on Default\IndexedDB or Default\Service Worker folder and see whether this changes the results of that IO benchmark.
I have run SSD benchmarks with and without it and it usually makes reading faster and writing slower.
Re: WASM compression benchmarks and the cost of missing compression APIs
#10Related to compressing data before storing on SSD: Blosc - faster than memcpy() https://github.com/Blosc/c-blosc On right circumstances Blosc is so fast that even speed ups reading data from RAM (read less, decompress in L1 and L2 caches)
[deleted]