Viewing profile — powturbo

Try TurboRC using 16-bits format: https://github.com/powturbo/Turbo-Range-Coder

Thank for your analyse. It seems it's a corner case, benchmarking a base64 encoded random file is not a typical case. igzip has no more advantage above libdeflate. It has only fast…

EDIT: I've seen the file you're referencing is 4Gi-Base64. This file is not very compressible (75% with gzip). It's possible that igzip is simply storing the file or some parts of …

Sorry, a digit was missing, but now https://github.com/powturbo/TurboBench/issues/43 Well, a correct benchmarking is not done with special data, but with datasets that represent a …

libdeflate compress better and has faster decompression than igzip. See the silesia single core in-memory benchmark here [1] comparing zlib,libdeflate,igzip,... https://github.com/…

file html8 : 100MB random html pages from a 1m Alexa Top sites corpus. Number of pages = 1178 Average length = 84886 bytes CPU: AMD 7840HS 3.8-5.1GHZ

You can use TurboBench [1] to benchmark libdeflate against zlib, igzip, zlib-ng and others. Download TurboBench from Releases [2] Here Some Benchmarks: - https://github.com/zlib-ng…

** Cython bindings for Turbo Base64 [1] ** - 20-30x faster than the standard library - Benchmarks faster than any other C base64 library - Fastest implementation of AVX, AVX2, and …

You can use TurboBench [1] to benchmark igzip, zlib-ng and others. Download TurbBench from Releases [2] Here Some Benchmarks: - https://github.com/zlib-ng/zlib-ng/issues/1486 - htt…

Yes, especially SIMD Neon where gcc producing horrible Neon code for all versions You can try it : https://github.com/powturbo/Turbo-Base64

> Obviously it's not impossible, since it's happening on my laptop. Well, zstd cli is more optimized for multithreading and asynchrounous i/o. zstd is overlapping cpu decompression…

"In memory" means without i/o (disk,ssd,network), only in RAM. Streaming means compressing/decompressing in chunks and can also be "IN memory".

No, No and No. It's algorithmically impossible that zstd is decompressing faster than lz4 by using the same environment. You are comparing the zstd with asynchrounous I/O + multith…

lz4 is ALWAYS decompressing faster (> 2x) thand zstd

LZAV now integrated into TurboBench [1] Download TurboBench for windows from releases [2] Lenovo IdeaPad 5 Pro - Ryzen 6600HS - DDR5 6400 - gcc-13.1 File: silesia.tar 220MB (Mixed …

see also Discussion with a brotli developer [2] and "Chrome: Implement zstd content-encoding" [3] : [1] - Benchmark App: https://github.com/powturbo/TurboBench [2] - https://twitte…

Thanks for the confirmaation. If your query engine can't exploit fine grained access, then general purpose compressors like lz4 or zstd with large compressed blocks are the better …

Gorilla [1] and Gorilla based algos [2] are simply overrated. Store these values as 32 bits floats instead of 64 bits and you get instant 50% reduction without any compression. Thi…

There is nothing to earn from non GPL license, maybe some small reputation and more stars on github, but that's all. Contribution is minimal if any, and when then you have more wor…

- Turbo-Base64 is used in clikhouse - Turbo-Base64 is available in archlinux: https://aur.archlinux.org/packages/turbo-base64