Just a side-comment, it would be really nice if we could get the browser vendors to support a newer compression algorithm beyond gzip and deflate. I know there have been a couple others implemented, but nothing that has been implemented by multiple browsers that has stuck. Really need to get MS, Google, Apple and Mozilla to come together on this. Should be patent free.
Apple Open-Sources its Compression Algorithm LZFSE
171–180 of 219 posts
Re: Apple Open-Sources its Compression Algorithm LZFSE
#172Earlier quoted context omitted.
That is my experience. Maths people are not renowned for their ability to write readable or maintainable code. I recently needed an implementation of the Simplex Noise algorithm (that I could port to Common Lisp). I ended up using this one, which works but the code certainly does nothing to help understanding: https://github.com/josephg/noisejs/blob/master/perlin.js Note that the Javscript implementation is also a po…
Hey, someone found those noise functions I ported to JS a few years ago! Sorry for not referencing the original code. The java version I translated from is here: http://webstaff.itn.liu.se/~stegu/simplexnoise/SimplexNoise.... And the paper describing the algorithm is here: http://webstaff.itn.liu.se/~stegu/simplexnoise/simplexnoise.... Glad it was useful to you!
It's funny, but nothing helps you understand something better than porting it to a new language... Even if you aren't particularly skilled in the shortcuts a given language/platform offers, it does help you understand a given algorithm.
Re: Apple Open-Sources its Compression Algorithm LZFSE
#173Earlier quoted context omitted.
the compilers are free to optimize. Creating a stack frame for each function is trivially avoided by inlining (esp. trivial for non-polymorph call sites)
It depends. In GCC/MSVC will only (attempt to) inline what you mark as inline. Then MSVC has a keyword which forces inlining. Unless you set a flag which tells the compiler to inline what ever it wants . But that being said Microsoft has a non-POSIX x64 ABI designed to allow better in-lining. How inlining works starts to dive pretty deep into the compiler rabbit hole.
This is not true. Gcc will inline everything it damn well wants to. It will definitely inline any function that is smaller than a function call, like small C++ accessors, and any other small function available at compile time like trivial constructors and the like. The "inline" keyword mainly serves to avoid ODR violations, not to inform the optimizer.
Re: Apple Open-Sources its Compression Algorithm LZFSE
#174Earlier quoted context omitted.
This disregards a common use of the Makefile: compiling software from source, bypassing the use of a package manager in the first place. Yes, software devs should be providing a Makefile that is friendly for package managers to wrap. But the best software provides a Makefile usable on all platforms it supports, without the expectation that a package manager will be involved. Sure, you can install each compiled packag…
> This disregards a common use of the Makefile: compiling software from source, bypassing the use of a package manager in the first place. Most of the executions (those actually used in the wild) of this strategy are quite stupid. Add to that the fact that building packages with distribution's tools is quite easy, and now on top of that add fpm, which produces terrible packages and should be banned for upstream maint…
> exaggregating
this could make for a cool portmanteau.
Re: Apple Open-Sources its Compression Algorithm LZFSE
#175Open source; but is it patent-free?
Re: Apple Open-Sources its Compression Algorithm LZFSE
#176Earlier quoted context omitted.
I've never seen this code base before, but this makes straightforward sense to me as I do keep my hand in with compression software, and I don't anticipate others who works with compression algorithms in general would have any trouble. It looks just like the style of code in all the other fast LZ codebases. They are all in this style. The "non-aligned access OK" comment litter is presumably to silence an LLVM perform…
I agree. The only questionable part is the stuff like *(uint16_t *)q = D; q += 2; // non-aligned access OK *(uint32_t *)q = literal; instead of memcpy(q, &D, sizeof(uint16_t)); q += 2; // non-aligned access OK memcpy(q, &literal, sizeof(uint32_t)); which is better defined behavior (i.e. doesn't violate -fstrict-aliasing) and possibly faster.
Why not test it and do a pull request?
Re: Apple Open-Sources its Compression Algorithm LZFSE
#177I feel like LZFSE is too little, too late. It would be great to have a proper comparison, but Zstd is stable, and offers a superior compression ratio with compression and decompression speeds that seem on par with LZFSE. And Zstd is not proprietary. (This issue is relevant in this regard: https://github.com/lzfse/lzfse/issues/21 ) https://github.com/Cyan4973/zstd Edit: here is a quick comparison I did on Linux with P…
Can somebody test the binary that ships with Mac OS X and iOS? I'm asking because the GitHub page says
"This is a reference C implementation of the LZFSE compressor introduced in the Compression library with OS X 10.11 and iOS 9."
My guess is that the versions on Mac OS X and iOS are different, and aren't even written in C (they might have started as C programs, but would have been hand-optimized later)
Re: Apple Open-Sources its Compression Algorithm LZFSE
#178Earlier quoted context omitted.
> This disregards a common use of the Makefile: compiling software from source, bypassing the use of a package manager in the first place. Most of the executions (those actually used in the wild) of this strategy are quite stupid. Add to that the fact that building packages with distribution's tools is quite easy, and now on top of that add fpm, which produces terrible packages and should be banned for upstream maint…
> exaggregating this could make for a cool portmanteau.
Oh, my. My spelling still leaves room for improvement. Sorry, I thought I know
how to write this word (now I know I don't know).
Re: Apple Open-Sources its Compression Algorithm LZFSE
#179Just a side-comment, it would be really nice if we could get the browser vendors to support a newer compression algorithm beyond gzip and deflate. I know there have been a couple others implemented, but nothing that has been implemented by multiple browsers that has stuck. Really need to get MS, Google, Apple and Mozilla to come together on this. Should be patent free.
I think Brotli is getting there, with support in Firefox and Chrome so far.
https://samsaffron.com/archive/2016/06/15/the-current-state-...
Re: Apple Open-Sources its Compression Algorithm LZFSE
#180I'm not fully up on the latest and greatest in compression technologies but my go to format these days is usually 7zip which I believe is just a container that uses LZMA. For whatever reason *nix people seem to hate it even though I get much better compression with it than tarballs, zlib/zips. Is there a similar container format that will or does use LZFSE? And how much better is it than 7zip/LZMA?
The Unixy version of LZMA is called XZ. Unix people prefer tar over zip, probably for historical/tribal reasons.
There are really three different compression "markets"; LZMA/XZ provides strong compression while LZFSE provides moderate or light compression so they don't really compete.