Apple Open-Sources its Compression Algorithm LZFSE

Post by eridius » Fri, Jul 08, 2016, 5:08 PM UTC

espadrine wrote: post #206

eridius wrote: post #192

Earlier quoted context omitted.

Surely LZ4 is not in fact more energy efficient, otherwise Apple would be pushing LZ4 as the algorithm to use on mobile devices instead of recommending LZFSE. If you see in that previous discussion, I was asking the same questions there and got no real answer. It looks to me like nobody (outside of Apple) has actually tested the energy efficiency.

LZ4 is a format. All you can say about how energy efficient a format is, is by looking at the average speeds of its implementations. However, zlib for instance (a DEFLATE library) is more energy efficient than zopfli (another DEFLATE library), just by virtue of taking less CPU cycles to crunch data. There is in fact a very high correlation between CPU cycles and energy efficiency, since compression algorithms don't s…

Post by dang » Sat, Jul 09, 2016, 12:54 AM UTC

cooper12 wrote: post #99

Oh wow dude you watch Silicon Valley also? Great reference! It sure added a lot to the conversation. Please continue making useful comments like this in the future, you really got us thinking with your pop culture reference. (And I sure patted myself on the back for recognizing such witticism)

Post by tmd83 » Sat, Jul 09, 2016, 4:58 AM UTC

twitch_checksum wrote: post #143

dchest wrote: post #41

Earlier quoted context omitted.

This is a baseless speculation. First of all, Apple provides LZ4 in libcompression. Secondly, LZFSE uses Lempel–Ziv algorithm and ANS coder invented by Jarek Duda ( https://arxiv.org/abs/1311.2540 ): https://developer.apple.com/library/ios/documentation/Perfor...

Well, many people nowadays pretend that, as if they read Jarek's paper, understood it and came out with a genuine implementation of their own. But Jarek's ANS paper was first out in 2007, and almost no one paid attention to it, because it was plain inscrutable. Many years later, it took an individual to create FSE ( https://github.com/Cyan4973/FiniteStateEntropy ), to prove that it could be transformed into something…

Post by rleigh » Sun, Jul 10, 2016, 4:07 PM UTC

mschuster91 wrote: post #121

vog wrote: post #111

Earlier quoted context omitted.

> I really appreciate the presence of a no-nonsense Makefile. Indeed, the current version of their Makefile is a great example of how to write a simple yet portable Makefile: https://github.com/lzfse/lzfse/blob/33629bc65f4b356072c9a7d5... > No autoconf, no pkgconfig, just plain and simple make. While I agree with your sentiment, I believe that your statement about pkg-config goes a bit over the top. Yes, the LZFSE pr…

> Indeed, the current version of their Makefile is a great example of how to write a simple yet portable Makefile: Yet it forgets the MOST important thing: make uninstall. Nothing worse than software where I have to reverse engineer a makefile in order to uninstall!

Post by aidenn0 » Sat, Jul 16, 2016, 7:55 PM UTC

eridius wrote: post #192

wmf wrote: post #168

Earlier quoted context omitted.

Based on Apple's own presentation it looks like energy efficiency is pretty proportional to performance, so it's very likely that a faster algorithm (e.g. LZ4) would also be more efficient. In modern "braniac" processors, 90% of the energy is spent on overhead and doesn't vary based on instruction mix. Memory accesses consume both energy and time, so fewer are better. (From the previous discussion: https://news.ycomb…

Surely LZ4 is not in fact more energy efficient, otherwise Apple would be pushing LZ4 as the algorithm to use on mobile devices instead of recommending LZFSE. If you see in that previous discussion, I was asking the same questions there and got no real answer. It looks to me like nobody (outside of Apple) has actually tested the energy efficiency.