Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Open-Sources its Compression Algorithm LZFSE

infoq.com

Post reply on HN
Print view View original

91–100 of 219 posts

Re: Apple Open-Sources its Compression Algorithm LZFSE

#91

Post by TULPU »

lokedhs wrote: post #33
barsonme wrote: post #28

Earlier quoted context omitted.

Blows my mind that people can come up with this stuff. I'm assuming they came up with the mathematical proofs first and translated that into code, so that has something to do with it, correct? It looks a lot like some crypto algorithms which are a nearly direct translation of the mathematical formulas. It's not that it's incredibly difficult to follow, but it's just very "math like".

That is my experience. Maths people are not renowned for their ability to write readable or maintainable code. I recently needed an implementation of the Simplex Noise algorithm (that I could port to Common Lisp). I ended up using this one, which works but the code certainly does nothing to help understanding: https://github.com/josephg/noisejs/blob/master/perlin.js Note that the Javscript implementation is also a po…

Looks like it is implemented from this http://stackoverflow.com/a/18516731 answer, which itself is inspired from another source

Re: Apple Open-Sources its Compression Algorithm LZFSE

#92

Post by rimantas »

throwaway2048 wrote: post #58
msvalkon wrote: post #46

Earlier quoted context omitted.

I don't really understand where the downvotes come from? I find the readability concerns legitimate, and would like to understand why compression algorithm developers feel like this is OK? Is it just the math heavy background? Can't think of any real benefits to this style.

If you don't understand the underlying mathematical algorithms its using, no amount of explicit varible names are going to help you. If you do, the concise structure makes things straightforward. The code is not meant to be read alone and understood, the papers published along with it need to be understood first.

Exactly. Not all code can be understandable to layman with zero effort.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#93

Post by rimantas »

MichaelBurge wrote: post #49
pilif wrote: post #29

It's 2016. How can you launch a reasonably high profile open source project with code that looks like this? This fulfills all the TODO list for unreadable code. One character variable names, one character parameter names, full of magic numbers... Yes. This is very performance critical code and I completely see the need to write very optimized code. That's fine. But optimizing code for speed shouldn't imply also optim…

I feel like spitting on other people's code is a good way to become an unhappy person. And this is coming from someone who's had a few head-scratching gdb sessions when a segfault occasionally appears in similar LZF compression code. And that's because doing the work is more valuable than being a snoot in the style aristocracy.

We have a proverb/parble which goes like this:

  —Lazy-bones, lazy-bones, would you like a boiled egg?
  —Is it peeled off?
  —Nope.
  —Throw it away.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#94

Post by eln1 »

svckr wrote: post #17

With energy efficiency as a primary goal I was expecting way more use of explicit SIMD instructions. The InfoQ post mentions xcodebuild, but there is also a Makefile. I really appreciate the presence of a no-nonsense Makefile. No autoconf, no pkgconfig, just plain and simple make. Also, because nobody mentioned it: yes, it compiles on Linux out of the box.

It is poorly optimized for 32bits - some benchmarks: http://encode.ru/threads/2221-LZFSE-New-Apple-Data-Compressi...

Re: Apple Open-Sources its Compression Algorithm LZFSE

#95

Post by userbinator »

pilif wrote: post #29

It's 2016. How can you launch a reasonably high profile open source project with code that looks like this? This fulfills all the TODO list for unreadable code. One character variable names, one character parameter names, full of magic numbers... Yes. This is very performance critical code and I completely see the need to write very optimized code. That's fine. But optimizing code for speed shouldn't imply also optim…

"It's 2016." So what? Have people lost the ability to use abbreviations? One letter is perfectly fine because they are abbreviations, the purpose of which you should recognise immediately if you understand the tiniest bit of what LZ algorithms do and the concepts surrounding them. D = distance, L = long-distance, M = medium-distance.

You may ask, "Why and what is q"? By only looking at the fragment posted, where q is the output pointer, I can already guess there is an input pointer named p, and glancing at the full file shows that is indeed the case. x is also a temporary. This is a very common convention.

Sadly, I'm increasingly finding that code these days is some bloated monstrosity with variable names that barely fit into 80 columns and plenty of ridiculous indirection that turns what really needs only a single line into a deep function-call-chain spanning dozens of lines (not all in the same place) and maybe even across multiple files. Reading "modern" C# and Java code makes my head hurt with all the verbiage --- there is so much code, but very little actual substance.

This code is essentially all substance and little verbiage, and I can comprehend it quite easily. Thus I find the style complaints entirely unfounded.

To borrow a sentiment from Linus Torvalds: the code is "unreadable" to you, because you are not (yet) qualified to read it.

/rant

Re: Apple Open-Sources its Compression Algorithm LZFSE

#96

Post by 2muchcoffeeman »

lokedhs wrote: post #33
barsonme wrote: post #28

Earlier quoted context omitted.

Blows my mind that people can come up with this stuff. I'm assuming they came up with the mathematical proofs first and translated that into code, so that has something to do with it, correct? It looks a lot like some crypto algorithms which are a nearly direct translation of the mathematical formulas. It's not that it's incredibly difficult to follow, but it's just very "math like".

That is my experience. Maths people are not renowned for their ability to write readable or maintainable code. I recently needed an implementation of the Simplex Noise algorithm (that I could port to Common Lisp). I ended up using this one, which works but the code certainly does nothing to help understanding: https://github.com/josephg/noisejs/blob/master/perlin.js Note that the Javscript implementation is also a po…

When I was doing numerical analysis and computational physics, this is also exactly what I did. Work out all the equations then port them straight into C or fortran.

And I was a comp sci major first and a physics major second. You spend over a decade doing math with single letter variables. Hard habit to break I guess.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#97

Post by randlet »

nicky0 wrote: post #59
pilif wrote: post #52

Earlier quoted context omitted.

> You could offer to rewrite if for them... Nope. Because I can't read it. I plainly do not have the information needed to understand what's going on here, nor is the design document part of the source code. As you seem to have no trouble understanding what's going on, can you enlighten me, for example, what's so special about the number 271 that we see M being compared to? > I don't see how giving the variables long…

OK but which is more readable ... (I know it's a bit silly, I juts made up names) } else if (D >= (1 34) { } else if (DirectWeightingFactor >= HUYGENS_LIMIT || MariachiBand == 0 || (xylemNonce + STANDARD_PZSH_INCREMENT) + MariachiBand > SWIM_RATE_B) { I guess your opinion differs to mine. I like the one that looks like math.

The more verbose name is infinitely more preferable to me. If I am tasked one day with doing some maintenance on this code and have never encountered it then at least I have a hint that I should be researching the aquatic properties of mariachi bands.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#98

Post by stinos »

nicky0 wrote: post #59
pilif wrote: post #52

Earlier quoted context omitted.

> You could offer to rewrite if for them... Nope. Because I can't read it. I plainly do not have the information needed to understand what's going on here, nor is the design document part of the source code. As you seem to have no trouble understanding what's going on, can you enlighten me, for example, what's so special about the number 271 that we see M being compared to? > I don't see how giving the variables long…

OK but which is more readable ... (I know it's a bit silly, I juts made up names) } else if (D >= (1 34) { } else if (DirectWeightingFactor >= HUYGENS_LIMIT || MariachiBand == 0 || (xylemNonce + STANDARD_PZSH_INCREMENT) + MariachiBand > SWIM_RATE_B) { I guess your opinion differs to mine. I like the one that looks like math.

I'd like the one where this whole thing is put in a function with a name describing what it does, no matter whether the math-like one or the one with long names is used. that I'd consider readable. And any sane compiler inlines it anyway.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#99

Post by cooper12 »

Oh wow dude you watch Silicon Valley also? Great reference! It sure added a lot to the conversation. Please continue making useful comments like this in the future, you really got us thinking with your pop culture reference. (And I sure patted myself on the back for recognizing such witticism)

Re: Apple Open-Sources its Compression Algorithm LZFSE

#100

Post by douche »

userbinator wrote: post #95
pilif wrote: post #29

It's 2016. How can you launch a reasonably high profile open source project with code that looks like this? This fulfills all the TODO list for unreadable code. One character variable names, one character parameter names, full of magic numbers... Yes. This is very performance critical code and I completely see the need to write very optimized code. That's fine. But optimizing code for speed shouldn't imply also optim…

"It's 2016." So what ? Have people lost the ability to use abbreviations? One letter is perfectly fine because they are abbreviations, the purpose of which you should recognise immediately if you understand the tiniest bit of what LZ algorithms do and the concepts surrounding them. D = distance, L = long-distance, M = medium-distance. You may ask, "Why and what is q"? By only looking at the fragment posted, where q i…

It's not like you have to pay a dime for every character in your source code. With halfway-decent autocomplete, longer identifiers are easier to use than shorter ones, I've found.

I'd hope you'd at least put a comment header to explain what the parameters actually are for anyone who doesn't have the paper handy, or god forbid, used a paper implementing the same thing using different nomenclature.

Post reply on HN