Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Open-Sources its Compression Algorithm LZFSE

infoq.com

Post reply on HN
Print view View original

61–70 of 219 posts

Re: Apple Open-Sources its Compression Algorithm LZFSE

#61

Post by pilif »

nicky0 wrote: post #59
pilif wrote: post #52

Earlier quoted context omitted.

> You could offer to rewrite if for them... Nope. Because I can't read it. I plainly do not have the information needed to understand what's going on here, nor is the design document part of the source code. As you seem to have no trouble understanding what's going on, can you enlighten me, for example, what's so special about the number 271 that we see M being compared to? > I don't see how giving the variables long…

OK but which is more readable ... (I know it's a bit silly, I juts made up names) } else if (D >= (1 34) { } else if (DirectWeightingFactor >= HUYGENS_LIMIT || MariachiBand == 0 || (xylemNonce + STANDARD_PZSH_INCREMENT) + MariachiBand > SWIM_RATE_B) { I guess your opinion differs to mine. I like the one that looks like math.

What about a middle ground?

} else if (D >= HUYGENS_LIMIT || M == 0 || (x+3)+M > SWIM_RATE_B)

Re: Apple Open-Sources its Compression Algorithm LZFSE

#62

Post by WalterBright »

pilif wrote: post #52
nicky0 wrote: post #45

Earlier quoted context omitted.

You could offer to rewrite if for them... It's an implementation of a mathematical algorithm. It doesn't need allTheVariables toBeNamed likeThis. Single letters map to meaningful concepts in the mathematical algorithm. I don't see how giving the variables longer names would make it more readable. Indeed I think long variable names would obscure the structure. Code like this has to be looked at in the concept of the a…

> You could offer to rewrite if for them... Nope. Because I can't read it. I plainly do not have the information needed to understand what's going on here, nor is the design document part of the source code. As you seem to have no trouble understanding what's going on, can you enlighten me, for example, what's so special about the number 271 that we see M being compared to? > I don't see how giving the variables long…

> what's so special about the number 271?

It's a Cuban Prime. Everybody knows that. Geez!

Re: Apple Open-Sources its Compression Algorithm LZFSE

#63

Post by guruz »

drinchev wrote: post #15
nodesocket wrote: post #9

If you want to see some crazy C code, check out this file from the GitHub repo: https://github.com/lzfse/lzfse/blob/master/src/lzvn_encode_b...

Excerpt from the link : if (D == D_prev) { if (L == 0) { *q++ = 0xF0 + (x + 3); // XM! } else { *q++ = (L >8 in 0..5 *q++ = (D >> 8) + (L = (1 34) { // Long dist *q++ = (L > 2) + (L

Browse this snippet with hover overlays and semantic linking: https://code.woboq.org/lzfse/lzfse/src/lzvn_encode_base.c.ht...

Re: Apple Open-Sources its Compression Algorithm LZFSE

#64

Post by chillydawg »

vertex-four wrote: post #53
johncolanduoni wrote: post #32

Earlier quoted context omitted.

> To release compression code in a non-safe language is risky enough At the moment, what's their real alternative? Rust is the only memory-safe language I can think of that could hope to meet their performance requirements, but even the Rust runtime would be a lot of overhead for this application. That said, I agree this isn't acceptable C code for something that runs on untrusted data while using tons of pointer ari…

Which bits of the Rust runtime(?) do you think are too high overhead for this?

I suspect any runtime, at all, would be too much overhead.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#65

Post by vertex-four »

chillydawg wrote: post #64
vertex-four wrote: post #53

Earlier quoted context omitted.

Which bits of the Rust runtime(?) do you think are too high overhead for this?

I suspect any runtime, at all, would be too much overhead.

There isn't a Rust runtime, though, in the sense that you seem to be implying. There's a standard library, which is what I assumed they meant.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#66

Post by verytrivial »

pilif wrote: post #29

It's 2016. How can you launch a reasonably high profile open source project with code that looks like this? This fulfills all the TODO list for unreadable code. One character variable names, one character parameter names, full of magic numbers... Yes. This is very performance critical code and I completely see the need to write very optimized code. That's fine. But optimizing code for speed shouldn't imply also optim…

It's 2016 and it on github. Fix it yourself!

Re: Apple Open-Sources its Compression Algorithm LZFSE

#68

Post by nodesocket »

Pica_soO wrote: post #44
gigatexal wrote: post #31

Earlier quoted context omitted.

I wonder why they use goto statements instead of just returning q1 like the statement evaluates to.

No function call overhead. Makes sense as long as you stay in the same state-machine. This thing doesn't have to be pretty. It has to be fast. Who cares for any oo-written implementation that takes half a hour to do the same job?

But there is no function call overhead right? Replacing goto OUT_FULL; with return q1; would seem to be faster right? No need for a goto, just invoke return.

What am I missing?

Re: Apple Open-Sources its Compression Algorithm LZFSE

#69

Post by johncolanduoni »

vertex-four wrote: post #53
johncolanduoni wrote: post #32

Earlier quoted context omitted.

> To release compression code in a non-safe language is risky enough At the moment, what's their real alternative? Rust is the only memory-safe language I can think of that could hope to meet their performance requirements, but even the Rust runtime would be a lot of overhead for this application. That said, I agree this isn't acceptable C code for something that runs on untrusted data while using tons of pointer ari…

Which bits of the Rust runtime(?) do you think are too high overhead for this?

Using rust 1.9.0, an empty (save for a function that adds two u32s) standalone dynamic library built in release mode on OS X is 1.6 MB. A static library is a whopping 2.4 MB. The comparable number for C are 4K and 800 bytes respectively.

Asking every client of the compression library to pull in that much overhead would likely make it rather unpopular. Until Rust gets better at eliminating unnecessary parts of the runtime when a program doesn't use it (something like GCC's gc-sections), it's not going to be feasible for small libraries to be written in it.

Re: Apple Open-Sources its Compression Algorithm LZFSE

#70

Post by jdub »

chillydawg wrote: post #64
vertex-four wrote: post #53

Earlier quoted context omitted.

Which bits of the Rust runtime(?) do you think are too high overhead for this?

I suspect any runtime, at all, would be too much overhead.

This C code doesn't use the C library. Code like this in Rust wouldn't engage with any part of the Rust standard library either. There's no runtime work to be done, in either language.
Post reply on HN