81–90 of 219 posts
Earlier quoted context omitted.
This is a baseless speculation. First of all, Apple provides LZ4 in libcompression. Secondly, LZFSE uses Lempel–Ziv algorithm and ANS coder invented by Jarek Duda ( https://arxiv.org/abs/1311.2540 ): https://developer.apple.com/library/ios/documentation/Perfor...
Can I just say I really appreciate your contributions on this thread. Answering crass company-bashing with well-referenced and informative replies.
It's 2016. How can you launch a reasonably high profile open source project with code that looks like this? This fulfills all the TODO list for unreadable code. One character variable names, one character parameter names, full of magic numbers... Yes. This is very performance critical code and I completely see the need to write very optimized code. That's fine. But optimizing code for speed shouldn't imply also optim…
> To release compression code in a non-safe language is risky enough At the moment, what's their real alternative? Rust is the only memory-safe language I can think of that could hope to meet their performance requirements, but even the Rust runtime would be a lot of overhead for this application. That said, I agree this isn't acceptable C code for something that runs on untrusted data while using tons of pointer ari…
Earlier quoted context omitted.
You could offer to rewrite if for them... It's an implementation of a mathematical algorithm. It doesn't need allTheVariables toBeNamed likeThis. Single letters map to meaningful concepts in the mathematical algorithm. I don't see how giving the variables longer names would make it more readable. Indeed I think long variable names would obscure the structure. Code like this has to be looked at in the concept of the a…
> You could offer to rewrite if for them... Nope. Because I can't read it. I plainly do not have the information needed to understand what's going on here, nor is the design document part of the source code. As you seem to have no trouble understanding what's going on, can you enlighten me, for example, what's so special about the number 271 that we see M being compared to? > I don't see how giving the variables long…
When the byte is 0xE1...0xEF (inclusive), the bottom 4 bits alone are the count - so after a run of 0xE0 sections this is how you write out any stragglers.
The decoder has comments, but I managed to figure the above out without them, so I doubt it can be that hard - I've just had a fair amount of practice at this bit twiddling C stuff. At some point when writing code you have to assume that the reader will be speaking your language.
https://github.com/lzfse/lzfse/blob/master/src/lzvn_decode_b...
Earlier quoted context omitted.
You could offer to rewrite if for them... It's an implementation of a mathematical algorithm. It doesn't need allTheVariables toBeNamed likeThis. Single letters map to meaningful concepts in the mathematical algorithm. I don't see how giving the variables longer names would make it more readable. Indeed I think long variable names would obscure the structure. Code like this has to be looked at in the concept of the a…
It's funny, because there's a whole couple of generations out there that have been taught that explicitVariableNames will fix their thinking deficit. I mean, if it weren't sad.
Waiting for first guy who take this implementation and run it through emscripten so we can actually use it in client -> server communication, eg sending compressed json payloads to the server.
Is running (not to mention downloading/parsing/compiling) emscriptened decompression code really that much faster than just transferring the data? I think a better strategy would be to add support for LZFSE to the browsers themselves.
With energy efficiency as a primary goal I was expecting way more use of explicit SIMD instructions. The InfoQ post mentions xcodebuild, but there is also a Makefile. I really appreciate the presence of a no-nonsense Makefile. No autoconf, no pkgconfig, just plain and simple make. Also, because nobody mentioned it: yes, it compiles on Linux out of the box.
Waiting for first guy who take this implementation and run it through emscripten so we can actually use it in client -> server communication, eg sending compressed json payloads to the server.
Is running (not to mention downloading/parsing/compiling) emscriptened decompression code really that much faster than just transferring the data? I think a better strategy would be to add support for LZFSE to the browsers themselves.
Earlier quoted context omitted.
That's not surprising, given that they went for compression and decompression speed and for energy usage. Their goal seems to have been to be at least as good as zlib at compressing stuff using less energy and doing it faster (that often correlates quite well with energy use on modern CPUs, as it allows them to drop to low energy states faster)
Could you give me some pointers on the actual numbers? My searches came back with nothing. I'm especially interested how they benchmarked the energy consumption.
With energy efficiency as a primary goal I was expecting way more use of explicit SIMD instructions. The InfoQ post mentions xcodebuild, but there is also a Makefile. I really appreciate the presence of a no-nonsense Makefile. No autoconf, no pkgconfig, just plain and simple make. Also, because nobody mentioned it: yes, it compiles on Linux out of the box.
I remember reading one paper whose conclusion was running SIMD instructions can be bad for power consumption: while you need the processor in a higher power state for longer without, you can keep the SIMD unit powered off.
[Edit: That said, I have no idea how true this is for the modern Intel CPUs yet alone Apple's custom SoCs.]
Earlier quoted context omitted.
But there is no function call overhead right? Replacing goto OUT_FULL; with return q1; would seem to be faster right? No need for a goto, just invoke return. What am I missing?
There is some. A few register need to be assigned, and everything else gets pushed/copied onto the stack. Then you jump to a location and copy everything back off the stack. This doesn't take many cycles, but it does take some. While a GOTO is just a jump.