The sorry state of Java deserialization
marginalia.nu
The sorry state of Java deserialization
1–10 of 54 posts
Re: The sorry state of Java deserialization
#2[deleted]
Re: The sorry state of Java deserialization
#3are the benchmarks done properly? whats the actual test code?
Re: The sorry state of Java deserialization
#4are the benchmarks done properly? whats the actual test code?
I don’t think the conclusion need a lot of precision in the benchmark. When the suggested code from the standard library (or some tutorial) is two orders of magnitude slower, something is not right.
The author is right that we are wasting something somewhere when we are only operating at 2% of the possible speed of the hard disk.
Re: The sorry state of Java deserialization
#5“I admit I don’t understand these results. There’s clearly nothing in the runtime itself that prevents these types of speeds.” Oh, there is. The default Java serialization is sort of like “pickle” module in Python - if you are familiar. It will deal with pretty much anything you throw at it, figuring the data structures and offsets to serialize or parse at runtime. More efficient methods trade universality for speed, where the offsets and calls to read/write the parts of the structure are determined in advance. Also, hard to say without source code but there is a high chance even more efficient methods like Protobuf create a lot of Java objects and that kills cache locality. With Java, you have to go out of your way to maintain good cache locality because you give up control over memory layout for automatic memory management.
Re: The sorry state of Java deserialization
#6I’m probably missing something obvious, but what’s wrong with Apache parquet-java for this use case?
Re: The sorry state of Java deserialization
#7I’m probably missing something obvious, but what’s wrong with Apache parquet-java for this use case?
The implementation is inexorably merged with hadoop, to the point where it is not useful outside of it.
Parquet-floor is a shim that replaces the hadoop depenencies with drop in java.io-ones.
Re: The sorry state of Java deserialization
#8are the benchmarks done properly? whats the actual test code?
I don’t think the conclusion need a lot of precision in the benchmark. When the suggested code from the standard library (or some tutorial) is two orders of magnitude slower, something is not right. The author is right that we are wasting something somewhere when we are only operating at 2% of the possible speed of the hard disk.
From the code samples it's hard to tell whether or not this has to do with de-serialization though. It would have been fun to see profiling results for tests such as these.
Re: The sorry state of Java deserialization
#9Earlier quoted context omitted.
I don’t think the conclusion need a lot of precision in the benchmark. When the suggested code from the standard library (or some tutorial) is two orders of magnitude slower, something is not right. The author is right that we are wasting something somewhere when we are only operating at 2% of the possible speed of the hard disk.
From the code samples it's hard to tell whether or not this has to do with de-serialization though. It would have been fun to see profiling results for tests such as these.
Author here, I'm away from my computer atm, but I can cook up a repo with each test in a few hours when I get home.
I designed the tests as a drag race because that mimics my real world usage.
Re: The sorry state of Java deserialization
#10“I admit I don’t understand these results. There’s clearly nothing in the runtime itself that prevents these types of speeds.” Oh, there is. The default Java serialization is sort of like “pickle” module in Python - if you are familiar. It will deal with pretty much anything you throw at it, figuring the data structures and offsets to serialize or parse at runtime. More efficient methods trade universality for speed,…
True, but none of the slow methods in the article involve this.