Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Breaking java.lang.String

wouter.coekaerts.be

Post reply on HN
Print view View original

81–90 of 206 posts

Re: Breaking java.lang.String

#81

Post by haspok »

coekie wrote: post #75
gizmo686 wrote: post #14

Is this actually a bug? The default assumption in Java is that types are not thread-safe unless otherwise specified. Attempting to use types in a way that exceeds their documented thread safety has always been allowed to leave your program in an inconsistent state.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

How would you fix this? Possibly by copying the input array to a local array as the first step? But then you might copy a lot of data, which is in 99% of the cases totally unnecessary. Have a separate threadsafe and a non-threadsafe constructor for char[]?

EDIT: another idea is to do a hash of the input array, and compare between start and end. But that also requires an O(n) walk through the input...

Re: Breaking java.lang.String

#82

Post by bonzini »

sorokod wrote: post #48
gizmo686 wrote: post #25

Earlier quoted context omitted.

As the article points out, the only thing the Javadoc guarantees is the subsequent modifications of the character array have no effect. It says nothing about concurrent modifications. The type whose thread safety is in question here is not actually String, but char[]. I'm not going to say it is always wrong to share char[] between threads (as a primitive array of something other than double and long, the Java Spec do…

"The type whose thread safety is in question here is not actually String, but char[]" Not quite, it is String constructor that has the race condition, char array is incidental there.

The thread-safety guarantees can only be obeyed if the arguments are thread-safe as well.

For example, take a java.util.concurrent.ConcurrentHashMap, which is thread-safe. If your value object is mutated concurrently, and is not thread-safe, you can't expect anything sensible to happen if you call containsValue().

Likewise, String's char[] constructor can only be thread safe if there are no concurrent changes to the argument.

Re: Breaking java.lang.String

#83

Post by silon42 »

haspok wrote: post #81
coekie wrote: post #75

Earlier quoted context omitted.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

How would you fix this? Possibly by copying the input array to a local array as the first step? But then you might copy a lot of data, which is in 99% of the cases totally unnecessary. Have a separate threadsafe and a non-threadsafe constructor for char[]? EDIT: another idea is to do a hash of the input array, and compare between start and end. But that also requires an O(n) walk through the input...

/s redesign a language to prevent parallel mutable use of the array

Re: Breaking java.lang.String

#84

Post by tpxl »

haspok wrote: post #81
coekie wrote: post #75

Earlier quoted context omitted.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

How would you fix this? Possibly by copying the input array to a local array as the first step? But then you might copy a lot of data, which is in 99% of the cases totally unnecessary. Have a separate threadsafe and a non-threadsafe constructor for char[]? EDIT: another idea is to do a hash of the input array, and compare between start and end. But that also requires an O(n) walk through the input...

> But that also requires an O(n) walk through the input...

Compressing requires an O(n) walk through the input either way.

Re: Breaking java.lang.String

#85

Post by ris58h »

coekie wrote: post #75
gizmo686 wrote: post #14

Is this actually a bug? The default assumption in Java is that types are not thread-safe unless otherwise specified. Attempting to use types in a way that exceeds their documented thread safety has always been allowed to leave your program in an inconsistent state.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

Accepting an issue doesn't mean that there is a bug.

> While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe.

If it's not documented it's just an assumption.

Re: Breaking java.lang.String

#86

Post by the_mitsuhiko »

haspok wrote: post #81
coekie wrote: post #75

Earlier quoted context omitted.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

How would you fix this? Possibly by copying the input array to a local array as the first step? But then you might copy a lot of data, which is in 99% of the cases totally unnecessary. Have a separate threadsafe and a non-threadsafe constructor for char[]? EDIT: another idea is to do a hash of the input array, and compare between start and end. But that also requires an O(n) walk through the input...

I'm not sure how much of the underlying implementation details of arrays are leaking into JNI, but one way to approach it would be with first class copy-on-write for arrays or opt-in immutability (freezing).

Re: Breaking java.lang.String

#87

Post by ris58h »

layer8 wrote: post #70
dundarious wrote: post #26

Calling this a "bug in java.lang.String" is silly. The same "bug" exists for all functions that take mutable objects. If you take a map and lookup two different keys, yep, that's a "bug". The bug is the other piece of code that introduces the data race in the first place. You can argue the case for languages like Rust with it's borrow system, or others that use linear types or something along those lines, to eliminat…

It’s a bug for constructor functions, in my book. I certainly always code defensively to prevent such misbehavior. A successfully constructed object should obey its documented interface contract, period.

> A successfully constructed object should obey its documented interface contract, period.

Could you provide the contract you are talking about?

Re: Breaking java.lang.String

#88

Post by _old_dude_ »

gizmo686 wrote: post #14

Is this actually a bug? The default assumption in Java is that types are not thread-safe unless otherwise specified. Attempting to use types in a way that exceeds their documented thread safety has always been allowed to leave your program in an inconsistent state.

It's a TOCTOU bug [1], a well known category of bugs.

[1] https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use

Re: Breaking java.lang.String

#89

Post by PedroBatista »

ris58h wrote: post #85
coekie wrote: post #75

Earlier quoted context omitted.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

Accepting an issue doesn't mean that there is a bug. > While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. If it's not documented it's just an assumption.

Yes, but at some point one has come out of the theory domain and face the real world. So in practical terms it’s closer to a bug and even the “official” guys from Java accepted it as a bug.

Re: Breaking java.lang.String

#90

Post by marginalia_nu »

coekie wrote: post #75
gizmo686 wrote: post #14

Is this actually a bug? The default assumption in Java is that types are not thread-safe unless otherwise specified. Attempting to use types in a way that exceeds their documented thread safety has always been allowed to leave your program in an inconsistent state.

Author of the blog here. Yes, this is a bug. While the javadoc doesn't state it explicitly, immutable classes in the core library are expected to be thread safe. Java tries to be and mostly succeeds at being a safe language, where (by default) the guarantees of its internals cannot be broken no matter what user code does. The JVM preserves its own integrity. There are some other deliberate holes in that safety, such…

Dunno, this seems like you're violating the java memory model and then then fully expected weird shit happens. If you're mutating a shared state between threads without proper synchronization, and there is no way for String to do this on its own, the CPU is permitted to do stuff like out-of-order execution as there is no happens-before relationship between the write and the read. The JVM is further permitted to optimize the code with the assumption that the data stays the same in the absence of volatile/synchronized/other synchronziation primitives, as long as the observable outcome is the same "as if serial" execution.

The understanding 'It first tries to encode the characters as latin-1 using StringUTF16.compress. If that fails, it returns null and the constructor falls back to using UTF-16. ' is incorrect in unsynchronized code. The reality is that it's permitted to do all these things at the same time in any order (or speculatively do both at the same time) as long as the observable consequences are the same. This relies on the assumption that the data stays the same. If you violate that assumption, you get bizarre and often unpredictable behavior.

I don't think this is a bug in String. It may be a security vulnerability in the JVM, in a log4j-esque "it's working as we intended but holy crap did we ever not intend on this interaction" manner.

Post reply on HN