Breaking java.lang.String

No idea why Thaxll and the other comments are mentioning mutexes.

The equivalent (*) API to this Java API in Rust does exist; it's `String::from_utf8(Vec) -> String`. And the bug in TFA does not exist there. Since the signature consumes the `Vec` it's impossible for the caller or any other code to still have access to it to be able to modify it concurrently.

Also consider the similar API `str::from_utf8(&[u8]) -> &str`. The bug in TFA does not exist here either. Since the signature takes a `&` borrow of the slice, it is not possible for anything else in the program to have a `&mut` borrow of that slice to be able to modify it concurrently. After the function returns other parts of the program could mutate the slice, but they would only be able to do after the `&str` that is derived from the slice is dropped. So once again nothing would be able to mutate the slice and observe the effects in the `&str` itself.

All these "unable to do" are enforced at compile-time, because "consuming a value makes it unavailable to other parts of the program" and "cannot get a `&mut` to a value as long as a `&` from that value is still in scope" are all typesystem concepts. No mutexes or other runtime checks are involved.

(*) "Equivalent" in that it's an API to convert a sequence of bytes into a string. The Rust API doesn't have the encoding thing of the Java API because the Rust String / str are required to be utf-8 internally. But if an exact equivalent of the Java API did exist in Rust, the signature would still be the same wrt consuming `Vec` / borrowing `[u8]`, so it doesn't change the overall point re: concurrent modification. Furthermore, concurrent modification would cause problems even with Rust Strings if it was possible, because it would allow a String / str to become invalid utf-8 after they'd already been checked to be valid utf-8, which Rust considers to be UB.

Another way of fixing it would be, whichever character caused 'StringUTF16.compress' to fail (we'd need to return it), make sure that character is kept in the final string (could just assign it at the end, remember it's value and location earlier).

By merging in the code of StringUTF16.compress and StringUTF16.toBytes into one function (and they are both very small), this wouldn't even have to slow things down, once you noticed location 'i' has char 'c' which isn't LATIN1, you make the new buffer, copy 0..i-1 over, then your known non-LATIN1 'i', then i+1..len.

This might be very slightly slower, but would fix (I think) the problem of breaking interning, which feels quite painful.

Rust absolutely helps here because in Rust it’s simply impossible for someone else to mutate something concurrently to you holding a reference to it. Code equivalent to that in the article simply won’t compile in Rust. This is, like, the very point of Rust’s borrow system. You can share, xor you can mutate, but not both at the same time. This holds equally for single and multi-threaded code.

// compareTo does consider them equal (even though its javadoc

// specifies that "compareTo returns 0 exactly when the

// equals(Object) method would return true")

Seems like at least a documentation error.

Clever! I like this! I feel a bit jealous that I didn't think of it :)

It's not quite as easy as it seems, because these methods are "intrinsics". That is, the pure Java code you see is not always the one that gets used; instead, the JIT compiler can use a faster implementation that uses vectorized assembly or whatever. (That's why you see "@IntrinsicCandidate" on compress() and toBytes() in StringUTF16.java)

But I think your idea would also be possible to implement in vectorized assembly, so it still works!

The rule for most things (such as ArrayList) in the JDK is: "if you use race conditions to break this thing, that's your problem, not ours". But String is different: it's one of the few things meant to be "rock-solid, can't break it even if you try", so I think this bug in String would qualify as a potential security issue in their mind: there are many places in Java that trust Strings not to act weird, and some of them are even in native code deep in the guts of the VM.

On the other hand, String is used all over the place so having to introduce a performance regression to fix this bug would be quite painful. All of the other proposed solutions in this thread introduce an extra copy, and an extra pass over the string. Your fix is basically no extra cost, and as a bonus, can be tweaked so each char in the array is read only once.

Which means that the bug can now be fixed "guilt-free", if anyone from the JDK team is reading this thread. Though they have some pretty clever people there too, so they might have thought of it eventually for all I know.