Most of software developers don't care about encodings all day. This will give them enough rope to hang half of the world.
Memory is dirt cheap these days anyway, why now? And if you have mounds of text, compress it.
JEP254: proposal to represent Java Strings as ISO-8859-1
11–20 of 55 posts
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#12Most of software developers don't care about encodings all day. This will give them enough rope to hang half of the world. Memory is dirt cheap these days anyway, why now? And if you have mounds of text, compress it.
Reducing memory usage also reduces time spent in garbage collection.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#13This sounds like a ridiculous idea. Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS). This proposal doesn't challenge that claim, nor could it. Java strings are sequences of Unicode code points, logically at least. This proposal is just about how to encode such a sequence internally. There are numerous alternatives for encoding sequences of Unicode code poin…
> Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS) unfortunately, Java treats text as data encoded in UCS-2 which uses a maximum of two bytes to represent a character and all the Java string APIs assume the UCS-2 encoding. The problem with UCS-2 is that it doesn't allow to encode characters outside of the Unicode BMP. In order to solve this, Java actually us…
> The only well-known contemporary languages that get this (somewhat) right are Python 3, Ruby 1.9 and Swift, though ES6 is in process of getting up to speed too.
And Perl.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#14Most of software developers don't care about encodings all day. This will give them enough rope to hang half of the world. Memory is dirt cheap these days anyway, why now? And if you have mounds of text, compress it.
Reducing memory usage also reduces time spent in garbage collection.
Why the downvote?
The smaller the strings, the more strings you can allocate before triggering a garbage collection. Also, the smaller the objects, the less copying is needed when promoting survivors.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#15Earlier quoted context omitted.
> Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS) unfortunately, Java treats text as data encoded in UCS-2 which uses a maximum of two bytes to represent a character and all the Java string APIs assume the UCS-2 encoding. The problem with UCS-2 is that it doesn't allow to encode characters outside of the Unicode BMP. In order to solve this, Java actually us…
> The only well-known contemporary languages that get this (somewhat) right are Python 3, Ruby 1.9 and Swift, though ES6 is in process of getting up to speed too. And Perl.
You are right of course (I knew this as you can see when you look at the PDF I linked - slide 50). I have edited my original post.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#16This sounds like a ridiculous idea. Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS). This proposal doesn't challenge that claim, nor could it. Java strings are sequences of Unicode code points, logically at least. This proposal is just about how to encode such a sequence internally. There are numerous alternatives for encoding sequences of Unicode code poin…
ISO 8859-1 does have the unique property that each of its 256 code points map directly to the first 256 Unicode code points.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#17I have always wondered why they didn't do that, it seems to be such a simple optimization. With UTF-8 you would give up some constant time operations like character lookups, while ISO-8859-1 shouldn't have any performance regressions I can think of. Because once you add a non-latin character to a String and the encoding has to change, the data has to be copied anyway due to String immutability.
>With UTF-8 you would give up some constant time operations like character lookups
if Java had a non-broken string API, even with the current UCS-2 based solution they couldn't do constant-time character lookups.
And even if Java went the extra mile of using characters big enough to encode every code point of a string, constant lookup would still not be possible because users are often interested graphemes, not characters.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#18Most of software developers don't care about encodings all day. This will give them enough rope to hang half of the world. Memory is dirt cheap these days anyway, why now? And if you have mounds of text, compress it.
The whole idea is that developers wouldn't need to know. Memory might be dirt cheap but GC pause time and surrounding latency definitely is not.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#19An interesting alternate idea, though I fear the constant factor may be too high for a language like Java, especially given Java's whole lack-of-value-types thing. (And Java 8's "value types" don't work for this, due to their immutability. Yay.)
You store a string as a self-balancing tree, preferably one that supports constant-time appends and prepends.(For example, a skew-binary random access list or a finger tree.) Each node of the tree has an encoding enum, an array (+ length of said array), and the length including children (alternatively, the length of the left child). A node's characters are all the same length given by the encoding of the node, and a node stores characters by grapheme clusters.
This allows efficient string building, among other things. About the only problems are that lookup/replacement in the middle of strings is now O(log n), and the constant factor.
So, for example, if you have the string "This is a t̴̟̟͙̞̑ͩ͌͝est", it'd (probably) be stored in three nodes: one that stores "This is a t" with an encoding of one byte per character, a direct lookup into the first 256 unicode characters, one that stores "̴̴̟̟͙̞̟̟͙̞̑ͩ͌̑ͩ͌͝͝e", with an encoding of, what, 20 bytes per character, utf8 within a character, and one that stores "st", same as the start.
Unfortunately, Java has too much overhead to make this practical.
Re: JEP254: proposal to represent Java Strings as ISO-8859-1
#20This sounds like a ridiculous idea. Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS). This proposal doesn't challenge that claim, nor could it. Java strings are sequences of Unicode code points, logically at least. This proposal is just about how to encode such a sequence internally. There are numerous alternatives for encoding sequences of Unicode code poin…
> Java treats text as all modern systems should: as a sequence of the open-ended Universal Character Set (UCS) unfortunately, Java treats text as data encoded in UCS-2 which uses a maximum of two bytes to represent a character and all the Java string APIs assume the UCS-2 encoding. The problem with UCS-2 is that it doesn't allow to encode characters outside of the Unicode BMP. In order to solve this, Java actually us…
> unfortunately, Java treats text as data encoded in UCS-2 which uses a maximum of two bytes to represent a character and all the Java string APIs assume the UCS-2 encoding.
Not entirely true. There are codePoint*, indexOf(int ch), and various other methods on String which do work correctly in code points. Unfortunately old methods can't be changed without affecting compatibility.