31–40 of 162 posts
Or am I missing something obviuos here?
This is why UTF-8 is great. If it works for any Unicode character it will work for them all. Surrogate pairs are rare enough that they are poorly tested. With UTF-8, if there are issues with multi-byte characters, they are obvious enough to get fixed. UTF-16 is not a very good encoding. It only exists for legacy reasons. It has the same major drawback as UTF-8 (variable-length encoding) but none of the benefits (ASCI…
The problem with UTF-8 is that lots of tools have 3 byte limits, and characters like Emoji take up 4 bytes in UTF-8.
This is why UTF-8 is great. If it works for any Unicode character it will work for them all. Surrogate pairs are rare enough that they are poorly tested. With UTF-8, if there are issues with multi-byte characters, they are obvious enough to get fixed. UTF-16 is not a very good encoding. It only exists for legacy reasons. It has the same major drawback as UTF-8 (variable-length encoding) but none of the benefits (ASCI…
The problem with UTF-8 is that lots of tools have 3 byte limits, and characters like Emoji take up 4 bytes in UTF-8.
Honest question, as the three byte limit seems rather arbitrary and no more logical than, say, a four byte one.
Sometimes you need to know about encodings, even if you're just a consumer. Putting just one non 7-bit character in your SMS message will silently change its encoding from 7-bit (160 chars) to 8-bit (140 chars) or even 16 bit (70 chars) which might make the phone split it into many chunks. The resulting chunks are billed as separate messages.
On iOS, using any non-basic Latin character in SMS makes it switch to 16 bit, even when there is no reason for that to happen. It's a thing that most foreign language speakers must live with. By doing this full of excuses write-up, this guy wasted a substantial amount of time that he could have spent better researching the issue. Your consumer doesn't care that Emoji is this much or that much bits, it doesn't matter…
JavaScript is a joke in this respect, and is keeping horrors like Shift-JIS alive long after they should have been retired.
>Wow, you read though all of that? You rock. I'm humbled that you gave me so much of your attention. That was actually really fun to read, even as a now non-technical guy. I can't put a finger on it, but there was something about his style that gave off a really friendly vibe even through all the technical jargon. That's a definite skill!
TLDR: node sucks
Sometimes you need to know about encodings, even if you're just a consumer. Putting just one non 7-bit character in your SMS message will silently change its encoding from 7-bit (160 chars) to 8-bit (140 chars) or even 16 bit (70 chars) which might make the phone split it into many chunks. The resulting chunks are billed as separate messages.
Here's the thread in the v8 bug tracker about this issue: http://code.google.com/p/v8/issues/detail?id=761 Is there a reason that the workaround in comment 8 won't address some of these issues?
If you read closely you'll see the original linked message is from January and there's an update on that issue from March when a fix was made in V8.
This is why UTF-8 is great. If it works for any Unicode character it will work for them all. Surrogate pairs are rare enough that they are poorly tested. With UTF-8, if there are issues with multi-byte characters, they are obvious enough to get fixed. UTF-16 is not a very good encoding. It only exists for legacy reasons. It has the same major drawback as UTF-8 (variable-length encoding) but none of the benefits (ASCI…
The problem with UTF-8 is that lots of tools have 3 byte limits, and characters like Emoji take up 4 bytes in UTF-8.
Earlier quoted context omitted.
TLDR: The V8 engine can't (supposedly) encode Unicode codepoints that are over 16-bits in length, because it uses the UCS-2 encoding.
TLDR: v8 "sucks" (and doesn't support Unicode code points outside of the lowest ~64k characters). Edit: v8 in general is pretty cool, but not supporting Unicode outside UCS-2 is pretty bad.