21–30 of 39 posts
I think the most realistic way to compress a tweet would be to replace words like "before" with "b4", "too"/"to" with "2", reduce whitespace (e.g. double spaces to single), and maybe start ripping out vowels ("vowels" -> "vwls"). Although not as efficient as demonstrated above, there are no external dependencies needed; the content can be decompressed by the reader's brain in-place at the slight cost of being difficu…
B4 bs ws 2 lt.
So Twitter allows 140 (UTF-8?) characters, regardless of the number bytes? The article wasn't clear about this.
Twitters allows 140 codepoints of a sequence in normalization form C.
The only reason I can think of is that they internally use UTF-32, so counting codepoints is more efficient. But I thought they used UTF-8.
Edit: the other reason I can think of is that conversion to normalization form C already counts the codepoints. Though I can't imagine making it also count characters would be nontrivial.
Earlier quoted context omitted.
There are a lot of code points that would need to be filtered out if you do this - Noncharacters, Control codes, High/Low surrogates, Private-Use, Whitespace, and then of course the ones that mutate other code points in the sequence - Bidirectional, Combining characters / diacritical marks. It isn't quite as simple as just combining random 32-bit characters, as I found when creating my URL shortener. If you want to p…
> High/Low surrogates Surrogates are not codepoints.
Earlier quoted context omitted.
There's also a social status cost, which could be great or small, for example if your readers have an intense dislike for netspeak and poor English.
Here in the UK it's interesting to note that, while text speak was all the rage a few years ago, it faded out. It was replaceddddd by making wordssss actuallllllly longerrrrrr for no reeeeaaaasonnnnn!!! Now the text speak is a bit more reined in and not totally incomprehensible like it used to get.
Earlier quoted context omitted.
Twitters allows 140 codepoints of a sequence in normalization form C.
Wait, really? I thought normalization form C was the form where composite characters were always used when possible. Why restrict by the number of codepoints (vs characters) if you're explicitly going to use the form which goes out of its way to use multi-codepoint characters? The only reason I can think of is that they internally use UTF-32, so counting codepoints is more efficient. But I thought they used UTF-8. Ed…
I'm sure it had some historical technical limitation to 140 ASCII or maybe 70 utf-8 chars (or something else logical), but they probably had to accomodate people who wanted to use non-English characters in a post and not get a lecture on unicode encoding; and some slightly offensive "so ... people like you only get 70 chars" message.
Earlier quoted context omitted.
Wait, really? I thought normalization form C was the form where composite characters were always used when possible. Why restrict by the number of codepoints (vs characters) if you're explicitly going to use the form which goes out of its way to use multi-codepoint characters? The only reason I can think of is that they internally use UTF-32, so counting codepoints is more efficient. But I thought they used UTF-8. Ed…
Most likely, because Twitter is a microblog, and 140 character posts is it's thing. I'm sure it had some historical technical limitation to 140 ASCII or maybe 70 utf-8 chars (or something else logical), but they probably had to accomodate people who wanted to use non-English characters in a post and not get a lecture on unicode encoding; and some slightly offensive "so ... people like you only get 70 chars" message.
Earlier quoted context omitted.
Most likely, because Twitter is a microblog, and 140 character posts is it's thing. I'm sure it had some historical technical limitation to 140 ASCII or maybe 70 utf-8 chars (or something else logical), but they probably had to accomodate people who wanted to use non-English characters in a post and not get a lecture on unicode encoding; and some slightly offensive "so ... people like you only get 70 chars" message.
You misunderstand. I'm wondering why they allow only 140 codepoints and not characters . Even in Unicode, 1 codepoint != 1 character.
I think Twitter should use whichever usually gives the user the most characters, to prevent them from getting burnt.
I think in many cases, the normalized form is more permissive, as it puts "character plus diacritic" together into one character. In a language with lots of diacritics, the number of codepoints might be more than the number of normalized characters (depending on the client). You wouldn't want to allow (say) ~70 Korean characters on one OS, and 140 on another, just because they use different codepoints to represent certain characters - one with character then diacritic (2 codepoints?), another with both crammed together in one codepoint.
But as I said, I'm not a unicode guru (and I don't know much about Korean, I just saw it as an example). This might be wrong.
Earlier quoted context omitted.
Wait, really? I thought normalization form C was the form where composite characters were always used when possible. Why restrict by the number of codepoints (vs characters) if you're explicitly going to use the form which goes out of its way to use multi-codepoint characters? The only reason I can think of is that they internally use UTF-32, so counting codepoints is more efficient. But I thought they used UTF-8. Ed…
Most likely, because Twitter is a microblog, and 140 character posts is it's thing. I'm sure it had some historical technical limitation to 140 ASCII or maybe 70 utf-8 chars (or something else logical), but they probably had to accomodate people who wanted to use non-English characters in a post and not get a lecture on unicode encoding; and some slightly offensive "so ... people like you only get 70 chars" message.
Earlier quoted context omitted.
You misunderstand. I'm wondering why they allow only 140 codepoints and not characters . Even in Unicode, 1 codepoint != 1 character.
Sorry, my Unicode is a bit weak. I think Twitter should use whichever usually gives the user the most characters, to prevent them from getting burnt. I think in many cases, the normalized form is more permissive, as it puts "character plus diacritic" together into one character. In a language with lots of diacritics, the number of codepoints might be more than the number of normalized characters (depending on the cli…