151–160 of 247 posts
Yes it's a pain, but the way the standard library designed its types force you to handle conversions correctly, for example when byte arrays are converted to UTF-8 strings and may contain invalid UTF-8 sequences.
https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
> Reverses a string. Technically, this function reverses the codepoints in a string and its main utility is for reversed-order string processing [...]. See also [...] `graphemes` from module Unicode to operate on user-visible "characters" (graphemes) rather than codepoints.
Properly reversing a string of flags (or any other grapheme clusters) is just a `using Unicode: grapheme` away.
Earlier quoted context omitted.
Exactly this. Humans have incredibly complicated writing systems, and all Unicode wants to do is encode them all. Keep in mind that the trivial toy system we're more familiar with, ASCII, already has some pretty strange features because even to half-arse one human writing system they needed those features. Case is totally wild, it only applies to like 5% of the symbols in ASCII, but in the process it means they each…
>Humans have incredibly complicated writing systems Not only that, there isn't even agreement about what's correct all the time! >it doesn't understand how to spell a bunch of common English words like naïve or café, pretty disappointing. A perfect example of this, since I would argue English doesn't have any diacritics at all. So the use of café is code switching. :)
So, in terms of acing interviews, increasingly one of the best answers to the question "Write some code that reverses a string" is that in a world of unicode, "reversing a string" is no longer possible or meaningful. You'll probably be told "oh, assume US ASCII" or something, but in the meantime, if you can back that up when they dig into it, you'll look really smart.
If you think the Unicode flag emoji take a lot of bytes, then consider the family emoji! ( https://unicode.org/emoji/charts/full-emoji-list.html#family ) I'm in the process of designing a scripting language and implementing it in C++. I plan to put together a YouTube series about it. (Doesn't everyone want to see Bison and Flex mixed with proper unit tests and C++20 code?) Due to my future intended use case, I needed…
Isn't it about time that we have some common language that every other language builds on?
Earlier quoted context omitted.
> It always feels like the most amount of work goes to the least used emoji. I always feel like those emoji were added on purpose in order to force implementations to fix their unicode support. Before emoji were added, most software had completely broken support for anything beyond the BMP (case study: MySQL's so-called "UTF8" encoding). The introduction of emoji, and their immediate popularity, forced many systems t…
WTF business do emojis have in Unicode? The BMP is all there ever should have been. Standardize the actual writing systems of the world, so everyone can write in their language. And once that is done, the standard doesn't need to change for a hundred years. What we need now is a standardized, sane subset of Unicode that implementations can support while rejecting the insane scope creep that got added on top of that.…
Unicode didn't invent emoji, they incorporated it because they were already popular in Japan, and if they didn't incorporate it, it would greatly reduce Japanese adoption.
Keep in mind that Unicode was intended to unify all the disparate encodings that had been brewed up to support different languages and which made exchanging documents between non-English speaking countries a nightmare. The term "mojibake" comes to mind [0] - Japan alone had so many encodings that a slang term for text encoded with something different than what your device expected (and subsequently got rendered as nonsensical/garbled text) came about. And they weren't alone, of course [1].
> What we need now is a standardized, sane subset of Unicode that implementations can support while rejecting the insane scope creep that got added on top of that.
Unicode wasn't intended to be pretty. It was intended to be the one system that everyone used, and a way to increase adoption was to do some less than ideal things, like duplicate characters (so it would be easier to convert to Unicode).
You may never need anything outside the BMP, but that doesn't make the rest of the planes worthless. Ignoring the value of including dead and nearing-extinct languages for preservation purposes (not being able to type a language will basically guarantee its extinction, with inventing a new encoding and storing text as jpgs being the only real alternatives), there are a lot of people speaking languages found in the SMP [2][3] ([2] has 83 million native speakers, for example).
[0]: https://en.wikipedia.org/wiki/Mojibake
[1]: https://segfault.kiev.ua/cyrillic-encodings/
Earlier quoted context omitted.
Exactly this. Humans have incredibly complicated writing systems, and all Unicode wants to do is encode them all. Keep in mind that the trivial toy system we're more familiar with, ASCII, already has some pretty strange features because even to half-arse one human writing system they needed those features. Case is totally wild, it only applies to like 5% of the symbols in ASCII, but in the process it means they each…
>Humans have incredibly complicated writing systems Not only that, there isn't even agreement about what's correct all the time! >it doesn't understand how to spell a bunch of common English words like naïve or café, pretty disappointing. A perfect example of this, since I would argue English doesn't have any diacritics at all. So the use of café is code switching. :)
If you think the Unicode flag emoji take a lot of bytes, then consider the family emoji! ( https://unicode.org/emoji/charts/full-emoji-list.html#family ) I'm in the process of designing a scripting language and implementing it in C++. I plan to put together a YouTube series about it. (Doesn't everyone want to see Bison and Flex mixed with proper unit tests and C++20 code?) Due to my future intended use case, I needed…
Why is this stuff even reinvented for every programming language? Isn't it about time that we have some common language that every other language builds on?
Earlier quoted context omitted.
WTF business do emojis have in Unicode? The BMP is all there ever should have been. Standardize the actual writing systems of the world, so everyone can write in their language. And once that is done, the standard doesn't need to change for a hundred years. What we need now is a standardized, sane subset of Unicode that implementations can support while rejecting the insane scope creep that got added on top of that.…
This argument was lost the moment Unicode was created. Japanese carriers had created their own standard for emoji encoding for sms. And they would not switch to Unicode unless the emoji were ported over. It’s a tricky situation. Maybe allowing an arbitrary bitmap char to represent any emoji would have been better but then we could have ended up in a situation where normal text or meaningful punctuation or perhaps eve…
Because emoji became characters, text rendering and font formats had to be extended to support them. There are four different ways to encode emoji in OpenType 1.8:
* Apple uses embedded PNG
* Google uses embedded colour bitmaps
* Microsoft uses flat glyphs in different colours layered on top of one-another
* Adobe and Mozilla use embedded SVG.