Earlier quoted context omitted.
Applications have for the most part proven that they cannot be trusted to get text encoding and decoding right, especially not in any consistent way. Operating systems definitely should make it possible to deal with the raw byte streams, but the default and preferred method of text handling should be a standard higher-level interface.
> Applications have for the most part proven that they cannot be trusted to get text encoding and decoding right, especially not in any consistent way. That's because text encoding and decoding is a mess. Operating systems doing it doesn't make it any less of a mess; it just inserts the mess deeper into everything. For example, look at all the quirks and edge cases in file name handling between different OS's, simply…
Re: Minoca OS: A new open source operating system
#141Regardless of whether operating systems get involved in tasks like re-encoding text, they really should at least carry along the metadata about encodings whenever they're handling bytes that represent strings. Completely ignoring the problem and leaving it up to applications further up the stack just ensures that there will be incompatible competing standards for how to tell applications how to decode the string data they get from the OS. You don't want some apps trying to write filenames in UTF-8 while others use UTF-16, but allowing it to happen silently is even worse.