Earlier quoted context omitted.
I've been toying with Zig over the summer, and I'm really impressed by the build system, general ergonomics and the explicitness of the language. The only thing I've found being a pain point is C's biggest pain point: the lack of a proper string type. Even though I've worked for years in the past with C and C++, I still get tripped up, and in Zig I keep on being confused whether I should use raw byte arrays, or senti…
Note that while strings are generally opaque bytes in Zig, there are standard library functions[0] to work with them as Unicode codepoints and such. But Unicode is quite large and complicated, and things like grapheme clusters aren't in the standard library (yet?). I also believe that that module is planned to be rewritten. So it's more the case that Zig plans to someday support Unicode at the standard library level,…
In something like Zig I think it's OK that it's merely a stated assumption that these bytes are UTF-8, not actually checked - so long as people take that seriously.
It's nice in code that maybe isn't very concerned with such things to be able to know that any "string" is actually text we can display, output to a console, something like that. Not for example a TCP/IP packet we haven't decoded yet, or the first 32 bytes of a JPEG image.
Programmers who aren't writing firmware for a vacuum cleaner or washing machine, probably want string literals like "this" in their code, and you'd naturally want those to have a type, for which a string type is the obvious fit. I believe in 2021 this type should obviously be UTF-8. "It's just some bytes" is far from useless, but it isn't much of a string type.
I was sceptical at first about Rust's choice to build in str (immutable UTF-8 string slices), because that seems like a relatively high level concept. But unlike std::string::String, str isn't that tricky after all. See, the only place such things would come from (not having std::string::String to make new ones) is the program source, and our compiler is necessarily already reading the program source, so it follows that the compiler must know how the source is encoded etc and thus it does actually know exactly what those literal strings are. If your literal wasn't Unicode, that wasn't a Rust program and it won't compile.