On “On Asm.js”

Endian-independent, more efficient than UTF-16 for most languages (often including CJK web pages: halved cost of HTML & URLs makes up for 33% extra text cost), supports easy and safe substring search, can detect cut characters, and all that with ASCII and C-string backwards-compatibility.

If I could redesign entire computing platform from scratch UTF-8 is one thing I wouldn't change.

This is a valid point, but hits on something that constantly grates with me. We already have something massively more cross platform and performance focused than JavaScript - C. The problem is that the standard library for C is utterly lacking, and there is a lack of focus on developing mechanisms to deploy native code /directly/ across the web.

Don't kid yourself either - all the data being thrown around at the moment is always (always, always, always otherwise it wouldn't even work!) translated into native code in some way - we are already throwing native code around the web, just in a horribly inefficient way. In todays world of sandboxing and virtualisation all of the security arguments that used to be completely reasonable are quite thoroughly invalidated.

I really do believe that fixing this from the technical perspective is not an impossible or even hard problem to solve - I can not stress this point enough. Game developers are constantly re-solving this problem in limited, optimisation focused ways. JavaScript implementations themselves are ultimately built on top of this technology or other technologies built on top of it. The standard libraries of modern scripting languages are possible to use from within the constraints of C. We have already solved all of these problems and there are myriad examples.

On the other hand, the political problem could be intractable... and thats sad.

Worst of all perhaps, C can be made better for performance without much thought or effort, and none of the new languages I see actually tackle this problem, which is quite real and measurable and impacts everything from productivity to the environment - they (quite rightly in many ways) focus utterly on ease of use and massive standard libraries. Why can't we focus this effort on fixing C, or providing a better alternative?

Why are we trying to catchup with native code performance instead of using just native code? Why aren't we improving native code performance in a serious way?

Another problem is that swapping a character in a string might cause the (byte) length of the string to change, which might force a reallocation (also slow) and if you're doing it in a loop over the string, your loop-ending condition might now have changed because the string (byte) length has changed.

In-fact, iterating over a UTF-8 strings characters is not something you can use a simple for loop any more, but something that requires at least one, possibly two function calls per character (one to find the next character, one for finding the end of the string which might just have moved due to your modification).

Finally, efficiency: For English texts, UTF8 is the most efficient Unicode encoding, but for other languages, that isn't true. A Chinese text would require three to four bytes per character, as opposed to just two in UCS-2 (which is what most OSes and languages use, even though it doesn't support encoding all of Unicode)

For these reasons, dealing with a fixed-length encoding is much more convenient (and speedier) while the string is loaded into memory. UTF8 is great for i/o and storage on disk, but in memory, it's inconvenient.

UCS-2 or UTF-16 is the reverse: it's very inconvenient on disk and for i/o (need I say more than BOM), but in-memory UCS-2 is very convenient, even though it doesn't support all of Unicode. It's in-fact so convenient that it's used by most programming environments (see yesterday's discussion about strings being broken)

Take python 3.3 and later for example: even though they now have full support for Unicode, also for characters outside of the BMP and thus requiring more than two bytes of storage, they didn't go with a variable length in-memory encoding, but they now chose the shortest width possible fixed length encoding that can be used for a particular string.

This seems like an awful lot of work to me, but they decided the fixed-lengthness was still worth it.

C......... isn't.

But things didn't go that way. Instead, each vendor built its own walled garden, with its own platform-specific libraries to do the same things.

The web, in this reality, is simply our attempt at encapsulating away the entirety of the OS, along with all its platform-specific libraries, as a sort of really-big-BIOS, and then building vendor-neutral platform on top with APIs that will work on every computer, running every OS.

But the fun thing is, once we get this web platform nailed down, and it can do everything? Once the OS beneath it is redundant in every respect? We can "molt" the outer layer away.

The future of Operating Systems will be the lineage of today's ChromeOS and FirefoxOS, not Windows or OSX.