Live data from Hacker News

Base-122 – A space efficient alternative to base-64

blog.kevinalbs.com

41–50 of 71 posts

Re: Base-122 – A space efficient alternative to base-64

#41
post #34

I'm looking for an efficient encoding that is guaranteed to not contain certain substrings, such as improper language.

Define improper language. Does your improper-language dictionary cover every possible translation of every possible "improper" word?

Re: Base-122 – A space efficient alternative to base-64

#42
post #27

It's an interesting technical exercise, but I don't think this is the right approach for optimizing HTML load times. If the goal is to reduce latency for small images, wouldn't it make it more sense to extend data URIs so the same base64 string can be referenced in multiple places? Actually, as HTTP2 can effectively return multiple resurces in the answer of one request, do we still need embedded images for latency re…

Compression takes care of exactly this.

Unless compression cannot be used: https://news.ycombinator.com/item?id=13049898

Also, there are still cases where compression cannot be applied, e.g. if a script naively queries innerHTML. (This wouldn't affect loading time but it could inflate the page's RAM unnecessarily)

Re: Base-122 – A space efficient alternative to base-64

#43

meanwhile the Unicode consortium has been hard at work since 1991 to make it possible to encode up to 2.8 MB -- more than enough for most images, short videos, or many PDF files -- in a single character.

Are you referring to UTF-8? If so, this is misleading as you can encode up to 2^21 + 2^16 + 2^11 + 2^7 = 2,164,864 code points, which is not the same as encoding bytes in a single character.

Re: Base-122 – A space efficient alternative to base-64

#44
post #42

Earlier quoted context omitted.

Compression takes care of exactly this.

Unless compression cannot be used: https://news.ycombinator.com/item?id=13049898 Also, there are still cases where compression cannot be applied, e.g. if a script naively queries innerHTML. (This wouldn't affect loading time but it could inflate the page's RAM unnecessarily)

Sure, but I'd consider those edge cases to be situations where the treatment is worse than the disease

Re: Base-122 – A space efficient alternative to base-64

#45
post #5

A lot of people have experimented with a lot of different ways of encoding binary data as printable text. Wikipedia has a list of different encoding schemes[0]. The most efficient one is yEnc[1]. Still the simplest ones such as base64 or good old hex may actually work better once compression comes into the picture. [0]: https://en.wikipedia.org/wiki/Binary-to-text_encoding [1]: https://en.wikipedia.org/wiki/YEnc

YEnc is unsuitable when the text will be transfered as UTF-8.

Re: Base-122 – A space efficient alternative to base-64

#46
Berkeley Unix (or was that SunOS?) used to cone with btoa that used base 85, packing 4 bytes into 5 uucp/mail/Usenet safe characters. For some reason, base 64 became the standard, and I don't think there's space for anything else these days. (Most things do just fine with binary now, anyway!)

Re: Base-122 – A space efficient alternative to base-64

#47
post #37

Does anyone else find it odd that both Base-122 was created with the web in mind. And As §3 shows, base-122 is not recommended to be used on gzip compressed pages, which is the majority of served web pages. Occur in just a few lines from each other? I get there's more use cases like email and such but if you're going to create something for the web but it can't be used on the majority of web pages that seems like a f…

I agree. Including:

> Base-122 encoded strings contain characters which did not seem to play well with copy-pasting.

A very important part of web development is being able to manipulate text documents. It seems that using UTF-8 in more places can reveal cracks in implementations for browsers/DE's/editors/terminals/etc.

Re: Base-122 – A space efficient alternative to base-64

#49
post #34

I'm looking for an efficient encoding that is guaranteed to not contain certain substrings, such as improper language.

You can probably get most of what you want by simply not using vowels. A few non-English ones will still get through like knnbccb and there's also shlt and so on....

Re: Base-122 – A space efficient alternative to base-64

#50

The use of codepoints below 32 (space, start of what's usually considered "printable") makes me a bit hesitant. A lot of systems won't preserve those characters. Base85 is a more efficient alternative to base64, and doesn't use that lower range: https://en.wikipedia.org/wiki/Ascii85

I was going to bring up base 85 as well, its a better choice for a variety of reasons. A long time ago I wrote a base encoder class in Java[1] mostly so that we could write a netnews reader in Java but also because I felt UUEncoding was not robust. The challenges of using unprintable characters is a lot more of a headache than anyone pays attention to initially. Lots (and I mean quite a few here) of systems consider unprintable characters "safe" to re-purpose into random uses. One display vendor had them changing the color of future characters in the display as an example.

Stick with the characters that nearly everyone assumes could legitimately come up in a document and your chances of running afoul of some "creative genius" who decided "Hey its unprintable so no one will try to print it, but when I do print it I want this thing to happen..."

[1] http://grepcode.com/file/repository.grepcode.com/java/root/j...

Post reply on HN