Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Base-122 – A space efficient alternative to base-64

blog.kevinalbs.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Base-122 – A space efficient alternative to base-64

#31

Post by bascule »

poizan42 wrote: post #5

A lot of people have experimented with a lot of different ways of encoding binary data as printable text. Wikipedia has a list of different encoding schemes[0]. The most efficient one is yEnc[1]. Still the simplest ones such as base64 or good old hex may actually work better once compression comes into the picture. [0]: https://en.wikipedia.org/wiki/Binary-to-text_encoding [1]: https://en.wikipedia.org/wiki/YEnc

Came here to post this... yEnc is about as compact as you can get, and has been around for over a decade.

Besides that, the Z85 encoding is the next runner up as a compact "string safe" encoding: https://rfc.zeromq.org/spec:32/Z85/

Re: Base-122 – A space efficient alternative to base-64

#32

Post by jws »

userbinator wrote: post #8

The use of codepoints below 32 (space, start of what's usually considered "printable") makes me a bit hesitant. A lot of systems won't preserve those characters. Base85 is a more efficient alternative to base64, and doesn't use that lower range: https://en.wikipedia.org/wiki/Ascii85

base 85 also has the interesting property that 4 original bytes fit in 5 encoded bytes. Depending on your processor's memory model and the cost of multiplies compared to shifts this can make it the best performer.

This was true on Vax 8200 hardware back in the day. In the same software, with Huffman decoding of JPEGs it was also fastest to create a finite state machine with an 8 bit symbol size. I suspect that is no longer true since it would kill your L1 cache and be well into your L2 cache on modern x86 machines. It is probably better to take the instruction count hit and process as bits or nibbles.

Re: Base-122 – A space efficient alternative to base-64

#35

Post by deckar01 »

kobeya wrote: post #28
CiPHPerCoder wrote: post #4

My problem with base-122 is simply that it's not an even power of 2. It's very easy to write a cache-timing-safe version of base{16,32,64} encoding for use in encoding/decoding cryptographic keys in configuration files. To wit: https://github.com/paragonie/constant_time_encoding Base-122? Not sure if it's even possible.

I'm pretty sure that Bitcoin has dealt with this issue for its base58 encoding. It might be worth checking if their algorithm is generalizable to other radix sizes.

Check out digit-array if you are interested in the generalized algorithm. The source code is commented with formal math notation for the operations.

https://github.com/deckar01/digit-array/blob/master/README.m...

Re: Base-122 – A space efficient alternative to base-64

#37

Post by daenney »

Does anyone else find it odd that both

Base-122 was created with the web in mind.

And

As §3 shows, base-122 is not recommended to be used on gzip compressed pages, which is the majority of served web pages.

Occur in just a few lines from each other? I get there's more use cases like email and such but if you're going to create something for the web but it can't be used on the majority of web pages that seems like a fairly large oversight/caveat.

Re: Base-122 – A space efficient alternative to base-64

#38

Post by zackmorris »

http://blog.kevinalbs.com/base122#a_minor_note_on_the_last_c...

Almost a perfect standard, but the prepended one byte header is a mistake IMHO. It makes it impossible to encode when the input size is unknown. Better to encode whether the last chunk is one byte or two at the end of the stream.

Please whoever is involved with this, revise the standard to not have a header and call this existing spec a beta. Otherwise, good work.

Edit: I have opened an issue for this: https://github.com/kevinAlbs/Base122/issues/3#issue-19188159...

Post reply on HN