Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did base64 win against uuencode?

retrocomputing.stackexchange.com

Post reply on HN
Print view View original

61–70 of 110 posts

Re: Why did base64 win against uuencode?

#61

Post by IshKebab »

Aardwolf wrote: post #38

A thing I wonder: why is using = padding required in the most common base64 variant? It's redundant since this info can be fully inferred from the length of the stream. Even for concatenations it is not necessary to require it, since you must still know the length of each sub stream (and = does not always appear so is not a separator). There's no way that using the = instead of per-byte length-checking gains any spee…

Perhaps to simplify implementations that read multiple characters at a time?

But I think it's likely just poor design taste.

Re: Why did base64 win against uuencode?

#62

Post by whoopdedo »

38 wrote: post #45

https://wikipedia.org/wiki/Binary-to-text_encoding

Not listed was a clever encoding for MS-DOS files, XXBUG[1]. DOS had a rudimentary debugger and memory editor. (It even stuck around all the way to Windows XP but didn't survive the transition to 64-bit.) Because it had the ability to write to disk you could convert any file to hexadecimal bytes and sprinkle some control commands about to create a script for DEBUG.EXE. The text-encoded file could then be sent anywhere without needing to download a decoder program first.

[1] http://justsolve.archiveteam.org/wiki/XXBUG

Re: Why did base64 win against uuencode?

#63

Post by OskarS »

jjeaff wrote: post #29
ekidd wrote: post #24

One reason that uuencode lost out to Base64 was that uuencode used spaces in its encoding. It was fairly common for Internet protocols in those days to mess with whitespace, so it was often necessary to patch up corrupted uuencode files by hand. Base64, on the other hand, was carefully designed to survive everything from whitespace corruption to being passed through non-ASCII character sets. And then it became widely…

and yet, Internet protocols (http, at least) don't play well with equal signs which are part of base64, sometimes. That little issue has caused lots of intermittent bugs for me over the years, either from forgetting to urlencode it or not urldecoding it at the right time.

And slashes as well, which is a magic character in both urls and file systems. Means you can't reliably use normal base64 for filenames, for instance. That might seem like a niche use-case, but it's really not, because you can use it for content-based addressing. Git does this, names all the blobs in the .git folder after their hash, but you can't encode the hash with regular base64.

Re: Why did base64 win against uuencode?

#64

Post by teo_zero »

Aardwolf wrote: post #38

A thing I wonder: why is using = padding required in the most common base64 variant? It's redundant since this info can be fully inferred from the length of the stream. Even for concatenations it is not necessary to require it, since you must still know the length of each sub stream (and = does not always appear so is not a separator). There's no way that using the = instead of per-byte length-checking gains any spee…

Without padding, how would you encode, for example, a message with just a single zero? To be more precise, how do you distinguish it from two zeroes and three zeroes?

Re: Why did base64 win against uuencode?

#65

Post by [deleted] »

jibal wrote: post #59
t-3 wrote: post #2

Base64 is very bizarre in general. Why did they use such a weird pattern of symbols instead of a contiguous section, or at least segments ordered from low->high (on that note, ASCII is also quite strange, I'm guessing due to some backwards compatibility idiocy that seemed like it made sense at some point (or maybe changing case was super important to a lot of workloads or something, making a compelling reason to fuck…

Base64 and ASCII both made perfect sense in terms of their requirements, and the future, while not fully anticipated at the time, is doing just fine, with ASCII being now incorporated into largely future-proof UTF-8. Considerably stranger in regard to contiguity was EBCDIC, but it too made sense in terms of its technological requirements, which centered around Hollerith punch cards. https://en.wikipedia.org/wiki/EBCD…

[deleted]

Re: Why did base64 win against uuencode?

#66

Post by BiteCode_dev »

eastbound wrote: post #57
jjeaff wrote: post #29

Earlier quoted context omitted.

and yet, Internet protocols (http, at least) don't play well with equal signs which are part of base64, sometimes. That little issue has caused lots of intermittent bugs for me over the years, either from forgetting to urlencode it or not urldecoding it at the right time.

So there are 7 base64 encodings, one with “+ / =“, one with “- _ =“, one with “+,” and no “=“… https://en.wikipedia.org/wiki/Base64#Variants_summary_table

TIL.

And Python uses RFC 4648

Re: Why did base64 win against uuencode?

#67

Post by eesmith »

mikecoles wrote: post #5
eesmith wrote: post #3

Earlier quoted context omitted.

The comments point out conversion issues with EBCDIC. You can't use ASCII characters like @ which are not in EBCDIC. https://datatracker.ietf.org/doc/html/rfc2045#section-6.8 says: This subset has the important property that it is represented identically in all versions of ISO 646, including US-ASCII, and all characters in the subset are also represented identically in all versions of EBCDIC. Other popular encodings,…

I thought the ASCII upper-case lower-case being a bit operation as being clever.

Yes, though in principle you could interleave AaBbCc and so on, which would also be a single bit difference, and the naive collation would be more like that people expect.

The design considerations at https://ia800606.us.archive.org/17/items/enf-ascii-1972-1975... show that 6-bit support was more important than naive collation support:

> A6.4 It is expected that devices having the capability of printing only 64 graphic symbols will continue to be important. It may be desirable to arrange these devices to print one symbol for the bit pattern of both upper and lower case of a given alphabetic letter. To facilitate this, there should be a single-bit difference between the upper and lower case representations of any given letter. Combined with the requirement that a given case of the alphabet be contiguous, this dictated the assignment of the alphabet, as shown in columns 4 through 7.

I just found and skimmed Bob Bemer's "A Story of ASCII", which includes personal recollections of the history. It seems that the 6-bit subset was firmed up first. From https://archive.org/details/ascii-bemer/page/n17/mode/2up?q=... :

> This is reflected in the set I proposed to X3 on 1961 September 18 (Table 3, column 3), and these three characters remained in the set from that time on. The lower case alphabet was also shown, but for some time this was resisted, lest the communications people find a need for more than the two columns then allocated for control functions.

but serious discussion of lower case wasn't taken up until later. From https://archive.org/details/ascii-bemer/page/n25/mode/2up?q=... :

> ISO/TC97/SC2 held its next meeting in 1963 October, at which time it was decided to add the lower case alphabet.

and at https://archive.org/details/ascii-bemer/page/n27/mode/2up?q=... :

> At the 1963 May meeting in Geneva, CCITT endorsed the principle of the 7-bit code for any new telegraph alphabet, and expressed general but preliminary agreement with the ISO work. It further requested the placement of the lower case alphabet in the unassigned area.

Bemer did not like interleaving lower- and upper-case. From https://archive.org/details/ascii-bemer/page/n5/mode/2up?q=l... :

> I had a great opportunity to start on the standards road when invited by Dr. Werner Buchholz to do the main design of the 120-character set [9,24] for the Stretch computer (the IBM 7030). I had help, but the mistakes are all mine (such as the interspersal of the upper and lower case alphabets). ...

> he didn't make the same mistake I made for STRETCH by interspersing both cases of the alphabet!

Re: Why did base64 win against uuencode?

#68

Post by 4gotunameagain »

Aardwolf wrote: post #38

A thing I wonder: why is using = padding required in the most common base64 variant? It's redundant since this info can be fully inferred from the length of the stream. Even for concatenations it is not necessary to require it, since you must still know the length of each sub stream (and = does not always appear so is not a separator). There's no way that using the = instead of per-byte length-checking gains any spee…

from Wikipedia:

  The padding character is not essential for decoding, since the number of missing bytes can be inferred from the length of the encoded text. In some implementations, the padding character is mandatory, while for others it is not used. An exception in which padding characters are required is when multiple Base64 encoded files have been concatenated.

Re: Why did base64 win against uuencode?

#69

Post by 4gotunameagain »

teo_zero wrote: post #64
Aardwolf wrote: post #38

A thing I wonder: why is using = padding required in the most common base64 variant? It's redundant since this info can be fully inferred from the length of the stream. Even for concatenations it is not necessary to require it, since you must still know the length of each sub stream (and = does not always appear so is not a separator). There's no way that using the = instead of per-byte length-checking gains any spee…

Without padding, how would you encode, for example, a message with just a single zero? To be more precise, how do you distinguish it from two zeroes and three zeroes?

The output padding is only relevant for decoding. For encoding, since the alphabet of Base64 is 6 bits wide, the padding is 0 when the input is not a multiple of 6 (e.g. encoding two bytes (16 bits) needs two more bits to become a multiple of 6 (18))

Refer to the "examples" section of the wikipedia page

Re: Why did base64 win against uuencode?

#70

Post by t-3 »

jibal wrote: post #59
t-3 wrote: post #2

Base64 is very bizarre in general. Why did they use such a weird pattern of symbols instead of a contiguous section, or at least segments ordered from low->high (on that note, ASCII is also quite strange, I'm guessing due to some backwards compatibility idiocy that seemed like it made sense at some point (or maybe changing case was super important to a lot of workloads or something, making a compelling reason to fuck…

Base64 and ASCII both made perfect sense in terms of their requirements, and the future, while not fully anticipated at the time, is doing just fine, with ASCII being now incorporated into largely future-proof UTF-8. Considerably stranger in regard to contiguity was EBCDIC, but it too made sense in terms of its technological requirements, which centered around Hollerith punch cards. https://en.wikipedia.org/wiki/EBCD…

I never questioned the competence of past engineers, I question the use of backwards compatibility.

Hardware has advanced, but software depends on standards and conventions formulated for far less capable hardware, and that's a problem.

The efficiency of string processing/generation is hugely important in terms of global energy consumption.

A simple and extremely common int->hex string conversion takes twice as many instructions as it would if ASCII was optimized for computability.

Bounds-checking for the English alphabet requires either an upfront normalization or twice the checking, so 50-100% more instructions for that.

There are also inconsistencies like front and back braces/(angle)brackets/parens not being convertible like the alphabet is.

[({ >})] would have been just as or more useful than the alphabet being convertible and saved a few instructions in common parsing loops.

Post reply on HN