128-bit storage: are you high?

I'm not sure that the computation is completely to the point -- he starts out asking "what is the _smallest_ storage unit that could fit this many bytes", and then goes on to consider how much energy it would take to do so. But if we are primarily worried about energy usage, we can write it cheaper (at the cost, I guess, of using more mass for the storage medium).

The usual limit that is mentioned for storage is Laundauer's principle (http://en.wikipedia.org/wiki/Landauer%27s_principle), which states that we need to use at least ln(2)kT Joules of energy to write one bit, where k is Boltzmann's constant and T is the temperature of the storage system.

Assuming the best case T=Cosmic Background Temperature=2.7K (although this is optimistic since we will also be heated up by the sun and by the Milky way), this give 8.8e15 J to write 2^128 bits, or 3.6e19 J to write 2^128 512-byte blocks.

That's much less than what Bonswick calculates, and less than the energy to boil the ocean!

Yes, this is theoretically cool and all, but when I hear something like:

"Let's start with the easy one: how do we know it's necessary? Some customers already have datasets on the order of a petabyte, or 250 bytes. Thus the 64-bit capacity limit of 264 bytes is only 14 doublings away."

It raises a huge red flag from a business point of view. In 2004 "some customers" had datasets that would cause issues sometime around 2015 - a problem that ZFS purported to solve (among admittedly much more relavant features). In planning for a future that was possibly ten years away for "some customers" instead of focusing on what was relevant to the larger customer base, Sun managed to continue their slow profitability death march until Oracle finally snatched them up this year.

Something to keep in mind when you decide to add "cool theoretical feature X" or "unparalleled scalability" to your four month old startup...

I lack the ability to downvote you, but this comment is clearly a series of fallacies. You've assumed at least two things I don't see in evidence:

a) That the customers Sun served in 2004 didn't care about anything ten years away. That their time scale for data on disk was less than a decade. And that marketing to them about how ZFS planned for the future was not effective.

b) That building a 128-bit filesystem (as opposed to a 64-bit filesystem) substantially impacted the amount of time it took to engineer the filesystem or impacted the adoption rate by customers. Clearly those are not facts in evidence. Since we know that an integer behaves pretty much the same regardless of whether it's 32, 64, or 128 bits, it's probably safer to assume the opposite.

I take the opposite lesson: I think when planning your startup, a little thought into "are we representing data in a way which we can extend into the future?" is not a terrible idea, especially when the choice is between a 32, 64, or 128 bit integer.

ZFS has been one of the biggest selling points for Solaris over the last 5 years. There are a huge number of places that use Solaris only for ZFS support. For example, the SmugMug people have written about it a number of times.

It can be useful to have a large address space, even if you don't have enough hardware to actually populate the whole thing.

For example, you might split the address space into smaller sections (executable, read only, read/write, shared, etc) which immediately cuts down the space you have. Or you might decide that your method for allocating space is to pick a random part of the address space, and check if it is taken (or perhaps something slightly cleverer). In this case, you will get better performance if there aren't as many collisions.