Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is the theoretical limit of information density?

physics.stackexchange.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: What is the theoretical limit of information density?

#21

Post by smosher »

vinceguidry wrote: post #4
itcmcgrath wrote: post #3

Earlier quoted context omitted.

No, let me paraphrase what you just said. 'If you can store infinite information outside of a system, then you can achieve infinite information density inside the system if you use outside the system as 'context'. I hope that paraphasing makes it clear enough the problem with this line of thinking. You have to include all the information when calculating absolute density. In your case, the context has to be stored so…

If all information has to be accounted for and stored somewhere, and context is part of the information, then you can't store any information without storing all information, everywhere. Because every bit of information exists inside the context of the entire universe.

Keep going, you're about to discover the holographic principle for yourself.

Re: What is the theoretical limit of information density?

#22

Post by sesqu »

vinceguidry wrote: post #2

This assumes that all information is stored physically. Let's say you get a telegram. The telegram itself can contain maybe a couple of paragraphs of text. You might think the bandwidth of the channel is at most a kilobyte or so. But there's plenty of other aspects that can raise the amount of information conveyed. Say you get one saying "Short Dow." Just going by the content, you'd have no idea what it was saying. I…

No.

If you were a stockbroker, that message would contain less information - and you probably would know it already. It's useful information to a stockbroker, but it would be more informative to almost anyone else, who doesn't encounter such messages daily. As another example, the information in the Rosetta stone was diminished - if more accessible - to its carvers, and a 700 MB disk holds 700 MB regardless of what those bits are.

What you're referring to is compression. High information density is indistinguishable from noise to any recipient who cannot translate it, which is why compression contests regularly require the extractor to be included in the message.

Re: What is the theoretical limit of information density?

#23

Post by andrewflnr »

Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

So I take my cubic meter and divide into 100^3=10^6 1cm^3 segments, and store 10^66 bits in each one. This brings me to 10^72 bits in the 1m^3 volume. Does something stop me from doing this? Or take it the other direction, store 10^66 bits in 10^6 little cubes and just stack them up. What gives?

Re: What is the theoretical limit of information density?

#24

Post by ars »

Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

Does your object have to be spherical? (Is it because of the black hole thing?)

A real 3D object has an infinite surface area anyway (just like a real 2D object has an infinite perimeter/circumference). https://en.wikipedia.org/wiki/Coastline_paradox

So if your computer is made out of atoms then it has an infinite information capacity.

Re: What is the theoretical limit of information density?

#25

Post by bjeanes »

ars wrote: post #24
Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

Does your object have to be spherical? (Is it because of the black hole thing?) A real 3D object has an infinite surface area anyway (just like a real 2D object has an infinite perimeter/circumference). https://en.wikipedia.org/wiki/Coastline_paradox So if your computer is made out of atoms then it has an infinite information capacity.

To quote from that very article:

> However, this figure [infinite perimeter] relies on the idea that space can be subdivided indefinitely. This fiction—which underlies Euclidean geometry and serves as a useful model in everyday measurement—almost certainly does not reflect the changing realities of 'space' and 'distance' on the atomic level.

Re: What is the theoretical limit of information density?

#26

Post by salmonellaeater »

itcmcgrath wrote: post #3
vinceguidry wrote: post #2

This assumes that all information is stored physically. Let's say you get a telegram. The telegram itself can contain maybe a couple of paragraphs of text. You might think the bandwidth of the channel is at most a kilobyte or so. But there's plenty of other aspects that can raise the amount of information conveyed. Say you get one saying "Short Dow." Just going by the content, you'd have no idea what it was saying. I…

No, let me paraphrase what you just said. 'If you can store infinite information outside of a system, then you can achieve infinite information density inside the system if you use outside the system as 'context'. I hope that paraphasing makes it clear enough the problem with this line of thinking. You have to include all the information when calculating absolute density. In your case, the context has to be stored so…

It's worse than that. There are a finite number of states of a message, and you can only reference that number of states of the outside system. If you think of the message as a pointer containing n bits, then you can only reference the first 2^n positions in the dictionary.

If the outside system is a probability distribution over output messages (a more general case of the dictionary you described) then the problem is synonymous with compression.

Re: What is the theoretical limit of information density?

#27

Post by ars »

bjeanes wrote: post #25
ars wrote: post #24

Earlier quoted context omitted.

Does your object have to be spherical? (Is it because of the black hole thing?) A real 3D object has an infinite surface area anyway (just like a real 2D object has an infinite perimeter/circumference). https://en.wikipedia.org/wiki/Coastline_paradox So if your computer is made out of atoms then it has an infinite information capacity.

To quote from that very article: > However, this figure [infinite perimeter] relies on the idea that space can be subdivided indefinitely. This fiction—which underlies Euclidean geometry and serves as a useful model in everyday measurement—almost certainly does not reflect the changing realities of 'space' and 'distance' on the atomic level.

I saw that, but it's not really accurate.

A subatomic particle has a physical size of zero (the "size" of a particle is essentially the range of the forces it participates in, but the spatial size is zero), so depending on how you measure it (i.e. which forces) you can get an area that really is infinite.

Re: What is the theoretical limit of information density?

#29

Post by maaku »

MildlySerious wrote: post #28

So the theories here assume qubits are the smallest unit of storage. I'm not into physics, so how's the evidence they are really the smallest units possible to store information as? After all, atoms were said to be the smallest thing at some point.

Sure, if you postulate new physics anything is possible.
Post reply on HN