Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is the theoretical limit of information density?

physics.stackexchange.com

Post reply on HN
Print view View original

31–39 of 39 posts

Re: What is the theoretical limit of information density?

#31

Post by yk »

Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

Since the Schwarzschild radius is proportional to the mass, it is probably better to note that the maximum entropy of a black hole seems to scale with mass squared, instead of being proportional to the mass as in an ideal gas. But this can sort of be understood by noting that ( using a really badly flawed model of black holes) the minimum energy states are all occupied and to add matter, it needs to sit at a energy proportional to the mass of the black hole. Additionally, arguing about black holes assumes that we can not get any information out of the computer.

Re: What is the theoretical limit of information density?

#32

Post by svantana »

vinceguidry wrote: post #2

This assumes that all information is stored physically. Let's say you get a telegram. The telegram itself can contain maybe a couple of paragraphs of text. You might think the bandwidth of the channel is at most a kilobyte or so. But there's plenty of other aspects that can raise the amount of information conveyed. Say you get one saying "Short Dow." Just going by the content, you'd have no idea what it was saying. I…

On the contrary, for the stockbroker there's only a single bit of information in there - the answer to the yes/no question "should I short Dow?"

Re: What is the theoretical limit of information density?

#33

Post by gibybo »

dizzystar wrote: post #14

Can anyone put this in terms like 100 100 Petabytes or something? Or is the number so large that it really isn't conceivable at this point? I'll admit that I don't understand the problem completely, but isn't this assuming the absolute maximum with little consideration of actual technology limits? How much does it change when we consider the limits of technology and our ability to store it on said technology?

There isn't a friendly name for 10^66 bits, but perhaps you can get an idea for the scale by considering the sizes of more commonly used names:

bit: 10^0, kilobit: 10^3, megabit: 10^6, gigabit: 10^9, terabit: 10^12, ... zettabit: 10^21, yottabit: 10^24

So we're at some pretty ridiculously large numbers and still nowhere near 10^66.

Re: What is the theoretical limit of information density?

#34

Post by powertower »

andrewflnr wrote: post #23
Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

So I take my cubic meter and divide into 100^3=10^6 1cm^3 segments, and store 10^66 bits in each one. This brings me to 10^72 bits in the 1m^3 volume. Does something stop me from doing this? Or take it the other direction, store 10^66 bits in 10^6 little cubes and just stack them up. What gives?

Looks like a glitch in the Matrix to me.

Information contained in a cubed meter should be equal to information contained in 100^3 cubed centimeters (the number of cubed centimeter parts in a cubed meter).

But it's not.

Re: What is the theoretical limit of information density?

#35

Post by Steuard »

Xcelerate wrote: post #16
Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

> This behavior is exceedingly non-intuitive, at least to me. I've been thinking about the notion of the Bekenstein Bound lately, and I'm not really sure how accurate the whole idea is. There seems to be a few problems with it, and I don't get the sense that it is a particurly heavily studied problem in the physics community (compared to something like general relativity or QFT). For one thing, nobody is certain that…

Don't count on finding big misconceptions here: in fact, questions about black hole entropy and the Beckenstein bound are some of the most intensely studied topics in theoretical physics today. Not that we understand the answers! But these issues are very near the heart of questions about quantum gravity.

As for your question about "actual" black holes, I think you may be missing some pieces of the puzzle, and I don't have the time to chat about them at length here. But one important point is that while outside observers may see infalling matter appear to stall out an infinitessimal distance from the black hole horizon, the standard classical or semiclassical result is that an actual infalling observer would pass through the event horizon without noticing anything special about it at all. (For a stellar-sized black hole, they would probably have already been ripped apart by tidal forces at that point ("spaghettification"), but that's unrelated to the horizon. For a truly massive galaxy-sized black hole where tidal forces are small at the horizon, the observer wouldn't notice anything special at all as she crossed the point of no return.) There is at this moment a massive debate raging in the particle physics community about whether quantum gravity effects might lead to a "firewall" at the horizon of a black hole that would change this story, but I don't think the question is settled yet.

As for localization of quantum states and information, again, all I can tell you is that people have been studying those issues a lot, too. The details of entanglement of states inside and outside a black hole are closely tied in with the current "black hole firewalls" debate that I mentioned earlier, for instance. So don't make the mistake of thinking that the physics community has neglected this sort of issue!

Re: What is the theoretical limit of information density?

#36

Post by Steuard »

andrewflnr wrote: post #23
Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

So I take my cubic meter and divide into 100^3=10^6 1cm^3 segments, and store 10^66 bits in each one. This brings me to 10^72 bits in the 1m^3 volume. Does something stop me from doing this? Or take it the other direction, store 10^66 bits in 10^6 little cubes and just stack them up. What gives?

See, that's the fun part. If you try to pack in more information than given by these limits, you inevitably form a black hole (and then you lose access to those individual cm^3 regions, and your plan falls apart).

Re: What is the theoretical limit of information density?

#37

Post by powertower »

Steuard wrote: post #36
andrewflnr wrote: post #23

Earlier quoted context omitted.

So I take my cubic meter and divide into 100^3=10^6 1cm^3 segments, and store 10^66 bits in each one. This brings me to 10^72 bits in the 1m^3 volume. Does something stop me from doing this? Or take it the other direction, store 10^66 bits in 10^6 little cubes and just stack them up. What gives?

See, that's the fun part. If you try to pack in more information than given by these limits, you inevitably form a black hole (and then you lose access to those individual cm^3 regions, and your plan falls apart).

How so? Those 1cc-cubes all have the same exact density, and placing them together does not affect/change the density of the formed object (the 1m-cube).

For what you're saying, they would have to be so massive and dense (in the first place) that they would gravitate-in any matter placed near them (to then form a black hole).

But does that even hold true when you run the numbers to see if that "max-info" 1cc-cube is anywhere near the required mass (given it's volume of 1cc^3) to form a black hole?

Re: What is the theoretical limit of information density?

#38

Post by snowwrestler »

Steuard wrote: post #15

The remarkable thing that this explanation doesn't really explore in depth is that the maximum information (~entropy) contained in a region scales according to its boundary area rather than to its volume. That means that "maximum information per unit volume" drops precipitously with size.[1] Just for example, this allows you to calculate an ultimate limit on Moore's law of only ~800 years, for any possible computer f…

The thing I've always wondered about here is how we distinguish between information density, and our ability to calculate information density. How can we tell the difference between the limits of our mathematics, and the actual limitations of the universe?

Re: What is the theoretical limit of information density?

#39

Post by Steuard »

powertower wrote: post #37
Steuard wrote: post #36

Earlier quoted context omitted.

See, that's the fun part. If you try to pack in more information than given by these limits, you inevitably form a black hole (and then you lose access to those individual cm^3 regions, and your plan falls apart).

How so? Those 1cc-cubes all have the same exact density, and placing them together does not affect/change the density of the formed object (the 1m-cube). For what you're saying, they would have to be so massive and dense (in the first place) that they would gravitate-in any matter placed near them (to then form a black hole). But does that even hold true when you run the numbers to see if that "max-info" 1cc-cube is…

One of the confusing issues here is that there is not a single, constant density necessary for the formation of a black hole. Instead, the Schwarzschild radius of a (potential) black hole is proportional to total mass. That means that as you increase its mass, the actual radius of a sphere of constant density grows only as m^(1/3) while its Schwarzschild grows much faster as m^1.

Thus, as you pile up more and more of your identical 1cc cubes, the size of the pile will grow more slowly than the size of its Schwarzschild radius. As soon as you add enough cubes for the Schwarzschild radius to exceed the actual radius, the system must inevitably form a black hole.

I think that this behavior is directly related to the information density limits that we've been talking about, but certainly the end result is the same: piling up lots of similar stuff in one place will eventually lead to gravitational collapse.

Post reply on HN