Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to fit any dataset with a single parameter

arxiv.org

Post reply on HN
Print view View original

61–70 of 155 posts

Re: How to fit any dataset with a single parameter

#61

Post by rnhmjoj »

tsimionescu wrote: post #51
mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

This is not just a false idea, but an obviously false one, contradicted by all the laws of physics. If you were right, then any finite volume would contain an infinite amount of information, which would mean it has infinite entropy, temperature, and energy. Also, by the same logic you apply to space, you could say that time is infinitely divisible, so you could create a computer which finishes an infinite amount of s…

There's an interesting paper[1] that argues that real numbers aren't physical, for the precise reasons you stated. That is, only the subset of real numbers that contain a finite amount of information (like constructive real numbers) are physically meaningful.

A consequence of this is that classical physics is not really deterministic: this is because, in general (ie. including chaotic systems), the evolution of a system depends on a set of initial conditions that are specified by full real numbers, impossible to measure with finite precision. So, the use of real numbers is hiding the indeterminism in the initial conditions, much like the function in this article encodes a dataset in a single parameter.

[1]: https://arxiv.org/abs/1803.06824

Re: How to fit any dataset with a single parameter

#62

Post by tbabb »

mjburgess wrote: post #22
air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

I think there are at least two things wrong with this take:

One is that I don't think it follows from the premise that the continuity of the physical world precludes AI, brain scanning, etc. Even if the physical world were continuous (likely not, see below), an arbitrary degree of approximation could be attained, in principle. At the very least I would not call the footing "firm".

The second is that the universe is very likely not continuous anyway. The Beckenstein bound[1] puts an upper limit on the number of bits of information a region of space may contain. If the ruler tickmark were either measured or localized to the precision required to encode the information, the information density would cause it to collapse into a black hole. This would happen once your measurement needs to be about as precise as a Planck length, which would allow you to encode about 115 bits of information with your tickmark.

(This in of itself is independent of the fact that you would need to construct the ruler out of objects that the universe permits; your ruler tickmark would need to be made of and measured with discrete fundamental particles, which by their very nature are quantized).

[1] https://en.wikipedia.org/wiki/Bekenstein_bound

Re: How to fit any dataset with a single parameter

#64

Post by tsimionescu »

mjburgess wrote: post #55
tsimionescu wrote: post #51

Earlier quoted context omitted.

This is not just a false idea, but an obviously false one, contradicted by all the laws of physics. If you were right, then any finite volume would contain an infinite amount of information, which would mean it has infinite entropy, temperature, and energy. Also, by the same logic you apply to space, you could say that time is infinitely divisible, so you could create a computer which finishes an infinite amount of s…

information here, ie log of a probability, is a continuous notion -- it is real-valued, as not least, log is a transcendental fn -- i specifically said with a /discrete/ conception , geometry is infinitely dense (of discrete states)

In all physical theories, any finite system has a finite number of distinguishable states. So it is not infinitely informationally dense, especially when working with discrete bits of information.

Not to mention, the finer the distinctions between two states of a system, the more energy you need to distinguish them. So, the less impact these differences can have, unless the system is extraordinarily energetic (and even then, you end up in fundamental limits of energy per volume, like the Schwarzschild radius).

So again, there is no sense in which a finite part of the universe is universally dense.

Even worse for your argument, all currently known laws of physics use computable functions (the randomness in QM notwithstanding). So, by definition, all known laws of physics can be simulated by an ideal Turing machine (again, give or take some randomness in QM, depending on the interpretation you chose to believe in and on how you chose to simulate the QM system).

Re: How to fit any dataset with a single parameter

#66

Post by gnramires »

cs702 wrote: post #23

Fun read. But if we allow ourselves as much precision as we need, we don't even need a parameter. Any constant that is a normal number should suffice. Such constants already contain every possible sequence of digits you could muster -- i.e., they already contain every possible dataset. EDIT: I replaced "transcendental" with "normal" after reading Scarblac's comment below: https://news.ycombinator.com/item?id=28699622…

Right, but any constant doesn't "encode" this information. To use a normal constant to encode information, you need to encode the location of the substring of interest. Generally, the location of the substring of interest needs to require as many bits as the substring itself (unless there's an intimate relationship between the number and the substrings of interest). So arguably it's the location that encodes the data, the number is irrelevant (and why not encode the data directly into this location?).

Re: How to fit any dataset with a single parameter

#67

Post by tsimionescu »

dlivingston wrote: post #37
mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

Even if space and time were continuous (which things like the Planck length would discredit), there are still discrete objects in that continuum. Elementary particles, for example, are discrete. You could argue that they have continuous effects vis a vis the EM field and spatial positioning, but ensemble effects usually render that irrelevant at large enough scales.

I will note that even in QM, both space and time are considered continuous - the Planck length is just a smallest measurable distance, but nothing in QM currently assumes that particles must be separated by an integer multiple of the Planck length (unlike spins, for example).

I believe there are some theories of quantum gravity that do rely on the idea that space-time is quantized in integer multiples of Planck's length, but these are far from definitive theories.

A much more relevant limit in terms of possible information density is Heisenberg's uncertainty principle, which essentially puts a limit on the maximum possible precision for any measurement.

Re: How to fit any dataset with a single parameter

#68

Post by Kranar »

yongjik wrote: post #60
mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

In theory, pi has infinite digits. You could publish a book of a trillion digits of pi, and you have barely scratched the surface: in fact you published a precisely 0.00000% of all digits of pi. In practice, you "only" need ~42 digits of pi to draw a circle spanning the entire known universe (diameter of 8.8 * 10^26 m) and it will deviate from the ideal circle by less than the size of a proton (0.8 * 10^−15 m). Havin…

Every number has infinite digits or can be made to have infinite digits for a given representation, but that's not the same as a number having an infinite amount of information. PI represents a finite amount of information, as opposed to say a number like Chaitin's constant which represents an infinite and irreducible amount of information:

https://en.wikipedia.org/wiki/Chaitin%27s_constant

Re: How to fit any dataset with a single parameter

#69

Post by tsimionescu »

mjburgess wrote: post #56
gnramires wrote: post #52

Earlier quoted context omitted.

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

the issue isn't brains no algorithm running on a cpu can move a muscle -- it is precisely that movement is a spatiotemporal property which means no turing machine can realise it movement isn't a symbolic operation

This is such a strange claim to make, I can't even tell what you could have meant. We are surrounded by simplistic mechanical computations moving muscles , and have been for more than a hundred years, since the industrial revolution at least.
Post reply on HN