Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to fit any dataset with a single parameter

arxiv.org

Post reply on HN
Print view View original

71–80 of 155 posts

Re: How to fit any dataset with a single parameter

#71

Post by Nition »

pveierland wrote: post #45
air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

My shot at the calculation for fun :-) Stick encoding with graphite resolution (0.335 * 10^-9 meter) [1]: "Uti" (31 bits -> 3 UTF8 characters) Stick encoding with Planck resolution (1.616255 * 10^-35 meter) [2]: "Utility ought " (115 bits -> 14 UTF8 characters) Complete first sentence: "Utility ought to be the principal intention of every publication." [3] It appears that this storage scheme may not be suited towards…

"Simply" scale up the stick to your desired minimum level of precision!

Re: How to fit any dataset with a single parameter

#73

Post by oever »

tbabb wrote: post #62
mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

I think there are at least two things wrong with this take: One is that I don't think it follows from the premise that the continuity of the physical world precludes AI, brain scanning, etc. Even if the physical world were continuous (likely not, see below), an arbitrary degree of approximation could be attained, in principle. At the very least I would not call the footing "firm". The second is that the universe is v…

Or use a larger stick. Every doubling of the stick length gives another bit of information. A stick of one light-year length would add about 53 bits.

Re: How to fit any dataset with a single parameter

#74

Post by tsimionescu »

rnhmjoj wrote: post #61
tsimionescu wrote: post #51

Earlier quoted context omitted.

This is not just a false idea, but an obviously false one, contradicted by all the laws of physics. If you were right, then any finite volume would contain an infinite amount of information, which would mean it has infinite entropy, temperature, and energy. Also, by the same logic you apply to space, you could say that time is infinitely divisible, so you could create a computer which finishes an infinite amount of s…

There's an interesting paper[1] that argues that real numbers aren't physical, for the precise reasons you stated. That is, only the subset of real numbers that contain a finite amount of information (like constructive real numbers) are physically meaningful. A consequence of this is that classical physics is not really deterministic: this is because, in general (ie. including chaotic systems), the evolution of a sys…

I think the precise definition of what exactly is physically meaningful is more interesting a bit, because pi is just as 'real' as 1 in some sense. You can't any more precisely measure 1m than pi meters. A perfect square doesn't exist any more or less precisely than a perfect circle.

Basically, the universe can just as well be approximated in two different ways. One, it's all straight lines, and anything that looks like a circle/curve is actually a piece of a veeery many sided polygon if you look closely enough at it. Two, it's all curves of some curvature, and anything that looks like a straight line is actually a segment of a veeery large circle. The first perspective corresponds to taking the rational numbers as physical. The second one corresponds to taking pi as physical (and then it's unclear whether e is also physical, or which other transcendental numbers are, but 1 is definitely un-physical then).

Probably the constructive framework is the best way to express this concept, just starting from whatever constant we chose to define as the unit.

Re: How to fit any dataset with a single parameter

#75

Post by tbabb »

oever wrote: post #73
tbabb wrote: post #62

Earlier quoted context omitted.

I think there are at least two things wrong with this take: One is that I don't think it follows from the premise that the continuity of the physical world precludes AI, brain scanning, etc. Even if the physical world were continuous (likely not, see below), an arbitrary degree of approximation could be attained, in principle. At the very least I would not call the footing "firm". The second is that the universe is v…

Or use a larger stick. Every doubling of the stick length gives another bit of information. A stick of one light-year length would add about 53 bits.

The main point I'm making is that the information density of the physical world is not unlimited as GP suggests.

But I think that example just shows how few bits you can really get out the exercise!

Re: How to fit any dataset with a single parameter

#76

Post by genewitch »

throwawaytemp27 wrote: post #53
michaelcampbell wrote: post #19

This reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_…

I wonder if you gave away your phone number there? Or how many 7 digit numbers match those criteria?

7 digits is barely a phone number in most of the United States anymore, most places outside of rural areas require 10 digit dialing and hide it behind carrier dial plans - I mean in the way a rural person can dial one, two, or three digits because all land lines in an area have the same area code, prefix, and possibly additional digits in common.

Re: How to fit any dataset with a single parameter

#77

Post by mr_toad »

bflesch wrote: post #7

This sounds like it could explain many ML models ;)

It’s actually possible in theory for part of a neural net to simply encode the training data and use it to regurgitate responses. In practice, more efficient parts of the network should win in the end.

Re: How to fit any dataset with a single parameter

#78

Post by rnhmjoj »

tsimionescu wrote: post #74
rnhmjoj wrote: post #61

Earlier quoted context omitted.

There's an interesting paper[1] that argues that real numbers aren't physical, for the precise reasons you stated. That is, only the subset of real numbers that contain a finite amount of information (like constructive real numbers) are physically meaningful. A consequence of this is that classical physics is not really deterministic: this is because, in general (ie. including chaotic systems), the evolution of a sys…

I think the precise definition of what exactly is physically meaningful is more interesting a bit, because pi is just as 'real' as 1 in some sense. You can't any more precisely measure 1m than pi meters. A perfect square doesn't exist any more or less precisely than a perfect circle. Basically, the universe can just as well be approximated in two different ways. One, it's all straight lines, and anything that looks l…

Note that π, like practically any constant or number normally found in mathemathics is constructive and contains a finite amount of information. What the author consider unphysical are the non-constructive reals: unfathomable numbers that can't be defined or written in any finite amount of words or paper. It's hard to argue they play some role in our physical world: it would basically amount to metaphysics.

However, even if rarely used outside of proofs, they are the vast majority of the reals, in fact there are only countably infinite constructive/computable numbers.

Re: How to fit any dataset with a single parameter

#79

Post by YeGoblynQueenne »

throwawaytemp27 wrote: post #53
michaelcampbell wrote: post #19

This reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_…

I wonder if you gave away your phone number there? Or how many 7 digit numbers match those criteria?

There can't be a better way to get a date with a number theory nerd.

Re: How to fit any dataset with a single parameter

#80

Post by analog31 »

cs702 wrote: post #23

Fun read. But if we allow ourselves as much precision as we need, we don't even need a parameter. Any constant that is a normal number should suffice. Such constants already contain every possible sequence of digits you could muster -- i.e., they already contain every possible dataset. EDIT: I replaced "transcendental" with "normal" after reading Scarblac's comment below: https://news.ycombinator.com/item?id=28699622…

You could use the digits of pi as an end-of-file marker, which would indeed make it transcendent. ;-)
Post reply on HN