How to fit any dataset with a single parameter • Hacker News Board

Re: How to fit any dataset with a single parameter

#51

Post by tsimionescu » Wed, Sep 29, 2021, 9:15 PM UTC

mjburgess wrote: post #22

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

This is not just a false idea, but an obviously false one, contradicted by all the laws of physics. If you were right, then any finite volume would contain an infinite amount of information, which would mean it has infinite entropy, temperature, and energy.

Also, by the same logic you apply to space, you could say that time is infinitely divisible, so you could create a computer which finishes an infinite amount of steps in a finite amount of time.

Re: How to fit any dataset with a single parameter

#52

Post by gnramires » Wed, Sep 29, 2021, 9:19 PM UTC

mjburgess wrote: post #22

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience).

But that doesn't mean brains aren't special -- it means brains are special and computers are special. Even more: it seems to imply computers, AI, etc. can be as special as ourselves, sentient, and perhaps even more special in ways we haven't realized yet.

It's difficult to even imagine a physical theory with unbounded local information. It seems to open the possibility to crazy things like hypercomputation, which do not seem very well defined. (For example: every 1-1/n seconds, (n>1) from now, flip a switch ON/OFF. At what state will the switch be at t>1s? An at exactly t=1s?)

Note: while information and information flow itself bounded (hence no hypercomputation), I don't know of any obvious objections to continuous time. (I'm not sure the continuity of time has any profound implications)

Re: How to fit any dataset with a single parameter

#53

Post by throwawaytemp27 » Wed, Sep 29, 2021, 9:24 PM UTC

michaelcampbell wrote: post #19

This reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_…

I wonder if you gave away your phone number there? Or how many 7 digit numbers match those criteria?

Re: How to fit any dataset with a single parameter

#54

Post by alittlesalami » Wed, Sep 29, 2021, 9:25 PM UTC

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

This is Arithmetic Coding: https://en.wikipedia.org/wiki/Arithmetic_coding

Re: How to fit any dataset with a single parameter

#55

Post by mjburgess » Wed, Sep 29, 2021, 9:29 PM UTC

tsimionescu wrote: post #51

mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

This is not just a false idea, but an obviously false one, contradicted by all the laws of physics. If you were right, then any finite volume would contain an infinite amount of information, which would mean it has infinite entropy, temperature, and energy. Also, by the same logic you apply to space, you could say that time is infinitely divisible, so you could create a computer which finishes an infinite amount of s…

information here, ie log of a probability, is a continuous notion -- it is real-valued, as not least, log is a transcendental fn --

i specifically said with a /discrete/ conception , geometry is infinitely dense (of discrete states)

Re: How to fit any dataset with a single parameter

#56

Post by mjburgess » Wed, Sep 29, 2021, 9:30 PM UTC

gnramires wrote: post #52

mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

the issue isn't brains

no algorithm running on a cpu can move a muscle --

it is precisely that movement is a spatiotemporal property which means no turing machine can realise it

movement isn't a symbolic operation

Re: How to fit any dataset with a single parameter

#57

Post by [deleted] » Wed, Sep 29, 2021, 9:32 PM UTC

mjburgess wrote: post #22

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

[deleted]

Re: How to fit any dataset with a single parameter

#58

Post by ohazi » Wed, Sep 29, 2021, 9:33 PM UTC

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

You jest, but this is almost exactly how arithmetic coding works.

If you arrange your symbols and contexts carefully, you can even use this as a technique for progressive or lossy compression -- i.e. the more accurately you specify the ratio, the higher fidelity your result.

Re: How to fit any dataset with a single parameter

#59

Post by gnramires » Wed, Sep 29, 2021, 9:36 PM UTC

mjburgess wrote: post #56

gnramires wrote: post #52

Earlier quoted context omitted.

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

the issue isn't brains no algorithm running on a cpu can move a muscle -- it is precisely that movement is a spatiotemporal property which means no turing machine can realise it movement isn't a symbolic operation

What do you suppose those computers are doing, then? ;)

https://www.youtube.com/watch?v=tF4DML7FIWk

https://www.youtube.com/watch?v=tfb6aEUMC04&t=115s

Re: How to fit any dataset with a single parameter

#60

Post by yongjik » Wed, Sep 29, 2021, 9:37 PM UTC

mjburgess wrote: post #22

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

In theory, pi has infinite digits. You could publish a book of a trillion digits of pi, and you have barely scratched the surface: in fact you published a precisely 0.00000% of all digits of pi.

In practice, you "only" need ~42 digits of pi to draw a circle spanning the entire known universe (diameter of 8.8 * 10^26 m) and it will deviate from the ideal circle by less than the size of a proton (0.8 * 10^−15 m).

Having a theoretically infinite precision does not mean that it makes a measurable difference.