How to fit any dataset with a single parameter • Hacker News Board

Re: How to fit any dataset with a single parameter

#101

Post by eru » Thu, Sep 30, 2021, 2:43 AM UTC

robbedpeter wrote: post #82

mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

Planck length and the speed of light define the limits. It's also a resolution of Xeno's Paradox.

I thought Xeno's classic Paradoxes were pretty well dealt with by limits already?

Re: How to fit any dataset with a single parameter

#102

Post by eru » Thu, Sep 30, 2021, 2:43 AM UTC

hammock wrote: post #84

mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

What you are saying is P/=NP

Huh? What does that have to do with anything here?

Re: How to fit any dataset with a single parameter

#103

Post by eru » Thu, Sep 30, 2021, 2:44 AM UTC

albertzeyer wrote: post #43

michaelcampbell wrote: post #19

This reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_…

I think it is still not proven that Pi contains all possible strings. https://math.stackexchange.com/questions/216343/does-pi-cont...

But it contains all possible phone numbers.

(Assuming that phone numbers are less than, say, 100 digits.)

Re: How to fit any dataset with a single parameter

#104

Post by still_grokking » Thu, Sep 30, 2021, 2:53 AM UTC

mjburgess wrote: post #22

air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

Only if real numbers are a "real" thing.

Re: How to fit any dataset with a single parameter

#105

Post by meroes » Thu, Sep 30, 2021, 3:15 AM UTC

gnramires wrote: post #52

mjburgess wrote: post #22

Earlier quoted context omitted.

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

> that's what consciousness is

How do you go from computation to feeling a toothache?

It's like telling me a cucumber is really a Porsche (but worse).

Re: How to fit any dataset with a single parameter

#106

Post by snek_case » Thu, Sep 30, 2021, 3:22 AM UTC

Nition wrote: post #71

pveierland wrote: post #45

Earlier quoted context omitted.

My shot at the calculation for fun :-) Stick encoding with graphite resolution (0.335 * 10^-9 meter) [1]: "Uti" (31 bits -> 3 UTF8 characters) Stick encoding with Planck resolution (1.616255 * 10^-35 meter) [2]: "Utility ought " (115 bits -> 14 UTF8 characters) Complete first sentence: "Utility ought to be the principal intention of every publication." [3] It appears that this storage scheme may not be suited towards…

"Simply" scale up the stick to your desired minimum level of precision!

You might have to wait for the universe to expand a bit more.

Re: How to fit any dataset with a single parameter

#107

Post by kirubakaran » Thu, Sep 30, 2021, 3:24 AM UTC

hnlmorg wrote: post #18

idiotsecant wrote: post #9

First, this is a fun implementation and I love it. Second, you could as easily embed an infinite size dataset into an infinitely long binary string and say that you've reproduced your dataset with a 'single' parameter! That's sort of what this is doing, with some extra steps.

This reminds me of the fun thought experiment of using /dev/random as storage. Given an infinite amount of time you'll find every file you need.

Infinite storage, horrendous seek time.

Re: How to fit any dataset with a single parameter

#108

Post by londons_explore » Thu, Sep 30, 2021, 3:45 AM UTC

throwawaytemp27 wrote: post #53

michaelcampbell wrote: post #19

This reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_…

I wonder if you gave away your phone number there? Or how many 7 digit numbers match those criteria?

Doubt it. OP effectively have us 5+4 bits of entropy, whereas a 7 digit phone number probably has 23 bits.

Re: How to fit any dataset with a single parameter

#109

Post by istjohn » Thu, Sep 30, 2021, 4:02 AM UTC

eru wrote: post #98

gnramires wrote: post #52

Earlier quoted context omitted.

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

> It's difficult to even imagine a physical theory with unbounded local information. It seems to open the possibility to crazy things like hypercomputation, which do not seem very well defined. (For example: every 1-1/n seconds, (n>1) from now, flip a switch ON/OFF. At what state will the switch be at t>1s? An at exactly t=1s?) I agree with the latter, but the former? Classical Newtonian mechanics is easy enough to i…

Chaos arises in very simple (Newtonian) systems, e.g. the double pendulum[1].

1. https://en.m.wikipedia.org/wiki/Double_pendulum

Re: How to fit any dataset with a single parameter

#110

Post by vokep » Thu, Sep 30, 2021, 4:06 AM UTC

meroes wrote: post #105

gnramires wrote: post #52

Earlier quoted context omitted.

I used to think there must be something 'special' to brains to distinguish us from computers. There isn't. Brains encode finite amounts of information (quantum mechanics seems to imply bounded local information). We are a huge information network ourselves -- that's what consciousness is (with some added bits like self-identity and various particulars structures that dictate the character of our experience). But that…

> that's what consciousness is How do you go from computation to feeling a toothache? It's like telling me a cucumber is really a Porsche (but worse).

Who says neural networks aren't in pain when their fitness (reward function) is subtracted from? or happy or orgasmic when the function is bumped up?

Pain/pleasure seem to have easy enough analogue to at least something like a reward function, but what really gets me is colors. I feel like if honestly considered, the word "color" is all that's necessary to disprove materialism. Color is. How? Dunno.