Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to fit any dataset with a single parameter

arxiv.org

Post reply on HN
Print view View original

121–130 of 155 posts

Re: How to fit any dataset with a single parameter

#121

Post by vidarh »

mjburgess wrote: post #22
air7 wrote: post #17

This reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...

I don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computation…

Depends how you define simulation. There's no reason to assume that you need, or even that it's desirable, to simulate more than "almost nothing" in order to simulate what you'd want to simulate - even a whole universe can be simulated with ease if your required fidelity is low enough.

So, if you want to simulate every possible interaction in realtime, sure. But you can increase overall capabilities by making sacrifices along several axes: duration, speed, precision, persistence of changes from a generative baseline and lots more.

In other words it depends entirely of your purpose for simulation.

Re: How to fit any dataset with a single parameter

#122

Post by eru »

goldenkey wrote: post #112
eru wrote: post #111

Earlier quoted context omitted.

Yes, Chaos arises in Newtonian systems just fine. They are non-linear. But Quantum Mechanics is linear. See eg https://physics.stackexchange.com/questions/33344/is-the-uni... for a discussion, and https://www.scottaaronson.com/papers/island.pdf for a longer treatment.

QM being linear doesn't make the universe linear. Schrodinger's equation doesn't work across measurements (or universes.) And if you try to weasel out of that by saying we will describe the multiverse instead, you have just created the unexplainable phenomena of being in a specific universe, with no way to accurately describe the process behind the random choice of the universe you reside in.

Measurements (and 'universes') are part of interpretations of QM. Not sure they are properly part of QM. I see what you mean though.

I'm not sure whether eg the Born rule is enough by itself to explain how (non-linear) chaos arises from linear QM.

Re: How to fit any dataset with a single parameter

#123

Post by DougBTX »

mjburgess wrote: post #119
drdeca wrote: post #94

Earlier quoted context omitted.

I agree that we can’t reject the hypothesis that reality is continuous (and, even if spacetime turns out to be discrete, it seems hard to imagine that the amplitudes of the wavefunction(s) have finitely many possible complex values, though I suppose we can’t rule it out) I disagree that this necessarily implies any difficulty for the possibility of brain scanning and AI. Just as things sampled faster than the nyquist…

Argument: 1) Intelligence requires being spatiotemproally acting-on and being acted-on by a spatiotemporally dynamic environment. 2) Dynamical spa'temp properties of the body enable (1) 3) These properties are continuous features of the bodily (eg., organic plasticity) 4) continuous properties are irreducible to measurably discrete ones 5) computers are systems which we build that have measurably discrete systems the…

It seems at point 3 that organic plasticity requires features smaller than a Planck length, is that right?

Re: How to fit any dataset with a single parameter

#124

Post by shoto_io »

shmageggy wrote: post #70

Prior work: "One parameter is always enough", Piantadosi (2018) https://www.google.com/search?q=one%20parameter%20piantadosi Haven't read the posted article but sounds like the same idea and motivation

It is the first reference in this article. So I guess they have built on it.

Re: How to fit any dataset with a single parameter

#125

Post by amelius »

cs702 wrote: post #34
Scarblac wrote: post #27

Earlier quoted context omitted.

I don't believe just being transcendental is sufficient for that property. E.g. I can't imagine that pi with all the '1' digits replaced by '2' isn't transcendental, but it clearly doesn't contain every sequence of digits. I think you mean normal numbers.

You're right : normal, not transcendental. IIRC, most real numbers are normal, and many important transcendental numbers are thought to be (but have not been proven to be) normal. I edited my comment. Thank you for the correction!

> most real numbers are normal

Which means that most real numbers violate all present and future copyrights.

Re: How to fit any dataset with a single parameter

#126

Post by mjburgess »

DougBTX wrote: post #123
mjburgess wrote: post #119

Earlier quoted context omitted.

Argument: 1) Intelligence requires being spatiotemproally acting-on and being acted-on by a spatiotemporally dynamic environment. 2) Dynamical spa'temp properties of the body enable (1) 3) These properties are continuous features of the bodily (eg., organic plasticity) 4) continuous properties are irreducible to measurably discrete ones 5) computers are systems which we build that have measurably discrete systems the…

It seems at point 3 that organic plasticity requires features smaller than a Planck length, is that right?

Whatever the length of the smallest discrete unit of spacetime, say, L -- then continuous spatial and temporal properties are those which have a state density O(1/L^2).

This may seem weird, and indeed, it's far less weird if you just say "continuous".

But here's an intuition: spatio-temporal continuity is "scale-free" in the sense that stuff happening at the sub-proton is affected by stuff happening at the galactic.

Thus reality has to be able to "zoom" from the sub-proton to the galactic.

In the case of organic plasticity, I do think that macroscopic effects which are whole-body distributed (including, eg., thoughts) have to drive protein expression at the sub-sub-celluar.

Consider simulating that with a low-state discrete computer: it is many orders of magnitude more data than a planet-sized computer could store and many more years than the lifetime of the universe (consider the number of molecules to store, and their interaction effects from whole-body down).

Running operations at anything in the nanoseconds makes this simulation impossible. It simply does need to be much closer to O(1/L^2).

Re: How to fit any dataset with a single parameter

#127

Post by adrianN »

mjburgess wrote: post #119
drdeca wrote: post #94

Earlier quoted context omitted.

I agree that we can’t reject the hypothesis that reality is continuous (and, even if spacetime turns out to be discrete, it seems hard to imagine that the amplitudes of the wavefunction(s) have finitely many possible complex values, though I suppose we can’t rule it out) I disagree that this necessarily implies any difficulty for the possibility of brain scanning and AI. Just as things sampled faster than the nyquist…

Argument: 1) Intelligence requires being spatiotemproally acting-on and being acted-on by a spatiotemporally dynamic environment. 2) Dynamical spa'temp properties of the body enable (1) 3) These properties are continuous features of the bodily (eg., organic plasticity) 4) continuous properties are irreducible to measurably discrete ones 5) computers are systems which we build that have measurably discrete systems the…

If meat is magical, just build computers out of meat then.

Re: How to fit any dataset with a single parameter

#128

Post by PeterisP »

vokep wrote: post #110
meroes wrote: post #105

Earlier quoted context omitted.

> that's what consciousness is How do you go from computation to feeling a toothache? It's like telling me a cucumber is really a Porsche (but worse).

Who says neural networks aren't in pain when their fitness (reward function) is subtracted from? or happy or orgasmic when the function is bumped up? Pain/pleasure seem to have easy enough analogue to at least something like a reward function, but what really gets me is colors. I feel like if honestly considered, the word "color" is all that's necessary to disprove materialism. Color is. How? Dunno.

Perceptions and feelings are real as you both experience them and could measure their signals in the brain, if you would have the proper tech to do that. And filtered, processed perceptions exist as well. In a "pipeline" in some brain - no matter if biological or artificial - somewhere in the middle you have processed signals of sensor input that have a complex meaning that has only some relation to the actual physical world "seen" by the sensors. Things like color and motion seem one of those; for an intuitive understanding it seems very close to the distill.pub analyses of what some specific middle-neurons in a convnet see.

Re: How to fit any dataset with a single parameter

#129

Post by WithinReason »

mjburgess wrote: post #119
drdeca wrote: post #94

Earlier quoted context omitted.

I agree that we can’t reject the hypothesis that reality is continuous (and, even if spacetime turns out to be discrete, it seems hard to imagine that the amplitudes of the wavefunction(s) have finitely many possible complex values, though I suppose we can’t rule it out) I disagree that this necessarily implies any difficulty for the possibility of brain scanning and AI. Just as things sampled faster than the nyquist…

Argument: 1) Intelligence requires being spatiotemproally acting-on and being acted-on by a spatiotemporally dynamic environment. 2) Dynamical spa'temp properties of the body enable (1) 3) These properties are continuous features of the bodily (eg., organic plasticity) 4) continuous properties are irreducible to measurably discrete ones 5) computers are systems which we build that have measurably discrete systems the…

Simulation doesn't mean exact simulation. The value of a continuous property is noise after a certain decimal and can be disregarded for the purposes of simulation. I also believe that 1) is false.

Re: How to fit any dataset with a single parameter

#130

Post by amelius »

hnlmorg wrote: post #18
idiotsecant wrote: post #9

First, this is a fun implementation and I love it. Second, you could as easily embed an infinite size dataset into an infinitely long binary string and say that you've reproduced your dataset with a 'single' parameter! That's sort of what this is doing, with some extra steps.

This reminds me of the fun thought experiment of using /dev/random as storage. Given an infinite amount of time you'll find every file you need.

The index might need more bits than the file.
Post reply on HN