Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What learning algorithms can predict that our physics theories might not

firstestprinciple.com

Post reply on HN
Print view View original

21–30 of 35 posts

Re: What learning algorithms can predict that our physics theories might not

#21

Post by ad510 »

jerf wrote: post #20
ad510 wrote: post #19

Earlier quoted context omitted.

Yes, I know that Solomonoff induction is completely impractical for real life machine learning. My point was that if you can survive in the simulation to the point where you see either the 0 or the 1, we don't have any way even in theory (let alone in practice) to guess the probabilities of seeing a 0 or 1, unless you use some sort of learning algorithm. You can use any learning algorithm for this; it doesn't have to…

But your argument seems to fundamentally rest on Solomonoff induction. Put any real algorithm in there, and now you need to ensure that 1. the biases of the algorithm encompass a hypothesis that matches the data and 2. the algorithm will be able to arrive at that hypothesis given a real data stream, and, ideally, a real amount of computation. Both of these are hard questions, in the strongest sense of the term. And o…

OK, I see what you're saying now. In that case, can you think of a better way of predicting whether you see a 0 or 1 in that situation?

Re: What learning algorithms can predict that our physics theories might not

#22

Post by jerf »

ad510 wrote: post #21
jerf wrote: post #20

Earlier quoted context omitted.

But your argument seems to fundamentally rest on Solomonoff induction. Put any real algorithm in there, and now you need to ensure that 1. the biases of the algorithm encompass a hypothesis that matches the data and 2. the algorithm will be able to arrive at that hypothesis given a real data stream, and, ideally, a real amount of computation. Both of these are hard questions, in the strongest sense of the term. And o…

OK, I see what you're saying now. In that case, can you think of a better way of predicting whether you see a 0 or 1 in that situation?

If I had an answer to that question, I probably wouldn't be putting it on HN. :) I'd be firing it at the market and making boodles of moolah.

Re: What learning algorithms can predict that our physics theories might not

#23

Post by eutectic »

cousin_it wrote: post #13
cel1ne wrote: post #8

Earlier quoted context omitted.

Could Solomonoff induction guess sequences of prime-numbers?

You can think of Solomonoff induction as a weighted set of experts. At each step, each expert makes a prediction, and the overall prediction is the one that gets the highest total weight. Then the true data comes in, and the weight of each expert is increased or decreased based on whether that expert was right or wrong. And here's the kicker, the experts are all possible prediction programs, initially weighted by 2^-…

More specifically, every incorrect expert is removed at each step, and the remaining experts have their probabilities uniformly rescaled to sum to 1.

Re: What learning algorithms can predict that our physics theories might not

#24

Post by [deleted] »

cousin_it wrote: post #13
cel1ne wrote: post #8

Earlier quoted context omitted.

Could Solomonoff induction guess sequences of prime-numbers?

You can think of Solomonoff induction as a weighted set of experts. At each step, each expert makes a prediction, and the overall prediction is the one that gets the highest total weight. Then the true data comes in, and the weight of each expert is increased or decreased based on whether that expert was right or wrong. And here's the kicker, the experts are all possible prediction programs, initially weighted by 2^-…

[deleted]

Re: What learning algorithms can predict that our physics theories might not

#25

Post by kordless »

jerf wrote: post #18
ad510 wrote: post #16

Well, I must say that I'm pleasantly surprised with the comments so far. I was actually expecting that this would be quickly shot down as either unoriginal or fundamentally flawed. But instead it seems no one has caught on to what this post is actually claiming, so I suppose I should now be very blunt about it. At the beginning of the blog post, it claims that it explains 2 things: 1. where exactly might you be able…

It is not clear to me that you realize that Solomonoff induction is a mathematical argument, not a practical algorithm. To run it at the level of generality necessary to discover the laws of physics is computationally infeasible. In fact, it's one of those cases where calling it "computationally infeasible" is an inadvertent understatement of the problem, because English doesn't have gradations for this level of diff…

I would note that "computation" is work done over time. Causality.

There may exist an alternate form of causality that isn't time bound, which may be exposed here over short periods of time. I would hesitate to judge it "computationally infeasible" until we know more. :)

Re: What learning algorithms can predict that our physics theories might not

#26

Post by naasking »

Strilanc wrote: post #6

I wasn't expecting this article to be about Sleeping Beauty problems and Solomonoff induction. There are people trying to apply real machine learning algorithms to real physics problems (e.g. [1][2]). This article is not about that. It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically. Brute forcing the space of all programs is ridiculously…

> It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically.

Indeed, just like Turing machines and the lambda calculus. Still an important theoretical step though!

Re: What learning algorithms can predict that our physics theories might not

#27

Post by marcosdumay »

Xcelerate wrote: post #4

This is a very interesting article. Scott Aaronson touches on some of these ideas related to quantum cloning and the concept of "you" in his blog posts. I've always thought this kind of concept might define the limits of standard science, as currently practiced. Science requires reproducibility. But by whom? Well, other scientists of course. If every scientist tries your experiment and gets the same result, then you…

Hum... That looks odd because it is. Things do not work this way.

Actually, the reproducibility applies to the hypothesis (conclusion), not to the experiment. It's only that, since the observer is irrelevant, people simplify and call it "reproducing the experiment".

If your experiment depends on the observer, and you get to know that, that dependence will go on the conclusion, and the people reproducing your experiment will expect to see the result your hypothesis say they would, not the same one you got. If you have a correct predicting model, everybody will conclude it's correct.

Re: What learning algorithms can predict that our physics theories might not

#28

Post by Strilanc »

goldenkey wrote: post #9
cel1ne wrote: post #8

Earlier quoted context omitted.

Could Solomonoff induction guess sequences of prime-numbers?

The output of a program could be infinite and thus it never halts. Without Chatlin's constant or the Busy Beaver values, brute forcing is not feasible in a countably computable universe. It is still interesting to talk about Oracles, ie. Somehow getting hold of Chatlin's constant and thereby easily solving the halting problem and being able to use the induction.

These are easily fixed by dove tailing through the programs and using an increasing-over-time cutoff. Not fixed in the "made practical" sense, but in the "made non-contradictory" sense.

Re: What learning algorithms can predict that our physics theories might not

#29

Post by jerf »

kordless wrote: post #25
jerf wrote: post #18

Earlier quoted context omitted.

It is not clear to me that you realize that Solomonoff induction is a mathematical argument, not a practical algorithm. To run it at the level of generality necessary to discover the laws of physics is computationally infeasible. In fact, it's one of those cases where calling it "computationally infeasible" is an inadvertent understatement of the problem, because English doesn't have gradations for this level of diff…

I would note that "computation" is work done over time. Causality. There may exist an alternate form of causality that isn't time bound, which may be exposed here over short periods of time. I would hesitate to judge it "computationally infeasible" until we know more. :)

I think it's important to distinguish arguments that hypothesize that our understanding of physics is fundamentally, deeply flawed, from arguments that are based on our current understanding of physics. I can't prove that our understanding of physics isn't deeply flawed and there isn't some source of infinite computation somehow available to us; for instance, one proposed explanation of the Fermi Paradox is that all civilizations escape to a physics/computation regime more congenial to civilization before colonizing the galaxy. But it's still important to know when we're engaging in flights of fancy vs. speculating based on what we know.

And given that the topic in question is plumbing the depths of physics in the first place, this is perhaps a notch more important than it might otherwise be. How would we discover that physics has an infinite/acausal computation mechanism if we first must use Solomonoff induction to discover that, when we can only afford to use Solomonoff induction to discover that if we harness that computation?

Re: What learning algorithms can predict that our physics theories might not

#30

Post by hacker42 »

eutectic wrote: post #23
cousin_it wrote: post #13

Earlier quoted context omitted.

You can think of Solomonoff induction as a weighted set of experts. At each step, each expert makes a prediction, and the overall prediction is the one that gets the highest total weight. Then the true data comes in, and the weight of each expert is increased or decreased based on whether that expert was right or wrong. And here's the kicker, the experts are all possible prediction programs, initially weighted by 2^-…

More specifically, every incorrect expert is removed at each step, and the remaining experts have their probabilities uniformly rescaled to sum to 1.

That relies on noiseless, unbiased data, right? What if an expert gets ruled out by accident?
Post reply on HN