Live data from Hacker News

What learning algorithms can predict that our physics theories might not

firstestprinciple.com

11–20 of 35 posts

Re: What learning algorithms can predict that our physics theories might not

#11

This is a very interesting article. Scott Aaronson touches on some of these ideas related to quantum cloning and the concept of "you" in his blog posts. I've always thought this kind of concept might define the limits of standard science, as currently practiced. Science requires reproducibility. But by whom? Well, other scientists of course. If every scientist tries your experiment and gets the same result, then you…

> But suppose that you manage to set up an experiment where the perception of which measurement resulted depends upon who is perceiving the result

One way that I think you could prove it to others is to have a way to predict the measurement depending on who's perceiving it, that is >=60% effective. This would consist of some form of proof to others at least.

Re: What learning algorithms can predict that our physics theories might not

#13
post #8
post #6

I wasn't expecting this article to be about Sleeping Beauty problems and Solomonoff induction. There are people trying to apply real machine learning algorithms to real physics problems (e.g. [1][2]). This article is not about that. It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically. Brute forcing the space of all programs is ridiculously…

Could Solomonoff induction guess sequences of prime-numbers?

You can think of Solomonoff induction as a weighted set of experts. At each step, each expert makes a prediction, and the overall prediction is the one that gets the highest total weight. Then the true data comes in, and the weight of each expert is increased or decreased based on whether that expert was right or wrong. And here's the kicker, the experts are all possible prediction programs, initially weighted by 2^-(program length). So yeah, in the long run Solomonoff induction will be at least as good as you at predicting any particular sequence, including prime numbers etc., because your own prediction algorithm is somewhere in the Solomonoff mixture. That also explains why approximating Solomonoff induction takes a huge amount of computation time. It's mostly a theoretical idea.

Re: What learning algorithms can predict that our physics theories might not

#15
post #8

Earlier quoted context omitted.

Could Solomonoff induction guess sequences of prime-numbers?

The output of a program could be infinite and thus it never halts. Without Chatlin's constant or the Busy Beaver values, brute forcing is not feasible in a countably computable universe. It is still interesting to talk about Oracles, ie. Somehow getting hold of Chatlin's constant and thereby easily solving the halting problem and being able to use the induction.

Solomonoff Induction starts by using Kolmogorov complexity to calculate its prior distribution, which already requires Chaitin's constant to be known.

So yeah.

Re: What learning algorithms can predict that our physics theories might not

#16
Well, I must say that I'm pleasantly surprised with the comments so far. I was actually expecting that this would be quickly shot down as either unoriginal or fundamentally flawed. But instead it seems no one has caught on to what this post is actually claiming, so I suppose I should now be very blunt about it.

At the beginning of the blog post, it claims that it explains 2 things:

1. where exactly might you be able to use learning algorithms where you can't just use existing physics theories instead 2. a hands on guide to applying learning algorithms in these situations

This is physicist code for "this blog post claims that it solves a major unsolved problem in physics." Let me explain.

Currently, we have the standard model and general relativity, which have been experimentally verified to extreme precision but are fundamentally incompatible with each other. So people have proposed theories of everything such as string theory, loop quantum gravity, and information/digital physics (which I'm obviously a fan of) to resolve these incompatibilities.

One of the biggest problems in fundamental physics right now is that the standard model and general relativity have been verified to such precision that it's hard to think of a practical experiment to show how they are wrong. The conventional wisdom is that this is only possible if we do things like measure the Planck scale or what happens inside a black hole, which are completely impractical on human timescales.

What this post proposes is that you actually don't need to measure the Planck scale or what happens in a black hole in order to test the proposed theories of everything, and instead you can do it with a sufficiently powerful computer simulation and a sufficiently good brain-computer interface. If our technology keeps improving exponentially, this may be possible in the next several decades.

So yeah, I made a bit of a white lie when I framed this post as a summary of recent research in information physics. I can back up almost everything in the post with the sources I linked to, but the part about the 0 or 1 experiment and predicting its outcome using Solomonoff induction is actually original research on my part, and I suspect it would actually be a very big deal if this works the way I think it does.

So here are the possible outcomes for this blog post:

1. The problem in physics I just described is actually already solved. 2. The blog post is fundamentally flawed, and/or it actually doesn't solve the problem that I'm claiming it solves. 3. The blog post actually does solve a major unsolved problem in physics, and this is a huge deal.

This is why I am so surprised at the comments I'm getting so far, since this proposal for experimentally testing theories of everything seems to be passing the internet commenter test. So if no one on HN finds anything seriously wrong with the blog post, can we get people like Scott Aaronson, John Baez, Juergen Schmidhuber, Stephen Hawking, or people of that caliber to look at it so we can get a more definitive answer to whether this actually solves an unsolved problem in physics?

Also, kudos to Xcelerate's comment, which is the closest to the point I was trying to get at with the blog post.

Re: What learning algorithms can predict that our physics theories might not

#17
post #16

Well, I must say that I'm pleasantly surprised with the comments so far. I was actually expecting that this would be quickly shot down as either unoriginal or fundamentally flawed. But instead it seems no one has caught on to what this post is actually claiming, so I suppose I should now be very blunt about it. At the beginning of the blog post, it claims that it explains 2 things: 1. where exactly might you be able…

[deleted]

Re: What learning algorithms can predict that our physics theories might not

#18
post #16

Well, I must say that I'm pleasantly surprised with the comments so far. I was actually expecting that this would be quickly shot down as either unoriginal or fundamentally flawed. But instead it seems no one has caught on to what this post is actually claiming, so I suppose I should now be very blunt about it. At the beginning of the blog post, it claims that it explains 2 things: 1. where exactly might you be able…

It is not clear to me that you realize that Solomonoff induction is a mathematical argument, not a practical algorithm. To run it at the level of generality necessary to discover the laws of physics is computationally infeasible. In fact, it's one of those cases where calling it "computationally infeasible" is an inadvertent understatement of the problem, because English doesn't have gradations for this level of difficulty. Merely a "singularity" doesn't help this problem; you need more computation than our physics appears to allow.

Re: What learning algorithms can predict that our physics theories might not

#19
post #18
post #16

Well, I must say that I'm pleasantly surprised with the comments so far. I was actually expecting that this would be quickly shot down as either unoriginal or fundamentally flawed. But instead it seems no one has caught on to what this post is actually claiming, so I suppose I should now be very blunt about it. At the beginning of the blog post, it claims that it explains 2 things: 1. where exactly might you be able…

It is not clear to me that you realize that Solomonoff induction is a mathematical argument, not a practical algorithm. To run it at the level of generality necessary to discover the laws of physics is computationally infeasible. In fact, it's one of those cases where calling it "computationally infeasible" is an inadvertent understatement of the problem, because English doesn't have gradations for this level of diff…

Yes, I know that Solomonoff induction is completely impractical for real life machine learning. My point was that if you can survive in the simulation to the point where you see either the 0 or the 1, we don't have any way even in theory (let alone in practice) to guess the probabilities of seeing a 0 or 1, unless you use some sort of learning algorithm. You can use any learning algorithm for this; it doesn't have to be Solomonoff induction.

Re: What learning algorithms can predict that our physics theories might not

#20
post #19
post #18

Earlier quoted context omitted.

It is not clear to me that you realize that Solomonoff induction is a mathematical argument, not a practical algorithm. To run it at the level of generality necessary to discover the laws of physics is computationally infeasible. In fact, it's one of those cases where calling it "computationally infeasible" is an inadvertent understatement of the problem, because English doesn't have gradations for this level of diff…

Yes, I know that Solomonoff induction is completely impractical for real life machine learning. My point was that if you can survive in the simulation to the point where you see either the 0 or the 1, we don't have any way even in theory (let alone in practice) to guess the probabilities of seeing a 0 or 1, unless you use some sort of learning algorithm. You can use any learning algorithm for this; it doesn't have to…

But your argument seems to fundamentally rest on Solomonoff induction. Put any real algorithm in there, and now you need to ensure that 1. the biases of the algorithm encompass a hypothesis that matches the data and 2. the algorithm will be able to arrive at that hypothesis given a real data stream, and, ideally, a real amount of computation. Both of these are hard questions, in the strongest sense of the term.

And once you open that door, well, all you've really done is restate the fact that learning how the universe works seems to be really difficult.

Post reply on HN