What learning algorithms can predict that our physics theories might not
firstestprinciple.com
What learning algorithms can predict that our physics theories might not
1–10 of 35 posts
Re: What learning algorithms can predict that our physics theories might not
#2Question about this paragraph:
> Is it possible to define a process that Solomonoff induction cannot predict? The short answer is yes, but the kinds of computers needed to simulate these processes don’t exist in the real world, and it’s unlikely that we’ll ever be able to build them.
Doesn't "pick a truly random number" qualify? And we can build those today.
Re: What learning algorithms can predict that our physics theories might not
#3Question about this paragraph: > Is it possible to define a process that Solomonoff induction cannot predict? The short answer is yes, but the kinds of computers needed to simulate these processes don’t exist in the real world, and it’s unlikely that we’ll ever be able to build them. Doesn't "pick a truly random number" qualify? And we can build those today.
When I wrote "predict" there, I was referring to whether the predicted probabilities approach the true underlying probabilities, and Solomonoff induction can "predict" a true random number generator in that sense because its predicted probabilities will approach those of the random number generator. [1] However, if you tried to use it to predict a halting oracle, the halting oracle would be deterministic but Solomonoff induction would never be able to predict it with complete confidence, and this is what I was referring to in that paragraph.
But you're right that random numbers are inherently unpredictable; maybe I should add another footnote explaining what I meant there. (Edit: I added a clarification to the paragraph you quoted.)
[1] http://twistedoakstudios.com/blog/Post5623_solomonoffs-mad-s... in the "Thinking with Programs: Random Data" section
Re: What learning algorithms can predict that our physics theories might not
#4This is a very interesting article. Scott Aaronson touches on some of these ideas related to quantum cloning and the concept of "you" in his blog posts.
I've always thought this kind of concept might define the limits of standard science, as currently practiced. Science requires reproducibility. But by whom? Well, other scientists of course. If every scientist tries your experiment and gets the same result, then you have a validated scientific theory.
But suppose that you manage to set up an experiment where the perception of which measurement resulted depends upon who is perceiving the result (I can think of a few ways that this scenario might arise if we could ever figure out a way to generate macroscopic, human-scale, superpositions [which is unlikely, I'll add]). That would really throw a wrench in things. In that case, you would have to have each scientist convincingly prove to each other scientist that they all see something different, in which case perhaps the result could still be universally accepted. But there may be a limit on how much consensus we can ultimately get.
Re: What learning algorithms can predict that our physics theories might not
#5Question about this paragraph: > Is it possible to define a process that Solomonoff induction cannot predict? The short answer is yes, but the kinds of computers needed to simulate these processes don’t exist in the real world, and it’s unlikely that we’ll ever be able to build them. Doesn't "pick a truly random number" qualify? And we can build those today.
If you feed biased random bits into Solomonoff induction, the shortest surviving programs at any given point would tend to be things like arithmetic encoders that match the bias and specify just a bit more output. (Assuming you're not using pseudo-random numbers, of course.) So it will at least start to predict probabilities with the correct bias.
If you're feeding in unbiased independent random bits, any and every process is already an optimal predictor.
Re: What learning algorithms can predict that our physics theories might not
#6I wasn't expecting this article to be about Sleeping Beauty problems and Solomonoff induction. There are people trying to apply real machine learning algorithms to real physics problems (e.g. [1][2]). This article is not about that.
It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically. Brute forcing the space of all programs is ridiculously mindbogglingly universe-crushingly expensive. (Actually, even if the process was magically tractable, there would still be limitations and dangers [3].)
1: http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.108...
Re: What learning algorithms can predict that our physics theories might not
#7> What if instead of connecting just 2 people’s brains together, we connected everyone’s brains like that to the internet? Would that mean that every human being on Earth would feel like they are just one small body part of a single, greater being?
I don't think that joining everyone's brains together would make everyone feel like a small body part of a single, greater being, because our brain architecture really wouldn't support that. You might be able to handle some shared "input" with another person or a small group of people (like these conjoined twins who share a thalamus[0]), but you're going to run into bandwidth issues pretty quickly given that there are only ~1-2 million nerves in each optic nerve[1]--if you're trying to split that 7 billion ways, you're going to have a difficult time getting coherent information through, let alone processing it.
The second major limitation to joining brains together is the speed of light--once we're able to open up communication between brains to allow "communicating via thoughts", we'll be communicating at the speed of our thoughts, which is much faster than physical speech. Connecting your brain with the brain of someone on the other side of the world might be a pretty disappointing experience because they wouldn't be nearly as responsive as someone physically nearby. Uploading brains and running them at higher "clock speeds" than biological hardware permits would make this limitation more significant, because you might subjectively experience a communication time lag that would feel like hours, days, or longer when connected to someone far away. In other words, there would be a limiting radius in physical reality for effective brain-connecting communication that varied depending on the speed of your subjective experience.
Those limitations aside, sign me up! Brain-AI merging and brain-brain communication are going to be the bees knees.
[0]: https://en.wikipedia.org/wiki/Krista_and_Tatiana_Hogan#Progr...
Re: What learning algorithms can predict that our physics theories might not
#8I wasn't expecting this article to be about Sleeping Beauty problems and Solomonoff induction. There are people trying to apply real machine learning algorithms to real physics problems (e.g. [1][2]). This article is not about that. It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically. Brute forcing the space of all programs is ridiculously…
Could Solomonoff induction guess sequences of prime-numbers?
Re: What learning algorithms can predict that our physics theories might not
#9I wasn't expecting this article to be about Sleeping Beauty problems and Solomonoff induction. There are people trying to apply real machine learning algorithms to real physics problems (e.g. [1][2]). This article is not about that. It should go without saying that Solomonoff induction is totally useless for practical applications, if interesting theoretically. Brute forcing the space of all programs is ridiculously…
Could Solomonoff induction guess sequences of prime-numbers?
The output of a program could be infinite and thus it never halts. Without Chatlin's constant or the Busy Beaver values, brute forcing is not feasible in a countably computable universe. It is still interesting to talk about Oracles, ie. Somehow getting hold of Chatlin's constant and thereby easily solving the halting problem and being able to use the induction.
Re: What learning algorithms can predict that our physics theories might not
#10Question about this paragraph: > Is it possible to define a process that Solomonoff induction cannot predict? The short answer is yes, but the kinds of computers needed to simulate these processes don’t exist in the real world, and it’s unlikely that we’ll ever be able to build them. Doesn't "pick a truly random number" qualify? And we can build those today.
When I wrote "predict" there, I was referring to whether the predicted probabilities approach the true underlying probabilities, and Solomonoff induction can "predict" a true random number generator in that sense because its predicted probabilities will approach those of the random number generator. [1] However, if you tried to use it to predict a halting oracle, the halting oracle would be deterministic but Solomono…
One of the principles of turing machines is that they are deterministic. In that vain, there exists no programmable RNG except for pseudo random ones. One has to ask oneself if piping a transform of the digits of a transcendental real number is a violation to this rule -- thus what is random really? Is random the lack of ability to find a correlation or program to reproduce it -- or is it something more like Komogrolov complexity? These are tough and inscrutable questions. Shannon, Turing, Curry, Church, Post, and others explored them deeply. Information theory gets extremely existential and esoteric. Is randomness a monad of our universe? Or is physicals and natural chaos just extremely leathery when it comes to extracting the generating program? Our lives depend on it. But either way, we'll carry on. Nature you goddamn enigma.