[dead]
Markov Chain Monte Carlo Without All the Bullshit (2015)
21–30 of 56 posts
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#22Earlier quoted context omitted.
Your overall point might be correct but your example does not prove your point. A Makrov chain is just the path taken through the course of a markov process. The terms 'chain' and 'process' are sometimes conflated in this context, but this is the most common distinction I've seen. As such you can run a markov process for some number of steps N times, and then ask how many generated chains contain the property you are…
Interesting. I'd not heard the term Markov Chain used to describe a path so I checked my copy of "All of Statistics" by Larry Wasserman and he (slightly) disagrees with both of us and says "A Markov chain is a stochastic process for which the distribution of X_t depends only on X_{t-1}." So he doesn't need it to be a discrete process but he also doesn't think it's a path. I guess the terminology is not 100% standardi…
A Markov chain is commonly understood to be a time-discrete Markov process. Intuitively, it’s a “chain” because you can “single out” its states in time rather than intervals. That’s also Markov’s original definition. Instead of Wassermann one can look up the notion in Wikipedia. A path is a notion relevant to the state space of a process - it’s a realization of states at every single point in time for the time span of interest.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#23Earlier quoted context omitted.
Your overall point might be correct but your example does not prove your point. A Makrov chain is just the path taken through the course of a markov process. The terms 'chain' and 'process' are sometimes conflated in this context, but this is the most common distinction I've seen. As such you can run a markov process for some number of steps N times, and then ask how many generated chains contain the property you are…
Interesting. I'd not heard the term Markov Chain used to describe a path so I checked my copy of "All of Statistics" by Larry Wasserman and he (slightly) disagrees with both of us and says "A Markov chain is a stochastic process for which the distribution of X_t depends only on X_{t-1}." So he doesn't need it to be a discrete process but he also doesn't think it's a path. I guess the terminology is not 100% standardi…
But a random walk is precisely a stochastic process for which the _next state_ depends only on the _current state_. In terms of graphs (where _random walk_ comes from), the next node is decided by randomly selecting a neighbor of the current node.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#24Earlier quoted context omitted.
Not all Markov processes have stationary distributions, and of those that do not all correspond to a non-normalized probability function. It therefore has some merit to think about MCMC as a random walk on a graph rather than Markov processes, because the “graph” needs to have some properties in order for the Markov process to be useful for MCMC. For example every “node” in the “graph” needs to be reachable from ever…
Could you explain further please? I agree with what you're saying but I don't' understand how it applies to what I said so there's definitely something I could learn here. Edit: Thanks.
Sure. As I see MCMC it’s basically a mathematical trick that lets you sample from probability distributions even if you only know the relative probability of different samples. It’s based on the observation that some Markov processes have a stationary distribution that is identical to a distribution you may want to sample from. But you need to carefully construct the Markov process for that observation to hold, and the properties you need to ensure are most easily understood as properties of a random walk graph. So the interesting subset of Markov processes are most easily understood as such random walks on a graph.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#25Hmm. I'm not an expert, but some of this seems definitely not to be accurate. Some of the "Bullshit" turns out perhaps to be quite important. Take the statement: > Markov Chain is essentially a fancy name for a random walk on a graph Is that really true? I definitely don't think so. To my understanding, a Markov process is a stochastic process that has the additional (aka "Markov") property that it is "memoryless". T…
Stochastic process with the Markov property: Past and future are conditionally independent given the present. The general version of conditionally independent is from probability theory based on measure theory and the Radon-Nikodym theorem (with von Neumann's novel proof in Rudin, Real and Complex Analysis ), but an easier introduction is in Erhan Çınlar, Introduction to Stochastic Processes . In a Poisson process th…
This Red/Blue submarine problem seems to be a better fit for ABM simulation, rather than Monte Carlo based on Markov processes.
IRL this will be a path dependent since both sides will learn from the past actions and probabilities will be changing, i.e. the memorylessness Markov property will not hold.
In ABM the ships (agents) can move on 2D space, which makes detection easier.
Also, obviously there are lots of externalities, like weapons, food, and sailors supply, ceasfires, surrenders, politics, etc.
All of the above is easier to simulate using ABM, rather than Monte Carlo.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#26Hmm. I'm not an expert, but some of this seems definitely not to be accurate. Some of the "Bullshit" turns out perhaps to be quite important. Take the statement: > Markov Chain is essentially a fancy name for a random walk on a graph Is that really true? I definitely don't think so. To my understanding, a Markov process is a stochastic process that has the additional (aka "Markov") property that it is "memoryless". T…
I won’t pretend to know the technical details (as the other replies do) but I want to make a point for the “pedagogical” effect here, which I agree with the author. The way I interpret the article, it’s not supposed to be a deep, theoretical treatise on the subject; more of an introductory, “intuitive” take on it. This works for those who need to either learn the concept to begin with, or refresh their memories if th…
I see, where you're coming from, but in that particular case, the "intuitive" explanation (walk on a graph) is far less intuitive for me than the proper explanation, that a Markov process is memoryless.
That said, I used MCMC in the past to do physics simulation, where the Markov property also applies to the underlying physical process. So maybe it's just me.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#27[deleted]
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#28Earlier quoted context omitted.
Interesting. I'd not heard the term Markov Chain used to describe a path so I checked my copy of "All of Statistics" by Larry Wasserman and he (slightly) disagrees with both of us and says "A Markov chain is a stochastic process for which the distribution of X_t depends only on X_{t-1}." So he doesn't need it to be a discrete process but he also doesn't think it's a path. I guess the terminology is not 100% standardi…
But a random walk is precisely a stochastic process for which the _next state_ depends only on the _current state_. In terms of graphs (where _random walk_ comes from), the next node is decided by randomly selecting a neighbor of the current node.
[deleted]
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#29Science communication is so important. I write scientific papers and I always write a blog post about the paper later, because nobody understands the scientific paper -- not even the scientists. The scientists regularly read my blog instead. The "scientific style" has become so obtuse and useless that even the professionals read the blog instead. True insanity.
Re: Markov Chain Monte Carlo Without All the Bullshit (2015)
#30Earlier quoted context omitted.
Interesting. I'd not heard the term Markov Chain used to describe a path so I checked my copy of "All of Statistics" by Larry Wasserman and he (slightly) disagrees with both of us and says "A Markov chain is a stochastic process for which the distribution of X_t depends only on X_{t-1}." So he doesn't need it to be a discrete process but he also doesn't think it's a path. I guess the terminology is not 100% standardi…
But a random walk is precisely a stochastic process for which the _next state_ depends only on the _current state_. In terms of graphs (where _random walk_ comes from), the next node is decided by randomly selecting a neighbor of the current node.
That's not true for random walks in general I don't think. A random walk is a process derived from taking random steps in some mathematical space. It can include jumps and it can include memory.
Take a "path-avoiding" random walk. At time t the distribution of the next step depends on whether or not I have at some point hit any of the adjacent nodes in the current path. That's not the current state, that's memory.