51–60 of 69 posts
Just pick a random place to start, read some stuff, and then take a guess as to which direction to go in next, based on what's probably a good next thing to read. Then keep repeating the process over and over again.
Earlier quoted context omitted.
Are you describing Markov chains or how to learn about Markov chains?
More specifically Markov chain with monte carlo method (MCMC)
So, the state of the system was the remaining Red/Blue inventories.
Some work by Koopmans showed that the encounter rates were a Poisson process. So, the time to the next encounter had exponential distribution, depending on the current state.
At an encounter, depending on the types, could have the Red die, the Blue die, both die, or neither die. Then after the encounter, the state of the system changed. So, the state of the system was a continuous time, discrete state space Markov process subordinated to a Poisson process. That is, in part, a Markov chain.
Yes, there is a closed form solution, but the combinatorial explosion of the discrete state space size meant that a direct attack via the closed form solution was not reasonable.
But it was easy enough to do Monte-Carlo, that is, generate a few hundred sample paths and average those, get confidence intervals, etc. While in grad school working on operations research I did that. While the state space was enormous, the Monte-Carlo was really fast. On any computer of today, the code would run before could get finger off the mouse button or the Enter key. And running off 1 million sample paths would be feasible. For the random numbers I looked in Knuth's appropriate volume of The Art ... and used
X(n + 1) = X(n) * 5^15 + 1 mod 2^47
programmed in assembler.
Work passed review by famous applied probabilist J. Keilson.
Apparently the work was sold to some intelligence agency. I could guess which one, but then I'd have to ...!
1. Elementary probability theory. 2. Poisson processes. 3. The Markov property. 4. Stochastic processes. 5. Realise that you’re missing a background in analysis, therefore you don’t know sh?t about measure theory but you actually need it to know anything deeper . Wonder to yourself if you really want to spend the next 3 years getting a maths background you don’t have. 6. Convince yourself that it’s all just engineeri…
1. Elementary probability theory. 2. Poisson processes. 3. The Markov property. 4. Stochastic processes. 5. Realise that you’re missing a background in analysis, therefore you don’t know sh?t about measure theory but you actually need it to know anything deeper . Wonder to yourself if you really want to spend the next 3 years getting a maths background you don’t have. 6. Convince yourself that it’s all just engineeri…
1. Elementary probability theory. 2. Poisson processes. 3. The Markov property. 4. Stochastic processes. 5. Realise that you’re missing a background in analysis, therefore you don’t know sh?t about measure theory but you actually need it to know anything deeper . Wonder to yourself if you really want to spend the next 3 years getting a maths background you don’t have. 6. Convince yourself that it’s all just engineeri…
Poisson processes are continuous time though. If you're interested in Markov chains you only need the discrete-time theory. In discrete time and discrete space, it mostly just reduces to linear algebra.
Markov chains in essence are simple. Instead of diverging and reading all the theory, I'd recommend do it on a need basis. Learn as you go. So pick up a problem and move ahead. I don't think it is fruitful to just learn everything about Markov Chains just for the sake of it. Markov Chain Monte Carlo to sample from probability distributions is a good start - https://arxiv.org/abs/1206.1901 if you are into sampling.
Betancourt's survey is at least as good, and more up to date. https://arxiv.org/pdf/1701.02434.pdf
https://soundcloud.com/jamesbradbury/9th-order-markov-chain-...