This one is great to me, less formula and more drawings :)
https://www.bzarg.com/p/how-a-kalman-filter-works-in-picture...
Derive Yourself a Kalman Filter
21–30 of 56 posts
Re: Derive Yourself a Kalman Filter
#22Earlier quoted context omitted.
Thanks for your comments! I don't see how what is discussed here conflicts with the notation I introduced into the post, do you still believe there is a soundness issue in what I have written?
I'm just saying that the notation of say A * X|Y * B seemed unfamiliar to me. I only know conditional notation within a P(...). Or an expectation, etc. Apparently your way of writing is used by others as well, but it may be good to know that it is not fully rigorous. Again, there are different people preferring different presentations. I as a student was often frustrated by abused notations and was often confused by…
> it may be good to know that it is not fully rigorous
What is the problem with A|B=b being a random variable? (Apart from you unfamiliarity with the concept, I mean.)
Edit: I don’t say there are no problems, I ask what do you think the problem is? There is no problem in the discrete case. In the continuous setting things are indeed more complicated (but if the limiting process is well defined there are no issues).
Note that the same lack of rigour that you find in conditional ramdom variables affects conditional probabilities. If you can accept the latter there is no reason to reject the former.
Re: Derive Yourself a Kalman Filter
#23Make sure to read this bit to make the formal explanation easier to understand.
Re: Derive Yourself a Kalman Filter
#24Earlier quoted context omitted.
I'm just saying that the notation of say A * X|Y * B seemed unfamiliar to me. I only know conditional notation within a P(...). Or an expectation, etc. Apparently your way of writing is used by others as well, but it may be good to know that it is not fully rigorous. Again, there are different people preferring different presentations. I as a student was often frustrated by abused notations and was often confused by…
> it may be good to know that it is not fully rigorous What is the problem with A|B=b being a random variable? (Apart from you unfamiliarity with the concept, I mean.) Edit: I don’t say there are no problems, I ask what do you think the problem is? There is no problem in the discrete case. In the continuous setting things are indeed more complicated (but if the limiting process is well defined there are no issues). N…
A random variable is different concept from a distribution. For me personally it is helpful to keep them separate, but I can see that others may not care about the complete conceptual picture.
In the PDF file linked above I can see conditional probabilities, conditional distributions and conditional expectation etc, which are all valid and rigorous. I can see that the author thinks it's a good idea to merge these into a single concept of conditional random variable for didactic reasons, but that's not a rigorous concept.
Practically, if you have two random variables then you can take their joint distribution. What would be the joint distribution of (A|B) and (C|D)? For actual random variables it's simple: you can take intersections in event space, but a "conditional random variable" does not correspond to any subset of the event space.
Very simply speaking (this is my working model, not the exact precise math definition which involves a lot of measure theory): in probability theory we have an event space containing atomic events that cover all possible outcomes for the whole experiment/observation. A random variable is a function that maps from each such potential (atomic) event to a number. That's right. The random variable is a function but not the mass function, which maps from a number to a probability.
Conditional probability P(A|B) is an expression defined to mean P(A,B)/P(B). That's a clear definition. I am yet to see the actual definition of a conditional random variable.
Again, disclaimer 1: I can see the practicality of disregarding formality. Still I argue this is best done only when you do know better but it would be tedious to be technically correct all the time. But as a beginner I find it more useful to keep track of the correct concepts. For example not distinguishing random variables and distributions can be very confusing when considering more advanced things, like mutual information and KL-divergence. The former operates on random variables, the latter on distributions. I remember this was a difficult realization for me because the material we used didn't emphasize the difference enough, probably in the name of practicality.
Disclaimer 2: my point is a minor one overall.
Re: Derive Yourself a Kalman Filter
#25"It's really quite simple. Here, let me show you all these formulas..." This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own." I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beg…
A Kalman filter is a specific instantiation of the Bayes filter, which is very elegant and intuitive. Then once you plug in a bunch of Gaussians and linearity in the Bayes filter and crunch the math, a Kalman filter falls out.
This is how it's treated in the book Probabilistic Robotics by Thrun, Burgard, Fox and I found it to be one of the best treatments on Kalman filters. A really great book overall.
Re: Derive Yourself a Kalman Filter
#26"It's really quite simple. Here, let me show you all these formulas..." This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own." I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beg…
You could, but you should follow it up with "where one of those sources is your (incomplete, noisy) information about what the value _used_ to be and how it changes over time". If you just have (say) three different ways of estimating the position of an aeroplane, then you don't need a Kalman filter. The place where the Kalman filter adds value is where you measure its position _repeatedly_ and make use of the fact t…
How do EKF and ensemble Kalman Filters fit into this?
Re: Derive Yourself a Kalman Filter
#27"It's really quite simple. Here, let me show you all these formulas..." This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own." I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beg…
There are no Kalman filters for beginners because no one is out there trying to make maths more complicated than it needs to be.
To understand what’s happening you need work linear algebra l, probability theory and distribution theory. There’s no way to explain it without because it makes no sense without. Kalman filters are born of those things.
Re: Derive Yourself a Kalman Filter
#28"It's really quite simple. Here, let me show you all these formulas..." This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own." I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beg…
"it can be insulting"
Insulting, really? Somebody cares to try to explain you something and the word you can find is "insulting"?
It's like everyone owes it to you to chew knowledge in small enough bits that you can swallow them without any effort.
What a kindergarden.
Re: Derive Yourself a Kalman Filter
#29Earlier quoted context omitted.
You could, but you should follow it up with "where one of those sources is your (incomplete, noisy) information about what the value _used_ to be and how it changes over time". If you just have (say) three different ways of estimating the position of an aeroplane, then you don't need a Kalman filter. The place where the Kalman filter adds value is where you measure its position _repeatedly_ and make use of the fact t…
How do EKF and ensemble Kalman Filters fit into this?
The Kalman filter assumes that all errors are gaussian, that all updates are linear (i.e., new state = linear function of old state), and that observations are linear (i.e., what you measure is a noisy version of a linear function of the state).
The extended Kalman filter allows for state updates and observations to be nonlinear, by the straightforward expedient of replacing them with linear approximations near to the current estimate.
The ensemble Kalman filter also allows for nonlinear updates and observations; instead of keeping track of the expectation and covariance of the state (i.e., everything you need to define a Gaussian model, things that behave nicely under linear transformations but not under nonlinear ones) it keeps track of an "ensemble" of sample values, applies the update and observation functions to those, and then estimates expectations and covariances from this ensemble. It's a sort of Monte Carlo Kalman filter.
A couple of other things worth knowing about:
Intermediate between the extended KF and the ensemble KF is the "unscented Kalman filter". Like the ensemble KF it estimates things using a number of samples; but instead of propagating those samples through repeated steps, it picks the sample points at each step on the basis of the estimated expectation and covariance, and uses them only to compute new expectations and covariances. More expensive than the EKF but copes better with substantial nonlinearities.
Extrapolating beyond the ensemble KF is the "particle filter", which uses the same "track an ensemble of samples" approach but gives up the assumption that all the errors are Gaussian. You need a larger ensemble to get good results, I think, but it can cope with a wider range of scenarios. (I find the name "particle filter" annoyingly distracting; the "particles" are the samples, which I guess you're supposed to think of as a cloud of points in possible-configuration-of-the-system space, and of course it's a "filter" in the same way as the Kalman filter is.)
Re: Derive Yourself a Kalman Filter
#30"It's really quite simple. Here, let me show you all these formulas..." This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own." I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beg…
When it comes to the woodworking: a japanese pull saw ( https://en.m.wikipedia.org/wiki/Japanese_saw ) is a great and fairly cheap way to get started. It enables high accuracy and is still quite fast.
Then you'll need a cheap but sturdy table, a hand plane, some vices, a tape measure, a carpenter's square, a cheap cordless drill. Screwing and glueing is the best way to get solid joints for a novice.