Live data from Hacker News

How a Kalman filter works, in pictures (2015)

bzarg.com

11–20 of 35 posts

Re: How a Kalman filter works, in pictures (2015)

#11
post #4

This is my favourite explanation of how a Kalman filter works http://htmlpreview.github.io/?https://github.com/aguaviva/Ka...

Here’s a good HN discussion of that article:

https://news.ycombinator.com/item?id=16575679

HN comments were quite informative too.

Re: How a Kalman filter works, in pictures (2015)

#12
Also check out Sebastian Thrun's videos from the "AI for robotics" Udacity course (2012). He explains with similar examples:

https://www.youtube.com/watch?v=LXJ5jrvDuEk

https://www.youtube.com/watch?v=HTL5-0DDqE4

https://www.youtube.com/watch?v=cUKlYjQEQGY

https://www.youtube.com/watch?v=hUnTg5v4tDU

Re: How a Kalman filter works, in pictures (2015)

#13

Nice pics for exam studies. But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation. I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.

We do control stuff in marine radar and target tracking in noisy environments. We use EKF and UKF all the time, and in our cases the covariance matrix is definitely sound, useful, and provides information that we use.

Re: How a Kalman filter works, in pictures (2015)

#15
post #10
post #7

It looks very similar to Bayesian estimation. I wonder what is really the difference. Does any of you know, or have a link that explains it?

There is really no difference. You can frame the Kalman filter as a Bayesian posterior inference problem. For example, for a stationary linear Gaussian model, you have a transition model of the form: z_t = A z_{t-1} + B u_t + e where e ~ Gaussian(0,Q) and an observation model of the form: x_t = C z_{t} + D u_t + d, where, d ~ Gaussian (0,R) Since, z_t and x_t are both multivariate gaussians in this model, you can com…

In the event of Gsussian noise, linear state transition and linear measurement, KF yields the exact posterior. Both unscented and extended refer to non-linear measurement or state transition equations. Neither are designed for non gaussian noise.

Though the KF and it's variants are one of the simplest, well-performing estimation methods out there, so it wouldn't suprise me if it's used for everything, appropriate or not.

Re: How a Kalman filter works, in pictures (2015)

#16
post #9
post #7

It looks very similar to Bayesian estimation. I wonder what is really the difference. Does any of you know, or have a link that explains it?

Yes it is a special case of Bayesian inference under the assumptions that the system is linear and that the priors and likelihoods are Gaussian.

I should have added that there's a very good reason for making these assumptions: they let you represent the system with extremely little data! In general for a system with a continuous state space (like position) then to represent a general Bayesian prior you need to store one data point for each of the infinitely many possible states. Obviously this is impossible so you might instead discretize the space into a grid and store the probability that it is at each that point in the grid. (Think of an image where each pixel's brightness tells you how likely you are to be there - you can store any shape of probability distributions this way but it's of limited resolution). Same for the likelihood and posterior distribution - this uses a ton of memory and is slow and limits your resolution.

Instead, if you assume that the priors are Gaussian, then you can store that information as just two numbers: the mean and the variance (or a matrix of numbers for higher dimensional state spaces). And Guassian's have the remarkable property that if you start with Gaussians and perform a Bayesian update, then you end up with a Gaussian that can also be represented with this same amount of data. Assuming that the system is linear means that you can also represent it's update from one time to the next as a matrix. Moreover, both of these approximations are pretty good for a large class of real-world problems.

There are other sophisticated ways to get around the downsides of the general approach. Namely, there's particle filters which discretize your distributions by a sample of points, but unlike the grid discretization above, the points aren't at specific fixed locations. They're allowed to move around and are constantly being resampled from the distribution you have. This allows lots of the points to get very close together and accurately represent the most interesting (most likely) parts of the state space without wasting tons of memory on extremely unlikely points in the state space. It's very clever and fun to watch in practice!

Re: How a Kalman filter works, in pictures (2015)

#18

Nice pics for exam studies. But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation. I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.

It's industry standard in aerospace, granted one usually uses the EKF with nonlinear dynamics so the covariance matrix is not estimated perfectly. That setup is also flexible enough to let you introduce of band measurements from the past but with the correct timestamp and correct.

It's kind of interesting that how particular fields chose to embrace certain particular methods; till I have worked with aerospace domain,I never had heard of the Kalman filter, whereas some of the newer and even more fundamental methods like wavelets and compressed sensing appear to have such low traction in Aerospace till now.

Re: How a Kalman filter works, in pictures (2015)

#19

Nice pics for exam studies. But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation. I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.

A while ago (around 2 or so years) I did quite a bit of mechatronics and Kalman Filters were used resonable effectively for data fusion in IMUs. I wouldn't be surprised if there were better methods around but all the standard chipsets used them at that time.

EDIT: I forgot but I've seen them used more recently by CERN for particle trajectory prediction as part of their hit tracking system. I only know this since they wrote about it in some of their supporting documentation for their latest kaggle comp [0].

[0]: https://www.kaggle.com/c/trackml-particle-identification

Re: How a Kalman filter works, in pictures (2015)

#20

Nice pics for exam studies. But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation. I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.

We do control stuff in marine radar and target tracking in noisy environments. We use EKF and UKF all the time, and in our cases the covariance matrix is definitely sound, useful, and provides information that we use.

Ye well maybe it's a domain thing. I do driveline programming for cars.

As soVeryTired wrote "In general I find Kalman filters a little suspect where the underlying dynamics of the system can change over time, and when the physical process that gives rise to the dynamics isn't known."

This maybe applies in a less degree to radars etc?

In general we use only one sensor source at a time for eg. rpm rather then fusing them. If one is obvously off just select another. Maybe it's rather about how exact you need the estimate to be.

Post reply on HN