Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a Kalman filter works, in pictures (2015)

bzarg.com

Post reply on HN
Print view View original

21–30 of 35 posts

Re: How a Kalman filter works, in pictures (2015)

#21

Post by tnecniv »

sudoscript wrote: post #15
tnecniv wrote: post #12

Earlier quoted context omitted.

Yes, they are a staple of modern robotics and other fields. If the Kalman filter assumptions about the system hold (linear model, Gaussian noise), the Kalman filter is an optimal filter and you can't do better. There are also more complicated variants like the extended Kalman filter and the unscented Kalman that can do better when the assumptions of the Kalman filter are not accurate. It's also worth noting that the…

In practical robotics, how often do the assumptions hold? Is it really usually true that the variables have only a linear relationship?

People are generally happy with the Gaussian noise assumption. In some cases, there are acceptable linearizations of the dynamical model. One example is linearizing the quadrotor dynamics about the hover state (no roll or pitch). This approximation works well enough for basic flying, but you won't be able to pull off any flashy maneuvers because any hard bank will move you too far away from the linearization point for it to hold. A better choice would be a more complicated KF (I've used an error-state KF in the past).

The real draw of these filters, though, is that they are very fast. In my experience, most of the compute time every update cycle is spent on sensing because your sensors dump a ton of data that you need to process as part of your CV / SLAM / whatever pipeline (the outputs of these then go into your KF). The dream is to get a 10ms update loop so your control algorithms can do a good job, but this is easier said than done.

Re: How a Kalman filter works, in pictures (2015)

#22

Post by tnecniv »

agustamir wrote: post #19

My ML professor once took hidden markov model and arrived at equations of a kalman filter. That just completely blew my mind, I could never think the concepts from estimation theory machine learning could be related so beautifully.

The Kalman filter algorithm is an EM algorithm.

Re: How a Kalman filter works, in pictures (2015)

#23

Post by agustamir »

tnecniv wrote: post #22
agustamir wrote: post #19

My ML professor once took hidden markov model and arrived at equations of a kalman filter. That just completely blew my mind, I could never think the concepts from estimation theory machine learning could be related so beautifully.

The Kalman filter algorithm is an EM algorithm.

At that point, I did not have an idea of the family of techniques that these belonged to.

Re: How a Kalman filter works, in pictures (2015)

#24

Post by JamesBarney »

For anyone who read this blog post and thought "Wow that's interesting, but I don't understand half of this. I need a whole youtube series to explain this to me."

Well we're in luck. https://www.youtube.com/watch?v=CaCcOwJPytQ.

It's a great set of 55 video lectures on Kalman filtering. (I'm only on number 5 but so far they've been great)

Re: How a Kalman filter works, in pictures (2015)

#25

Post by emcq »

amelius wrote: post #4

Pardon my ignorance, I'm just wondering about some context, since the Kalman filter was invented in the 60s. Are Kalman filters still highly relevant, or are they (in practice and/or in theory) obsoleted by other techniques, such as general ML?

Yes it is used. But there is a whole wealth of more general additional techniques based on probabilistic graphical models. A good reference for those curious is probabilistic robotics: http://robots.stanford.edu/probabilistic-robotics/

Re: How a Kalman filter works, in pictures (2015)

#26

Post by platz »

amelius wrote: post #4

Pardon my ignorance, I'm just wondering about some context, since the Kalman filter was invented in the 60s. Are Kalman filters still highly relevant, or are they (in practice and/or in theory) obsoleted by other techniques, such as general ML?

-- Kalman Filters vs HMM (Hidden Markov Model):

"In both models, there's an unobserved state that changes over time according to relatively simple rules, and you get indirect information about that state every so often. In Kalman filters, you assume the unobserved state is Gaussian-ish and it moves continuously according to linear-ish dynamics (depending on which flavor of Kalman filter is being used). In HMMs, you assume the hidden state is one of a few classes, and the movement among these states uses a discrete Markov chain. In my experience, the algorithms are often pretty different for these two cases, but the underlying idea is very similar." - THISISDAVE

-- HMM vs LSTM/RNN:

"Some state-of-the-art industrial speech recognition [0] is transitioning from HMM-DNN systems to "CTC" (connectionist temporal classification), i.e., basically LSTMs. Kaldi is working on "nnet3" which moves to CTC, as well. Speech was one of the places where HMMs were _huge_, so that's kind of a big deal." -PRACCU

"HMMs are only a small subset of generative models that offers quite little expressiveness in exchange for efficient learning and inference." - NEXTOS

"IMO, anything that be done with an HMM can now be done with an RNN. The only advantage that an HMM might have is that training it might be faster using cheaper computational resources. But if you have the $$$ to get yourself a GPU or two, this computational advantage disappears for HMMs." - SHERJILOZAIR

Re: How a Kalman filter works, in pictures (2015)

#27

Post by icholy »

JamesBarney wrote: post #24

For anyone who read this blog post and thought "Wow that's interesting, but I don't understand half of this. I need a whole youtube series to explain this to me." Well we're in luck. https://www.youtube.com/watch?v=CaCcOwJPytQ . It's a great set of 55 video lectures on Kalman filtering. (I'm only on number 5 but so far they've been great)

Unfortunately those lectures are not complete.

Re: How a Kalman filter works, in pictures (2015)

#28

Post by wyc »

JamesBarney wrote: post #24

For anyone who read this blog post and thought "Wow that's interesting, but I don't understand half of this. I need a whole youtube series to explain this to me." Well we're in luck. https://www.youtube.com/watch?v=CaCcOwJPytQ . It's a great set of 55 video lectures on Kalman filtering. (I'm only on number 5 but so far they've been great)

Wow, this lecturer is great! I think this is the most intuitive explanation of Kalman filters I've seen yet. I've seen other lecturers try to quickly jump into state space and lose students with big matrices.

I would have paid good money for this if I wasn't already familiar with the material. The clarity is up there with The Essence of Linear Algebra: https://www.youtube.com/watch?v=kjBOesZCoqc

Trading math videos on the Internet--definitely nerds.

Re: How a Kalman filter works, in pictures (2015)

#29

Post by jessaustin »

icholy wrote: post #27
JamesBarney wrote: post #24

For anyone who read this blog post and thought "Wow that's interesting, but I don't understand half of this. I need a whole youtube series to explain this to me." Well we're in luck. https://www.youtube.com/watch?v=CaCcOwJPytQ . It's a great set of 55 video lectures on Kalman filtering. (I'm only on number 5 but so far they've been great)

Unfortunately those lectures are not complete.

Well there are at least 42 videos, so that's fairly complete. Nearly a semester already...

Re: How a Kalman filter works, in pictures (2015)

#30

Post by deepitee1 »

ajross wrote: post #5
amelius wrote: post #4

Pardon my ignorance, I'm just wondering about some context, since the Kalman filter was invented in the 60s. Are Kalman filters still highly relevant, or are they (in practice and/or in theory) obsoleted by other techniques, such as general ML?

They're not really applicable to the same problem areas. Kalman filters are most often applied in small/embedded system control environments where latency matters more, control is continuous, and you don't have a cloud to host your autopilot.

>Kalman filters are most often applied in small/embedded system control environments where latency matters more

For example, missile guidance.

Post reply on HN