Live data from Hacker News

Implementing a Kalman Filter in Postgres

neon.com

11–14 of 14 posts

Re: Implementing a Kalman Filter in Postgres

#11
Wow this is extremely cool/impressive, but if my manager asked me to implement this I'd quit lol. The "state" headaches alone seem like a nightmare, nevermind all the whacky linear algebra you're going to have hand-roll (Like does Postgres even have a matrix type?? Did you have to implement matrix inversion in SQL from scratch?? I get nauseous just thinking about it.)

edit: I guess in 2D a lot of this becomes simpler than in general high-dimensions.

Re: Implementing a Kalman Filter in Postgres

#12
post #10
post #8

Earlier quoted context omitted.

Yeah. Using the Kalman filter just to determine the position from noisy position measurements really undercuts the capability of the filter to use system physics to estimate the true state. In one of the most common applications of Kalman filters, autonomous robots (e.g., a robot vacuum or a commercial drone), the filters are around 9 to 12 dimensions.

Right, in addition to the position you usually want the velocity, and sometimes also the acceleration, in all dimensions. More ambitious (or optimistic) practitioners could add more sensor measurements, like gyroscopes.

You are right of course and I was out of my depth. I wonder if the vector types now being added to databases for ML/AI stuff could help with this.

Re: Implementing a Kalman Filter in Postgres

#13
post #4

Interestingly, in image 2, the filtered data seems to be worse than the actual noisy data? Sure, the large spikes from sensor data were reduced, as seen with the blue line up in north which was considerably reduced, but seemingly at the cost of the more accurate tracks. We can see some "ground truth" - namely the map roads. I think if the source of the tracks are someone moving on a road (in a car etc.), it is safe t…

Yeah, this sounds like a way to "smooth" the GPS trail to remove anomalies quickly, without paying attention to the road network. The problem of snapping a noisy GPS trail to the road network is known as map-matching. Good map-matching algorithms tend to use hidden Markov models, which are sort of like discrete Kalman filters. The state of the model is something like "which road segment is the truck on", and the pred…

So apps like Google maps do this? I'm always surprised when it jumps between roads. Like... You knew I've been on this road for the last ten minutes, you think I'm going to teleport into the tunnel beneath me?

Re: Implementing a Kalman Filter in Postgres

#14
post #13
post #4

Earlier quoted context omitted.

Yeah, this sounds like a way to "smooth" the GPS trail to remove anomalies quickly, without paying attention to the road network. The problem of snapping a noisy GPS trail to the road network is known as map-matching. Good map-matching algorithms tend to use hidden Markov models, which are sort of like discrete Kalman filters. The state of the model is something like "which road segment is the truck on", and the pred…

So apps like Google maps do this? I'm always surprised when it jumps between roads. Like... You knew I've been on this road for the last ten minutes, you think I'm going to teleport into the tunnel beneath me?

I'm not sure about Maps to be honest, but that sort of glitch is a strong indicator that they're just snapping to the nearest current road rather than doing proper routing calculations.

My Toyota has a speed limit symbol on the dashboard which will occasionally show the speed of a slip-road going onto the motorway I'm already on. I'm guessing it's a similar phenomenon.

Post reply on HN