Implementing A Kalman Filter In PostgreSQL To Smooth GPS Data

Usually databases are treated primarily as fairly dumb data storage systems, but they can be capable of much more. Case in point the PostgreSQL database and its – Ada-based – PL/pgSQL programming language, which allows you to perform significantly more complex operations than would be realistically possible with raw SQL. Case in point the implementation of a Kalman Filter by the folk over at Traconiq, which thus removes the necessity for an external filtering pipeline.

Using a Kalman Filter is highly desirable when you’re doing something like vehicle tracking using both dead-reckoning and GPS coordinates, as it filters out noise that can be the result of e.g. GPS reception issues. As noted in the article, transferring state from one row to the next requires a bit of lateral thinking, but is doable with some creative SQL usage. As PL/pgSQL is very similar to Oracle’s PL/SQL, this same code should work there too without too much porting required.

The code for the different implementations and associated benchmarks can be found on GitHub, though the benchmark results make it abundantly clear that the most efficient approach is to run an offline aggregate processing routine. This coincides with the other batch processing tasks that are typically performed by a database server to e.g. optimize storage, so this isn’t entirely unsurprising.

Get Your SQL Statements Right The First Time With SQL Lint

What’s your average success rate of getting a SQL statement right on the first try? In best case, you botched a simple statement without side effects and just have to try again with correct syntax or remove that typo from a table name, but things can easily go wrong fast here. But don’t worry, the days of fixing it on the fly can be over, thanks to [Joe Reynolds] who wrote a linter for SQL.

A linter parses code to tell you where you screwed up. While checking SQL syntax itself is somewhat straightforward, [Joe]’s sql-lint tool will also check the semantics of it by looking up the actual database and performing sanity checks on it. Currently supporting PostgreSQL and MySQL, it can be either run on a single SQL file or a directory of files, or take input directly from the command line. Even better, it also integrates within your editor of choice — assuming it supports external plugins — and the documentation shows how to do that specifically for Vim.

If you can look past the fact that it’s written in TypeScript and consequentially results in a rather large executable (~40 MB), it might serve as an interesting starting point for the language itself, or adds a new perspective on writing this type of analyzer. And if databases aren’t your terrain, how about shell scripts?

Continue reading “Get Your SQL Statements Right The First Time With SQL Lint”