Delete search term

Header

Main navigation

School of Engineering

Localization with stereo camera and IMU

Localization with stereo camera and IMU

As part of a master's thesis, an algorithm for visual inertial odometry was developed and tested at the ISC. This created a prerequisite for realizing VIO solutions on different hardware platforms.

Visual-Inertial-Odometry (VIO) is a technique for determining the position and orientation or pose of a rigid body. Only camera images and inertial signals, i.e. rotation rate and acceleration, serve as sensor signals. These signals can be acquired autonomously by the object to be localized; the visible environment is sufficient as infrastructure. In contrast, classical localization systems require additional infrastructure, which is associated with cost and effort and reduced flexibility. For example, GNSS requires a direct line of sight to a sufficient number of satellites. The method of visual inertial odometry overcomes these disadvantages and is therefore considered the ideal localization principle for many applications. First commercial products like virtual reality systems and smartphones or the tracking camera T265 from Intel already used this technology successfully. This application is made possible by the rapid development of computing power and algorithms during the last decades.

Operating principle of Visual Inertial Odometry (VIO)

Our eyes and sense of balance allow us to determine our location and orientation. This biological principle served as inspiration for visual odometry. In the technical implementation, the sense of sight is realized by cameras and the sense of balance by a so-called Inertial Measurement Unit (IMU). During the movement a 3D map of prominent visual landmarks is generated. This process is called mapping. The location of these landmarks in a new camera image suggests the position and orientation of the camera, see image below. The successive determination of the position and the map is also called simultaneous localization and mapping (SLAM). The use of an IMU allows an approximate prediction of the landmarks in the new camera image, which speeds up their search.

Main results

The implemented VIO algorithm was tested on the known EuRoC data set of the Autonomous Systems Lab of ETH. For a 1 min and 55 m long trajectory, an RMS error of 0.25 m or a drift of 1% of the travelled path was achieved. Orientation was estimated at a total rotation of 381° with an RMS error of 2° and a drift of 1.3%. Depending on the configuration, the VIO algorithm should be real-time capable on the processor of a smartphone or even a Raspberry Pi.
The master thesis of our MSE graduate Philipp Schmid was awarded the GMFH Prize 2019 for the most convincing and challenging use of mathematics in a practical application.

At a glance

Participating institutes and centers

        Institute of Signal Processing and Wireless Communications (ISC)

Project status:  finished

Award:   GMFH Award 2019