Mobile 3D Vision: LiDAR Assisted Car Navigation and Mapping using Deep Learning on Point Clouds

Source: orbis360

Keywords:

LiDAR, 3D Correspondences, Deep Learning, Point Cloud, Mobile Laser Scanning, Navigation, Trajectory Estimation

Introduction:

In Mobile Laser Scanning (MLS), LiDAR scanners are mounted on cars to scan along roads with various applications such as road inspections, mapping or navigation. The accuracy of point clouds is a key requirement of such measurement campaigns, of which a prerequisite is the careful modeling of the trajectory and its accurate estimation. This is challenging to obtain in practice because loss of GNSS signal or degradation of its quality are common events when scanning from the road.

In this project, you will contribute on improving a real scanning system operated by a Swiss company, orbis360. The project will focus on adapting a deep learning methodology developed in our lab and able to recognize correspondences (i.e. recognizable points scanned multiple times in the point cloud). The end goal is to improve the robustness of the trajectory estimation and point cloud generation pipeline when GNSS signal degradation occurs, allowing for more accurate 3D digitization of scanned areas.

Example of MLS point cloud in St-Saphorin (VD), source: orbis360

Challenges:

Several challenges are associated with LiDAR-based 3D correspondences in Mobile Laser Scanning, including:

  1. Nature of the data: The current pipeline to establish 3D correspondences has been developed for Aerial Laser Scanning, which generates point clouds that are very different in terms of scale, resolution and noise level.

  2. Occlusions and overlapping objects: Occlusions and overlapping objects in urban scenes can hinder the accurate matching of corresponding points, leading to erroneous correspondences.

  3. Computational efficiency: The considered LiDAR system is generating very dense and thus very heavy point clouds. Efficient ways to extract correspondences from such dataset is essential for practical deployment in real-world scenarios.

Objectives:

The main objectives of this project are the following:

  1. Identification and extraction of overlapping areas in the MLS point clouds

  2. Adaptation of the existing pipeline of correspondences extraction, including network training and adaptation

  3. Evaluation of the method compared to the baseline model

  4. Other contributions that motivate the student are very welcome!

Prerequisites:

The main objectives of this project are the following:

  1. Identification and extraction of overlapping areas in the point clouds

  2. Adaptation of the existing pipeline of correspondences extraction

  3. Evaluation of several training strategies and variants compared to the available model

  4. Other contributions that motivates the student are very welcome!

Contacts:

Interested candidates are requested to send a brief motivation statement and if available their CV via email to the following contacts:

Aurelien Brun, Jan Skaloud

 

References:

  1. A. Brun et. al., 2022, Lidar point–to–point correspondences for rigorous registration of kinematic scanning in dynamic networks