3D vision : point-cloud based room segmentation algorithm for accurate indoor odometry

Keywords:

Point-Cloud, LiDAR, Computer Vision, Segmentation, Machine learning

Introduction:

Room segmentation on point clouds is an important task in computer vision and robotics that has numerous applications in assisted odometry, which in turns is crucial in several fields such as autonomous navigation or virtual reality.

The goal of this project is to develop a methodology that allows the segmentation of point clouds of buildings’ interior into their constituent rooms. Those point clouds are acquired using a portable high-end LiDAR device named BLK2Go, that is carried by a human during the scan.

The results of the segmentation will be used to estimate accurately the trajectory of the agent during the scan. This will in turn allow to improve both the quality of the estimated trajectory and the accuracy of the LiDAR point-cloud, making it more reliable.

from: Ochmann S., et al., 2019. Automatic reconstruction of fully volumetric 3D building models from oriented point clouds

Challenges:

There are several challenges associated with room segmentation on point clouds, including:

Noisy and incomplete data: Point clouds often contain noise and missing data, making it challenging to segment some sections of the scan.

Overlapping rooms: The rooms in a building may overlap, with large opennings, making it difficult to differentiate between them.

Occlusion : some rooms might contains large and numerous object, which makes it challenging for the alogirhm to properly identify walls and ceiling

Foreseen Solutions:

To overcome these challenges, various techniques can be used, including:

Data preprocessing: Removing noise and filling in missing data can significantly improve the accuracy of the segmentation.

Machine learning: Machine learning algorithms (posibly deep learning), could be used to learn the features of the rooms (walls, floor/ceiling, furnitures) and segment them accurately. Nevertheless, non-machine learning algorithm are also considered.

Iterative refinement: Refining the segmentation iteratively using multiple algorithms could improve the results.

Objectives:

Analysis of the state of the art room segmentation point cloud and choice of one (or multiple) algorithm(s) to investigate further.

Implementation and evaluation of the chosen algorithm on the provided data

Prerequisites:

Fluency in Python, experience in computer vision, image processing, machine learning.

For master students affiliated to Data Science, Computer Science, SysCom., Robotics, Environmental Sciences and Engineering programs.

Contact:

Interested candidates are kindly asked to send us by email their CV and a short motivation statement.

Aurelien Brun, Jan Skaloud

References:

Room segmentation algorithm publications :

  1. Ochmann S., et al., 2019. Automatic reconstruction of fully volumetric 3D building models from oriented point clouds
  2. Yang, F., et al., 2021. Volumetric Representation and Sphere Packing of Indoor Space for Three-Dimensional Room Segmentation
  3. Frías, E., et al, 2020. Point cloud room segmentation based on indoor spaces and 3D mathematical morphology

Point cloud segmentation reviews :

  1. Xie. Y, et al, 2020. Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
  2. Guo Y., et al, 2020. Deep Learning for 3D Point Clouds: A Survey (Section 5.1, 5.2 & 5.3)