Texture-less Deformable Surfaces Dataset

The dataset features deformable objects with uniform albedo captured under varying lighting conditions using depth camera Microst Kinect Xbox 360. The main challenge is to recover a 3D shape of a deforming surface observed in a single RGB image. Each sample thus contain an RGB image and corresponding ground truth (GT) normal map and depth map. A small subset of the dataset also contains GT triangulated meshes. Please refer to [1] for the detailed description of the dataset.

There are five distinct objects and a total of 26445 samples:

Hardware and Data Acquisition

The dataset was captured using Microsoft Kinect Xbox 360 which provides both the RGB images and corresponding depth maps with resolution 640 x 480 px. The RGB and depth data were automatically aligned using freely available library OpenKinect Freenect. Each sequence captures an object which is gradually deformed so as to present as much variability in shape as possible. In order to decrease the number of almost identical frames, we recorded the sequences at 5 FPS.

Four different complex lighting scenarios were considerd, please refer to Figure 1. All the RGB images and their corresponding GT normal maps and depth maps are subsampled to 224 x 224 px and segmentation of the foreground objects is provided. The normal maps are computed by differentiating the smoothed depth maps. Both the meshes and the normal maps are given in standard OpenCV coordinate frame: X-right, Y-down, Z-forward. Please refer to [1] for further details.

Description of Available Files

We provide:

• RGB images, resolution 224 x 224 px, 8bit depth.
• Normal maps, resolution 224 x 224 px.
• Depth maps, resolution 224 x 224 px.
• Triangulated meshes (only for 7 cloth sequences), 961 vertices.
• Camera intrinsic parameters (corresponding to the subsampled and cropped images).
• Lists of RGB images used as test sets in [1].

# Tested with Python 3.5.
import numpy as np
import matplotlib.pyplot as plt

path_img = '...'
path_nmap = '...'
path_dmap = '...'
path_mesh = '...'
path_simplices = '.../simplices.tri'

# Load an image.
img = plt.imread(path_img)

# Load a normal map and a depth map.
nmap = np.load(path_nmap)['normals']
dmap = np.load(path_dmap)['depth']

# Load a mesh.
mesh = np.load(path_mesh)['mesh']
simplices = np.loadtxt(path_simplices)

# Reshape the mesh to its template shape.
mesh = mesh.reshape((31, 31, 3))

Download

textureless_deformable_surfaces.zip (~3.8 GB)

Contact

Jan Bednarik

Pascal Fua

Mathieu Salzmann

References

[1] J. Bednarik, P. Fua and M. Salzmann, “Learning to Reconstruct Texture-less Deformable Surfaces from a Single View,” in CoRR, 2018.