Click on the images to jump to corresponding results.
Starting from our earlier approach to fast keypoint matching, we developed real-time techniques to detect deformable surfaces. Using wide-baseline correspondences between an input image and a model image, we recover quickly and accurately the 2D or 3D shape of a deforming surface. In addition, we also model the lighting and detect occlusions.
Results
Painting a Platypus on a Deforming TShirt [top]
Given the TShirt and platypus images shown below, we recover the deformations of the tshirt in the video. We then deform the platypus image accordingly, relight it, and add the result to the original images while taking occlusions into account.
Magic Book [top]
The same technology was used by Camille Scherrer, an designer, to create a magic book.
Augmenting a 3D Surface [top]
In the examples above, we only recover 2D surface deformations. We can also use our techniques for monocular 3D shape recovery to recover the deformations of the paper and augment it with fully 3D objects.
Code
References
Please note that the publication lists from Infoscience integrated into the EPFL website, lab or people pages are frozen following the launch of the new version of platform. The owners of these pages are invited to recreate their publication list from Infoscience. For any assistance, please consult the Infoscience help or contact support.
Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation
International Journal of Computer Vision. 2008. Vol. 76, num. 2, p. 109-122. DOI : 10.1007/s11263-006-0017-9.Please note that the publication lists from Infoscience integrated into the EPFL website, lab or people pages are frozen following the launch of the new version of platform. The owners of these pages are invited to recreate their publication list from Infoscience. For any assistance, please consult the Infoscience help or contact support.