Images co-segmentation with Conditional Random Fields
Synopsis: Image segmentation serves as a key step for many applications. However, most segmentation algorithms needs to a large amount of already segmented images as training data. It is tedious and expensive to get such data. Another way of providing supervision information is by providing a bunch of unlabeled images with common objects among these images. By exploring the common information, one could segment out the common objects. This type of task is called co-segmentation ( also referred as co-saliency in some literature)
In this project you will:
1. Find related work on co-segmentation
2. Propose an algorithm to combine the co-segmentation pipeline with conditional random fields to generate reliable segments.
3. Test and compare the propose method over existing co-segmentation algorithms
References:
[1] Faktor, Alon, and Michal Irani. “Co-segmentation by composition.” Proceedings of the IEEE International Conference on Computer Vision. 2013.
[2] Alvarez, Jose M., Mathieu Salzmann, and Nick Barnes. “Large-scale semantic co-labeling of image sets.” Applications of Computer Vision (WACV), 2014 IEEE Winter Conference on. IEEE, 2014.
[3] Krähenbühl, Philipp, and Vladlen Koltun. “Efficient inference in fully connected crfs with gaussian edge potentials.” arXiv preprint arXiv:1210.5644 (2012).
Deliverables: In the end of the semester, the student should provide a written report on the work done as well as the implemented algorithm that can perform images co-segmentation
Prerequisites:
Basic knowledge of image processing and computer vision, programming with Matlab or python or C/C++ using OpenCV.
Type of work: 60% research and 40% implementation
Level: MS Semester project
Supervisor: Bin Jin (firstname.lastname@epfl.ch).
Comparison of structured light encoding methods for precise depth estimation
Synopsis:
3D sensing is gaining momentum with the possibility to display and fabricate the results with high fidelity. The most commonly used methods for optical depth estimation are based on the triangulation or the time-of-flight principle. We are interested in the former principle, more precisely, in methods based on structured light [1].
In this project you will implement three different classes of structured light encoding for depth estimation. The classes that we are interested in are: time-multiplexing [2], phase-shifting [3] and frequency multiplexing [4].
You will then compare the different classes in terms of accuracy, robustness and complexity by implementing the complete structured light processing pipeline. The pipeline involves:
– calibrating a projector-camera pair
– capturing and decoding the projected patterns
– performing the triangulation (depth estimation)
References:
[1] Geng, Jason. “Structured-light 3D surface imaging: a tutorial.” Advances in Optics and Photonics 3.2 (2011): 128-160.
[2] Gupta, Mohit, et al. “A practical approach to 3D scanning in the presence of interreflections, subsurface scattering and defocus.” International journal of computer vision 102.1-3 (2013): 33-55.
[3] Gupta, Mohit, and Shree K. Nayar. “Micro phase shifting.” Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012.
[4] Takeda, Mitsuo, and Kazuhiro Mutoh. “Fourier transform profilometry for the automatic measurement of 3-D object shapes.” Applied optics 22.24 (1983): 3977-3982.
Deliverables:
– code of the complete structured light processing pipeline (see above)
– written report explaining the three methods and comparing their pros and cons
Prerequisites:
– knowledge of image processing / computer vision
– coding skills in Matlab / Python / C / C++ / Java
Level: MS semester project
Type of work: 25% research and 75% implementation
Supervisor: Marjan Shahpaski (firstname.lastname@epfl.ch)
Depth map fusion in the presence of vibrations
Synopsis:
Depth map fusion is the process of fusing (averaging) multiple similar observations of a scene in order to decrease the noise that is present in the individual depth maps [1][2]. Consequently, it also renders a more detailed scene.
In this project we would like to investigate if having vibrations during the depth scanning can be used for improving the quality of the measured depth. The project will focus on the selection and implementation of suitable depth fusion algorithm(s). The depth maps will be acquired by a Kinect or a structured light system. Finally, the output of this system will be compared to that of a static depth scanning system.
References:
[1] Newcombe, Richard A., et al. “KinectFusion: Real-time dense surface mapping and tracking.” Mixed and augmented reality (ISMAR), 2011 10th IEEE international symposium on. IEEE, 2011.
[2] Blais, François, Michel Picard, and Guy Godin. “Accurate 3D acquisition of freely moving objects.” Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004. 2004.
Deliverables:
– implemented depth fusion algorithm(s)
– written report explaining the implemented algorithm(s) and analyzing of the obtained results
Prerequisites:
– knowledge of image processing / computer vision
– coding skills in Matlab / Python / C / C++ / Java
Level: BS / MS semester project
Type of work: 25% research and 75% implementation
Supervisor: Marjan Shahpaski (firstname.lastname@epfl.ch)
Exploring color representation in deep neural networks
Synopsis: Deep convolutional neural networks consistently achieve state of the art results on various tasks, yet their complex internal workings remain poorly understood. Recent work has begun to shed light on what these networks learn by using optimisation techniques to visualise the features learned by deep layer of the network. While first-layer filters exhibit edge detectors with clear color-sensitivity or invariance, deeper layers’ use of color information remains largely unexplored.
In this project you will:
1. Work with state of the art pre-trained deep neural networks.
2. Implement advanced visualisation techniques.
3. Synthesise input and manipulate network connectivity to examine color sensitivity.
4. Propose a hypothesis of color representation in neural networks based on empirical analysis.
References:
[1] Simonyan, Karen, and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition.” (2014).
[2] Yosinski, Jason, et al. “Understanding neural networks through deep visualization.” (2015).
Deliverables: Written report and source code.
Prerequisites:
Familiarity with neural networks and a software framework capable of running them.
Type of work: 50% research and 50% implementation
Level: Master
Supervisor: Edo Collins (firstname.lastname@epfl.ch)
Color spaces and preprocessing for object classification
Synopsis: Many successful object recognition techniques, such as those based on convolutional neural networks, operate directly on the pixels of an image. Most such models operate in RGB space, often employing preprocessing steps such as centering and whitening in order to remove 1st and 2nd-order statistics as to allow the model to ‘focus’ on more interesting patterns in the data. Similar effects could be partially achieved by considering an image representation in a different color space.
In this project you will:
1. Implement a neural network for object classification.
2. Implement various preprocessing techniques and observe impact on model performance.
3. Study the effect of operating in color spaces other than standard RGB, e.g. LAB, HSV, etc.
References:
[1] Krizhevsky, Alex, and Geoffrey Hinton. “Learning multiple layers of features from tiny images.” (2009).
[2] Bora, Dibya Jyoti, Anil Kumar Gupta, and Fayaz Ahmad Khan. “Comparing the Performance of L* A* B* and HSV Color Spaces with Respect to Color Image Segmentation.” (2015).
Deliverables: Written report and source code.
Prerequisites:
Basic knowledge of machine learning and image processing. Programming with python (preferred) or Matlab.
Type of work: 30% research and 70% implementation
Level: Bachelor or Master
Supervisor: Edo Collins (firstname.lastname@epfl.ch)