Jelenlegi hely
Multicue Image and Video Segmentation: a Multi-layer MRF Framework
The human visual system is not treating different features sequentially. Instead, multiple cues are perceived simultaneously and then they are integrated by our visual system in order to explain the observations. Therefore different image features has to be handled in a parallel fashion. In this project, we attempt to develop such a model in a Markovian framework. The model has a multi-layer structure (see image on the left showing the layers in the case of motion and color based segmentation): Each feature has its own layer, called feature layer, where an MRF model is defined using only the corresponding feature. A special layer is assigned to the combined MRF model. This layer interacts with each feature layer and provides the segmentation based on the combination of different features. Unlike previous methods, our approach doesn’t assume common boundaries for different features. The uniqueness of the proposed method is the ability to detect boundaries that are visible only in one of the features.
The proposed model consists of 3 layers. At each layer, we use a first order neighborhood system and extra inter-layer cliques. The image features are represented by multivariate Gaussian distributions. The combined layer only uses the features indirectly, through inter-layer cliques, thus the model is not fusioning the feature data directly, rather it combines label proposals coming from the individual feature layers. The combined layer model also estimates the number of classes and chose those label pairs which are actually present in the input image.
The proposed algorithm has been tested on real video sequences using motion and color features as well as on color textured images using color and texture features. Experimental results demonstrate that the proposed method is quite powerful in combining different features in order to detect boundaries visible only in one of them.
Results
First, we present some segmentation examples using color and texture features. More results are available here.
 |
 |
 |
 |
Below, we show some segmentation results based on combined color and motion features. On the sythetic example, we also show detected occlusion boundaries in black color.
 |
 |
 |
 |
 |
 |
 |
 |
 |