You are here

Automatic Color Image Segmentation via Reversible Jump MCMC

Lifetime from: 
Lifetime to: 
Short description: 
The goal of this project is to propose a method which is able to segment a color image without any human intervention. The only input is the observed image, all other parameters are estimated during the segmentation process. The algorithm finds the most likely number of classes, their associated model parameters and generates a segmentation of the image by classifying the pixels into these classes.

Image segmentation is an important early vision task where pixels with similar features are grouped into homogeneous regions. As a matter of fact, the simplest statistical model for an image consists of the probabilities of pixel classes. The knowledge on the dependencies between nearby pixels can be modeled by a MRF. Such models are much more powerful, even if it is not easy to determine the values of the parameters which specify a MRF. If each pixel class is represented by a different model then the observed image may be viewed as a sample from a realization of an underlying label field. Unsupervised segmentation can therefore be treated as an incomplete data problem where the color values are observed, the label field is missing and the associated class model parameters, including the number of classes, need to be estimated.

Due to the difficulty of estimating the number of pixel classes (or clusters), unsupervised algorithms often assume that this parameter is known a priori. When the number of pixel classes is also being estimated, the unsupervised segmentation problem becomes a model selection problem over a combined model space.

Our approach consists of building a Bayesian color image model using a first order MRF. The observed image is represented by a mixture of multivariate Gaussian distributions while inter-pixel interaction favors similar labels at neighboring sites. In a Bayesian framework, we are interested in the posterior distribution of the unknowns given the observed image. Herein, the unknowns comprise the hidden label field configuration, the Gaussian mixture parameters, the MRF hyperparameter, and the number of mixture components (or classes). Then a MCMC algorithm is used to sample from the whole posterior distribution in order to obtain a MAP estimate via simulated annealing. However, classical MCMC methods are restricted to problems where the dimensionality of the parameter vector is fixed. Therefore, the estimation of the number of mixture components is not possible. Recently, a novel method, called Reversible Jump MCMC (RJMCMC), has been proposed by P. Green. This method makes it possible to construct reversible Markov chain samplers that jump between parameter subspaces of different dimensionality. It has also been applied to univariate Gaussian mixture identification, intensity based image segmentation, and computing medial axes of 2D shapes. Herein, RJMCMC allows us the direct sampling of the whole posterior distribution defined over the combined model space thus reducing the optimization process to a single simulated annealing run. Another advantage is that no coarse segmentation neither exhaustive search over a parameter subspace is required. Although we present the model in the case of 3 dimensional observations, it is straightforward to extend it to higher dimensions.


We have conducted a test on the Berkeley segmentation dataset. The public benchmark data consists of grayscale and color human segmentations for 300 images. The images are divided into a training set of 200 images, and a test set of 100 images. Since our method doesn't need training, we have used the 100 color test images. The benchmark results of other segmentation algorithms are available at the Berkeley segmentation benchmark. The results of our algorithm with different beta parameter settings as well as results produced by JSEG, another unsupervised segmentation algorithm by Yining Deng and B.S.Manjunath, are available here.

Below we show some segmentation results obtained by the proposed method. Note that no user input was needed for these segmentations, only the input color image has been specified, the rest was done automatically by our algorithm.


Results on real color images


Markov Models