Fields of Experts

S. Roth and M. J. Black, “Fields of experts,” International Journal of Computer Vision (IJCV), vol. 82, no. 2, pp. 205–229, Apr. 2009. [preprint (opens in new tab) ]

Conference version:

S. Roth and M. J. Black, “Fields of experts: A framework for learning image priors,” in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, San Diego, California, Jun. 2005, pp. 860–867. [preprint (opens in new tab) ]

Abstract

We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach provides a practical method for learning high-order Markov random field (MRF) models with potential functions that extend over large pixel neighborhoods. These clique potentials are modeled using the Product-of-Experts framework that uses non-linear functions of many linear filter responses. In contrast to previous MRF approaches all parameters, including the linear filters themselves, are learned from training data. We demonstrate the capabilities of this Field-of-Experts model with two example applications, image denoising and image inpainting, which are implemented using a simple, approximate inference scheme. While the model is trained on a generic image database and is not tuned toward a specific application, we obtain results that compete with specialized techniques.

Training Data

We used a subset of the training images of the Berkeley Segmentation Database in order to train the FoE model. This database is itself a subset of the Corel image database.

Note: The image data available here is provided for research and educational purposes only. The copyright and distribution terms of the original copyright holder apply.

Test Data (“68 Images”)

We used a subset of the test section of the Berkeley Segmentation Database to evaluate the denoising performance of the trained FoE model. We also measured the denoising performance on a standard set of images commonly used in the image processing community (Lena, Boats, etc.).

Example Code & Models

Note: The code and model data provided here underlies the copyright and licensing terms of Brown University. The code and data may be used for non-commercial purposes only. See the licensing terms included in the source package for more details.

Note 2: If you are looking for a recent implementation of Fields of Experts and/or trained models, please refer to our CVPR 2010 paper (opens in new tab) and the accompanying code instead of the code linked here. This newer work substantially outperforms the implementation of the original FoE provided below. The code here is mainly intended to promote reproducible research related to the original work.

We have a set of MATLAB functions available that illustrate how to use the Fields of Experts model for image denoising and image inpainting. The package includes trained FoE models with cliques of size 3x3 (with 8 filters) and 5x5 (with 24 filters). Each MATLAB function comes with a brief documentation (type “help foe_demo” for an overview). Please also see the included README file.

  • Source and data package (version 1.0, 06/09/2005) [package ]

Detailed package contents:

  • Trained FoE models (3x3 with 8 filters, 5x5 with 24 filters).
  • Code to compute the log density of an image under the FoE model.
  • Code to compute the gradient of the log density with respect to an image.
  • Code for image denoising with denoising demo and example images.
  • Code for image inpainting with image inpainting demo and example images.

Example Results