6. Validation and Evaluation for Visual Attention Models

  1. Liming Zhang1 and
  2. Weisi Lin2

Published Online: 20 MAR 2013

DOI: 10.1002/9780470828144.ch6

Selective Visual Attention: Computational Models and Applications

Selective Visual Attention: Computational Models and Applications

How to Cite

Zhang, L. and Lin, W. (2013) Validation and Evaluation for Visual Attention Models, in Selective Visual Attention: Computational Models and Applications, John Wiley & Sons (Asia) Pte Ltd, Singapore. doi: 10.1002/9780470828144.ch6

Author Information

  1. 1

    Fudan University, P. R. China

  2. 2

    Nanyang Technological University, Singapore

Publication History

  1. Published Online: 20 MAR 2013
  2. Published Print: 27 MAR 2013

ISBN Information

Print ISBN: 9780470828120

Online ISBN: 9780470828144

SEARCH

Keywords:

  • performance evaluation;
  • ground-truth;
  • human-labelled images;
  • eye tracking data;
  • eye fixation;
  • databases;
  • true positive rate (TPR);
  • false positive rate (FPR);
  • positive predictive value(PPV);
  • F-measure;
  • receiver operating characteristic curve (ROC);
  • area under the mean ROC curve(AUC) score;
  • Kullback-Leibler (KL) distance;
  • normalized scanpath salience (NSS);
  • Spearman's rank-order correlation with visual conspicuity

Summary

This chapter assesses the performance of the saliency detection models described in the previous chapters. As with many other cases in engineering, a developed visual attention model needs to be critically benchmarked against other models, and then fully tested before being used in particular applications and situations. A number of qualitative and quantitative evaluation methods, as well as related ground-truth databases, are introduced in this chapter. Common benchmarks include simple man-made visual patterns, human-labelled images and eye tracking data, which are first given in Sections 6.1–6.3. The quantifying estimation of performance of the computational models is listed in Sections 6.4–6.6. The most commonly used criteria are PPV, TPR, F-measure, ROC and AUC, as introduced in Section 6.4. The statistical criteria for both static and dynamic scene – NNS and KL distance – are presented in Section 6.5. Then Section 6.6 shows the criterion of Spearman's rank-order correlation with visual conspicuity.

Each type of ground-truth, the associated evaluation methods and their advantages and disadvantages are discussed whenever needed and possible.