6. Validation and Evaluation for Visual Attention Models
Published Online: 20 MAR 2013
Copyright © 2013 John Wiley & Sons Singapore Pte. Ltd.
Selective Visual Attention: Computational Models and Applications
How to Cite
Zhang, L. and Lin, W. (2013) Validation and Evaluation for Visual Attention Models, in Selective Visual Attention: Computational Models and Applications, John Wiley & Sons (Asia) Pte Ltd, Singapore. doi: 10.1002/9780470828144.ch6
- Published Online: 20 MAR 2013
- Published Print: 27 MAR 2013
Print ISBN: 9780470828120
Online ISBN: 9780470828144
- performance evaluation;
- human-labelled images;
- eye tracking data;
- eye fixation;
- true positive rate (TPR);
- false positive rate (FPR);
- positive predictive value(PPV);
- receiver operating characteristic curve (ROC);
- area under the mean ROC curve(AUC) score;
- Kullback-Leibler (KL) distance;
- normalized scanpath salience (NSS);
- Spearman's rank-order correlation with visual conspicuity
This chapter assesses the performance of the saliency detection models described in the previous chapters. As with many other cases in engineering, a developed visual attention model needs to be critically benchmarked against other models, and then fully tested before being used in particular applications and situations. A number of qualitative and quantitative evaluation methods, as well as related ground-truth databases, are introduced in this chapter. Common benchmarks include simple man-made visual patterns, human-labelled images and eye tracking data, which are first given in Sections 6.1–6.3. The quantifying estimation of performance of the computational models is listed in Sections 6.4–6.6. The most commonly used criteria are PPV, TPR, F-measure, ROC and AUC, as introduced in Section 6.4. The statistical criteria for both static and dynamic scene – NNS and KL distance – are presented in Section 6.5. Then Section 6.6 shows the criterion of Spearman's rank-order correlation with visual conspicuity.
Each type of ground-truth, the associated evaluation methods and their advantages and disadvantages are discussed whenever needed and possible.