Chapter 13. An evaluation, comparison, and accurate benchmarking of several publicly available MS/MS search algorithms: Sensitivity and specificity analysis

  1. Prof. Dr. Gilbert S. Omenn
  1. Eugene A. Kapp,
  2. Frédéric Schütz,
  3. Lisa M. Connolly,
  4. John A. Chakel,
  5. Jose E. Meza,
  6. Christine A. Miller,
  7. David Fenyo,
  8. Jimmy K. Eng,
  9. Joshua N. Adkins,
  10. Dr. Gilbert S. Omenn2 and
  11. Professor Richard J. Simpson1

Published Online: 2 NOV 2006

DOI: 10.1002/9783527609482.ch13

Exploring the Human Plasma Proteome

Exploring the Human Plasma Proteome

How to Cite

Kapp, E. A., Schütz, F., Connolly, L. M., Chakel, J. A., Meza, J. E., Miller, C. A., Fenyo, D., Eng, J. K., Adkins, J. N., Omenn, G. S. and Simpson, R. J. (2006) An evaluation, comparison, and accurate benchmarking of several publicly available MS/MS search algorithms: Sensitivity and specificity analysis, in Exploring the Human Plasma Proteome (ed G. S. Omenn), Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany. doi: 10.1002/9783527609482.ch13

Editor Information

  1. Internal Medicine, University of Michigan, A520 MSRB 1 Bldg, 1150 West Medical Center Dr. Ann Arbor, MI 48109–0626, USA

Author Information

  1. 1

    Joint ProteomicS Laboratory, Ludwig Institute for Cancer Research, P.O. Box 2008, Royal Melbourne Hospital, Parkville, Victoria 3050, Australia

  2. 2

    Internal Medicine, University of Michigan, A520 MSRB 1 Bldg, 1150 West Medical Center Dr. Ann Arbor, MI 48109–0626, USA

  1. Originally published in Proteomics 2005, 13, 3475–3490

Publication History

  1. Published Online: 2 NOV 2006
  2. Published Print: 6 OCT 2006

ISBN Information

Print ISBN: 9783527317578

Online ISBN: 9783527609482

SEARCH

Keywords:

  • human plasma proteome;
  • evaluation;
  • comparison;
  • accurate benchmarking of several publicly available MS/MS search algorithms;
  • sensitivity analysis;
  • specificity analysis;
  • heuristic algorithms;
  • HUPO-PPP reference specimens

Summary

This chapter contains sections titled:

  • Introduction

    • Heuristic algorithms

    • Probabilistic algorithms

  • Materials and methods

    • HUPO-PPP reference specimens

    • Sample preparation and MS analysis

    • Protein sequence databases

    • MS/MS database search strategy

      • SEQUEST and MASCOT workflow performed by the JPSL research group

      • SEQUEST and PeptideProphet workflow performed by the ISB research group

      • Spectrum Mill workflow performed by the Agilent group

      • Sonar and X!Tandem workflow performed by David Fenyo

    • Web interface for data validation, integration, and cross annotation

    • ROC curve generation

  • Results and discussion

    • Comparison of MS/MS search algorithms

      • Sensitivity and concordance between MS/MS search algorithms

      • Specificity and discriminatory power of the primary score statistic for the different MS/MS search algorithms: Distribution of scores and ROC plots

      • Calculation of score thresholds based on specified FP identification error rates

      • Benchmarking of the different MS/MS search algorithms at 1% FP error rate

      • Effect of database size and search strategy

      • Utility of reversed sequence searches

      • Consensus scoring between MS/MS search algorithms

  • Concluding remarks

  • References