• 1
    McHorney CA. Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann Intern Med 1997;127:74350.
  • 2
    Shiffman S, Hufford MR. Subject experience diaries in clinical research, Part 2: ecological momentary assessment. Appl Clin Trials 2001;10:428.
  • 3
    Shiffman S, Hufford MR, Paty J. Subject experience diaries in clinical research, Part 1: the patient experience movement. Appl Clin Trials 2001;10:4656.
  • 4
    Wiklund I. Assessment of patient-reported outcomes in clinical trials: the example of health-related quality of life. Fundam Clin Pharmacol 2004;18:35163.
  • 5
    Leidy NK, Revicki DA, Geneste B. Recommendations for evaluating the validity of quality of life claims for labeling and promotion. Value Health 1999;2:11327.
  • 6
    Revicki DA, Osoba D, Fairclough D, et al. Recommendations on health-related quality of life research to support labeling and promotional claims in the United States. Qual Life Res 2000;9:887900.
  • 7
    Santanello NC, Baker D, Cappelleri JC. Regulatory issues for health-related quality of life—PhRMA Health Outcomes Committee Workshop, 1999. Value Health 2002;5:1425.
  • 8
    Acquadro C, Berzon R, Dubois D, et al. Incorporating the patient's perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) harmonization group meeting at the Food and Drug Administration, February 16, 2001. Value Health 2003;5: 52133.
  • 9
    Revicki DA. FDA draft guidance and health outcomes research. Lancet 2007;369:5402.
  • 10
    Revicki DA, Gnanasakthy A, Weinfurt K. Documenting the rationale and psychometric characteristics of patient reported outcomes for labeling and promotional claims: the PRO evidence dossier. Qual Life Res 2007;16:71723.
  • 11
    Sloan JA, Halyard MY, Frost MH, et al. The Mayo Clinic manuscript series relative to the discussion, dissemination, and operationalization of the Food and Drug Administration guidance on patient-reported outcomes. Value Health 2007; 10(Suppl. 2):S5963.
  • 12
    Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials 2004;25: 53552.
  • 13
    Szende A, Leidy NK, Revicki D. Health-related quality of life and other patient-reported outcomes in the European centralized drug regulatory process: a review of guidance documents and performed authorizations of medicinal products 1995 to 2003. Value Health 2005;8:53448.
  • 14
    Shiffman S. Delivering on the eDiary promise. Appl Clin Trials 2005;14:64.
  • 15
    Wiechers OA. The move to EDC. Appl Clin Trials 2002;11:3840.
  • 16
    Saponjic RM, Freedman S, Sadighian A. What monitors think of EDC: results of a survey of U.S. monitors. Appl Clin Trials 2003;12:502.
  • 17
    Getz KA. The imperative to support site adoption of EDC. Appl Clin Trials 2006;15:3840.
  • 18
    US Food and Drug Administration. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims (DRAFT). February 2006. Available from:[Accessed June 1, 2008.
  • 19
    Rothman ML, Beltran P, Cappelleri JC, et al. Patient-reported outcomes: conceptual issues. Value Health 2007;10(Suppl 2):S6675.
  • 20
    Snyder CF, Watson ME, Jackson JD, et al. Patient-reported outcome instrument selection: designing a measurement strategy. Value Health 2007;10(Suppl 2):S7685.
  • 21
    Turner RR, Quittner AL, Parasuraman BM, et al. Patient-reported outcomes: instrument development and selection issues. Value Health 2007;10(Suppl 2):S8693.
  • 22
    Frost MH, Reeve BB, Liepa AM, et al. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health 2007;10(Suppl 2):S94105.
  • 23
    Sloan JA, Dueck AC, Erickson PA, et al. Analysis and interpretation of results based on patient-reported outcomes. Value Health 2007;10(Suppl 2):S10615.
  • 24
    Revicki DA, Erickson PA, Sloan JA, et al. Interpreting and reporting results based on patient-reported outcomes. Value Health 2007;10(Suppl 2):S11624.
  • 25
    Patrick DL, Burke LB, Powers JH, et al. Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health 2007;10(Suppl 2):S12537.
  • 26
    Tourangeau R, Smith TW. Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opin Q 1996;60:275304.
  • 27
    Taenzer PA, Speca M, Atkinson MJ, et al. Computerized quality-of-life screening in an oncology clinic. Cancer Practice 1997;5:16875.
  • 28
    Bloom DE. Technology, experimentation, and the quality of survey data. Science 1998;280:8478.
  • 29
    Velikova G, Wright EP, Smith AB, et al. Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J Clin Oncol 1999;17:9981007.
  • 30
    Stone AA, Shiffman S, Schwartz JE, et al. Patient noncompliance with paper diaries. BMJ 2002;324:11934.
  • 31
    Bushnell DM, Reilly MC, et al. Validation of electronic data capture of the Irritable Bowel Syndrome—Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol. Value Health 2006;9:98105.
  • 32
    US Food and Drug Administration. Code of federal regulations—title 21—food and drugs. Available from:[Accessed June 1, 2008.
  • 33
    US Food and Drug Administration. Guidance for industry: E6 good clinical practice: consolidated guidance. April 1996. Available from:[Accessed June 1, 2008.
  • 34
    US Food and Drug Administration. Guidance for industry: computerized systems used in clinical investigations. May 2007. Available from:[Accessed June 1, 2008.
  • 35
    US Food and Drug Administration. Guidance for industry—part 11, electronic records; electronic signatures—scope and application, August 2003. Available from:[Accessed June 1, 2008.
  • 36
    Raymond SA, Meyer GF. Interpretation of regulatory requirements by technology providers. Appl Clin Trials 2002;11:508.
  • 37
    Farrell J, Cooper M. Navigating the new 21 CFR 11 guidelines. Appl Clin Trials 2004;13:6770.
  • 38
    Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J Clin Epidemiol 2007;60:817.
  • 39
    Hahn EA, Cella D, Dobrez D, et al. The talking touchscreen: a new approach to outcomes assessment in low literacy. Psychooncology 2004;13:8695.
  • 40
    Crow JT. Receptive vocabulary acquisition for reading comprehension. Mod Lang J 1986;70:24250.
  • 41
    Henriksen B. Three dimensions of vocabulary development. Stud Second Lang Acq 1999;21:30317.
  • 42
    Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11:32233.
  • 43
    Krystal AD, Walsh JK, Laska E, et al. Sustained efficacy of eszopiclone over 6 months of nightly treatment: results of a randomized, double-blind, placebo-controlled study in adults with chronic insomnia. Sleep 2003;26:7939.
  • 44
    Mundt JC, Marks IM, Shear MK, Greist JH. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry 2002;180:4614.
  • 45
    Mundt JC, Kobak KA, Taylor LV, et al. Administration of the Hamilton Depression Rating Scale using interactive voice response technology. MD Comput 1998;15:319.
  • 46
    Mundt JC, Katzelnick DJ, Kennedy SH, et al. Validation of an IVRS version of the MADRS. J Psychiatr Res 2006;40:2436.
  • 47
    Rush A, Bernstein I, Trivedi M, et al. An evaluation of the quick inventory of depressive symptomalogy and the hamilton rating scale for depression: a sequenced treatment alternatives to relieve depression trial report. Biol Psychiatry 2006;59:493501.
  • 48
    Alemi F, Stephens R, Parran T, et al. Automated monitoring of outcomes: application to the treatment of drug abuse. Med Decis Making 1994;14:1807.
  • 49
    Agel J, Greist JH, Rockwood T, et al. Comparison of interactive voice response and written self-administered patient surveys for clinical research. Orthopedics 2001;24: 115557.
  • 50
    Dunn JA, Arakawa R, Greist JH, Clayton AH. Assessing the onset of antidepressant-induced sexual dysfunction using interactive voice response technology. J Clin Psychiatry 2007;68: 52532.
  • 51
    Fricker RD, Schonlau M. Advantages and disadvantages of internet research surveys: evidence from the literature. Field Methods 2002;14:34767.
  • 52
    Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. J Asthma 2003;40:75162.
  • 53
    Ryan JM, Corry JR, Attewell R, Smithson MJ. A comparison of an electronic version of the SF-36 General Health Questionnaire to the standard paper version. Qual Life Res 2002;11:1926.
  • 54
    Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res 2001;10:1521.
  • 55
    Caro JJ Sr, Caro I, Caro J, et al. Does electronic implementation of questionnaires used in asthma alter responses compared to paper implementation? Qual Life Res 2001;10:68391.
  • 56
    Crawley JA, Kleinman L, Dominitz J. User preferences for computer administration of quality of life instruments. Drug Inf J 2000;34:13744.
  • 57
    Allenby A, Matthews J, Beresford J, McLachlan SA. The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting. Eur J Cancer Care 2002;11:24553.
  • 58
    Kleinman L, Leidy NK, Crawley J, et al. A comparative trial of paper-and-pencil versus computer administration of the Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire. Med Care 2001;39:1819.
  • 59
    Cook AJ, Roberts DA, Henderson MD, et al. Electronic pain questionnaires: a randomized, crossover comparison with paper questionnaires for chronic pain assessment. Pain 2004;110: 3107.
  • 60
    Drummond HE, Ghosh S, Ferguson A, et al. Electronic quality of life questionnaires: a comparison of pen-based electronic questionnaires with conventional paper in a gastrointestinal study. Qual Life Res 1995;4:216.
  • 61
    Palermo TM, Valenzuela D, Stork PP. A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain 2004;107:2139.
  • 62
    Shields A, Gwaltney C, Tiplady B, et al. Grasping the FDA's PRO Guidance. Appl Clin Trials 2006;15:6972.
  • 63
    Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage Publications, 2005.
  • 64
    Quigley D, Elliott MN, Hays RD, et al. Bridging from the Picker Hospital Survey to the CAHPS® Hospital Survey. Med Care 2008;46:65461.
  • 65
    Willis G, Reeve BB, Barofsky I. The use of cognitive interviewing techniques in quality of life and patient-reported outcomes assessment. In: LipscombJ, GotayCC, SnyderC, eds. Outcomes Assessment in Cancer: Measures, Methods, and applications. Cambridge: Cambridge University Press, 2005.
  • 66
    Willis GB, DeMaio TJ, Harris-Kojetin B. Is the bandwagon headed to the methodological promised land? Evaluating the validity of cognitive interviewing techniques. In: SirkenMG, HerrmannDJ, SchecterS, et al., eds. Cognition and Survey Research. New York: John Wiley & Sons, 1999.
  • 67
    Ojanen V, Gogates G. A briefing on cognitive debriefing. Good Clin Prac J 2006;12:259.
  • 68
    Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigourous methods. BMJ 1996;313: 369.
  • 69
    Fleiss JL, Kingman A. Statistical management of data in clinical research. Crit Rev Oral Biol Med 1990; 1:5466. Available from:[Accessed June 13, 2008.
  • 70
    Blackwelder WC. Current issues in clinical equivalence trials. J Dent Res 2004;83(Spec Iss C):C1135.
  • 71
    Atherton SP, Sloan JA. Design and analysis of equivalence trials via the SAS system. SUGI Proc 1998;23:116671.
  • 72
    Sloan JA, Novotny P, Loprinzi CL, Ghosh M. Graphical and analytical tools for two-period crossover clinical trials. SUGI Proc 1997;22:13127. Available from:[Accessed June 13, 2008.
  • 73
    Sloan JA, Dueck A. Issues for statisticians in conducting analyses and translating results for quality of life end points in clinical trials. J Biopharm Stat 2004;14:7396.
  • 74
    Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1988.
  • 75
    Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med 1998;17:10110.
  • 76
    Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision. Stat Med 2002;21:13315.
  • 77
    Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use (3rd ed.). New York: Oxford University Press, 2003.
  • 78
    Bland JM, Altman DG. Comparing two methods of clinical measurement: a personal history. Int J Epidemiol 1995;24:714.
  • 79
    Gonin R, Lloyd S, Cella DF. Establishing equivalence between scaled measures of quality of life. Qual Life Res 1996;5:206.
  • 80
    Marshall GN, Hays RD, Nicholas R. Evaluating agreement between clinical assessment methods. Int J Methods Psychiatr Res 1994;4:24957.
  • 81
    Sloan JA. Statistical issues in the application of cancer outcome measures. In: LipscombJ, GotayCC, SnyderC, eds. Outcomes Assessment in Cancer: Measures, Methods, and Applications. New York: Cambridge University Press, 2005;36285.
  • 82
    Smith DJ, Huntington J, Sloan JA. Choosing the “correct” assessment tool. Curr Probl Cancer 2005;29:27282.
  • 83
    Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:4208.
  • 84
    Anastasi A, Urbina S. Psychological Testing (7th ed.). Upper Saddle River, NJ: Prentice Hall, 1996.
  • 85
    Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:3746.
  • 86
    Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 1968;70: 21320.
  • 87
    Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:6139.
  • 88
    Fleiss JL. Statistical Methods for Rates and Proportions (2nd ed.). New York: Wiley, 1981.
  • 89
    Nunnally JC, Bernstein IH. Psychometric Theory (3rd ed.). New York: McGraw-Hill, 1994. Theory.
  • 90
    Weiner EA, Stewart BJ. Assessing Individuals. Boston: Little Brown, 1984.
  • 91
    Ramachandran S, Lundy JJ, Coons SJ. Testing the measurement equivalence of paper and touch-screen versions of the EQ-5D visual analog scale (EQ VAS). Qual Life Res 2008;17:111720.
  • 92
    Guyatt GH, Osoba D, Wu AW, et al. Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc 2002;77: 37183.
  • 93
    Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003;41:58292.
  • 94
    Sloan J, Symonds T, Vargas-Chanes D, Fridley B. Practical guidelines for assessing the clinical significance of health-related quality of life changes within clinical trials. Drug Information Journal 2003;37:2331.
  • 95
    Sloan JA, Cella D, Hays RD. Clinical significance of patient-reported questionnaire data: another step toward consensus. J Clin Epidemiol 2005;58:12179.
  • 96
    Farivar SS, Liu H, Hays RD. Half standard deviation estimate of the minimally important difference in HRQOL scores? Expert Rev Pharmacoeconomics Outcomes Res 2004.51523.4.
  • 97
    Hays RD, Farivar SS, Liu H. Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. COPD 2005;2:637.
  • 98
    Teresi JA. Overview of quantitative measurement methods: equivalence, invariance, and differential item functioning in health applications. Med Care 2006;44(Suppl. 3):S3949.
  • 99
    Teresi JA, Fleishman JA. Differential item functioning and health assessment. Qual Life Res 2007;16:3342.
  • 100
    Crane PK, Gibbons LE, Ocepek-Welkson K, et al. A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression. Qual Life Res 2007;16:6984.
  • 101
    Feldt LS, Woodruff KJ, Saith FA. Statistical inference for coefficient alpha. Appl Psychol Meas 1987;11:93103.
  • 102
    Hays RD, Revicki D. Reliability and validity (including responsiveness). In: FayersP, HaysRD, eds. Quality of Life Assessment in Clinical Trials (2nd ed.). Oxford, UK: Oxford University Press, 2005.
  • 103
    Lohr K. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193205.
  • 104
    Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcome Measurement Information System (PROMIS). Med Care 2007;45:S2231.
  • 105
    Hepner KA, Brown JA, Hays RD. Comparison of mail and telephone in assessing patient experiences in receiving care from medical group practices. Eval Health Prof 2005;28:37789.
  • 106
    Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46:141732.
  • 107
    Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcome (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005;8:94104.
  • 108
    Marquis P, Keininger D, Acquadro C, De La Loge C. Translating and evaluating questionnaires: cultural issues for international research. In: FayersP, HaysRD, eds. Quality of Life Assessment in Clinical Trials (2nd ed.). Oxford, UK: Oxford University Press, 2005.
  • 109
    Hahn EA, Cella D. Health outcomes assessment in vulnerable populations: measurement challenges and recommendations. Arch Phys Med Rehabil 2003;84(Suppl):S3542.