FORTY QUESTIONS ABOUT VALIDITY GENERALIZATION AND META‐ANALYSIS
COMMENTARY ON FORTY QUESTIONS ABOUT VALIDITY GENERALIZATION AND META‐ANALYSIS
The opinions expressed herein are those of the authors and do not necessarily reRect the official policy of the organizations with which they are affiliated. We would like to thank Ann Hurd for her heroic efforts in typing and retyping our response to the commentators under the pressures of the tight time constraints imposed by the editor. Frank Schmidt is now in the Department of Human Resources and Industrial Relations, University of Iowa, Iowa City. Reprints are not available from the authors; they can be purchased, however, from Personnel Psychology.
You do not have access to the full version of this article.View access options below.
Log in with Open Athens, Shibboleth, or your institutional credentials.
If you have previously obtained access with your personal account, Please log in.
If you previously purchased this article, Log in to Readcube.
You previously purchased this article on READCUBE_PURCHASE_DATE. Click on an option below to access.
Logged in as READCUBE_USER. Log out of ReadCube.
Citing Literature
Number of times cited according to CrossRef: 108
- Gerhard Blickle, Personalauswahl, Arbeits- und Organisationspsychologie, 10.1007/978-3-662-56666-4_17, (271-302), (2018).
- Flávia Schwartz Maranho and Ricardo Leal, Corporate governance and firm performance in Latin America: a meta-analysis, Academia Revista Latinoamericana de Administración, 10.1108/ARLA-04-2017-0126, 31, 1, (195-211), (2018).
- Daniel Tzabbar, Shay Tzafrir and Yehuda Baruch, A bridge over troubled water: Replication, integration and extension of the relationship between HRM practices and organizational performance using moderating meta-analysis, Human Resource Management Review, 27, 1, (134), (2017).
- Ernest H. O'Boyle, Validity Generalization as a Continuum, Industrial and Organizational Psychology, 10, 03, (485), (2017).
- Robert P. Tett, Nathan A. Hundley and Neil D. Christiansen, Meta-Analysis and the Myth of Generalizability, Industrial and Organizational Psychology, 10, 03, (421), (2017).
- James H. Dulebohn, Dongyuan Wu and Chenwei Liao, Does liking explain variance above and beyond LMX? A meta-analysis, Human Resource Management Review, 27, 1, (149), (2017).
- Sooyeon Choi and Richard A. Feinberg, The LOHAS Lifestyle and Marketplace Behavior, Handbook of Engaged Sustainability, 10.1007/978-3-319-53121-2_10-1, (1-19), (2017).
- Varsha Verma, Dheeraj Sharma and Jagdish Sheth, Does relationship marketing matter in online retailing? A meta-analytic approach, Journal of the Academy of Marketing Science, 44, 2, (206), (2016).
- Jeffrey M. Cucina and Michael A. McDaniel, Pseudotheory proliferation is damaging the organizational sciences, Journal of Organizational Behavior, 37, 8, (1116-1125), (2016).
- Thomas A. O’Neill, Matthew J. W. McLarnon and Julie J. Carswell, Variance Components of Job Performance Ratings, Human Performance, 28, 1, (66), (2015).
- Frederik Anseel, Adam S. Beatty, Winny Shen, Filip Lievens and Paul R. Sackett, How Are We Doing After 30 Years? A Meta-Analytic Review of the Antecedents and Outcomes of Feedback-Seeking Behavior, Journal of Management, 41, 1, (318), (2015).
- Frank L. Schmidt, History and development of the Schmidt–Hunter meta‐analysis methods, Research Synthesis Methods, 6, 3, (232-239), (2015).
- Hannah R. Rothstein, Form meets function: A commentary on meta‐analytic history, Research Synthesis Methods, 6, 3, (290-292), (2015).
- Christopher R. Huber, Nathan R. Kuncel, Paul R. Sackett and Adam S. Beatty, Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates, International Journal of Selection and Assessment, 23, 3, (237-246), (2015).
- Max Reinwald, Hendrik Hüttermann, Julia Kröll and Sabine Boerner, Gender Diversity in Führungsteams und Unternehmensperformanz: Eine Metaanalyse, Schmalenbachs Zeitschrift für betriebswirtschaftliche Forschung, 67, 3, (262), (2015).
- Allen I. Huffcutt, Satoris S. Culbertson and William S. Weyhrauch, Moving Forward Indirectly: Reanalyzing the validity of employment interviews with indirect range restriction methodology, International Journal of Selection and Assessment, 22, 3, (297-309), (2014).
- In‐Sue Oh, Russell P. Guay, Kwanghyun Kim, Crystal M. Harold, Jong‐Hyun Lee, Chang‐Goo Heo and Kang‐Hyun Shin, Fit Happens Globally: A Meta‐Analytic Comparison of the Relationships of Person–Environment Fit Dimensions with Work Attitudes and Performance Across East Asia, Europe, and North America, Personnel Psychology, 67, 1, (99-152), (2013).
- Dana Balas-Timar, Valentina E. Balas and Alina Costin, Fuzzy Expert System Applied in Hiring Ratings and Long Term Job Performance Prediction, Journal of Human Resource and Sustainability Studies, 02, 03, (157), (2014).
- Kamran Ahmed, Keryn Chalmers and Hichem Khlif, A Meta-analysis of IFRS Adoption Effects, The International Journal of Accounting, 48, 2, (173), (2013).
- M. Travis Maynard, John E. Mathieu, Lucy L. Gilson, Ernest H. O’Boyle and Konstantin P. Cigularov, Drivers and outcomes of team psychological empowerment, Organizational Psychology Review, 3, 2, (101), (2013).
- André A. de Waal, Evergreens of excellence, Journal of Management History, 19, 2, (241), (2013).
- Patrice Laroche and Sébastien Soulez, Meta-Analysis for Marketing Research, Recherche et Applications en Marketing (English Edition), 27, 1, (79), (2012).
- MeowLan Evelyn Chan and Richard D. Arvey, Meta-Analysis and the Development of Knowledge, Perspectives on Psychological Science, 7, 1, (79), (2012).
- Jesse S. Michel, Chockalingam Viswesvaran and Jeffrey Thomas, Conclusions from meta‐analytic structural equation models generally do not change due to corrections for study artifacts, Research Synthesis Methods, 2, 3, (174-187), (2011).
- Herman Aguinis, Charles A. Pierce, Frank A. Bosco, Dan R. Dalton and Catherine M. Dalton, Debunking Myths and Urban Legends About Meta-Analysis, Organizational Research Methods, 14, 2, (306), (2011).
- Charles E. Lance, More Statistical and Methodological Myths and Urban Legends, Organizational Research Methods, 14, 2, (279), (2011).
- Gerhard Blickle, Personalauswahl, Arbeits- und Organisationspsychologie, 10.1007/978-3-642-16972-4_17, (225-252), (2011).
- David S. DeGeest and Frank L. Schmidt, The impact of research synthesis methods on industrial–organizational psychology: The road from pessimism to optimism about cumulative knowledge, Research Synthesis Methods, 1, 3‐4, (185-197), (2011).
- Jeff W. Johnson, Piers Steel, Charles A. Scherbaum, Calvin C. Hoffman, P. Richard Jeanneret and Jeff Foster, Validation Is Like Motor Oil: Synthetic Is Better, Industrial and Organizational Psychology, 3, 03, (305), (2010).
- JEFF W. JOHNSON, PIERS STEEL, CHARLES A. SCHERBAUM, CALVIN C. HOFFMAN, P. RICHARD JEANNERET and JEFF FOSTER, Validation Is Like Motor Oil: Synthetic Is Better, Industrial and Organizational Psychology, 3, 3, (305-328), (2010).
- Ioannis Tsaousis, Circadian preferences and personality traits: A meta‐analysis, European Journal of Personality, 24, 4, (356-373), (2010).
- Mark Cook, References, Personnel Selection, (310-338), (2010).
- Brian D. Blume, J. Kevin Ford, Timothy T. Baldwin and Jason L. Huang, Transfer of Training: A Meta-Analytic Review, Journal of Management, 36, 4, (1065), (2010).
- Andrew Li and Russell Cropanzano, Do East Asians Respond More/Less Strongly to Organizational Justice Than North Americans? A Meta‐Analysis, Journal of Management Studies, 46, 5, (787-805), (2009).
- FRANK L. SCHMIDT, JONATHAN A. SHAFFER and IN‐SUE OH, INCREASED ACCURACY FOR RANGE RESTRICTION CORRECTIONS: IMPLICATIONS FOR THE ROLE OF PERSONALITY AND GENERAL MENTAL ABILITY IN JOB AND TRAINING PERFORMANCE, Personnel Psychology, 61, 4, (827-868), (2008).
- Frank Schmidt, Meta-Analysis, Organizational Research Methods, 11, 1, (96), (2008).
- Chockalingam Viswesvaran and Deniz S. Ones, Job Performance: Assessment Issues in Personnel Selection, The Blackwell Handbook of Personnel Selection, (354-375), (2017).
- Piers D. G. Steel and John Kammeyer-Mueller, Bayesian Variance Estimation for Meta-Analysis, Organizational Research Methods, 11, 1, (54), (2008).
- FRANK L. SCHMIDT, IN‐SUE OH and HUY LE, INCREASING THE ACCURACY OF CORRECTIONS FOR RANGE RESTRICTION: IMPLICATIONS FOR SELECTION PROCEDURE VALIDITIES AND OTHER RESEARCH RESULTS, Personnel Psychology, 59, 2, (281-305), (2006).
- Andrew Li and Jessica Bagger, Using the BIDR to Distinguish the Effects of Impression Management and Self‐Deception on the Criterion Validity of Personality Measures: A Meta‐Analysis, International Journal of Selection and Assessment, 14, 2, (131-141), (2006).
- FRANK L. SCHMIDT, IN-SUE OH and HUY LE, INCREASING THE ACCURACY OF CORRECTIONS FOR RANGE RESTRICTION: IMPLICATIONS FOR SELECTION PROCEDURE VALIDITIES AND OTHER RESEARCH RESULTS, Personnel Psychology, 10.1111/j.1744-6570.2006.00037.x, 59, 2, (281-305), (2006).
- Laszlo Tihanyi, David A Griffith and Craig J Russell, The effect of cultural distance on entry mode choice, international diversification, and MNE performance: a meta-analysis, Journal of International Business Studies, 36, 3, (270), (2005).
- Huy Le, Alex Casillas, Steven B. Robbins and Ronelle Langley, Motivational and Skills, Social, and Self-Management Predictors of College Outcomes: Constructing the Student Readiness Inventory, Educational and Psychological Measurement, 65, 3, (482), (2005).
- , Book Reviews: 17, Personnel Psychology, 58, 4, (1104-1108), (2005).
- Eugene F. Stone-Romero and Dimitri Liakhovitski, Strategies for detecting moderator variables: A review of conceptual and empirical issues, , 10.1016/S0742-7301(02)21008-7, (333-372)
- Timothy A Judge, Daniel M Cable and Chad A Higgins, The Employment Interview: A Review of Recent Research and Recommendations for Future Research, Human Resource Management Review, 10, 4, (383), (2000).
- Alexander D. Stajkovic, Fitting Parametric Fixed Effect Categorical Models to Effect Sizes: A Neglected Meta-Analytic Approach in Organizational Studies, Organizational Research Methods, 2, 1, (88), (1999).
- JOHN HATTIE, RICHARD M. JAEGER and LLOYD BOND, Chapter 11 : Persistent Methodological Questions in Educational Testing, Review of Research in Education, 24, 1, (393), (1999).
- ALICE F. STUHLMACHER and AMY E. WALTERS, GENDER DIFFERENCES IN NEGOTIATION OUTCOME: A META‐ANALYSIS, Personnel Psychology, 52, 3, (653-677), (2006).
- William P. Dunlap and Ronald S. Landis, Interpretations of Multiple Regression Borrowed From Factor Analysis and Canonical Correlation, The Journal of General Psychology, 125, 4, (397), (1998).
- Stephen G. West and John F. Finch, Personality Measurement, Handbook of Personality Psychology, 10.1016/B978-012134645-4/50007-X, (143-164), (1997).
- LYLE F. SCHOENFELDT and KAREN J. JANSEN, Methodological Requirements for Studying Creativity in Organizations, The Journal of Creative Behavior, 31, 1, (73-90), (2011).
- Jennifer M. Verive and Michael A. McDaniel, Short-term memory tests in personnel selection: Low adverse impact and high validity, Intelligence, 23, 1, (15), (1996).
- AMIR EREZ, MATTHEW C. BLOOM and MARTIN T WELLS, USING RANDOM RATHER THAN FIXED EFFECTS MODELS IN META‐ANALYSIS: IMPLICATIONS FOR SITUATIONAL SPECIFICITY AND VALIDITY GENERALIZATION, Personnel Psychology, 49, 2, (275-306), (2006).
- Jesús F. Salgado and Silvia Moscoso, Meta-Analysis of Interrater Reliability of Job Performance Ratings in Validity Studies of Personnel Selection, Perceptual and Motor Skills, 83, 3_suppl, (1195), (1996).
- Paul M. Muchinsky, The Correction for Attenuation, Educational and Psychological Measurement, 56, 1, (63), (1996).
- Craig J. Russell and Stephen W. Gilliland, Why Meta-Analysis Doesn’t Tell Us What the Data Really Mean: Distinguishing between Moderator Effects and Moderator Processes, Journal of Management, 21, 4, (813), (1995).
- Kenneth S. Law, The Use of Fisher’s Z in Schmidt-Hunter–type Meta-Analyses, Journal of Educational and Behavioral Statistics, 20, 3, (287), (1995).
- J. F. Wang, S. Ohuchi and M. Taguri, HIGHER ORDER ACCURATE CONFIDENCE INTERVALS FOR SMOOTH FUNCTIONS OF MEANS WITH APPLICATION TO THE CORRELATION COEFFICIENT, Journal of the Japanese Society of Computational Statistics, 8, 1, (17), (1995).
- Raymond W. Preiss and Mike Allen, Understanding and using Meta-Analysis, Evaluation & the Health Professions, 18, 3, (315), (1995).
- CALVIN C. HOFFMAN, APPLYING RANGE RESTRICTION CORRECTIONS USING PUBLISHED NORMS: THREE CASE STUDIES, Personnel Psychology, 48, 4, (913-923), (2006).
- Fred M. Rafilson, Steven P. Allscheid and Joan G. Weiss, Transportability of Validity of the National Police Officer Selection Test for a Large Eastern Metropolitan Police Department, Psychological Reports, 75, 2, (707), (1994).
- Celeste Sturdevant Reed, Willard R. Young and Patrick P. McHugh, A Comparative Look at Dual Commitment: An International Study, Human Relations, 47, 10, (1269), (1994).
- MENI KOSLOWSKY and ABRAHAM SAGIE, COMPONENTS OF ARTIFACTUAL VARIANCE IN META‐ANALYTIC RESEARCH, Personnel Psychology, 47, 3, (561-574), (2006).
- John M. Cornwell and Robert T. Ladd, Power and Accuracy of the Schmidt and Hunter Meta-Analytic Procedures, Educational and Psychological Measurement, 53, 4, (877), (1993).
- Beryl Hesketh and Ivan Robertson, Validating Personnel Selection: A Process Model for Research and Practice, International Journal of Selection and Assessment, 1, 1, (3-17), (2007).
- Charles R. Schwenk and Charles B. Shrader, Effects of Formal Strategic Planning on Financial Performance in Small Firms: A Meta-Analysis, Entrepreneurship Theory and Practice, 17, 3, (53), (1993).
- David B. Allison and Bernard S. Gorman, Calculating effect sizes for meta-analysis: The case of the single case∗, Behaviour Research and Therapy, 31, 6, (621), (1993).
- Samuel Messick, FOUNDATIONS OF VALIDITY: MEANING AND CONSEQUENCES IN PSYCHOLOGICAL ASSESSMENT, ETS Research Report Series, 1993, 2, (i-18), (2014).
- David S. Cordray, Strengthening causal interpretations of nonexperimental data: The role of meta‐analysis, New Directions for Program Evaluation, 1993, 60, (59-96), (2004).
- Kevin R. Murphy, The Situational Specificity of Validities: Correcting for Statistical Artifacts Does Not Always Reduce the Trans‐Situational Variability of Correlation Coefficients, International Journal of Selection and Assessment, 1, 3, (158-162), (2007).
- David A. Dye, Martin Reck and Michael A. McDaniel, The Validity of Job Knowledge Measures, International Journal of Selection and Assessment, 1, 3, (153-157), (2007).
- Joseph J. Martocchio and Ellen M. Whitener, Fairness in Personnel Selection: A Meta-Analysis and Policy Implications, Human Relations, 45, 5, (489), (1992).
- Lawrence M. Rudner, Pre-Employment Testing and Employee Productivity, Public Personnel Management, 21, 2, (133), (1992).
- Maury Getkate, Peter Hausdorf and Steven F. Cronshaw, Transnational Validity Generalization of Employment Tests from the United States to Canada, Canadian Journal of Administrative Sciences / Revue Canadienne des Sciences de l'Administration, 9, 4, (324-335), (2009).
- STEVEN D. ASHWORTH, H. G. OSBURN, JOHN C. CALLENDER and KRISTIN A. BOYLE, THE EFFECTS OF UNREPRESENTED STUDIES ON THE ROBUSTNESS OF VALIDITY GENERALIZATION RESULTS, Personnel Psychology, 45, 2, (341-361), (2006).
- Particia Patterson, The validity generalization of skinfolds as measures of body density, American Journal of Human Biology, 4, 1, (115-124), (2005).
- Robert S. Kennedy and Janet J. Turnage, Reliability Generalization: A Viable Key for Establishing Validity Generalization, Perceptual and Motor Skills, 72, 1, (297), (1991).
- Joseph G. Rosse, Howard E. Miller and Laurie Keitel Barnes, Combining personality and cognitive ability predictors for hiring service-oriented employees, Journal of Business and Psychology, 10.1007/BF01014493, 5, 4, (431-445), (1991).
- Robert A. Roe and Martin A. M. Greuter, Developments in Personnel Selection Methodology, Advances in Educational and Psychological Testing: Theory and Applications, 10.1007/978-94-009-2195-5_7, (187-226), (1991).
- Heinz Schuler, Uwe Funke and Jutta Baron‐Boldt, Predictive Validity of School Grades ‐A Meta‐analysis, Applied Psychology, 39, 1, (89-103), (2008).
- Mike Allen and Raymond W. Preiss, Using meta‐analyses to evaluate curriculum: An examination of selected college textbooks, Communication Education, 39, 2, (103), (1990).
- Donna M. Randall, The consequences of organizational commitment: Methodological investigation, Journal of Organizational Behavior, 11, 5, (361-378), (2006).
- Samuel Messick, VALIDITY OF TEST INTERPRETATION AND USE, ETS Research Report Series, 1990, 1, (1487-1495), (2014).
- Kenneth P. Carson, Chester A. Schriesheim and Angelo J. Kinicki, The Usefulness of the "Fail-Safe" Statistic in Meta-Analysis, Educational and Psychological Measurement, 50, 2, (233), (1990).
- Samuel Messick, Meaning and Values in Test Validation: The Science and Ethics of Assessment, Educational Researcher, 18, 2, (5), (1989).
- Hannah R. Rothstein and Michael A McDaniel, Guidelines for Conducting and Reporting Meta-Analyses, Psychological Reports, 65, 3, (759), (1989).
- MICHAEL M. HARRIS, RECONSIDERING THE EMPLOYMENT INTERVIEW: A REVIEW OF RECENT LITERATURE AND SUGGESTIONS FOR FUTURE RESEARCH, Personnel Psychology, 42, 4, (691-726), (2006).
- Louise F Fitzgerald and James B Rounds, Vocational behavior, 1988: A critical analysis, Journal of Vocational Behavior, 35, 2, (105), (1989).
- Stephen B. Knouse, The Role of Attribution Theory in Personnel Employment Selection: A Review of the Recent Literature, The Journal of General Psychology, 116, 2, (183), (1989).
- MICHAEL A. McDANIEL, FRANK L. SCHMIDT and JOHN E. HUNTER, A META‐ANALYSIS OF THE VALIDITY OF METHODS FOR RATING TRAINING AND EXPERIENCE IN PERSONNEL SELECTION, Personnel Psychology, 41, 2, (283-309), (2006).
- Ramon J. Aldag and Timothy M. Steams, Issues in Research Methodology, Journal of Management, 14, 2, (253), (1988).
- MICHAEL M. HARRIS and JOHN SCHAUBROECK, A META-ANALYSIS OF SELF-SUPERVISOR, SELF-PEER, AND PEER-SUPERVISOR RATINGS, Personnel Psychology, 41, 1, (43), (1988).
- Samuel Messick, MEANING AND VALUES IN TEST VALIDATION: THE SCIENCE AND ETHICS OF ASSESSMENT1, ETS Research Report Series, 1988, 2, (i-28), (2014).
- Frank L Schmidt, The problem of group differences in ability test scores in employment selection, Journal of Vocational Behavior, 33, 3, (272), (1988).
- Roger E. Millsap, Tolerance Intervals: Alternatives to Credibility Intervals in Validity Generalization Research, Applied Psychological Measurement, 12, 1, (27), (1988).
- A. R. Montazemi and S. Wang, The Effects of Modes of Information Presentation on Decision-Making: A Review and Meta-Analysis, Journal of Management Information Systems, 5, 3, (101), (1988).
- RALPH A. ALEXANDER, GROUP HOMOGENEITY, RANGE RESTRICTION, AND RANGE ENHANCEMENT EFFECTS ON CORRELATIONS, Personnel Psychology, 41, 4, (773-777), (2006).
- Michael A. McDaniel and John W. Jones, Predicting employee theft: A quantitative review of the validity of a standardized measure of dishonesty, Journal of Business and Psychology, 10.1007/BF01013765, 2, 4, (327-345), (1988).
- Samuel Messick, VALIDITY, ETS Research Report Series, 1987, 2, (i-208), (2014).
- Philip M. Podsakoff and Dan R. Dalton, Research Methodology in Organizational Studies, Journal of Management, 13, 2, (419), (1987).
- Margaret J. Safrit, M. Glaucia Costa and Linda M. Hooper, The Validity Generalization Model: An Approach to the Analysis of Validity Studies in Physical Education, Research Quarterly for Exercise and Sport, 57, 4, (288), (1986).
- John F. Duffy and Andrew C. Peacock, Contingency Factors in the Effects of Rater Training on Interrater Agreement: Some Lint in the Bellybutton of Past Research, Canadian Journal of Administrative Sciences / Revue Canadienne des Sciences de l'Administration, 3, 2, (317-328), (2009).
- NEAL SCHMITT and RAYMOND A. NOE, ON SHIFTING STANDARDS FOR CONCLUSIONS REGARDING VALIDITY GENERALIZATION, Personnel Psychology, 39, 4, (849-851), (2006).
- HANNAH ROTHSTEIN HIRSH, LOIS C. NORTHROP and FRANK L. SCHMIDT, VALIDITY GENERALIZATION RESULTS FOR LAW ENFORCEMENT OCCUPATIONS, Personnel Psychology, 39, 2, (399), (1986).
- Julia E. Hoch, William H. Bommer, James H. Dulebohn and Dongyuan Wu, Do Ethical, Authentic, and Servant Leadership Explain Variance Above and Beyond Transformational Leadership? A Meta-Analysis, Journal of Management, 10.1177/0149206316665461, (014920631666546), (2016).
- Timothy A. Judge, Ronald F. Piccolo and Remus Ilies, The Forgotten Ones? The Validity of Consideration and Initiating Structure in Leadership Research., Journal of Applied Psychology, 10.1037/0021-9010.89.1.36, 89, 1, (36-51), (2004).
- Tammy D. Allen, Lillian T. Eby, Mark L. Poteet, Elizabeth Lentz and Lizzette Lima, Career Benefits Associated With Mentoring for Proteges: A Meta-Analysis., Journal of Applied Psychology, 10.1037/0021-9010.89.1.127, 89, 1, (127-136), (2004).
