Volume 21, Issue 4

APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS

DAVID J. WEISS

Corresponding Author

University of Minnesota

DAVID J. WEISS, Professor, Department of Psychology, N660 Elliott Hall, University of Minnesota, Minneapolis, MN 55455. Degrees: BA, University of Pennsylvania; PhD, University of Minnesota. Specializations: Computerized adaptive testing, latent trait test theory.

G. GAGE KINGSBURY, Testing and Evaluation Specialist, Portland Public Seheols, Education Department, 501 N. Dixon St., PC) Box 3107, Portland, OR 97208. Degrees: BA, Ohio State University; PhD, University of Minnesota. Specializations: Computerized adaptive testing, item response theory.

Search for more papers by this author
G. GAGE KINGSBURY

Corresponding Author

University of Minnesota

DAVID J. WEISS, Professor, Department of Psychology, N660 Elliott Hall, University of Minnesota, Minneapolis, MN 55455. Degrees: BA, University of Pennsylvania; PhD, University of Minnesota. Specializations: Computerized adaptive testing, latent trait test theory.

G. GAGE KINGSBURY, Testing and Evaluation Specialist, Portland Public Seheols, Education Department, 501 N. Dixon St., PC) Box 3107, Portland, OR 97208. Degrees: BA, Ohio State University; PhD, University of Minnesota. Specializations: Computerized adaptive testing, item response theory.

Search for more papers by this author
First published: December 1984
Citations: 175

The methods and data presented in this paper were developed under contract N00014‐79‐C‐0172, NR 150‐433, from the Office of Naval Research, with additional funding from the Air Force Office of Scientific Research, the Air Force Human Resources Laboratory, and the Army Research Institute.

Abstract

Three applications of computerized adaptive testing (CAT) to help solve problems encountered in educational settings are described and discussed. Each of these applications makes use of item response theory to select test questions from an item pool to estimate a student's achievement level and its precision. These estimates may then be used in conjunction with certain testing strategies to facilitate certain educational decisions. The three applications considered are (a) adaptive mastery testing for determining whether or not a student has mastered a particular content area, (b) adaptive grading for assigning grades to students, and (c) adaptive self‐referenced testing for estimating change in a student's achievement level. Differences between currently used classroom procedures and these CAT procedures are discussed. For the adaptive mastery testing procedure, evidence from a series of studies comparing conventional and adaptive testing procedures is presented showing that the adaptive procedure results in more accurate mastery classifications than do conventional mastery tests, while using fewer test questions.

Number of times cited according to CrossRef: 175

  • Pass/Fail and Other Dichotomies, The Art of Modelling the Learning Process, 10.1007/978-3-030-43082-5_5, (83-102), (2020).
  • Technologiegestütztes Assessment, Online Assessment, Handbuch Bildungstechnologie, 10.1007/978-3-662-54368-9, (493-513), (2020).
  • Integrating Educational Content into Game: An Encapsulation Method, Cognitive Cities, 10.1007/978-981-15-6113-9_42, (367-372), (2020).
  • Machine Learning–Driven Language Assessment, Transactions of the Association for Computational Linguistics, 10.1162/tacl_a_00310, 8, (247-263), (2020).
  • undefined, 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), 10.1109/ICIRCA48905.2020.9183068, (723), (2020).
  • Adapting cognitive diagnosis computerized adaptive testing item selection rules to traditional item response theory, PLOS ONE, 10.1371/journal.pone.0227196, 15, 1, (e0227196), (2020).
  • Stopping rules for multi‐category computerized classification testing, British Journal of Mathematical and Statistical Psychology, 10.1111/bmsp.12202, 0, 0, (2020).
  • Procedures to Develop a Computerized Adaptive Testing to Advance the Measurement of Narcissistic Personality, Frontiers in Psychology, 10.3389/fpsyg.2020.01437, 11, (2020).
  • Hypothesis Testing Methods for Multivariate Multi-Occasion Intra-Individual Change, Multivariate Behavioral Research, 10.1080/00273171.2020.1730739, (1-17), (2020).
  • Development of a Multidimensional Computerized Adaptive Test based on the Bifactor Model, International Journal of Assessment Tools in Education, 10.21449/ijate.707199, (323-342), (2020).
  • An Intelligent Recommender System for Personalized Test Administration Scheduling With Computerized Formative Assessments, Frontiers in Education, 10.3389/feduc.2020.572612, 5, (2020).
  • Look‐ahead content balancing method in variable‐length computerized classification testing, British Journal of Mathematical and Statistical Psychology, 10.1111/bmsp.12165, 73, 1, (88-108), (2019).
  • Computerized adaptive testing and short form development for child and adolescent oral health patient‐reported outcomes measurement, Clinical and Experimental Dental Research, 10.1002/cre2.259, 6, 1, (124-133), (2019).
  • Development of a computerized adaptive screening tool for overall psychopathology (“p”), Journal of Psychiatric Research, 10.1016/j.jpsychires.2019.05.028, (2019).
  • Measuring Burden of Care After Catastrophic Illness or Injury, Physical Medicine and Rehabilitation Clinics of North America, 10.1016/j.pmr.2019.03.005, (2019).
  • A Report on the Application of Adaptive Testing in a First Year University Course, Learning Technology for Education Challenges, 10.1007/978-3-030-20798-4_38, (439-449), (2019).
  • Applying Adaptive Approaches to Talent Management Practices, The Cambridge Handbook of Technology and Employee Behavior, 10.1017/9781108649636, (131-150), (2019).
  • On the Design and Development of an Assessment System with Adaptive Capabilities, Methodologies and Intelligent Systems for Technology Enhanced Learning, 8th International Conference, 10.1007/978-3-319-98872-6_23, (190-199), (2019).
  • Strumentazioni e pratiche digitali nella valutazione degli apprendimenti, RIV Rassegna Italiana di Valutazione, 10.3280/RIV2018-070005, 70, (75-97), (2019).
  • Efficiency of Targeted Multistage Calibration Designs Under Practical Constraints: A Simulation Study, Journal of Educational Measurement, 10.1111/jedm.12203, 56, 1, (121-146), (2019).
  • Summative Game-Based Assessment, Game-Based Assessment Revisited, 10.1007/978-3-030-15569-8_3, (37-65), (2019).
  • Operationalizing Computerized Testing in Mathematics Competition, Redesigning Higher Education Initiatives for Industry 4.0, 10.4018/978-1-5225-7832-1.ch012, (204-220), (2019).
  • Bob - A General Culture Game with Voice Interaction, Procedia Computer Science, 10.1016/j.procs.2019.09.187, 159, (323-332), (2019).
  • Personalized adaptive learning: an emerging pedagogical approach enabled by a smart learning environment, Smart Learning Environments, 10.1186/s40561-019-0089-y, 6, 1, (2019).
  • Termination Rules for Variable-Length CD-CAT From the Information Theory Perspective, Frontiers in Psychology, 10.3389/fpsyg.2019.01122, 10, (2019).
  • Dynamic Multistage Testing: A Highly Efficient and Regulated Adaptive Testing Method, International Journal of Testing, 10.1080/15305058.2019.1621871, (1-21), (2019).
  • Investigating Consequences of Using Item Pre-knowledge in Computerized Multistage Testing, Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 10.17152/gefad.535376, (2019).
  • A LASSO-Based Method for Detecting Item-Trait Patterns of Replenished Items in Multidimensional Computerized Adaptive Testing, Frontiers in Psychology, 10.3389/fpsyg.2019.01944, 10, (2019).
  • Investigation of Item Selection Methods According to Test Termination Rules in CAT Applications, Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 10.21031/epod.530528, (315-326), (2019).
  • Computer Adaptive Testing Simulations in R, International Journal of Assessment Tools in Education, 10.21449/ijate.621157, (44-56), (2019).
  • A Hybrid Multi-Agent based BFPSO Algorithm for Optimization of Benchmark Functions, Journal of Circuits, Systems and Computers, 10.1142/S0218126620501121, (2019).
  • So many questions, so little time: Integrating adaptive inventories into public opinion research, Journal of Survey Statistics and Methodology, 10.1093/jssam/smz027, (2019).
  • Controlling construct-irrelevant factors through computer-based testing: disengagement, anxiety, & cheating, Education Inquiry, 10.1080/20004508.2018.1490127, 10, 1, (21-33), (2018).
  • Technologiegestütztes Assessment, Online Assessment, Lernen mit Bildungstechnologien, 10.1007/978-3-662-54373-3, (1-21), (2018).
  • Dynamic Assessment and Computerized Adaptive Tests in Reading Processes, Journal of Cognitive Education and Psychology, 10.1891/1945-8959.17.1.70, 17, 1, (70-96), (2018).
  • Constructing Shadow Tests in Variable-Length Adaptive Testing, Applied Psychological Measurement, 10.1177/0146621617753736, 42, 7, (538-552), (2018).
  • Online assessment, adaptive feedback and the importance of visual learning for students. The advantages, with a few caveats, of using MapleTA, International Review of Economics Education, 10.1016/j.iree.2018.03.002, 28, (11-28), (2018).
  • Adapting Marketing Surveys to Individual Respondents, Market Research Society. Journal., 10.1177/147078539203400202, 34, 2, (1-19), (2018).
  • Adaptive Designs for Likert-Type Data: An Approach for Implementing Marketing Surveys, Journal of Marketing Research, 10.1177/002224379002700305, 27, 3, (304-321), (2018).
  • Measuring Consumer Attitudes toward the Marketplace with Tailored Interviews, Journal of Marketing Research, 10.1177/002224378902600305, 26, 3, (311-326), (2018).
  • Comparing computer adaptive testing stopping rules under the generalized partial-credit model, Behavior Research Methods, 10.3758/s13428-018-1068-x, (2018).
  • Bireyselleştirilmiş Bilgisayarlı Sınıflama Testi Kriterlerinin Test Etkililiği ve Ölçme Kesinliği Açısından Karşılaştırılması, Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 10.21031/epod.401077, (162-178), (2018).
  • A genetic algorithm to find optimal reading test word subsets for estimating full-scale IQ, PLOS ONE, 10.1371/journal.pone.0205754, 13, 10, (e0205754), (2018).
  • Beyond test scores: A framework for reporting mathematics assessment results to enhance teaching and learning, Pythagoras, 10.4102/pythagoras.v39i1.393, 39, 1, (2018).
  • Variable-Length Stopping Rules for Multidimensional Computerized Adaptive Testing, Psychometrika, 10.1007/s11336-018-9644-7, (2018).
  • Plus ÇA Change, plus C’Est Pareil: Making a Continued Case for the Use of MCQs in Medical Education, Medical Teacher, 10.1080/0142159X.2018.1505035, (1-9), (2018).
  • Projection-Based Stopping Rules for Computerized Adaptive Testing in Licensure Testing, Applied Psychological Measurement, 10.1177/0146621617726790, 42, 4, (275-290), (2017).
  • undefined, Proceedings of the 2nd international Conference on Big Data, Cloud and Applications - BDCA'17, 10.1145/3090354.3090448, (1-8), (2017).
  • Is a Computerized Adaptive Test More Motivating Than a Fixed-Item Test?, Applied Psychological Measurement, 10.1177/0146621617707556, 41, 7, (495-511), (2017).
  • Computerized Adaptive Testing for Public Opinion Surveys, Political Analysis, 10.1093/pan/mps060, 21, 2, (172-192), (2017).
  • Overview and current management of computerized adaptive testing in licensing/certification examinations, Journal of Educational Evaluation for Health Professions, 10.3352/jeehp.2017.14.17, 14, (17), (2017).
  • Basit ve Karmaşık Test Desenlerinde Çok Boyutlu Madde Seçme Yöntemlerinin Karşılaştırılması, Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 10.21031/epod.286956, (34-34), (2017).
  • The impacts of computer adaptive testing from a variety of perspectives, Journal of Educational Evaluation for Health Professions, 10.3352/jeehp.2017.14.12, 14, (12), (2017).
  • Development and Psychometric Evaluation of the Hansen Research Services Matrix Adaptive Test: A Measure of Nonverbal IQ, Journal of Autism and Developmental Disorders, 10.1007/s10803-016-2932-0, 49, 7, (2721-2732), (2016).
  • A Computer Adaptive Measure of Delay Discounting, Assessment, 10.1177/1073191116680448, 25, 8, (1036-1055), (2016).
  • Strategic planning tools for large-scale technology-based assessments, Assessment in Education: Principles, Policy & Practice, 10.1080/0969594X.2016.1173013, 25, 2, (200-223), (2016).
  • The Impact of Item Dependency on the Efficiency of Testing and Reliability of Student Scores From a Computer Adaptive Assessment of Reading Comprehension, Journal of Research on Educational Effectiveness, 10.1080/19345747.2016.1178361, 10, 2, (408-423), (2016).
  • The Measurement of Intelligence in the XXI Century using Video Games, The Spanish Journal of Psychology, 10.1017/sjp.2016.84, 19, (2016).
  • Common Core State Standards and Response to Intervention: The Importance of Assessment, Intervention, and Progress Monitoring, Handbook of Response to Intervention, 10.1007/978-1-4899-7568-3, (165-184), (2016).
  • An Item Response Theory–Based, Computerized Adaptive Testing Version of the MacArthur–Bates Communicative Development Inventory: Words & Sentences (CDI:WS), Journal of Speech, Language, and Hearing Research, 10.1044/2015_JSLHR-L-15-0202, 59, 2, (281-289), (2016).
  • On the functional form of temporal discounting: An optimized adaptive test, Journal of Risk and Uncertainty, 10.1007/s11166-016-9242-y, 52, 3, (233-254), (2016).
  • Multidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality, Applied Psychological Measurement, 10.1177/0146621616648931, 40, 6, (387-404), (2016).
  • Computerized Adaptive Diagnosis and Testing of Mental Health Disorders, Annual Review of Clinical Psychology, 10.1146/annurev-clinpsy-021815-093634, 12, 1, (83-104), (2016).
  • undefined, Proceedings of the 16th Koli Calling International Conference on Computing Education Research - Koli Calling '16, 10.1145/2999541.2999544, (145-149), (2016).
  • Item Response Theory and its Applications to Patient-Reported Outcomes Measurement, Evaluation & the Health Professions, 10.1177/0163278705278275, 28, 3, (264-282), (2016).
  • Book Review: Computer-Optimized Multimedia Intelligence Test, Journal of Psychoeducational Assessment, 10.1177/073428290101900309, 19, 3, (279-285), (2016).
  • Computerized Adaptive Testing for Classifying Examinees into three Categories, Educational and Psychological Measurement, 10.1177/00131640021970862, 60, 5, (713-734), (2016).
  • Putting the process in developmental processes, International Journal of Behavioral Development, 10.1080/01650250050118277, 24, 3, (295-300), (2016).
  • Strategies for Computerized Adaptive Grading Testing, Applied Psychological Measurement, 10.1177/01466219922031266, 23, 2, (136-146), (2016).
  • Comparability of the Paper-and-Pencil and Computerised Adaptive Versions of the General Scholastic Aptitude Test (GSAT) Senior, South African Journal of Psychology, 10.1177/008124639802800104, 28, 1, (21-27), (2016).
  • Comparison of SPRT and Sequential Bayes Procedures for Classifying Examinees Into Two Categories Using a Computerized Test, Journal of Educational and Behavioral Statistics, 10.3102/10769986021004405, 21, 4, (405-414), (2016).
  • Conducting Self-Adapted Testing Using Microcat, Educational and Psychological Measurement, 10.1177/0013164496056005008, 56, 5, (821-827), (2016).
  • Multidimensional Computerized Adaptive Testing in a Certification or Licensure Context, Applied Psychological Measurement, 10.1177/014662169602000406, 20, 4, (389-404), (2016).
  • Computerized Adaptive Testing: Tracking Candidate Response Patterns, Journal of Educational Computing Research, 10.2190/GNB7-5HQ0-GD8M-P0B2, 13, 2, (151-162), (2016).
  • The Effect of Review on Student Ability and Test Efficiency for Computerized Adaptive Tests, Applied Psychological Measurement, 10.1177/014662169201600103, 16, 1, (33-40), (2016).
  • The Influence of Dimensionality on Cat Ability Estimation, Educational and Psychological Measurement, 10.1177/0013164492052003002, 52, 3, (513-528), (2016).
  • Confidence in Pass/Fail Decisions for Computer Adaptive and Paper and Pencil Examinations, Evaluation & the Health Professions, 10.1177/016327879201500407, 15, 4, (453-464), (2016).
  • Analysis of Patterns in Time: A Method of Recording and Quantifying Temporal Relations in Education, American Educational Research Journal, 10.3102/00028312027001180, 27, 1, (180-204), (2016).
  • Estimating Reliabilities of Computerized Adaptive Tests, Applied Psychological Measurement, 10.1177/014662168901300204, 13, 2, (145-149), (2016).
  • Bayesian Adaptation during Computer-Based Tests and Computer-Guided Practice Exercises, Journal of Educational Computing Research, 10.2190/60HH-77DG-WK36-PG43, 5, 1, (89-114), (2016).
  • Chapter 6: Recent Developments in Item Response Theory with Implications for Teacher Certification, Review of Research in Education, 10.3102/0091732X014001239, 14, 1, (239-275), (2016).
  • A Comparison of the Relative Efficiency and Validity of Tailored Tests and Conventional Quizzes, Evaluation & the Health Professions, 10.1177/016327878701000106, 10, 1, (67-79), (2016).
  • An Estimator of Examinee-Level Measurement Error Variance That Considers Test Form Difficulty Adjustments, Applied Psychological Measurement, 10.1177/014662168601000209, 10, 2, (175-186), (2016).
  • An Item Bank of Diabetes-Related Test Questions, The Diabetes Educator, 10.1177/014572178501100306, 11, 3, (37-41), (2016).
  • On Computing the Key Probability in the Stochastically Curtailed Sequential Probability Ratio Test, Applied Psychological Measurement, 10.1177/0146621615611633, 40, 2, (142-156), (2015).
  • Improving Comprehension Assessment for Middle and High School Students: Challenges and Opportunities, Improving Reading Comprehension of Middle and High School Students, 10.1007/978-3-319-14735-2_6, (119-151), (2015).
  • Development and Simulation Testing of a Computerized Adaptive Version of the Philadelphia Naming Test, Journal of Speech, Language, and Hearing Research, 10.1044/2015_JSLHR-L-14-0297, 58, 3, (878-890), (2015).
  • Components of Education Quality Monitoring: Problems and Prospects, Procedia - Social and Behavioral Sciences, 10.1016/j.sbspro.2015.11.600, 214, (103-111), (2015).
  • Optimization of High-Dimensional Functions through Hypercube Evaluation, Computational Intelligence and Neuroscience, 10.1155/2015/967320, 2015, (1-11), (2015).
  • Considering the Use of General and Modified Assessment Items in Computerized Adaptive Testing, Applied Measurement in Education, 10.1080/08957347.2014.1002921, 28, 2, (156-167), (2015).
  • Teste Adaptativo Computadorizado Multidimensional com propósitos educacionais: princípios e métodos, Ensaio: Avaliação e Políticas Públicas em Educação, 10.1590/S0104-40362015000100016, 23, 87, (389-414), (2015).
  • Validation of the Neuro-QoL measurement system in children with epilepsy, Epilepsy & Behavior, 10.1016/j.yebeh.2015.02.038, 46, (209-214), (2015).
  • Educational Testing: Measuring and Remedying Achievement Gaps, Emerging Trends in the Social and Behavioral Sciences, 10.1002/9781118900772, (1-14), (2015).
  • DISEÑO Y VALIDACIÓN DE UN MODELO DE EVALUACIÓN DINÁMICA BASADO EN LA TEORÍA DE RESPUESTA AL ÍTEM, Revista de la Facultad de Ciencias, 10.15446/rev.fac.cienc.v4n2.53174, 4, 2, (58-73), (2015).
  • Education, learning and assessment: current trends and best practice for medical educators, Irish Journal of Medical Science (1971 -), 10.1007/s11845-014-1069-4, 184, 1, (1-12), (2014).
  • New Item Selection Methods for Cognitive Diagnosis Computerized Adaptive Testing, Applied Psychological Measurement, 10.1177/0146621614554650, 39, 3, (167-188), (2014).
  • Stochastic Curtailment in Adaptive Mastery Testing, Applied Psychological Measurement, 10.1177/0146621614561314, 39, 4, (278-292), (2014).
  • An Automated Dialog System for Conducting Security Interviews for Access Control, Critical Infrastructure Protection VIII, 10.1007/978-3-662-45355-1_8, (111-125), (2014).
  • Toward Understanding the Potential of Games for Learning: Learning Theory, Game Design Characteristics, and Situating Video Games in Classrooms, Computers in the Schools, 10.1080/07380569.2014.890879, 31, 1-2, (2-22), (2014).
  • Examining the ™ Assessment for Job Fit and Core Capabilities of ™, ETS Research Report Series, 10.1002/ets2.12040, 2014, 2, (1-43), (2014).
  • See more

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.