Northwest Evaluation Association Researchers Develop Framework for Evaluating Computerized Adaptive Tests

Share Article

K-12 assessment experts’ proposed process for evaluating alignment of computerized adaptive tests to standards was recently published in the peer-reviewed journal Educational Measurement: Issues and Practice.

News Image

Northwest Evaluation Association (NWEA) Senior Research Fellow Steven Wise, along with former NWEA Vice President and psychometrician Dr. Gage Kingsbury and Dr. Norman Webb, a leading developer of alignment models for assessment, has developed a new framework for evaluating the alignment of computerized adaptive tests (CATs). The proposed process is the first of its kind to be published, and recently appeared in the National Council on Measurement in Education’s Educational Measurement: Issues and Practice.

Psychometricians have long relied on established methods to evaluate how well a given traditional or “fixed-form” assessment measures student knowledge, but these methods are not well suited for adaptive assessments. CATs are designed to provide efficient measurement, by adjusting to the individual student’s level of skill and knowledge. In practice, every student might experience a different set of questions or test items, depending on his or her performance. In this paper, researchers Wise, Kingsbury and Webb address the challenge of evaluating how well a CAT measures student performance against a particular set of standards.

The authors emphasize that evaluators must consider the purpose of an adaptive assessment in order to understand how alignment should be evaluated. If, for example, the purpose is to measure whether or not students have achieved grade-level proficiency, limiting items to those aligned to a student’s grade-level would be appropriate. If, instead, the assessment is intended to measure the full range of what a given student knows and can do, it may be more appropriate to include items below or above the grade-level standards. The paper, “Evaluating Content Alignment in Computerized Adaptive Testing,” provides specific criteria for quantifying both alignment to content as specified by a test plan and the degree to which the items administered to a given student were of appropriate difficulty and challenge.

States, school districts and assessment developers may soon be taking a closer look at CAT alignment, given the recent reauthorization of the federal Elementary and Secondary Education Act. The Every Student Succeeds Act (ESSA) provides greater flexibility for states and districts to use computerized adaptive testing to meet accountability requirements.

###
About Northwest Evaluation Association
Northwest Evaluation Association™ (NWEA™) is a global not-for-profit educational services organization known for our suite of assessments, including our flagship interim assessment, Measures of Academic Progress® (MAP®), and Skills Navigator®, a skills mastery and progress monitoring tool. More than 7,600 partners in U.S. schools, school districts, education agencies, and international schools trust us to offer pre‑kindergarten through grade 12 assessments that accurately measure student growth and mastery, professional development that fosters educators' ability to accelerate student learning, and research that supports assessment validity and data interpretation. To better inform instruction and maximize every learner's academic growth, educators currently use NWEA assessments with nearly eight million students. Learn more at nwea.org.

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Jean Fleming
Visit website