Design and validation of a dynamic assessment model based on the item response theory

Design and validation of a dynamic assessment model based on the item response theory

The goal of this paper is to describe an assessment model based on an item selection iterative process using a dynamic estimation of the student knowledge. The idea behind such a model is to achieve equilibrium between the progress demonstrated by the student and the items that are used during the assessment process. In this way two common phenomena are avoided: the boredom of 'advanced' students, and the frustration of the 'straggler' ones. For the design of such a model the Computerized Adaptive Testing approach was followed using the Item Response Theory and in particular the logistic model with three parameters. To validate the proposal a study was performed with 59 students in a real educational environment, speci cally within a college course in data structures. Such a study demonstrated, through hypothesis tests, not only a signi cant raise in the measuring accuracy with regard to the knowledge level of the examinees, but also a dropout in the mean assessment time.

Saved in:
Bibliographic Details
Main Authors: Moreno Cadavid, Julián, Montoya Gómez, Luis Fernando
Format: Digital revista
Language:spa
Published: Universidad Nacional de Colombia - Sede Medellín - Facultad de Ciencias 2015
Online Access:https://revistas.unal.edu.co/index.php/rfc/article/view/53174
Tags: Add Tag
No Tags, Be the first to tag this record!
id oai:www.revistas.unal.edu.co:article-53174
record_format ojs
spelling oai:www.revistas.unal.edu.co:article-531742018-06-09T22:32:33Z Design and validation of a dynamic assessment model based on the item response theory Diseño y validación de un modelo de evaluación dinámica basado en la teoría de respuesta al ítem Moreno Cadavid, Julián Montoya Gómez, Luis Fernando Item response theory assessment dynamic estimation 3PL Teoría de respuesta al ítem evaluación estimación dinámica 3PL The goal of this paper is to describe an assessment model based on an item selection iterative process using a dynamic estimation of the student knowledge. The idea behind such a model is to achieve equilibrium between the progress demonstrated by the student and the items that are used during the assessment process. In this way two common phenomena are avoided: the boredom of 'advanced' students, and the frustration of the 'straggler' ones. For the design of such a model the Computerized Adaptive Testing approach was followed using the Item Response Theory and in particular the logistic model with three parameters. To validate the proposal a study was performed with 59 students in a real educational environment, speci cally within a college course in data structures. Such a study demonstrated, through hypothesis tests, not only a signi cant raise in the measuring accuracy with regard to the knowledge level of the examinees, but also a dropout in the mean assessment time. El objetivo de este artículo es describir un modelo de evaluación basado en un proceso iterativo de selección de ítems a partir de una estimación dinámica del conocimiento del estudiante. La idea tras dicho modelo es lograr un equilibrio entre el progreso demostrado por el estudiante y la dificultad de los ítems que son usados durante el proceso de evaluación. De esta manera se logra evitar de cierta manera dos fenómenos comunes: el tedio en los estudiantes ‘avanzados’, y la frustración en los ‘rezagados’. Para el diseño de dicho modelo se siguió el enfoque de la Evaluación Adaptativa Computarizada usando la Teoría de Respuesta al Ítem y en particular el modelo logístico de tres parámetros. Para validar la propuesta se llevó a cabo un estudio con una muestra de 59 estudiantes en un ambiente educativo real, específicamente dentro de un curso universitario de estructuras de datos. Dicho estudio demostró, por medio de pruebas de hipótesis, no solo un incremento significativo en la precisión de la medición respecto al nivel de conocimiento de los examinados, sino también una disminución en el tiempo promedio de evaluación. Universidad Nacional de Colombia - Sede Medellín - Facultad de Ciencias 2015-07-01 info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion application/pdf https://revistas.unal.edu.co/index.php/rfc/article/view/53174 10.15446/rev.fac.cienc.v4n2.53174 Revista de la Facultad de Ciencias; Vol. 4 No. 2 (2015); 58-73 Revista de la Facultad de Ciencias; Vol. 4 Núm. 2 (2015); 58-73 2357-5549 0121-747X spa https://revistas.unal.edu.co/index.php/rfc/article/view/53174/54886 Derechos de autor 2015 Revista de la Facultad de Ciencias
institution UNAL
collection OJS
country Colombia
countrycode CO
component Revista
access En linea
databasecode rev-rfc
tag revista
region America del Sur
libraryname Sistema Nacional de Bibliotecas de la UNAL
language spa
format Digital
author Moreno Cadavid, Julián
Montoya Gómez, Luis Fernando
spellingShingle Moreno Cadavid, Julián
Montoya Gómez, Luis Fernando
Design and validation of a dynamic assessment model based on the item response theory
author_facet Moreno Cadavid, Julián
Montoya Gómez, Luis Fernando
author_sort Moreno Cadavid, Julián
title Design and validation of a dynamic assessment model based on the item response theory
title_short Design and validation of a dynamic assessment model based on the item response theory
title_full Design and validation of a dynamic assessment model based on the item response theory
title_fullStr Design and validation of a dynamic assessment model based on the item response theory
title_full_unstemmed Design and validation of a dynamic assessment model based on the item response theory
title_sort design and validation of a dynamic assessment model based on the item response theory
description The goal of this paper is to describe an assessment model based on an item selection iterative process using a dynamic estimation of the student knowledge. The idea behind such a model is to achieve equilibrium between the progress demonstrated by the student and the items that are used during the assessment process. In this way two common phenomena are avoided: the boredom of 'advanced' students, and the frustration of the 'straggler' ones. For the design of such a model the Computerized Adaptive Testing approach was followed using the Item Response Theory and in particular the logistic model with three parameters. To validate the proposal a study was performed with 59 students in a real educational environment, speci cally within a college course in data structures. Such a study demonstrated, through hypothesis tests, not only a signi cant raise in the measuring accuracy with regard to the knowledge level of the examinees, but also a dropout in the mean assessment time.
publisher Universidad Nacional de Colombia - Sede Medellín - Facultad de Ciencias
publishDate 2015
url https://revistas.unal.edu.co/index.php/rfc/article/view/53174
work_keys_str_mv AT morenocadavidjulian designandvalidationofadynamicassessmentmodelbasedontheitemresponsetheory
AT montoyagomezluisfernando designandvalidationofadynamicassessmentmodelbasedontheitemresponsetheory
AT morenocadavidjulian disenoyvalidaciondeunmodelodeevaluaciondinamicabasadoenlateoriaderespuestaalitem
AT montoyagomezluisfernando disenoyvalidaciondeunmodelodeevaluaciondinamicabasadoenlateoriaderespuestaalitem
_version_ 1798167560469872640

Similar Items