Can Item Analysis of MCQs Accomplish the Need of a Proper Assessment Strategy for Curriculum Improvement in Medical Education?

Yogesh R. Pawade*, Dipti S. Diwase**
* Assistant Professor, Department of Biochemistry, Seth G.S. Medical College & K.E.M. Hospital, Mumbai, India.
** Assistant Professor, Department of Information Technology, K.J. Somaiya College of Engineering, Vidyavihar, Mumbai, India.
Periodicity:April - June'2016
DOI : https://doi.org/10.26634/jet.13.1.6017

Abstract

Item analysis of Multiple Choice Questions (MCQs) is the process of collecting, summarizing and utilizing information from students' responses to evaluate the quality of test items. Difficulty Index (p-value), Discrimination Index (DI) and Distractor Efficiency (DE) are the parameters which help to evaluate the quality of MCQs used in an examination. This study has been postulated to investigate the relationship of items having good p-value and DI with their DE and their utility to frame 'ideal questions'. This study further evaluates the MCQs as a tool of assessment so as to improve the curricula in Medical Education. In this study, 20 test items of ‘Type A’ MCQ tests of assessment were selected. The p-values, DI and DE were estimated. The relationship between the p-value and DI for each test item was determined by Pearson correlation analysis. Mean p-value and DI of the test were 66.53 ± 16.82% and 0.41 ± 0.16% respectively. Only 20% of total test items crossed the p-value of 80% indicative of their easy difficulty level. 95% of the test items showed acceptable (> 0.2) DI. 12 out of 20 test items showed excellent DI (≥ 0.4). 8 (40%) test items were regarded as ‘ideal’ having p-value from 30- 70, and DI > 0.24. Correlation studies revealed that, DI associated with p-value (r = -0.288; P = 0.219). Mean DE of the test was 76.25 ± 22.18%. The DE was directly related to the DI. Items with good and excellent DI had DE of 66.67 ± 14.43% and 83.33 ± 19.46% respectively. In conclusion, an acceptable level of test difficulty and discrimination was maintained in the type A MCQ test. The test items with excellent discrimination tend to be in the moderately difficult range. There was a consistent spread of difficulty in type A MCQ items used for the test. Much more of these kinds of analysis should be carried out after each examination to identify the areas of potential weakness in the type A MCQ tests to improve the standard of assessment.

Keywords

Item Analysis, Difficulty Index, Discrimination Index, Distractor Efficiency, Type A MCQ.

How to Cite this Article?

Pawade,Y.R., and Diwase, D.S. (2016). Can Item Analysis of MCQs Accomplish the Need of a Proper Assessment Strategy for Curriculum Improvement in Medical Education?. i-manager’s Journal of Educational Technology, 13(1), 44-53. https://doi.org/10.26634/jet.13.1.6017

References

[3]. E.N. Skakun, E. M. Nanson, W.C. Taylor, and S. Kling, (1977). “An investigation of three types of multiple choice questions”. Ann Conf. Res. Med Educ., Vol. 16, pp. 111-116.
[4]. A.M. Zubairi, and N.L.A. Kassim, (2006). “Classical and Rasch analysis of dichotomously scored reading comprehension test items”. Malaysian J. of. ELT. Res., Vol. 2, pp. 1-20.
[5]. A. Davies, (1990). Principles of Language Testing. Cambridge. Oxford: Basil blackwell Ltd.
[6]. S.L. Fowell, L.J. Southgate, and J.G. Bligh, (1999). “Evaluating assessment: the missing link?” Med Educ., Vol. 33, pp. 276-281.
[7]. R.G. Carroll, (1993). “Evaluation of vignette-type examination items for testing medical physiology”. Am. J. Physiol., Vol. 264, pp. S11-S15.
[8]. R.J. Dufresne, and W.J. Leonard, W.J. Gerace, (2002). “Making sense of student's answers to multiple-choice questions”. Phys Teach., Vol. 40, pp. 174-180.
[9]. M. Tarrant, J. Ware, (2008). “Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments”. Med. Educ., Vol. 42, pp. 198-206.
[10]. R.G. Carroll, (1993). “Evaluation of vignette-type examination items for testing medical physiology”. Am. J. Physiol., Vol. 264, pp. S11-S15.
[12]. Sim Si-Mui, and R.I. Rasiah, (2006). “Relationship between item difficulty and discrimination indices in true/false type multiple choice questions of a para-clinical multi disciplinary paper”. Ann. Acad. Med., Vol. 35, pp. 67-71.
[13]. T. L. Kelley, (1939). “The selection of upper and lower groups for validation of test items”. J. Educ. Psychol., Vol. 30, pp. 17-24.
[14]. R.L. Ebel, (1972). Essentials of Educational st Measurement. 1 Ed, New Jersey; Prentice Hall.
[15]. F. G. Brown, (1983). Principles of Educational and rd Psychological Testing, 3 Ed. New York: Holt, Rinehart and Winston.
[16]. L. Crocker, and J. Algina, (1986). Introduction to Classical and Modern Test Theory. New York: Holt, Rinehart and Winston.
[17]. Z. Meshkani, and H. Abadie, (2005).“Multivariate analysis of factors influencing reliability of teacher made tests”. Journal of Med. Ed., Vol. 6, pp. 149-159.
[18]. R.N. Marso, and F.L. Pigge, (1991).“An analysis of teacher made tests and item construction errors”. J. Contemp. Edu. Psych., Vol. 16, pp. 279-286.
[19]. D. Baner, V. Kopp, M.R. Fischer, (2007). “Answer changing in multiple choice assessment change that answer when in doubt and spread the word”. BMC Med. Education., Vol. 7, pp. 28.
[20]. J. Pellegrino, N. Chudowsky, R. Glaser, (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academic Press.
[21]. M.C. Rodriguez, (2005). “Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research”. Educational Measurement: Issues and Practice., Vol. 24, pp. 3-13.
[22]. M. Tarrant, J. Ware, and A. M. Mohammed, (2009). “An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis”. BMC Med. Educ., Vol. 9, pp. 40.
[23]. M. Tarrant, and J. Ware, (2010). “A comparison of the psychometric properties of three- and four-option multiplechoice questions in nursing assessments”. Nurse Educ. Today., Vol. 30, pp. 539-543.
[24]. R. Vyas, and A. Supe, (2008). “Multiple choice questions: a literature review on the optimal number of options”. Nat. Med. J. India., Vol. 21, pp. 130-133.
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Pdf 35 35 200 20
Online 35 35 200 15
Pdf & Online 35 35 400 25

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.