i-manager Publications

Measurement and Evaluation Challenges in Distance Education: Systematic Review and Qualitative Meta-Synthesis

N. Bilge Uzun*, Mehtap Aktaş**, Cenk Akay***

*-*** Mersin University, Turkey.

** Trakya University, Turkey.

Periodicity:January - March'2023
DOI : https://doi.org/10.26634/jet.19.4.19368

Abstract

The challenges experienced in measurement and evaluation during the distance education process among student and instructor groups are discussed in the study. A qualitative meta-synthesis method is used in this research. Twenty studies were included in the meta-synthesis. The challenges experienced by the instructors are program utilization, technological inadequacies, a lack of in-service seminars, online exam diversity, inconsistency between what is measured and what is intended to be measured, interference of other features with the measurement, unsuitability of the measurement tools, academic dishonesty, lack of control over online exam diversity, preparation and grading challenges, infrastructural issues, and a lack or latency of feedback. Challenges experienced by learners include alienation to the electronic environment, language, technological inadequacies, lack of materials, validity-reliability issues, application difficulty, lack of guidance in exams, infrastructural issues, network problems, lack of interest and motivation, anxiety about online exams, and avoidance and resistance to self-disclosure in virtual environments.

Keywords

Measurement and Evaluation, Distance Education, Challenges, Meta-Synthesis, Systematic Review.

How to Cite this Article?

Uzun, N. B., Aktaş, M., and Akay, C. (2023). Measurement and Evaluation Challenges in Distance Education: Systematic Review and Qualitative Meta-Synthesis. i-manager’s Journal of Educational Technology, 19(4), 1-23. https://doi.org/10.26634/jet.19.4.19368

References

[1]. Allen, I. E., & Seaman, J. (2008). Staying the Course: Online Education in the United States, 2008. Sloan Consortium.

[2]. Badu-Nyarko, S. K., & Amponsah, S. (2017). Assessment of challenges in distance education at University of Ghana. Indian Journal of Open Learning, 25(2), 87-103.

[3]. Bozkurt, A., Jung, I., Xiao, J., Vladimirschi, V., Schuwer, R., Egorov, G., ... & Paskevicius, M. (2020). A global outlook to the interruption of education due to COVID-19 pandemic: Navigating in a time of uncertainty and crisis. Asian Journal of Distance Education, 15(1), 1-126.

[4]. Brown, K., & Lally, V. (2018). Rhetorical relationships with students: A higher education case study of perceptions of online assessment in mathematics. Research in Comparative and International Education, 13(1), 7-26.

[5]. Buheji, M., & Ahmed, D. (2020). Foresight of coronavirus (COVID-19) opportunities for a better world. American Journal of Economics, 10(2), 97-108.

[6]. Cabı, E. (2016). Uzaktan eğitimde e-değerlendirme üzerine öğrenci algıları. Yükseköğretim ve Bilim Dergisi, (1), 94-101.

[7]. Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., & Zheng, J. (2020). The psychological impact of the COVID- 19 epidemic on college students in China. Psychiatry Research, 287, 112934.

[8]. Chang, F. C. I. (2002). Intelligent assessment of distance learning. Information Sciences, 140(1-2), 105-125.

[9]. Chapman, K. J., Davis, R., Toy, D., & Wright, L. (2004). Academic integrity in the business school environment: I'll get by with a little help from my friends. Journal of Marketing Education, 26(3), 236-249.

[10]. Chaudhary, S., & Dey, N. (2013). Assessment in open and distance learning system (ODL): A challenge. Open Praxis, 5(3), 207-216.

[11]. Clariana, R. B. (1993). A review of multiple-try feedback in traditional and computer-based instruction. Journal of Computer-Based Instruction, 20(3), 67-74.

[12]. Crocker, L., & Algina, J. (1986). Introduction to Classical and Modern Test Theory. Holt, Rinehart and Winston.

[13]. d'Orville, H. (2020). COVID-19 causes unprecedented educational disruption: Is there a road towards a new normal? Prospects, 49(1-2), 11-15.

[14]. Dietel, R. J., Herman, J. L., & Knuth, R. A. (1991). What does research say about assessment. North Central Regional Educational Laboratory, 1, 18.

[15]. Erwin, E. J., Brotherson, M. J., & Summers, J. A. (2011). Understanding qualitative metasynthesis: Issues and opportunities in early childhood intervention research. Journal of Early Intervention, 33(3), 186-200.

[16]. Fei, W. F., Abd Aziz, M. S., Azman, H., & Razak, N. (2005). An online assessment of language proficiency: development and challenges. Internet Journal of e-Language Learning & Teaching, 2(1), 13-24.

[17]. Finfgeld-Connett, D. (2018). A Guide to Qualitative Meta-Synthesis. Routledge.

[18]. Ghilay, Y., & Ghilay, R. (2013). OTLA: A new model for online teaching, learning and assessment in higher education. i-manager's Journal of Educational Technology, 10(1), 10-21.

[19]. Gikandi, J. W. (2012). Online Formative Assessment in Higher Education: Enhancing Continuing Teacher Education in E-Learning (Doctoral Dissertation, University of Canterbury).

[20]. Gil-Jaurena, I., Domínguez-Figaredo, D., & Ballesteros-Velázquez, B. (2022). Learning outcomes based assessment in distance higher education. A case study. Open Learning: The Journal of Open, Distance and e-Learning, 37(2), 193-208.

[21]. Guilford, J. P., & Fruchter, B. (1973). Fundamental Statistics in Psychology and Education. McGraw-Hill.

[22]. Gulliksen, H. (1950). Theory of Mental Tests. John Wiley & Sons.

[23]. Hannafin, M., Hill, J. R., Oliver, K., Glazer, E., & Sharma, P. (2003). Cognitive and learning factors in webbased distance learning environments. Handbook of Distance Education, 245-260.

[24]. Hatzipanagos, S., & Warburton, S. (2009). Feedback as dialogue: Exploring the links between formative assessment and social software in distance learning. Learning, Media and Technology, 34(1), 45-59.

[25]. Henly, D. C. (2003). Use of Web based formative assessment to support student learning in a metabolism/nutrition unit. European Journal of Dental Education, 7(3), 116-122.

[26]. Hillier, M. (2014, January). The very idea of e-Exams: student (pre) conceptions. In Proceedings of ASCILITE 2014-Annual Conference of the Australian Society for Computers in Tertiary Education (pp. 77-88).

[27]. Iskenderoglu, M., Iskenderoglu, T. A., & Palanci, M. (2012). Opinion of teaching staff in distance education systems, regarding the assessment and evaluation process. Procedia-Social and Behavioral Sciences, 46, 4661-4665.

[28]. Jian, N., & Hamp Lyons, L. (2006). Progress assessment in Chinese distance education: the voices of learners. Open Learning: The Journal of Open, Distance and e-Learning, 21(2), 111-123.

[29]. Jung, I. Y., & Yeom, H. Y. (2009). Enhanced security for online exams using group cryptography. IEEE Transactions on Education, 52(3), 340-349.

[30]. Karal, H., Çebi, A., Pekşen, M., Turgut, Y. E., & Merkezi, U. E. (2010). Sözel problemlerin anlamlandırılması ve çözümünde web tabanlı eğitsel simülasyonların etkisi. University of Gaziantep Journal of Social Sciences, 9(1), 147-162.

[31]. Kaya, Z. (2002). Uzaktan Eğitim. Pegem A Yayıncılık.

[32]. Kearns, L. R. (2012). Student assessment in online learning: Challenges and effective practices. Journal of Online Learning and Teaching, 8(3), 198-208.

[33]. King, C. G., Guyette Jr, R. W., & Piotrowski, C. (2009). Online exams and cheating: An empirical analysis of business students' views. Journal of Educators Online, 6(1), 1-11.

[34]. Koutsoupidou, T. (2014). Online distance learning and music training: benefits, drawbacks and challenges. Open Learning: The Journal of Open, Distance and e- Learning, 29(3), 243-255.

[35]. Kürtüncü, M., & Aylin, K. (2020). COVID-19 pandemisi döneminde hemşirelik öğrencilerinin uzaktan eğitim konusunda yaşadiklari sorunlar. Avrasya Sosyal ve Ekonomi Araştırmaları Dergisi, 7(5), 66-77.

[36]. Lanier, M. M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244-261.

[37]. Livingston, S. A. (1988). Reliability of Test Results. J. P. Keeves (ed.) Educational Research, Methodology and Measurement: An International Handbook (pp. 386-392). Pergamon Press, Oxford, England.

[38]. Lord, F. M., Novick, M. R., & Birnbaum, A. (1968). Statistical Theories of Mental Test Scores. Addison-Wesley.

[39]. Magnusson, D. (1967). Test Theory. Addison-Wesley.

[40]. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta- analyses: The PRISMA statement. PLoS Med, 6(7), 1-8.

[41]. Moore, M. G., & Kearsley, G. (2011). Distance Education: A Systems View of Online Learning. Cengage Learning.

[42]. Nyika, R. (2016). Use of E-Formative assessment in open [9] and Distance Learning (ODL): Possibilities and challenges of open and distance university tutors in Zimbabwe. International Journal of Advanced Research in Education & Technology, 3(4), 127-131.

[43]. Ogange, B. O., Agak, J. O., Okelo, K. O., & Kiprotich, P. (2018). Student perceptions of the effectiveness of formative assessment in an online learning environment. Open Praxis, 10(1), 29-39.

[44]. Ozer, B., & Ustun, E. (2020). Evaluation of students' views on the covid-19 distance education process in music departments of fine arts faculties. Asian Journal of Education and Training, 6(3), 556-568.

[45]. Pace, D. S. (2020). The use of Formative Assessment (FA) in Online Teaching and Learning during the COVID-19 compulsory education school closure: the Maltese experience. Malta Review of Educational Research, 14(2), 243-271.

[46]. PHA. (2020). COVID-19 (Coronavirus). Lymphatic Research and Biology, 18(2), 99-100.

[47]. Ramu, T., & Arivoli, T. (2013). A framework of secure biometric based online exam authentication: an alternative to traditional exam. International Journal of Scientific & Engineering Research, 4(11), 52-60.

[48]. Rofiqo, K. (2019). Students' challenges in working on online assessment using blog in Computer Assissted Language Learning (CALL) Class: A case study at English Language Education Department in UIN Sunan Ampel Surabaya (Doctoral dissertation, Uin Sunan Ampel Surabaya).

[49]. Rossiter, S. (2020, June). Cheating Becoming an Unexpected COVID-19 Side Effect for Universities.

[50]. Saavedra, J. (2020). Educational Challenges and Opportunities of the Coronavirus (COVID-19) Pandemic.

[51]. Sahu, P. (2020). Closure of universities due to coronavirus disease 2019 (COVID-19): impact on education and mental health of students and academic staff. Cureus, 12(4), e7541.

[52]. Sandelowski, M., & Barroso, J. (2007). Handbook for Synthesizing Qualitative Research. Springer publishing company.

[53]. Sibeoni, J., Orri, M., Valentin, M., Podlipski, M. A., Colin, S., Pradere, J., & Revah-Levy, A. (2017). Metasynthesis of the views about treatment of anorexia nervosa in adolescents: perspectives of adolescents, parents, and professionals. Plos One, 12(1), e0169493.

[54]. Simon, W. E. (2019). Evaluation of Online Formative Assessment Practices at Higher Education Institutions (Doctoral dissertation, University of Pretoria).

[55]. Simonso, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and Learning at a Distance. Ohio Columbus.

[56]. Tondeur, J., Van Braak, J., Ertmer, P. A., & Ottenbreit- Leftwich, A. (2017). Understanding the relationship between teachers' pedagogical beliefs and technology use in education: a systematic review of qualitative evidence. Educational Technology Research and Development, 65, 555-575.

[57]. Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology, 12(1), 1-8.

[58]. Turgut, M. F., & Baykul, Y. (2019). Eğitimde Ölçme ve Değerlendirme Metotları. Pegem Akademi.

[59]. Turner, J., & Briggs, G. (2018). To see or not to see? Comparing the effectiveness of examinations and end of module assessments in online distance learning. Assessment & Evaluation in Higher Education, 43(7), 1048-1060.

[60]. UNESCO. (2020). COVID-19 Crisis and Curriculum: Sustaining Quality Outcomes in the Context of Remote Learning.

[61]. Uşun, S. (2006). Uzaktan Eğitim. Nobel Yayınları.

[62]. Van Vollenhoven, W. J., & Combrinck, M. (2020). Computer-assisted assessment: an old remedy for challenges in open distance learning. The Independent Journal of Teaching and Learning, 15(1), 22-34.

[63]. Velan, G. M., Kumar, R. K., Dziegielewski, M., & Wakefield, D. (2002). Web-based self-assessments in pathology with Questionmark Perception. Pathology, 34(3), 282-284.

[64]. Wang, C., Cheng, Z., Yue, X. G., & McAleer, M. (2020). Risk management of COVID-19 by universities in China. Journal of Risk and Financial Management, 13(2), 36.

[65]. Watson, C., Wilson, A., Drew, V., & Thompson, T. L. (2017). Small data, online learning and assessment practices in higher education: a case study of failure? Assessment & Evaluation in Higher Education, 42(7), 1030-1045.

[66]. Xiaotang, Y., & Nuersan, R. (2008). The study and practice of web-based learning assessment in open and distance education. Asian Association of Open Universities Journal, 3(1), 27-43.

[67]. Yue, W. S., & Kaur, S. (2016). A survey of students' perceptions on online assessment in the modular objectoriented dynamic learning environment (moodle). Advanced Science Letters, 22(12), 4082-4086.

If you have access to this article please login to view the article or kindly login to purchase the article

Single Article

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Online	15	15

ADD TO CART

Options for accessing this content:

If you would like institutional access to this content, please recommend the title to your librarian.
If you already have i-manager's user account: Login above and proceed to purchase the article.
New Users: Please register, then proceed to purchase the article.