ESET: [Eagles Student Evaluation Of Teaching] - An Online Andragogical Student Ratings Of Instruction Tool That Is An In–Depth Systemic Statistical Mechanism Designed To Inform, Enhance, And Empower Higher Education

James Edward Osler II*, Mahmud A. Mansaray**
* Faculty Member, North Carolina Central University, USA.
** Research Analyst, North Carolina Central University, USA.
Periodicity:September - November'2015
DOI : https://doi.org/10.26634/jsch.11.2.3703

Abstract

This paper seeks to provide an epistemological rationale for the establishment of ESET (an acronym for: “Eagles Student Evaluation of Teaching”) as a novel universal SRI [Student Ratings of Instruction] tool. Colleges and Universities in the United States use Student Ratings of Instruction [SRI] for course evaluation purposes (Osler and Mansaray, 2013a). This research investigation is the third part of a post hoc study that psychometrically examines the reliability and validity of the items used in an Historic Black College and University (HBCU) SRI instrument. The ESET sample under analysis consisted of the responses to 56,451 total items extracted from 7,919 distributed Student Ratings Instruments that were delivered electronically (at HBCU) to students who completed the ESET tool. The ESET methodology provides a statistically valid SRI/SET survey instrument along with a variety of post hoc statistical measures to determine the efficacy of collegiate instruction. This research is also the continuation of research conducted on innovative statistical metrics introduced in the i-managre’s Journal on Mathematics.

Keywords

Advanced Statistics, Construct Validity, Cronbach's Alpha, Efficacy, Formative Assessment, Goodman & Kruskal's Lambda, Principal Component Factor Analysis, Reliability, Statistics, Student Ratings of Instruction [SRI], Summative Assessment, Tri–Squared Test and Validity.

How to Cite this Article?

Osler, J. E., II. and Mansaray.M.A. (2015). ESET: [Eagles Student Evaluation Of Teaching] - An Online Andragogical Student Ratings Of Instruction Tool That Is An In–Depth Systemic Statistical Mechanism Designed To Inform, Enhance, And Empower Higher Education. i-manager’s Journal on School Educational Technology, 11(2), 24-42. https://doi.org/10.26634/jsch.11.2.3703

References

[1]. Agbetsiafa, D. (2010). “Evaluating effective teaching in college level Economics using student ratings of instruction: A factor analytic approach”, Journal of College Teaching and Learning, 7(5), 57-66. Retrieved from http://journals.cluteonline.com/index.php/TLC
[2]. Amrein-Beardsley, A., and Haladyna, T. (2012). “ Validating a theory-based survey to evaluate teaching effectiveness in higher education”, Journal on Excellence in College Teaching, Vol. 23(1), pp. 17-42. Retrieved from http://celt.muohio.edu/ject/issue.php?v=20&n=2
[3]. Anastasiadou, S. D. (2011). “Reliability and validity testing of a new scale for measuring attitudes toward learning statistics with technology”, Acta Didactica Napocensia, Vol. 4(1), 1-10. Retrieved from http://adn.teaching.ro/
[4]. Arthur, L. (2009). “From per formativity to professionalism: lecturers' responses to student feedback”, Teaching in Higher Education, Vol.14(4), pp. 441-454. doi:10.1080/13562510903050228
[5]. Baker, L. R., Pollio, D. E., and Hudson, A. (2011). “Assessing student perception of practice evaluation knowledge in introductory research methods”. Journal of Social Work Education, Vol. 47(3), pp.555-564. Retrieved From http: //www.tandfonline.com/toc/uswe20/ current#.U5yC8WXD8dU
[6]. Beran, T., and Rokosh, J. (2009). “Instructors' perspectives on the utility of student ratings of instruction”, Instructional Science, Vol. 37(2), pp. 171-184. doi:10.1007/s11251-007-9045-2
[7]. Beran, T., Violato, C., and Kline, D. (2007). “What's the "Use" of student ratings of instruction foradministrators? One university's experience”, Canadian Journal of Higher Education, Vol. 37(1), pp. 27-43. Retrieved from http://ojs.library.ubc.ca/index.php/cjhe/index
[8]. Beran, T., Violato, C., Kline, D., and Frideres, J. (2009). “What do students consider useful aboutstudent ratings?”, Assessment & Evaluation in Higher Education, Vol. 34(5), pp. 519-527. doi:10.1080/02602930802082228
[9]. Berk, R. A. (2005). “Survey of 12 strategies to measure teaching effectiveness”, International Journal of Teaching Learning in Higher Education, Vol. 17(1), pp. 48–62. Retrieved from http://www.isetl.org/ijtlhe/
[10]. Brookfield, S. (1986). Understanding and facilitating adult learning. San Francisco CA: Jossey-Bass.
[11]. Carrell, S. E., and West, J. E. (2010). “Does professor quality matter? Evidence from random assignment of students to professors”, Journal of Political Economy, Vol. 118(3), pp. 409-432. Retrieved from http://www.jstor.org/action/showPublication?journalCode =jpoliecon&
[12]. Chen, G., and Watkins, D. (2010). “Stability and correlates of student evaluations of teaching at a Chinese university”, Assessment & Evaluation in Higher Education, Vol. 35(6), pp.675-685. doi:10.1080/02602930902977715
[13]. Chulkov, D. V., and Van Alstine, J. (2012). “Challenges in designing student teaching evaluations in a business program”, International Journal of Educational Management, Vol. 26(2), pp.162-174. doi:10.1108/ 09513541211201979
[14]. Clayson, D. E. (2009). “Student evaluations of teaching: Are they related to what student learn?”, Journal of Marketing Education, Vol. 31(1), pp.16-30. Retrieved from http://jmd.sagepub.com
[15]. Cozby, P. C., and Bates, S. C. (2012). Methods in behavioral research, 11th ed., New York, NY: McGraw-Hill.
[16]. Davenport, J., and Davenport, J. A. (1985). “A chronology and analysis of the andragogy debate”, Adult Education Quarterly, Vol. 35(3), pp. 152-159. Retrieved from http://aeq.sagepub.com
[17]. Donnon, T., Delver, H., and Beran, T. (2010). “Student and teaching characteristics related to ratings of instruction in medical sciences graduate programs”, Medical Teacher, Vol. 32(4), pp.327-332. doi:10.3109/01421590903480097
[18]. Dorasamy, N., and Balkaran, R. (2013). “Role of student ratings of lecturers in enhancing teaching at higher education institutions: A case study of the durban university of technology”, Journal of Economics and Behavioral Studies, Vol. 5(5), pp. 268-281. Retrieved from http://ifrnd. org/JournalDetail.aspx?JournalID=2
[19]. El Hassan, K. (2012). “Investigating substantive and consequential validity of student ratings of instruction”. Higher Education Research & Development, Vol. 28(3), pp. 319-333. doi:10.1080/07294360902839917
[20]. Frick, T., Chadha, R., Watson, C., Wang, Y., and Green, P. (2009). “College student perceptions of teaching and learning quality”, Educational Technology Research & Development, Vol. 57(5), pp .705-720. doi:10.1007/s11423-007-9079-9
[21]. Fitzpatrick, J. L., and Miller-Stevens, K. (2009). “A case study of measuring outcomes in an MPA program”, Journal of Public Affairs Education, Vol. 15(1), pp.17-31. Retrieved from http://www.naspaa.org/initiatives/jpae/jpae.asp
[22]. Forrest III, S., and Peterson, T. (2006). “It's called andragogy. Academy of Management Learning and Education”, Vol. 5(1), pp. 113-122. doi:10.5465/ AMLE.2006. 20388390
[23]. Galbraith, C. S., and Merrill, G. B. (2012). “Predicting student achievement in university-level business and economics classes: Peer observation of classroom instruction and student ratings of teaching effectiveness”, College Teaching, Vol. 60(2), pp. 48-55. doi:10.1080/8756 7555.2011.627896
[24]. Galbraith, C., Merrill, G., and Kline, D. (2012). “Are student evaluations of teaching effectiveness valid for measuring student learning outcomes in business related classes? A neural network and bayesian analyses”, Research in Higher Education, Vol. 53(3), pp. 353-374. doi:10.1007/s11162-011-9229-0
[25]. Gilstrap, D. (2013). “Why do we teach? Adult learning theory in professional standards as a basis for curriculum development”. College and Research Libraries, Vol. 74(5), pp. 501-518. Retrieved from http://crl.acrl.org/
[26]. Haladyna, T., and Amrein-Beardsley, A. (2009). “Validation of a research-based student survey of instruction in a college of education”, Educational Assessment, Evaluation & Accountability, Vol. 21(3), pp. 255-276. doi:10.1007/s11092-008-9065-8
[27]. Hartree. A. (1984). “Mialcolm Knowles' theory of androgogy: A critique”, International Journalof Lifelong Education, Vol. 3(3), pp. 203-210. Retrieved from http://www.tandfonline.com
[28]. Hatfield, C. L., and Coyle, E. A. (2013). “Factors that influence student completion of course and faculty evaluations”, American Journal of Pharmaceutical Education, Vol. 77(2), pp.1-4. Retrieved from http://www.aj pe.org/
[29]. Ibrahim, A. M. (2011). “Using generalizability theory to estimate the relative effect of class sizeand number of items on the dependability of student ratings of instruction”, Psychological Reports, Vol. 109(1), pp. 252-258. doi:10. 2466/03.07.11.PR0.109.4.252-258
[30]. Johnson, M. D., Narayanan, A., and Sawaya, W. J. (2013). “Effects of course and instructor characteristics on student evaluation of teaching across a college of engineering”, Journal of Engineering Education, Vol. 102(2), pp. 289-318. doi:10.1002/jee.20013
[31]. Keeley, J., Furr, R., and Buskist, W. (2010). “Differentiating psychology students' perceptions of teachers using the teacher behavior checklist”, Teaching of Psychology, Vol. 37(1), pp. 16-20. doi:10.1080/00986280903426282
[32]. Knowles, M. S. (1974). “Human resources development in OD”, Public Administration Review, Vol. 34(2), pp. 115-123. Retrieved from http://onlinelibrary. wiley.com/ journal/10.1111/(ISSN)1540-6210
[33]. Knowles. M . S. (1980). The Modernl Practice of Adult Education: From Pedagogy to Androgogy. (2nd ed.) New York, NY: Cambridge Books.
[34]. Knowles, M. (1984). The adult learner: A neglected species. 3rd ed., Houston, TX: Gulf.
[35]. Kozub, R. M. (2010). “Relationship of course, instructor, and student characteristics to dimensions of student ratings of teaching effectiveness in business schools”, American Journal of Business Education, Vol. 3(1), pp.33-40. Retrieved from http://journals.cluteonline.com/index. php/AJBE
[36]. Leedy, P. D., and Ormrod, J. E. (2009). Practical research: Planning and design, (9th ed.). UpperSaddle River, NJ: Pearson Education Inc.
[37]. Liu, O. L. (2012). “Student evaluation of instruction: In the new paradigm of distance education”, Research in Higher Education, 53(4), 471-486. doi:http://dx.doi.org/ 10.1007/s11162-011-9236-1
[38]. Madden, T. J., Dillon, W. R., and Leak, R. L. (2010). “Students' evaluation of teaching: Concernsof item diagnosticity”, Journal of Marketing Education, Vol. 32(3), pp. 264-274. doi:10.1177/0273475310377759
[39]. Marsh, H. W., Ginns, P., Morin, A. S., Nagengast, B., and Martin, A. J. (2011). “Use of student ratings to benchmark universities: Multilevel modeling of responses to the Australian Course Experience Questionnaire (CEQ)”, Journal of Educational Psychology, Vol. 103(3), pp. 733- 748. doi:10.1037/a0024221
[40]. Mau, R. R., and Opengart, R.A. (2012). “Comparing ratings: In-class (paper) vs. out of class (online) student evaluations”. Higher Education Studies, Vol. 2(3), pp. 55. doi:10.5539/hes.v2n3p5
[41]. Merriam, S B. Mott, V. W., and Lee, M. (1996). “Learning that comes from the negative interpretation of life experience”, Studies in Continuing Education, Vol. 18(1), pp.1-23. Retrieved from http://www.tandfonline.com/
[42]. O'Shaughnessy, L. (2013). 6 Challenges facing state universities.CBSNews.Retrievedfrom http://www.cbsnews.com/news/6-challenges-facingstate- universities/
[43]. Osler, J. E. and Mansaray M. A. (2013a). “The Application of Online Psychometric Tests: An Analysis of Student Ratings of Instruction”, i-manager's Journal on Educational Psychology, Vol. 7 (3), pp. 32–50.
[44]. Osler, J. E., and Mansaray, M. A. (2013b). “Applied mathematical statistical modeling to determine the validity and reliability of independent instructional measures”. imanager's Journal on Mathematics, Vol. 2(3), pp. 12-30. Retrieved from http://www.imanagerpublications.com/ JournalIntroduction.aspx?journalid=102
[45]. Otani, K., Kim, B., and Jeong-IL, C. (2012). “Student evaluation of teaching (SET) in higher education: How to use SET more effectively and efficiently in public affairs education”, Journal of Public Affairs Education, Vol. 18(3), pp. 531-544. Retrieved from http://www.naspaa.org/ initiatives/jpae/jpae.asp
[46] Pritchard, R. E., and Potter, G. C. (2011). “Adverse changes in faculty behavior resulting from use of student evaluations of teaching: A case study”, Journal of College Teaching and Learning, Vol. 8(1), pp. 1-7. Retrieved from http://journals.cluteonline.com/index.php/TLC
[47]. Ruppert, B., and Green, D. A. (2012). “Practicing what we teach: Credibility and alignment in the business communication classroom”, Business Communication Quarterly, Vol. 75(1), pp. 29-44. doi:10.1177/10805699114 26475
[48]. Safavi, S., Bakar, K., Tarmizi, R., and Alwi, N. (2012). “The role of student ratings of instruction from perspectives of the higher education administrators”, International Journal of Business and Social Science, Vol. 3(9), pp. 233- 239. Retrieved from http://www.ijbssnet.com/
[49]. Savicevic, D. M. (1991). “Modern conceptions of andragogy: A European framework”. Studies in the Education of Adults, Vol. 23(2), pp. 179. Retrieved from http://www.niace.org.uk/publications/academicjournals/ studies
[50]. Seyedeh, A. S., and Kamariah, A. B. (2013). “The utility of student evaluations for medical sciences teachers and administrators”, International Journal of Business and Social Science, Vol. 4(13). Retrieved from http://www. ijbssnet.com/
[51]. Seyedeh, A. S., Kamariah, A. B., Rohani, A. T., and Alwi, N. H. (2012). “The role of student ratings of instruction from perspectives of the higher education administrators”, International Journal of Business and Social Science, Vol.3(10). Retrieved from http://www.ijbssnet.com/
[52]. Skowronek, J., Friesen, B., and Masonjones, H. (2011). “Developing a statistically valid and practically useful student evaluation instrument”, International Journal for the Scholarship of Teaching and Learning, Vol. 5(1), pp. 1- 19. Retrieved from http://digitalcommons.georgiasouthern .edu/ij-sotl//
[53]. Slocombe, T., Miller, D., and Hite, N. (2011). “A survey of student perspectives toward faculty evaluations”, American Journal of Business Education, Vol. 4(7), pp. 51- 57. Retrieved from http://journals.cluteonline. com/index.php/AJBE
[54]. Smith, D. L., Cook, P., and Buskist, W. (2011). “An Experimental analysis of the relation between assigned grades and instructor evaluations”, Teaching of Psychology, Vol. 38(4), 225-228. doi:10.1177/009862831 1421317
[55]. Spooren, P. (2010). “On the credibility of the judge: A cross-classified multilevel analysis on students' evaluation of teaching”, Studies in Educational Evaluation, Vol. 36(4), pp. 121-131.
[56]. Spooren, P., Brockx, B., and Mortelmans, D. (2013). “On the validity of student evaluation of teaching: The state of the art”, Review of Educational Research, Vol. 83(4), pp. 598. doi:10.3102/0034654313496870
[57]. Spooren, P. P., Mortelmans, D. D., and Denekens, J. J. (2007). “Student evaluation of teaching quality in higher education: Development of an instrument based on 10 Likert-scales”, Assessment & Evaluation in Higher Education, 32(6), 667-679. doi:10.1080/0260293 0601117191
[58]. Stowell, J. R., Addison, W. E., and Smith, J. L. (2012). “Comparison of online and classroom-based student evaluations of instruction”, Assessment and Evaluation in Higher Education, Vol. 37(4), pp. 465-473. doi:10.1080/02 602938.2010.545869
[59]. Taylor, C. L., Grey, N., and Satterthwaite, J. D. (2013). “Assessing the clinical skills of dental students: A review of the literature”, Journal of Education and Learning, Vol. 2(1), pp. 20-31. Retrieved from http://www.ccsenet.org/ journal /index.php/jel
[60]. Venette, S., Sellnow, D., and McIntyre, K. (2010). “Charting new territory: assessing the online frontier of student ratings of instruction”. Assessment and Evaluation in Higher Education, Vol. 35(1), pp. 97-111. doi:10.1080/0260 2930802618336
[61]. Young, N. (2012). “Effective learning and teaching of RDA: Applying adult learning theory”. Journal of Library Metadata, Vol. 12(2/3), pp. 188-198. doi:10.1080/1938638 9.2012.699836
[62]. Young, S., and Duncan, H. E. (2014). “Online and face-to-face teaching: How do student ratings differ?”. Journal of Online Learning and Teaching, Vol. 10(1), pp. 70- 79. Retrieved from http://jolt.merlot.org/
[63]. Zhao, J., and Gallant, D. J. (2012). “Student evaluation of instruction in higher education: exploring issues of validity and reliability”, Assessment & Evaluation in Higher Education, Vol.37(2), pp.227-235. doi:10.1080/02602938. 2010.523819
[64]. Zmeyov, S. I. (1998). “Andragogy: Origins, developments and trends”, International Review of Education/InternationaleZeitschriftFür Erziehungswissenschaft, Vol. 44(1), pp. 103-108. Retrieved from http://link.springer.com/journal/11159
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Online 15 15

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.