i-manager Publications

Future Prediction of L2 Writing Performance: A Machine Learning Approach

Kutay Uzun*

Department of Foreign Languages, Trakya University, Turkey.

Periodicity:July - September'2020
DOI : https://doi.org/10.26634/jet.17.2.16840

Abstract

Writing in L2 is both crucial and difficult for teachers and students since most of the assessment in higher education is in written form. The production of texts as well as providing feedback to them requires time and effort on both sides. For this reason, prediction of future L2 writing performance in advance may prove quite useful both for teachers and students and the information obtained can be used for failure prevention. In this respect, the present study aims to train a machine learning model which used demographic and psychometric data to predict end-of-term L2 writing performance on a Pass/Fail basis. Secondly, the study aimed to find out if synthetically augmenting available data would increase prediction accuracy. The data were collected in the beginning of the semester from 102 students of English Language Teaching in Turkey. L2 writing performance was measured at the end of the semester. The findings indicated that Instance-Bases Learning with Parameter K algorithm could accurately predict L2 writing performance in advance. Moreover, artificial augmentation of the data was seen to increase prediction accuracy. The findings bear implications for the effective prevention of student failure by detecting poor performance in advance.

Keywords

Educational Data Mining, Early Warning Systems, L2 Writing, Machine Learning, Performance Prediction.

How to Cite this Article?

Uzun, K. (2020). Future Prediction of L2 Writing Performance: A Machine Learning Approach. i-manager’s Journal of Educational Technology, 17(2), 1-13. https://doi.org/10.26634/jet.17.2.16840

References

[1]. Akbani R., Kwek S., & Japkowicz N. (2004). Applying support vector machines to imbalanced datasets. In J. F. Boulicaut, F. Esposito, F. Giannotti, & D. Pedreschi (Eds.), Machine learning: ECML 2004 lecture notes in computer science. Springer, Berlin, Heidelberg.

[2]. Amruthnath, N., & Gupta, T. (2018, April). A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance. In 2018 5th International Conference on Industrial Engineering and Applications (ICIEA) (pp. 355-361). IEEE.

[3]. Angeline, D. M. D. (2013). Association rule generation for student performance analysis using apriori algorithm. The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), 1(1), 12-16.

[4]. Arias, J. P., Yoma, N. B., & Vivanco, H. (2010). Automatic intonation assessment for computer aided language learning. Speech Communication, 52(3), 254-267.

[5]. Baker, R. S. J. D. (2010). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.), International Encyclopedia of Education (3rd ed., pp. 112-118). Oxford, UK: Elsevier.

[6]. Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. JEDM Journal of Educational Data Mining, 1(1), 3-17.

[7]. Ben-David, A. (2008). Comparison of classification accuracy using Cohen's Weighted Kappa. Expert Systems with Applications, 34(2), 825-832.

[8]. Bharadwaj, B. K., & Pal, S. (2011). Mining educational data to analyze students' performance. International Journal of Advance Computer Science and Applications, 2(6), 63–69.

[9]. Burdick, H., Swartz, C. W., Stenner, A. J., Fitzgerald, J., Burdick, D., & Hanlon, S. T. (2013). Measuring students' writing ability on a computer-analytic developmental scale: An exploratory validity study. Literacy Research and Instruction, 52(4), 255-280.

[10]. Bydzovska, H. (2016). A comparative analysis of techniques for predicting student performance. In T. Barnes, M. Chi, & M. Feng (Eds.). In Proceedings of the 9^th International Conference on Educational Data Mining (pp. 306–311). North Carolina, NC: Raleigh.

[11]. Chen, C. M., & Li, Y. L. (2010). Personalised contextaware ubiquitous learning system for supporting effective English vocabulary learning. Interactive Learning Environments, 18(4), 341-364.

[12]. Chen, X., & Meurers, D. (2016, December). CTAP: A web-based tool supporting automatic complexity analysis. In Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC) (pp. 113-119), Osaka, Japan.

[13]. Cheng, Y. S. (2002). Factors associated with foreign language writing anxiety. Foreign Language Annals, 35(6), 647-656.

[14]. Cheng, Y. S. (2004). A measure of second language writing anxiety: Scale development and preliminary validation. Journal of Second Language Writing, 13(4), 313-335.

[15]. Chun, D. M. (2007). Technological advances in researching and teaching phonology. In Phonology in Context (pp. 274-299). Palgrave Macmillan, London.

[16]. Costa, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, 73, 247-256.

[17]. Cotos, E. (2014). Genre-based automated writing evaluation for L2 research writing: From design to evaluation and enhancement. UK: Palgrave Macmillan, Springer.

[18]. Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading, 35(2), 115-135.

[19]. Cumming, A. (1989). Writing expertise and secondlanguage proficiency. Language Learning, 39(1), 81-135.

[20]. Davies, A. (1989). Communicative competence as language use. Applied Linguistics, 10(2), 157-170.

[21]. Eibe, F., Mark, A. H., & Witten, I. H. (2016). The WEKA Workbench in Online Appendix for Data Mining: Practical Machine Learning Tools and Techniques (4th ed.). Morgan Kaufmann.

[22]. Erdogan, T. (2013). The effect of creative drama method on pre-service classroom teachers' writing skills and attitudes towards writing. Australian Journal of Teacher Education, 38(1), 45-61.

[23]. Ferrier, J., Horne, J., & Singleton, C. (2013). Factors affecting the speed of free writing. Journal of Research in Special Educational Needs, 13(1), 66-78.

[24]. Goggins, S. P., Xing, W., Chen, X., Chen, B., & Wadholm, B. (2015). Learning analytics at small" scale: Exploring a complexity-grounded model for assessment automation. Journal of Universal Computer Science, 21 (1), 66-92.

[25]. Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445-476.

[26]. Graham, S., Berninger, V., & Fan, W. (2007). The structural relationship between writing attitude and writing achievement in first and third grade students. Contemporary Educational Psychology, 32(3), 516-536.

[27]. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2^nd ed.). Springer, New York, NY.

[28]. Heiner, C., Heffernan, N., & Barnes, T. (2007). Educational data mining. In Supplementary Proceedings of the 12^th International Conference of Artificial Intelligence in Education.

[29]. Hirose, K., & Sasaki, M. (1994). Explanatory variables for Japanese students' expository writing in English: An exploratory study. Journal of Second Language Writing, 3(3), 203-229.

[30]. Hoyle, D. C. (2008). Automatic PCA dimension selection for high dimensional data and small sample sizes. Journal of Machine Learning Research, 9, 2733-275 9.

[31]. Hwang, G. J., Hsiao, C. L., & Tseng, J. C. (2003). A computer-assisted approach to diagnosing student learning problems in science courses. Journal of Information Science and Engineering, 19(2), 229–248.

[32]. Losifidis, V., & Ntoutsi, E. (2018). Dealing with bias via data augmentation in supervised learning scenarios. In J. Bates, P. D. Clough, R. Jäschke, & J. Otterbacher, (Eds.). In Proceedings of the International Workshop on Bias in Information, Algorithms, and Systems Co-Located with 13th International Conference on Transforming Digital Worlds (pp. 24-29).

[33]. Kabakchieva, D. (2013). Predicting student performance by using data mining methods for classification. Cybernetics and Information Technologies, 13(1), 61-72.

[34]. Kira, K. & Rendell, L. (1992). A practical apprach to feature selection. In D. Sleeman & P. Edwards (Eds.), Machine Learning: Proceedings of International Conference (ICML'92) (pp. 249-256). San Francisco, CA: Morgan Kaufmann.

[35]. Klingler, S. (2017). Sensor-free learner models for trait discovery and identification in intelligent tutoring systems (Unpublished Doctoral dissertation), ETH Zurich, Switzerland.

[36]. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (Vol 2, pp. 1137-1143) San Francisco, CA: Morgan Kaufmann Publishers Inc.

[37]. Kotsiantis, S., Pierrakeas, C., Pintelas, P. (2004). Prediction of student's performance in distance learning using machine learning techniques. Applied Artificial Intelligence, 18(5), 411-426.

[38]. Lee, S. (2005). Facilitating and inhibiting factors in English as a foreign language writing performance: A model testing with structural equation modeling. Language Learning, 55(2), 335-374.

[39]. Luan, J. (2002). Data mining and its applications in higher education. In A. Serban & J. Luan (Eds.). Knowledge management: Building a competitive advantage for higher education (pp. 17-36). San Francisco, CA: Jossey Bass.

[40]. Matsumoto, K. (1995). Research paper writing strategies of professional Japanese EFL writers. TESL Canada Journal, 13(1), 17-27.

[41]. Merceron, A., & Yacef, K. (2004). Mining student data captured from a web-based tutoring tool: Initial exploration and results. Journal of Interactive Learning Research, 15(4), 319–346.

[42]. Merceron, A., & Yacef, K. (2005). Educational data mining: A case study. In C.-K. Looi, G. McCalla, B. Bredeweg & J. Breuker (Eds.). Proceedings of the 12th international conference on artificial intelligence (pp. 467–474). Amsterdam: IOS Press.

[43]. Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT Press.

[44]. Nandeshwar, A., & Chaudhari, S. (2009). Enrollment Prediction Models Using Data Mining.

[45]. Nguwi, Y., & Cho, S. (2010). An unsupervised selforganizing learning with support vector ranking for imbalanced datasets. Expert Systems with Applications, 37(12), 8303-8312.

[46]. Noel-Levitz White Paper, (2008). Qualifying Enrollment Success: Maximizing Student Recruitment and Retention through Predictive Modeling. Noel-Levitz, Inc.

[47]. Olinghouse, N. G., & Graham, S. (2009). The relationship between the discourse knowledge and the writing performance of elementary-grade students. Journal of Educational Psyhology, 101(1), 37-50.

[48]. Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature. Reading & Writing Quarterly, 19(2), 139-158.

[49]. Pratiwi, N. E., Yufrizal, H., & Sukirlan, M. (2017). The relationship between students' self assessment of communicative competence and their actual performance. UNILA Journal of English Teaching, 6(7), 1-10.

[50]. Rai, S., & Jain, A. K. (2013). Students' dropout risk assessment in undergraduate courses of ICT at Residential University – A case study. International Journal of Computer Applications, 84(14), 31-36.

[51]. Ramaswami, M., & Bhaskaran, R. (2010). A CHAID based performance prediction model in educational data mining. IJCSI International Journal of Computer Science Issues, 7(1), 10-18.

[52]. Romero, C., & S. Ventura (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135-146.

[53]. Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews, 40(6), 601-618.

[54]. Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27.

[55]. Sakai, A., Minoda, Y., & Morikawa, K. (2017). Data augmentation methods for machine-learning-based classification of bio-signals. In 10th Biomedical Engineering International Conference, Hokkaido, Japan.

[56]. Shmueli, G., Patel, N., & Bruce, P. (2007). Data Mining for Business Intelligence. Canada: John Wiley & Sons, Inc.

[57]. Shrivastava, A., Sondhi, J., & Kumar, B. (2017). Machine learning technique for product classification in ecommerce data using Microsoft Azure Cloud. International Research Journal of Engineering & Applied Sciences, 5(2), 11-13.

[58]. Simard, P. Y., Steinkraus, D., & Platt, J. C. (2003). Best practices for convolutional neural networks applied to visual document analysis, "ICDAR, 3, 958-962. " to " In Proceedings of Seventh International Conference on Document Analysis and Recognition, 2003. (pp. 958-963).

[59]. Simon, P. (2013). Too Big to Ignore: The Business Case for Big Data. Hoboken, NJ: Wiley.

[60]. Sung, Y., Lin, W., Dyson, S., Chang, K., & Chen, Y. (2015). Leveling L2 texts through readability: Combining multilevel linguistic features with the CEFR. The Modern Language Journal, 99(2), 371-391.

[61]. Troia, G. A., Harbaugh, A. G., Shankland, R. K., Wolbers, K. A., & Lawrence, A. M. (2013). Relationships between writing motivation, writing activity, and writing performance: Effects of grade, sex, and ability. Reading and Writing, 26(1), 17-44.

[62]. Tsai, Y. R., Chang, Y., & Ouyang, C. S. (2011). Mining error patterns of engineering students' English reading comprehension. In C. Patrick & D. Chunru (Eds.), Proceedings of 2011 International Conference on Machine Learning and Cybernetics (pp. 68–72).

[63]. Van den Kerkhof, T., Maesschalck, R., Vanhoutte, K., & Coene, M. C. (2006). Augmentation of near infrared diffuse reflectance and transmittance spectral data for the development of robust PLSBC models for classifying double blind clinical trial. Journal of Pharmaceutical and Biomedical Analysis, 42(4), 517-522.

[64]. Vieira, S. M., Kaymak, U., & da Costa Sousa, J. M. (2010). Cohen's Kappa coefficient as a performance measure for feature selection. In Proceedings 2010 IEEE World Congress on Computational Intelligence (pp. 1-8). Piscataway: Institute of Electrical and Electronics Engineers (IEEE).

[65]. Villalón, J., Kearney, P., Calvo, R. A., & Reimann, P. (2008). Glosser: Enhanced feedback for student writing tasks. In 22nd Australasian Joint Conference on Artificial Intelligence, Melbourne, Australia.

[66]. West, M. (1994). Discovery sampling and selection models. In S. S. Gupta & J. O. Berger (Eds.). Statistical Decision Theory and Related Topics V (pp. 221-235). New York: Springer-Verlag.

[67]. Witten, I. H., & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques (2nd ed.). Elsevier, San Francisco, CA.

[68]. Wolfe-Quintero, K., Inagaki, S., & Kim, H. Y. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy, & Complexity. Honolulu, HI: University of Hawaii Press.

[69]. Wu, J., Chang, Y., Mitamura, T., & Chang, J. (2010). Automatic collocation suggestion in academic writing. In Proceedings of the ACL Conference, Short paper track, Uppsala, Finland.

[70]. Yang, X, Kong, X, Hasegawa-Johnson, M, & Xie, Y. (2016). "Landmark-based pronunciation error identification on L2 Mandarin Chinese. Proc. Speech Prosody 2016, 247- 251.

[71]. Yavuz-Erkan, D. (2004). Efficacy of cross-cultural email exchange for enhancing EFL writing: A perspective for tertiary-level Turkish EFL learners (Unpublished doctoral dissertation). Çukurova University, Turkey.

[72]. Yu, P., Own, C., & Lin, L. (2001). On learning behavior analysis of web based interactive environment. In Proceedings of the implementing Curricular Change in Engineering Education (pp. 1-10), Oslo, Norway.

[73]. Zabihi, R. (2017). The role of cognitive and affective factors in measures of L2 writing. Written Communication, 35(1), 32-57.

If you have access to this article please login to view the article or kindly login to purchase the article

Single Article

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Online	15	15

ADD TO CART

Options for accessing this content:

If you would like institutional access to this content, please recommend the title to your librarian.
If you already have i-manager's user account: Login above and proceed to purchase the article.
New Users: Please register, then proceed to purchase the article.