Identifying Critical Features for Formative Essay Feedback with Artificial Neural Networks and Backward Elimination

Dec 3, 2019·
Mohsin Abbas
,
Peter van Rosmalen
Marco Kalz
Marco Kalz
· 1 min read
Abstract
For predicting and improving the quality of essays, text analytic metrics (surface, syntactic, morphological and semantic features) can be used to provide formative feedback to the students. In this study, the intent was to find a small number of features that exhibit a fair proxy of the scores given by the human raters. Using an existing corpus and a text analysis tool for the Dutch language, a large number of features were extracted. Artificial neural networks, Levenberg Marquardt algorithm and backward elimination were used to reduce the number of extracted features automatically. Irrelevant features were eliminated based on the inter-rater agreement between predicted and human scores calculated using Cohen’s Kappa (κ). By using our algorithm, the number of features in this study was reduced from 457 to 23. The selected features were grouped into six different categories. Of these categories, we believe that the features present in the groups “Word Difficulty” and “Lexical Diversity” are most useful for providing automated formative feedback to the students. The approach presented in this research paper is the first step towards our ultimate goal of providing meaningful for-mative feedback to the students for enhancing their writing skills and capabilities.
Type
Publication
Transforming Learning with Meaningful Technologies, 396-408. Springer International Publishing
publications publications

Abbas, M., van Rosmalen, P., & Kalz, M. (2019). Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. In Scheffel, M., Broisin, J., Pammer-Schindler, V., Ioannou, A., Schneider, J. (eds) Transforming Learning with Meaningful Technologies. EC-TEL 2019. (pp. 396-408). Lecture Notes in Computer Science, vol 11722. Springer, Cham. https://doi.org/10.1007/978-3-030-29736-7_30

Formative Feedback Natural Language Processing Neural Networks Backward Elimination Feature Selection Dimensionality Reduction
Marco Kalz
Authors
Marco Kalz
Professor of Educational Technology

I am a researcher in educational technology whose work explores how digital technologies reshape learning, teaching, and educational institutions. My research combines educational technology, learning sciences, feedback research, and critical perspectives on digital transformation.Current areas of interest include peer feedback and feedback literacy, open and networked learning, AI and misinformation in education, digital learning ecologies, and the societal implications of data-driven and platform-based education. Methodologically, my work spans empirical learning research, psychometric scale development, design-oriented research, and conceptual analyses of digital transformation in education.

I am working as a full professor of educational technology and Chief Information/Chief Digital Officer (CIO/CDO) at the Heidelberg University of Education. I serve as associate editor of the International Journal of Artificial Intelligence in Education and editorial board member of the Journal of Computing in Higher Education. I am a senior-fellow of the Interuniversity Center for Educational Sciences (ICO) and the Dutch research school on information and knowledge systems (SIKS). I work as director of the study program E-Learning and Media Education and director of the Heidelberg Centre for Digital Transformation in Education. Over the years I could secure approx. 4 Mio EUR of research funding for my institutions from competitive projects with a total budget of 36 Mio EUR. I have been an invited keynote speaker on more than 60 conferences and events. Besides European projects I am regularly involved in educational innovation and consulting projects with partners inside and outside of my institutions including clients like the International Labour Organisation, United Nations Environment Program, the European Commission, UNESCO, OECD or other international and national organizations.