Validity evidence for the internal structure of the peer feedback orientation scale

Apr 6, 2026·
Marco Kalz
Marco Kalz
,
Annette Schulze
,
Holger Meeh
,
Joshua Weidlich
· 1 min read
Abstract
Peer feedback has the potential to contribute to a transformation of the feedback culture in higher education. The disposition of students for peer feedback is a crucial factor for the successful implementation of peer feedback. In earlier work we have developed through a mixed-method sequential research approach an initial instrument to measure the peer feedback orientation of students. The current study has the goal produce further evidence for the validity of the factor structure of the initial instrument, compare competing models and to cross-validate the model across samples and sub-populations. For this purpose, the Peer feedback Orientation instrument (PFOS) has been implemented in a mandatory course on media education in the teacher education program of the institution of the first author. 269 students have participated in the study. A confirmatory factor analysis has confirmed the measurement model from an earlier exploratory factor analysis. A competing two-factor model focusing on the role-change in peer feedback processes could not achieve comparable benchmarks. A test of measurement invariance and a retesting of this measurement provides further evidence of the validity of the internal structure. These results show that the peer feedback orientation scale is a robust measure for students in higher education and that peer feedback orientation is a promising area for future research. By providing an empirically supported measurement model, the study offers a theoretically grounded framework for examining how students’ orientations toward peer feedback shape their participation in feedback activities. This framework may help to explain variability in peer-feedback engagement and outcomes observed in higher education settings and thus contributes to the broader effort to better understand the learner dispositions that support effective feedback processes.
Type
Publication
Technology, Knowledge and Learning, 1-16. Springer
publications publications

Kalz, M., Schulze, A., Meeh, H., & Weidlich, J. (2026). Validity evidence for the internal structure of the peer feedback orientation scale. Technology, Knowledge and Learning, 1–16. https://doi.org/10.1007/s10758-026-09972-9

Peer-Feedback Higher Education Feedback Scale Development PFOS Peer-Feedback Orientation Dispositions Confirmatory Factor Analysis Individual Differences
Marco Kalz
Authors
Marco Kalz
Professor of Educational Technology

I am a researcher in educational technology whose work explores how digital technologies reshape learning, teaching, and educational institutions. My research combines educational technology, learning sciences, feedback research, and critical perspectives on digital transformation.Current areas of interest include peer feedback and feedback literacy, open and networked learning, AI and misinformation in education, digital learning ecologies, and the societal implications of data-driven and platform-based education. Methodologically, my work spans empirical learning research, psychometric scale development, design-oriented research, and conceptual analyses of digital transformation in education.

I am working as a full professor of educational technology and Chief Information/Chief Digital Officer (CIO/CDO) at the Heidelberg University of Education. I serve as associate editor of the International Journal of Artificial Intelligence in Education and editorial board member of the Journal of Computing in Higher Education. I am a senior-fellow of the Interuniversity Center for Educational Sciences (ICO) and the Dutch research school on information and knowledge systems (SIKS). I work as director of the study program E-Learning and Media Education and director of the Heidelberg Centre for Digital Transformation in Education. Over the years I could secure approx. 4 Mio EUR of research funding for my institutions from competitive projects with a total budget of 36 Mio EUR. I have been an invited keynote speaker on more than 60 conferences and events. Besides European projects I am regularly involved in educational innovation and consulting projects with partners inside and outside of my institutions including clients like the International Labour Organisation, United Nations Environment Program, the European Commission, UNESCO, OECD or other international and national organizations.