Developing Students' Engagement in Online Learning Questionnaire using Rasch Measurement Model
Abstract
Student engagement is an essential factor in online learning. Hence, it needs to be assessed. This study developed a questionnaire to measure student engagement in online learning. Initially, 24 items were developed. All items were measured using a five-point Likert scale. Experts reviewed the items; revisions were conducted based on experts’ appraisals. Then, Rasch analysis was conducted to determine the construct validity and reliability. One hundred seventeen undergraduate students participated in the trial phase. Based on the Rasch analysis using Winsteps, the item and person reliabilities were 0.98 and 0.82, respectively. The item reliability can be considered very good. The item and person separation indexes were 6.37 and 2.15, respectively. The mean-square (MNSQ), standardized as a z-score (ZSTD), and point measure correlation were used as criteria to refine items. Five items should be dropped because they did not meet the requirements. The final questionnaire consisted of 19 items.
Keywords: student engagement, online learning, Rasch Measurement model
References
[1] Al-shaikhli D, Courtenage S. A literature review of personalisation, search, and navigation methods in open e-learning. International Journal of Information and Education Technology. 2018;8(1):26–33. https://doi. org/10.18178/ijiet.2018.8.1.1007
[2] Dhawan S. Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology System. 2020;49(1):5–22. https://doi.org/10.1177/0047239520934018
[3] M. Hu and H. Li, “Student Engagement in Online Learning: A Review.,” In: 2017 International Symposium on Educational Technology (ISET). pp. 39–43. IEEE (2017).
[4] Hu S, Kuh GD. Being (dis)engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education. 2002;43(5):555–75.
[5] Zhoc KCH, Webster BJ, King RB, Li JCH, Chung TSH. Higher education student engagement scale (HESES): Development and psychometric evidence. Research in Higher Education. 2018;60:219–44. https://doi.org/ 10.1007/s11162-018-9510-6
[6] J.A. Fredricks, P.C. Blumenfeld, and A.H. Paris, “School Engagement: Potential of the Concept, State of the Evidence.,” Review of Educational Research. vol. 74, no. 1, pp. 59–109, 2004..
[7] S. Gunuc and A. Kuzu, “Student engagement scale: development, reliability and validity.,” Assessment & Evaluation in Higher Education. vol. 40, no. 4, pp. 587–610, 2015.
[8] E.R. Wester, L.L. Walsh, S. Arango-Caro, and K.L. Callis-Duehl, “Student Engagement Declines in STEM Undergraduates during COVID-19–Driven Remote Learning.,” Journal of Microbiology & Biology Education. vol. 22, no. 1, pp. 1–11, 2021.
[9] Greene BA. Measuring cognitive engagement with self-report scales: Reflections over 20 years of research. Educational Psychologist. 2015;50:14–30.
[10] Fredricks JA, McColskey W. Handbook of research on student engagement. Christenson S, Reschy AL, Wylie C, editors. New York: Springer; 2012.
[11] M.-T. Wang, J.A. Fredricks, F. Ye, T.L. Hofkens, and J.S. Linn, “The Math and Science Engagement Scales: Scale development, validation, and psychometric properties.,” Learning and Instruction. vol. 43, pp. 16–26, 2016
[12] Aziz NFA, Ahmad H, Nashir IM. Validation of technical and vocational teachers’ competency evaluation instrument using the Rasch model. Jurnal Pendidikan Sains & Matematik Malaysia. 2019;9(1):18–25. https://doi.org/10.37134/jpsmm.vol9.1.3.2019
[13] Ee NS, Yeo KJ, Kosnin AM. Item analysis for the adapted motivation scale using Rasch model. International Journal of Evaluation and Research in Education. 2018;7(4):264– 69. https://doi.org/10.11591/ijere.v7.i4.pp264-269
[14] Mokshein SE, Ishak H, Ahmad H. The use of Rasch measurement model in English testing. Cakrawala Pendidikan. 2019;38(1):16–32. https://doi.org/10.21831/cp.v38i1.22750
[15] Tabatabaee-Yazdi M, Motallebzadeh K, Ashraf H, Bagheai P. Development and validation of a teacher success questionnaire using the Rasch model. International Journal of Instruction. 2018;11(2):129–144. https://doi.org/10. 12973/iji.2018.11210a
[16] Fisher WP. Rating scale instrument quality criteria. Institute for Objective Measurement; Australia; 2018. Available from: https://www. rasch.org/rmt/rmt211m.htm
[17] Linacre JM. Winsteps help for Rasch analysis. Winsteps, USA; 2012.
[18] G.P. Utari, W. Liliawati, and J.A. Utama, “Design and validation of six-tier astronomy diagnostic test instruments with Rasch Model analysis.,” Journal of Physics: Conference Series. vol. 1806, no. 1, pp. 1–7, 2021 DOI