It's All in the Data - But What is It? Learning Analytics and Data Mining of Multimedia Physics Courses

Gerd Kortemeyer

doi:10.51724/ijpce.v11i1.16

Authors

Gerd Kortemeyer Michigan State University, United States; ETH Zurich https://orcid.org/0000-0001-6643-9428

DOI:

https://doi.org/10.51724/ijpce.v11i1.16

Keywords:

Learning analytics, Data mining, Online, Hybrid

Abstract

Before “Learning Analytics” became the buzzword that it is today, fueled by the advent of "big data" MOOCs and increased institutional attention to retention and time-to-degree, fine-grained transactions within courses were analyzed as a unique window into student learning: what resources do learners access in which order, how much time do they spend with each resource, to what degree do different resources contribute to success in formative assessment, what is the effect of online discussion forums, how many attempts does it take which learners to solve online homework, and what is the extend and effect of unproductive behavior such as procrastination, guessing, and copying? We will focus on virtual and blended physics courses and use data mining methods to extract various predictive signatures of eventual learner success on exams. We also use classical test theory and item response theory to estimate both quality parameters of online homework and latent learner traits such as ability and the propensity to copy or guess answers. Based on these findings, we will make recommendations for course design.

Downloads

Download data is not yet available.

References

Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. JEDM| Journal of Educational Data Mining, 1(1), 3-17.

Baker, R. S. J. D., & Inventado, P. S. (2014). Educational data mining and learning analytics. In J.A. Larusson, & B. White (Eds.), Learning analytics: from research to practice (pp. 61-75). New York: Springer.

Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom: Research into edX's first MOOC. Research & Practice in Assessment, 8.

Bruckman, A. (2006). Analysis of log file data to understand behavior and learning in an online community. In The International handbook of virtual learning environments, 1449-1465.

Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 40.

Castro, F., Vellido, A., Nebot, À., & Mugica, F. (2007). Applying data mining techniques to e-learning problems. Evolution of teaching and learning paradigms in intelligent environment, 183-221.

Chatti, M. A., Lukarov, V., Thüs, H., Muslim, A., Yousef, A. M. F., Wahid, U., ... & Schroeder, U. (2014). Learning analytics: Challenges and future research directions. eleed, Iss. 10 (urn:nbn:de:0009-5-40350).

Clow, D. (2013). MOOCs and the funnel of participation. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 185-189). ACM.

Cruz, É., Dias, H., & Kortemeyer, G. (2011). The effect of formative assessment in Brazilian university physics courses. Revista Brasileira de Ensino de Física, 33(4), 4315-4315.

Duval, E. (2011). Attention please!: learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 9-17). ACM.

Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and implementation of a learning analytics toolkit for teachers. Journal of Educational Technology & Society, 15(3), 58.

EDUCAUSE Learning Initiative Learning Analytics Resources ... (n.d.). Retrieved July 10, 2016, from https://net.educause.edu/section_params/conf/eli124/LearningAnalyticsResources.pdf

Eichhorn, S., & Matkin, G. W. (2016). Massive open online courses, big data, and education research. New Directions for Institutional Research, 2015(167), 27-40.

Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.

Fournier, H., Kop, R., & Sitlia, H. (2011). The value of learning analytics to networked learning on a personal learning environment. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 104-109). ACM.

García Peñalvo, F. J., Conde García, M. Á., Alier Forment, M., & Casany Guerrero, M. J. (2011). Opening learning management systems to personal learning environments. Journal of Universal Computer Science, 17(9), 1222-1240.

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68-84.

Gönülateş, E., & Kortemeyer, G. (2017). Modeling unproductive behavior in online homework in terms of latent student traits: An approach based on item response theory. Journal of Science Education and Technology, 26(2), 139-150.

Gok, T. (2010). The general assessment of problem solving processes in physics education. Eurasian Journal of Physics and Chemistry Education, 2, 110-122.

Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Journal of Educational Technology & Society, 15(3), 42.

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.

Heiner, C. E., Banet, A. I., & Wieman, C. (2014). Preparing students for class: How to get 80% of students reading the textbook before class. American Journal of Physics, 82, 989-996.

Kashy, E., Sherrill, B. M., Tsai, Y., Thaler, D., Weinshank, D., Engelmann, M., & Morrissey, D. J. (1993). CAPA—An integrated computer‐assisted personalized assignment system. American Journal of Physics, 61, 1124-1130.

Kashy, D. A., Albertelli, G., Bauer, W., Kashy, E., & Thoennessen, M. (2003). Influence of nonmoderated and moderated discussion sites on student success. Journal of Asynchronous Learning Networks, 7, 31-36.

Khribi, M. K., Jemni, M., & Nasraoui, O. (2008). Automatic recommendations for e-learning personalization based on web usage mining techniques and information retrieval. In Advanced Learning Technologies, 2008. ICALT'08. Eighth IEEE International Conference on (pp. 241-245). IEEE.

Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014). Understanding in-video dropouts and interaction peaks inonline lecture videos. Proceedings of the First ACM Conference on Learning @ Scale Conference - L@S '14. ACM.

Kizilcec, R. F., Piech, C., & Schneider, E. (2013, April). Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the third international conference on learning analytics and knowledge (pp. 170-179). ACM.

Kortemeyer, G., Hall, M., Parker, J., Minaei-Bidgoli, B., Albertelli II, G., Bauer, W., & Kashy, E. (2005). Effective feedback to the instructor from online homework. Journal of Asynchronous Learning Networks, 9(2), 19-28.

Kortemeyer, G. (2006). An analysis of asynchronous online homework discussions in introductory physics courses. American Journal of Physics, 74, 526.

Kortemeyer, G. (2007). Correlations between student discussion behavior, attitudes, and learning. Physical Review Special Topics-Physics Education Research, 3(1), 010101.

Kortemeyer, G., Kashy, E., Benenson, W., & Bauer, W. (2008). Experiences using the open-source learning content management and assessment system LON-CAPA in introductory physics courses. American Journal of Physics, 76, 438.

Kortemeyer, G. (2009). Gender differences in the use of an online homework system in an introductory physics course. Physical Review Special Topics - Physics Education Research, 5(1).

Kortemeyer, G., & Riegler, P. (2010). Large-Scale E-Assessments, Prüfungsvor-und-nachbereitung: Erfahrungen aus den USA und aus Deutschland. Zeitschrift für E-learning-Lernkultur und Bildungstechnologie, 8, 22.

Kortemeyer, G. (2014a). Extending item response theory to online homework. Physical Review Special Topics - Physics Education Research, 10(1).

Kortemeyer, G. (2014b). Over two decades of blended and online physics courses at Michigan State University. eleed, Iss. 10. (urn:nbn:de:0009-5-40115)

Kortemeyer, G. (2015). An empirical study of the effect of granting multiple tries for online homework. American Journal of Physics, 83(7), 646-653.

Kortemeyer, G. (2016a). The losing battle against plug-and-chug. The Physics Teacher, 54(1), 14-17.

Kortemeyer, G. (2016b). Work Habits of Students in Traditional and Online Sections of an Introductory Physics Course: A Case Study. Journal of Science Education and Technology, 25(5), 697-703.

Kortemeyer, G. (2016c). Scalable continual quality control of formative assessment items in an educational digital library: an empirical study. International Journal on Digital Libraries, 17(2), 143-155.

Kortemeyer, G. (2017). The Spectrum of Learning Analytics. eleed, 12(1).

Laverty, J. T., & Kortemeyer, G. (2012). Function plot response: A scalable system for teaching kinematics graphs. American Journal of Physics, 80(8), 724.

Laverty, J. T., Bauer, W., Kortemeyer, G., & Westfall, G. (2012). Want to reduce guessing and cheating while making students happier? Give more exams! The Physics Teacher 50(9), 540.

Mason, A., & Singh, C. (2009). Reflection and self‐monitoring in quantum mechanics. In AIP Conference Proceedings (Vol. 1179, No. 1, pp. 197-200). AIP.

Mattingly, K. D., Rice, M. C., & Berge, Z. L. (2012). Learning analytics as a tool for closing the assessment loop in higher education. Knowledge Management & E-Learning: An International Journal, 4, 236-247.

Mazza, R., Bettoni, M., Faré, M., & Mazzola, L. (2012). Moclog–monitoring online courses with log data, in Proc. 1st Moodle Research Conference, Heraklion, Greece, 132-139.

Minaei-Bidgoli, B., Kashy, D., Kortemeyer, G., & Punch, W. (2003.). Predicting student performance: An application of data mining methods with an educational web-based system. 33rd Annual Frontiers in Education, 2003. FIE 2003.

Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4, 11-48.

Novak, G. M. (1999). Just-in-time teaching: Blending active learning with web technology. Upper Saddle River: Prentice Hall.

Papamitsiou, Z. K., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49-64.

Podolefsky, N., & Finkelstein, N. (2006). The perceived value of college physics textbooks: Students and instructors may not see eye to eye. The Physics Teacher, 44, 338-342.

Psaromiligkos, Y., Orfanidou, M., Kytagias, C., & Zafiri, E. (2011). Mining log data for the analysis of learners’ behaviour in web-based learning management systems. Operational Research, 11, 187-200.

Rayyan, S., Seaton, D. T., Belcher, J., Pritchard, D. E., & Chuang, I. (2014). Participation and Performance in 8.02x Electricity and Magnetism: The First Physics MOOC from MITx. 2013 Physics Education Research Conference Proceedings. comPADRE.

R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/

Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135-146.

Romero, C., Ventura, S., Delgado, J., & De Bra, P. (2007). Personalized links recommendation based on data mining in adaptive educational hypermedia systems. Creating New Learning Experiences on a Global Scale, 292-306.

Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12-27.

Seaton, D. T., Bergner, Y., Kortemeyer, G., Rayyan, S., Chuang, I., & Pritchard, D. E. (2014a). The impact of course structure on eText use in large-lecture introductory-physics courses. 2013 Physics Education Research Conference Proceedings. comPADRE.

Seaton, D. T., Kortemeyer, G., Bergner, Y., Rayyan, S., & Pritchard, D. E. (2014b). Analyzing the impact of course structure on electronic textbook use in blended introductory physics courses. American Journal of Physics, 82, 1186-1197.

Siemens, G., & Baker, R. S. (2012). Learning analytics and educational data mining. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK '12. ACM.

Stelzer, T., Gladding, G., Mestre, J. P., & Brookes, D. T. (2009). Comparing the efficacy of multimedia modules with traditional textbooks for learning introductory physics content. American Journal of Physics, 77, 184-190.

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57, 1500-1509.

Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. In Proceedings of the third international conference on learning analytics and knowledge (pp. 145-149). ACM.

Zaïane, O. R. (2002). Building a recommender agent for e-learning systems. In Proceedings. International Conference on Computers in Education 2002 (pp. 55-59). IEEE.