A Methodology for Generating Subquestions for the Force Concept Inventory (and Other Research-Based Assessments)
Abstract views: 39 / PDF downloads: 26
DOI:
https://doi.org/10.51724/ijpce.v16i2.351Keywords:
Force Concept Inventory, subquestions, false positives, Newtonian mechanicsAbstract
In this paper, we discuss the methodology we have developed and have been using for creating subquestions for the Force Concept Inventory (FCI). The FCI is a research-based assessment that is used internationally to assess student understanding of Newtonian mechanics. The assessment has been investigated from a number of perspectives and many suggestions have been made for its improvement. One challenge that is becoming more and more pervasive as more learning has transitioned to an online environment and more students are completing the FCI without a proctor is that of test security. If the answers to FCI items become easily accessible, then students will be able to provide correct answers despite lacking in understanding of Newtonian mechanics. One approach to mitigate the effects of items leaking into the public sphere and maintaining test security is the creation and administration of subquestions in place of the original FCI questions. Subquestions have an additional benefit of reducing false positives (answering a survey item correctly without correct understanding) and false negatives (answering incorrectly despite correct understanding). In this paper, we will discuss how we created subquestions for four items on the FCI, informed by survey-based interviews with students and the original intended targets of the items.
Downloads
References
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74. https://doi.org/10.1119/1.18809
Hestenes, D., & Halloun, I. (1995). Interpreting the force concept inventory: A response to March 1995 critique by Huffman and Heller. The Physics Teacher, 33(8), 502. https://doi.org/10.1119/1.2344278
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141-158. https://doi.org/10.1119/1.2343497
Lawson, A. E. (1978). The development and validation of a classroom test of formal reasoning. Journal of Research in Science Teaching, 15(1), 11-24. https://doi.org/10.1002/tea.3660150103
Low, D. J., & Wilson, K. F. (2017). The role of competing knowledge structures in undermining learning: Newton’s second and third laws. American Journal of Physics, 85(1), 54-65. https://doi.org/10.1119/1.4972041
Planinic, M., Ivanjek, L., & Susac, A. (2010). Rasch model based analysis of the Force Concept Inventory. Physical Review Special Topics - Physics Education Research, 6(1), Article 010103. https://doi.org/10.1103/PhysRevSTPER.6.010103
Scott, T. F., & Schumayer, D. (2017). Conceptual coherence of non-Newtonian worldviews in Force Concept Inventory data. Physical Review Physics Education Research, 13(1), Article 010126. https://doi.org/10.1103/PhysRevPhysEducRes.13.010126
Scott, T. F., Schumayer, D., & Gray, A. R. (2012). Exploratory factor analysis of a Force Concept Inventory data set. Physical Review Special Topics - Physics Education Research. 8(2), Article 020105. https://doi.org/10.1103/PhysRevSTPER.8.020105
Taniguchi, M.-a., & Yasuda, J.-i. (2014). Quantitative validation of Japanese translation of Force Concept Inventory using subquestions [Japanese]. Journal of the Physics Education Society of Japan, 62(4), 226-231. https://doi.org/10.20653/pesj.62.4_226
Thornton, R. K., Kuhl, D., Cummings, K., & Marx, J. (2009). Comparing the force and motion conceptual evaluation and the force concept inventory. Physical Review Special Topics - Physics Education Research, 5(1), Article 010105. https://doi.org/10.1103/PhysRevSTPER.5.010105
Traxler, A., Henderson, R., Stewart, J., Stewart, G., Papak, A., & Lindell, R. (2018). Gender fairness within the Force Concept Inventory. Physical Review Physics Education Research, 14(1), Article 010103. https://doi.org/10.1103/PhysRevPhysEducRes.14.010103
Wainer, H., Bradlow, E. T., & Wang, X. (2007). What’s a testlet and why do we need them? In Testlet response theory and its applications (pp. 44–59). Cambridge University Press. https://doi.org/10.1017/CBO9780511618765.005
Wang, J., & Bao, L. (2010). Analyzing force concept inventory with item response theory. American Journal of Physics, 78(10), 1064-1070. https://doi.org/10.1119/1.3443565
Wilson, K. F., & Low, D. J. (2015). “On second thoughts…”: Changes of mind as an indication of competing knowledge structures. American Journal of Physics, 83(9), 802-808. https://doi.org/10.1119/1.4928131
Yasuda J.-i, Hull, M. M., & Mae, N. (2023). Visualizing depth of student conceptual understanding using subquestions and alluvial diagrams. Physical Review Physics Education Research, 19(2), Article 020121. https://doi.org/10.1103/PhysRevPhysEducRes.19.020121
Yasuda, J.-i., & Taniguchi, M.-a. (2013). Validating two questions in the Force Concept Inventory with subquestions. Physical Review Special Topics - Physics Education Research, 9(1), Article 010113. https://doi.org/10.1103/PhysRevSTPER.9.010113
Yasuda, J.-i., Hull, M. M., & Mae, N. (2022). Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory. Physical Review Physics Education Research, 18(1), Article 010112. https://doi.org/10.1103/PhysRevPhysEducRes.18.010112
Yasuda, J.-i., Mae, N., Hull, M. M., & Taniguchi, M.-a. (2018). Analyzing false positives of four questions in the Force Concept Inventory. Physical Review Physics Education Research, 14(1), Article 010112. https://doi.org/10.1103/PhysRevPhysEducRes.14.010112
Yasuda, J.-i., Mae, N., Hull, M. M., & Taniguchi, M.-a. (2021). Optimizing the length of computerized adaptive testing for the Force Concept Inventory. Physical Review Physics Education Research, 17(1), Article 010115. https://doi.org/10.1103/PhysRevPhysEducRes.17.010115
Yasuda, J.-i., Uematsu, H., & Nitta, H. (2011). Validating a Japanese version of Force Concept Inventory [Japanese]. Journal of the Physics Education Society of Japan, 59(2), 90-95. https://doi.org/10.20653/pesj.59.2_90
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Michael M. Hull, Jun-ichiro Yasuda, Naohiro Mae
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright © Authors