Using Process Features to Investigate Scientific Problem-solving in Large-scale Assessments

Overview

Journal Front Psychol

Date 2023 May 5

PMID 37143587

Authors

Tao Gong

Lan Shuai

Yang Jiang

Burcu Arslan

Affiliations

Soon will be listed here.

Abstract

Introduction: This study investigates the process data from scientific inquiry tasks of [requiring test-takers to manipulate a target variable while keeping other(s) constant] and (requiring test-takers to construct all combinations of given variables) in the National Assessment of Educational Progress program.

Methods: We identify significant associations between item scores and temporal features of preparation time, execution time, and mean execution time.

Results: Reflecting, respectively, durations of action planning and execution, and execution efficiency, these process features quantitatively differentiate the high- and low-performing students: in the fair tests, high-performing students tended to exhibit shorter execution time than low-performing ones, but in the exhaustive tests, they showed longer execution time; and in both types of tests, high-performing students had shorter mean execution time than low-performing ones.

Discussion: This study enriches process features reflecting scientific problem-solving process and competence and sheds important light on how to improve performance in large-scale, online delivered scientific inquiry tasks.

References

Liu O, Bridgeman B, Gu L, Xu J, Kong N . Investigation of Response Changes in the GRE Revised General Test. Educ Psychol Meas. 2018; 75(6):1002-1020. PMC: 5965601. DOI: 10.1177/0013164415573988. View

Griffiths T, Lieder F, Goodman N . Rational use of cognitive resources: levels of analysis between the computational and the algorithmic. Top Cogn Sci. 2015; 7(2):217-29. DOI: 10.1111/tops.12142. View

Chen Z, Klahr D . All other things being equal: acquisition and transfer of the control of variables strategy. Child Dev. 1999; 70(5):1098-120. DOI: 10.1111/1467-8624.00081. View

Kuhn D, Dean Jr D . Is developing scientific thinking all about learning to control variables?. Psychol Sci. 2005; 16(11):866-70. DOI: 10.1111/j.1467-9280.2005.01628.x. View

Klahr D, Nigam M . The equivalence of learning paths in early science instruction: effect of direct instruction and discovery learning. Psychol Sci. 2004; 15(10):661-7. DOI: 10.1111/j.0956-7976.2004.00737.x. View

Dominik Guss C, Burger M, Dorner D . The Role of Motivation in Complex Problem Solving. Front Psychol. 2017; 8:851. PMC: 5440455. DOI: 10.3389/fpsyg.2017.00851. View

Han Z, He Q, von Davier M . Predictive Feature Generation and Selection Using Process Data From PISA Interactive Problem-Solving Items: An Application of Random Forests. Front Psychol. 2019; 10:2461. PMC: 6882413. DOI: 10.3389/fpsyg.2019.02461. View

Tang X, Wang Z, He Q, Liu J, Ying Z . Latent Feature Extraction for Process Data via Multidimensional Scaling. Psychometrika. 2020; 85(2):378-397. DOI: 10.1007/s11336-020-09708-3. View

Gardner E . Instruction in mastery goal orientation: developing problem solving and persistence for clinical settings. J Nurs Educ. 2006; 45(9):343-7. DOI: 10.3928/01484834-20060901-03. View

10.

Lieder F, Griffiths T . Strategy selection as rational metareasoning. Psychol Rev. 2017; 124(6):762-794. DOI: 10.1037/rev0000075. View

11.

Gong T, Jiang Y, Saldivia L, Agard C . Using Sankey diagrams to visualize drag and drop action sequences in technology-enhanced items. Behav Res Methods. 2021; 54(1):117-132. DOI: 10.3758/s13428-021-01615-4. View