Computer Technology and Complex Problem Solving: Issues in the Study of Complex Cognitive Activity

Abstract: Goals and plans organize much complex problem solving behavior and are often inferable from action sequences. This paper addresses the strengths and limitations of inferring goals and plans from information that can be derived from computer traces of software used to solve mathematics problems. We examine mathematics problems solving activity about distance, rate, time relationships in a computer software environment designed to support understanding of functional relationships amoung these variables (e.g. distance = rate x time; time = distance/rate) using graphical representations of the results of simulations. Ten adolescent-aged students used the software to solve two distance, rate, time problems, and provided think out loud protocols. To determine the inferability of understanding from the action traces, coders analyzed students' understanding from the computer traces alone (Trace-only raters) and compare these to analyses based on the traces plus the verbal protocols (Trace-plus raters). Inferability of understanding from the action traces was related to level of student understanding how they used the graphing tool. When students had a good understanding of distance, rate, time relationships, it could be accurately inferred from the computer traces if they used the simulation tool in conjunction with the graphing tool. When students had a weak understanding, the verbal protocol were necessary to make accurate inferences about what was and was not understood. The computer trace also failed to capture dynamic exploration of the visual environment, so students who relied on the graphing tool were inaccurately characterized by the Trace-only coders. Discussion concerns, types of scaffolds that would be helpful learning environment for complex problems, the kind of information that is needed to adequately track student understanding in this content domain, and instructional models for integrating learning environments like these into classrooms.

Key words: computers, complex problem solving

*Members of the Cognition and Technology Group at Vanderbilt who have contributed to this project are (in alphabetical order) Helen Bateman, John Bransford, Thaddeus Crews, Allison Moore, Mitchell Nathan, and Stephen owens.