FINAL REPORT

Assessing Student Understanding of Concepts in Electricity

to inform Instructional Decisions

 

ONR Research Group

 

Gautam Biswas, Daniel Schwartz, Bharat Bhuva, John Bransford, Doug Holton, Amit Verma, and Jay Pfaffman.

 

Vanderbilt University

Box 1679, Station B

Nashville, TN 37235

 

 

 

 

 

 

1.       INTRODUCTION       

 

            We have been investigating students' knowledge and understanding of basic concepts in electricity and their application to solving electrical circuit problems. In previous work, we identified and characterized domain concepts that students had difficulty applying correctly to problem solving tasks mainly in the DC domain. We found that student knowledge was “in pieces,” and their lack of understanding could be broadly classified into four different categories: (i) undifferentiated concepts, (ii) experiential impoverishment, i.e., the inability to link physical processes and parameters to abstract circuit models, (iii) incomplete metaphors, and (iv) simplifying assumptions of minimum causality [Biswas, et al., 1997]. Moreover, the “invisible” nature of electricity made it difficult to comprehend, and beginning students came into the domain with very few preconceptions (and, therefore, misconceptions). Most of what a student knew was picked up from instruction. We also discovered that the range of misconceptions and student learning styles were best handled by employing different perspectives and instructional resources. We developed a learning environment that provided resources for self-assessment along with learning, and pilot studies showed that approach to be quite useful in learning difficult concepts.

 

In this phase of the project, our initial focus was on characterizing the AC circuit domain, and analyzing student understanding and problem solving ability in this domain.  Protocol analyses on beginning and more advanced undergraduate students in the Electrical Engineering (EE) program revealed that students have very little physical intuition of AC circuit concepts.  Students’ problem solving primarily involved the generation of mathematical formulations (equations), and manipulating these formulations to derive numerical solutions to problems. Appeal to everyday physical phenomena did not seem to clarify or improve the students’ understanding of these concepts. For example, one of the most prevalent misconceptions among beginning EE students is that the sinusoidal waveform represents a spatial property of the voltage and current in a wire as opposed to a time-varying description of behavior that occurs simultaneously at every point in the wire. These students also have no notion of what it means for voltage and current to take on negative values. Some of this may be attributed to the students’ lack of understanding of the physical nature of voltage and current. However, a more direct reason may be the natural mapping those students create from the visual representation of the sinusoidal waveform to the spatial dimension of a wire. Unlike the DC domain, our attempts to link AC waveforms to everyday phenomena, such as the operation of radio receivers, and the transmission of signals from different radio stations to a receiver, did not help in clarifying misconceptions.  Students also had a lot of difficulty in understanding the behavior of components, such as capacitors and inductors, which exhibit time varying behavior in AC circuits. In such situations, most students could not correctly formulate and explain the equations for analyzing AC circuit behavior. We have developed a test in the AC domain to capture the primary misconceptions that students exhibit in understanding AC circuit behavior. This is discussed in greater detail in Section 3.

 

Our protocol analyses and misconceptions studies have established that students have very little understanding of AC circuit concepts. As a result, they exhibit a lot of difficulty in formulating and solving problems in this domain. Moreover, a number of the students’ misconceptions and difficulties can be linked to instruction, as opposed to pre-conceived notions of domain concepts. These observations have led us to turn to dynamic assessment approaches (Feurestein, 1979; Campione and Brown, 1985 and 1987; Bransford, et al, 1987) and focus more on how to prepare students to learn through instruction. This methodology, called the Assessment for Domain Learnability is described in greater detail in Section 4.

 

 In the last year, we have decided to adopt a systematic methodology for instruction, learning, and assessment in this domain. We adopt a generic framework for describing physical systems in terms of their structure, behavior, and function. We link this descriptive framework to three broad categories of problems that engineers and technicians encounter in their everyday work: analysis, diagnosis, and design. These three tasks can also be looked upon as mappings between the structure, behavior, and function of a circuit, and the formulation is used to develop sets of questions for students to assess their understanding of the various concepts in the domain.

 

To aid students in developing a systematic and well-structured problem-solving paradigm, we have adopted an instructional strategy that emulates expert problem solving behavior. An important component of this process is to get students to reason about phenomena using qualitative techniques, so that their focus is on the application of the laws that govern circuit behavior, and not on mathematical manipulations. We introduce the notion of invariants that capture the fundamental laws and concepts that govern electrical circuit behavior. We have developed a web-based software system for self-assessment and learning called Inductor (described in Section 5) that presents students with a set of multiple-choice questions about a variety of AC circuits. The sequence goes from simpler questions to progressively more difficult ones, and starts from purely resistive circuits, and then goes onto RC and RLC circuits linked to different real-world applications. Students are required to pick the relevant invariant relations and analyze them qualitatively to derive the solution to the problem.  A detailed description of the system, and preliminary experiments that we have conducted with the system are described in Section 5.

 

The report ends with a summary of the current status of our work, and proposes directions for future research. A set of appendices provides details of the Inductor system.

           

2.       AC DOMAIN DESCRIPTION

 

Like DC circuits, the fundamentals of the AC domain are represented in terms of voltage, current, and power. In AC circuits, these values are time varying, and described visually as waveforms, most typically sinusoidal waveforms. Two parameters of these waveforms, the frequency and the phase, play an important role in characterizing the behavior of AC circuits. Typically beginning students are able to reproduce voltage and current values in mathematical (the sinusoidal equation) and visual forms (sine waves), but do not really understand the link between the waveforms and the voltage drops and current flows in a given circuit.

 

The time-varying nature of voltage and current is the basis for the differences in AC and DC circuit analysis. For purely resistive circuits, this difference is not significant because voltage and current remain in phase, and resistance values are not affected by frequency changes. Therefore, voltage and current computations are based on simple algebraic relations. Power computations in AC circuits have an equivalent DC expression when voltages and currents are expressed as root mean square (RMS) values.

 

Capacitor and inductor elements exhibit significantly different behaviors in AC circuits. Their impedance values (the equivalent of resistance) are a function of the frequency of the AC waveform, and this property is exploited in the design of a number of applications. Capacitor and inductor elements also cause a phase difference between voltage and current, and this is used in the design of applications like filters, oscillators, and signal generators.

 

Our approach to analyzing student understanding of DC and AC concepts is based on the observation that the two domains share a number of fundamental concepts. Our protocol studies on AC understanding were divided into two phases. The first phase focused on these basic concepts.  The second phase looked at more advanced AC concepts in the context of applications. The primary applications of AC systems are in power transmission, broadcasting, and communication. AC is still the most effective way for power generation and transmission, but in the present day digital generation, most equipment, such as computers, convert the input AC voltage to a DC voltage before use.  Communication systems use AC waveforms superimposed on DC signals for their operation. In keeping with our previous protocol studies (Biswas et al., 1997; Schwartz, Biswas, Bransford, Bhuva, Balac, & Brophy, 2000), where we studied DC concepts in the context of real-world devices, our study of student understanding of advanced AC concepts has been in the context of the applications discussed above.

 3.      MISCONCEPTION STUDIES

 

We briefly review previous work in analyzing misconceptions in the domain of electricity. Most of this work has been targeted to DC circuits. We extend the analysis of DC misconceptions to the AC circuit domain, and present the results of our protocol studies. To analyze misconceptions in a more systematic way, we have developed a misconceptions test for AC concepts. We briefly describe the test in this report.  The complete set of test question can be accessed at http://relax.ltc.vanderbilt.edu/onr/ac-misconceptions.doc.

 

Previous Work

The DC misconception literature lists the erroneous conceptions students have about the domain as well as the omissions of knowledge that they demonstrate.  In our previous work (Biswas, et al, 1997; Schwartz, et al, 2000) we categorize and report most of the known misconceptions and omissions that students have about the notion of voltage, current, resistance, power and other electrical circuit concepts.

 

Cohen, Eylon, and Ganiel (1982) found that students think of current as the primary concept (potential difference is regarded as a consequence of current flow, and not as its cause), and that the battery is often regarded as a source of constant current rather than constant voltage.  They also observed students' "difficulties in analyzing the effect that a change in one component has on the rest of the circuit" and dealing with a simultaneous change of several variables.  These misconceptions cause major problems in students' reasoning about electrical circuits.          Other literature in the field concentrated on student understanding using analogical models. For example, Gentner and Gentner (1993) dealt with two different analogical models:  (i) the “flowing water model,” where the flow of current through wires is analogical to the flow of water through pipes and (ii) the “teeming crowds model,” where the analogy was made between current or the flow of charged particles and the movement of crowds through passageways.  Magnusson, Temple, and Boyle (1997) discovered eight different students' models of the path of electric current in parallel circuits and adapted six different models of students' conceptions of current from work reported in Osborne (1983), Russell (1980), and Arnold and Millar (1987).

 

Hunt & Minstrell (1994) have generated a list of pre-scientific knowledge pieces, or facets, that students may have, including misconceptions about concepts in electricity.  They developed a program (DIAGNOSER) that targets and assesses these misconceptions with carefully constructed test questions.  Upon identifying a specific difficulty a student has, DIAGNOSER also provides some instruction and resources addressing this misconception.

 

AC Misconception Studies

We have begun extending the work on student understanding and misconceptions in the DC circuit domain to the AC and DC domains.  We generated a series of circuit questions relevant to the AC domain and interviewed students as they worked through these problems.  As a result of these structured interviews, we identified specific areas in which students had misconceptions or lacked experience (listed later in this section).  More recently we also constructed part of a misconceptions multiple choice test that targets these misconceptions, in cooperation with Steve Parchman and other researchers (also described later in this section). 

 

Experimental Setup

For the protocol analysis studies, we made up a number of AC problems, starting from the simple DC flashlight, but replacing the DC source with an AC source. The first set of problems were set up for students to analyze contrasting cases, such as what happens in the flashlight circuit when the DC source is replaced by an AC source, and where would you place fuses to protect a component in identical DC and AC circuits. In this study, we were specifically looking for misconceptions that students had exhibited in an earlier study on DC circuits, such as  (i) the empty pipe and sequential flow misconceptions, (ii) the inability to recognize the differences between voltage and current, and (iii) the belief that current remained constant in a circuit, and what impact these misconceptions may have on their understanding of AC circuits. In addition, there were questions that asked students to analyze the effect of changing source frequency on power consumed in a circuit.  In some cases, the students were asked to plot the voltage and current waveforms at different points in a circuit. The students involved in this study were beginning Electrical Engineering (EE) students at Vanderbilt University who had completed their first circuits course. We also interviewed students in the Navy training center at Memphis.

 

We also developed a second, more advanced AC problem set, where students were asked to explain how a particular device worked, and especially why it exhibited certain behaviors and functionality.  The second set of problems tested student understanding of capacitors and inductors in AC circuits, and the use of RC and RLC circuits in practical applications. This set of problems was presented to senior undergraduate students and some graduate students. We also interviewed an electrical technician. The focus was on whether students could analyze the circuits and produce a qualitative explanation of the observed system functionality. Our last report [Biswas, et al., 1999] describes the problem sets in greater detail. 

 

AC Misconceptions

The analysis of student responses provided interesting results.  We interviewed a total of 18 subjects at Vanderbilt University, and about 6 trainees in their first EE technician course at the Memphis naval center. All 12 Vanderbilt students in the first group were in the beginning electrical engineering course (EECE 112), and the 6 students in the second group were juniors, seniors, and graduate students.  In our protocol analysis we found a variety of erroneous knowledge about basic AC concepts.  They are summarized below.  We divided the misconceptions into three categories:

 

1.      Those directly related to characteristics of AC waveforms,

2.      General classes of difficulties that are linked to cognitive difficulties, and

3.      Lack of knowledge of general domain principles.

 

These are discussed in greater detail below.

 

 

List of Misconceptions specific to AC waveforms.

 

1.      Spatial AC misconception.  The sinusoidal AC voltage and current waveforms are not a representation of variation of these variables at a point in time.  Rather they depict a variation of their magnitudes along the length of the wire in which the current is flowing. For example, students said that a string of identical light bulbs in series when connected to an AC source would light up in sequence, and some of the light bulbs may be on when others are off.  At the same instant of time, the brightness of the bulbs would vary depending on their position in the circuit.

 

2.      Negative part of AC cycle is just a mathematical artifact. No current flowing in circuit or power delivered during negative part of AC cycle.  For example, a number of students said that a light bulb only lights up during the positive part of the sinusoidal cycle.  Others said that there could be “no such thing as negative current. That is just a mathematical artifact.  If current reverses, the electrons would reverse direction too. They would then run into each other, stopping flow, which implies there could be no current.”

 

3.      Alternate form of this misconception. The negative current "cancels" out the positive current.  So bulb will never light up when you connect to true AC source.

 

4.      Empty pipe misconception. During AC cycle electrons stop, turn around, and go the other way. In some cases when you have very long wires, they may never reach the light bulb connected to the end of the wire. Students thought that you would need two fuses to provide protection in an AC circuit, where you could do with one in a DC circuit.

 

5.      Incorrectly importing DC models to explain AC.

A.      Students often surmised that the alternating current going through a resistor was constant in time.

B.      Students often hypothesized that a capacitor behaved the same in AC and DC circuits.

 

6.      Difficulties understanding circuit behavior when AC and DC signals are combined.  Students had difficulty “separating” or recognizing the AC and DC components of a signal in problems in which the midpoint of a sinusoidal voltage was not zero.

 

7.      More generally, difficulty thinking of circuit behavior when multiple waveforms, frequencies are combined.  Even advanced students stated that the number of channels you can got from cable TV was a function of the number of wires in the cable, or the thickness of the cable.

 

General classes of difficulties that are not specific to AC. [Schwartz, et al. 2000]

 

8.      Failure to differentiate among concepts. Examples, voltage and current, series and parallel configurations, role of capacitor in DC versus AC circuits.

 

9.      Minimum causality error.  (Incorrect simplifying assumptions). Single change in outcome must be a result of single change in cause. (e.g., a 10W bulb must have greater resistance than a 5W bulb).

 

10.  Overly local reasoning.  Not thinking of global constraints, invariants.

 

11.  Bad framing.  Incorrect generalizations, trouble switching from equations to physical explanations to analogical models.

 

12.  Experiential impoverishment. Electricity is invisible except for its end products.

 

Lack of basic circuit knowledge.

 

13.  Lack of Ohm's law (how resistance affects current when voltage is constant)

 

14.  Lack of KCL  (current through all components of a loop must be equal).

 

15.  Lack of KVL  (the voltage drop across components of a loop must sum to zero).

 

Note that 14 and 15 together represent the conservation laws: (i) charge cannot disappear, and (ii) energy must be conserved.

 

16.  Lack of knowledge of the behavior of capacitors (such as C=Q/V)

 

17.  Lack of knowledge of Capacitor and Inductor impedance as a function of frequency.

 

18.  Topographic misunderstanding of the circuit (e.g. unable to differentiate series from parallel).

 

 

Misconceptions Test

            Using the above list of AC misconceptions, we developed a set a number of multiple choice test questions to target these misconceptions in cooperation with Steve Parchman’s group in Florida, and other researchers.  An example question is shown below in Figure 1.

This question focuses on the spatial misconception that students have regarding electricity (water-pipe model, and the spatial variation of AC signals).  The question also addresses the notion of electricity as a substance, i.e., electricity gets consumed as it goes along the string of lights.

The misconceptions test has been conducted with naval students, and Steven Parchman’s group is currently analyzing the results.  The full test can retrieved from http://relax.ltc.vanderbilt.edu/onr/ac-misconceptions.doc.

 

Discussion

            Our preliminary study of student understanding in the AC domain has proven to be quite revealing.  Beginning students seem to have very little understanding of the time-varying nature of AC voltage and current. This can be attributed to a combination of problems they exhibit in their basic understanding of concepts. The empty pipe misconception affects their understanding of current flow, and makes it especially difficult for them to reason about current that reverses direction periodically. The inability to differentiate between voltage and current and the lack of understanding in mapping from physical concepts to abstract circuit parameters compounds students’ problems. They are often stuck with beliefs such as a source provides constant current, and a source cannot deliver power unless the current flows in one direction from one of its terminals to another. These misconceptions and lack of knowledge are not unique to the AC domain; in fact students exhibited the same problems when reasoning in the DC domain. 

 


 


 A Christmas light strand contains 50 identical light bulbs connected in series to form a light string. When it is plugged into a 110 volt AC power socket of frequency 60Hz, which light will burn the brightest?

 

a)       The first bulb is always the brightest.

b)       The 50th bulb always burns the brightest.

c)       Since the current is alternating, each of the bulbs starting from the first to the 50th is the brightest in turn.

d)       All of the bulbs are equally bright at all times.

Correct answer:

d) All of the bulbs are equally bright at all times.

 

Figure 1:  A Misconceptions Test question and accompanying figure

 

 

From the point of view of instruction, these observations can be interpreted in many ways. On the one hand, one can make the argument that since DC instruction traditionally precedes AC instruction, it is very important to ensure that students do not develop misconceptions and omissions described above during DC instruction. Careful contrasts also need to be made when making the transition from the DC to the AC domain. On the other hand, one could say that the similarity of the basic concepts in the two domains imply that the most effective form of teaching should focus on the concepts and their implications in problem solving rather than spend a lot of effort in focusing on the differences. For resistive circuits, the time-varying nature of AC voltage and current has no strong implications on behavior. Students need to understand the concept of power delivered, and how to compute the power delivered.  As discussed earlier, the time-varying nature of current and voltage has important implications in circuits with capacitors and inductors, and it may be best to introduce these concepts by demonstrating their use in real applications and devices. The latter approach may be further justified by the observation that a number of the misconceptions of the beginning students seemed to go away as they moved on to more advanced courses.

Another issue of importance that we have observed among students is their reliance on mathematical formulations and solving of equations to derive answers to problems. As discussed earlier, this implies the students lack understanding of the underlying physical phenomena, and therefore, do not develop a deep understanding of the basic concepts in the domain.  This problem is even further compounded in the AC domain, especially when students have to deal with the more complex phenomena associated with real world devices and systems. When dealing with the questions in problem set 2, a number of students attempted to convert the given circuit or problem description into mathematical equations. However, the resultant differential equations were hard to analyze, and did not directly provide the information required to solve the problem. The implication here is that students need to develop a better qualitative understanding of phenomena, and how these phenomena combine to produce circuit and system functionality. In our protocol studies on the second problem set, a number of students had to be coached to reason about a problem qualitatively.  Only then were they able to analyze the problem, and generate the desired solutions and explanations. Developing qualitative reasoning skills and function-level understanding may also contribute to the development of better troubleshooting skills, a long-term goal of this research.

 

In the next section of the report, we develop a methodology for instruction that combines learning with assessment. The goal is to exploit computer technology to provide students with an environment for selecting from a set of available resources depending on their self-identified needs.

 

 

 

4.       FROM PROTOCOL ANALYSIS TO INSTRUCTION:

 The Assessment of Domain Learnability Framework

               

Our studies of student understanding in AC and DC circuit problem solving suggested that student misconceptions and difficulties could be linked to instruction as opposed to the preconceived notions of domain concepts.  These observations led us to turn to dynamic assessment approaches (Feurestein, 1979; Campione and Brown, 1985 and 1987; Bransford, et al, 1987) and focus more on how to prepare students to learn through instruction. Our first step in this direction was to build a computer-based tools using the STAR.Legacy framework to help students self-assess their understanding of concepts linked to DC circuit problem solving, and to provide resources to help students learn these concepts they found difficult to learn.

 

Assessing Domain Learnability

 It appears that some electricity concepts may be more difficult to learn than others. With respect to the instruction in this domain, we believe that an important research task is to identify features and concepts that influence learnability of concepts that affect problem solving tasks.   We will call this task "assessing domain learnability” or ADL for short. By trying to remediate people's misconceptions and missing conceptions, we may determine which are particularly difficult to remediate given our methods of instruction (e.g., Heller & Finley, 1992), and which type of understanding has the greatest impact on subsequent learning.   The basic observation is that not all misconceptions are equally strong or equally relevant to future instruction.   For example, although we have rarely seen it in the literature (Cooke & Breedin, 1994), it would be interesting to ask people to compare their confidence in answers where they exhibit misconceptions relative to those that they do not.  We suspect that for many of the misconceptions that have been documented, people are reasonably aware that they do not know what they are talking about.   For those misconceptions that are of low confidence, should we expect that people would be more likely to overcome their misconceptions and learn?  Much of the research on misconceptions has no handle on this question.  An ADL approach seems more likely to provide an answer.

            There may be limitations to ADL as we have conceptualized it so far.  One possible weakness of ADL is that it is particularly prone to the ways that we assess whether someone has learned a correct conception or not.  For example, if we ask the exact same question that we taught, does this mean that people have learned in any meaningful sense?  The problem of assessing and deciding upon ecologically satisfactory understanding, however, is a problem faced by much educational research.  ADL actually fairs better than most in this regard.  This is because the ultimate test for ADL is whether a given concept has implications for future learning.  For example, consider the typical course sequence in electrical engineering where students begin with direct current (DC) circuits and then move to study alternating current (AC) circuits.  Students start with many misconceptions about DC circuits.  Are all the misconceptions and their correct counterparts equally important in shaping students' ability to learn AC circuits?  This is the question that ADL is designed to answer.

            A second potential weakness to ADL is that if our instruction fails to teach a correct conception of a domain, it is hard to determine whether this was a function of the domain's difficulty or a function of our teaching methods.  On the one hand, we can never disentangle these two possibilities beyond a reasonable appraisal.  On the other hand, it is the interactions of the instruction and the domain that constitute the important parameters of assessing domain learnability.   The emphasis of ADL is not on domain learnability in the abstract, but rather domain learnability with respect to the state of the art in instruction.  The next section describes a computer environment that captures many of our ideas about the state of the art.

 

A Computer-Based Learning Environment for DC Problem Solving

A computer-based environment provides an integrated learning-assessment tool for pulling together different instructional techniques and resources that can be applied to a domain.  A single instructional technique would be too restrictive for ADL.  For example, one might use a dynamic tutoring system to teach the procedural knowledge of a domain, but there are other types of knowledge that are important to assess as well, like, do people have difficulty constructing a mental model of the domain (Lajoie & Lesgold, 1992).  Similarly, one might create a system that matches an individual's misconceptions against a known "bug list" and teaches to those bugs directly, but this typically assumes that misconceptions are non-interacting.

 

            Our software environment for assessing the learnability of DC concepts was created using the STAR.Legacy framework (Schwartz, et al. 2000).  In line with the test-teach-retest model of dynamic assessment, students begin with a question in the look ahead problem and end with the same question when they reflect back.  In this case, the look ahead and reflect back problem asks students to explain what happens in a simple flashlight circuit when a 5-watt bulb is replaced by a 10-watt bulb. The overall assessment and learning task is divided into three challenges, which were chosen on the basis of our protocol research described earlier (Biswas, et al, 1997).  We found three problem situations that were particularly good at making students' thinking visible.  Challenge 1 asked students to reason about the possible causes of a dim bulb.  This problem was intended to help students differentiate voltage and current, to help them overcome the minimum causality error, and to give them some increased experience in the domain and its analogies.  Challenge 2 asked students to design a battery operated drill that could run at different speeds.  In this design problem, students progressively deepen their understanding of the topics raised by challenge 1 while adding the issues of local reasoning and framing.  Finally, challenge 3 tried to bring the lessons together into a single problem.  In this challenge, students were asked to reason about a flashlight that has two bulbs, one that points forward and one that points to the ground.  They are told that somebody wants to change the forward bulb to a higher wattage.  How will that effect the flashlight overall?  These challenges are intended to bring forward the different classes of misconceptions that students may possess.  At the same time, we expect the interaction of the challenges and instruction to reveal other conceptual hot spots. This is one of the attractive features of ADL -- it can reveal misconceptions in the context of instruction.

            After reading a challenge, students try to generate their first thoughts about how to prepare for solving the challenge.  These initial thoughts usually provide both instructors and the student with a sense of the strengths and weaknesses of the student, and it helps the student choose which of the multiple perspectives to listen to.  Each perspective directly targets key learning difficulties with a 10-15 second comment by an expert.  For example, one of the perspectives has an expert explain the minimum causality error, although not in those terms.  The expert states, "a common mistake that people make with these problems is that they often do not realize that when the power changes, two other things in the circuit must change.”   Another perspective tries to tie the perceptual phenomena (a dim bulb) to relevant electrical concepts by pointing out that a dim bulb means less power is being consumed.  Another perspective, under the assumption that the students have been taught some form of water analogy, tries to get students to think how voltage and current map into the water domain.

 

 

            When students listen to the perspectives, instructors may ask the student to explain whether they understand what the experts are saying.  This provides them with valuable knowledge about which aspects of the domain the student may be having trouble with.  For example, some students do not know that "two things must change,” whereas others may not know how to draw the analogy between water and electricity.  This becomes important when the interview proceeds to Research & Revise.  The student and interviewer choose which resources to work with depending on the gaps in knowledge.

            Figure 2 shows the resources that are available for challenge 1.  A chalk talk on Ohm's law explains why two things must change if the power changes.  There is also a set of multiple-choice problems that allow students to practice using Ohm's law.  These problems include automated feedback that states the qualitative implications of the student's incorrect answers.  For example, one feedback comment reads, "This answer implies that as you increase the voltage across the circuit, current will decrease!  For example, if we used a more powerful battery, the current in the flashlight circuit would decrease.  Does that make sense?"   This form of feedback helps the students to think about qualitative relationships as opposed to simply making algebraic manipulations of numbers.

           


 


Figure 2: Resources for challenge 1: by clicking on an image, a learner can gain access to its resources

 

Another resource is a brief presentation of a mnemonic that helps students memorize that current is a "through" property whereas voltage is an "across" property.  There are also pairings of simulations of a circuit and an analogous water system.  The resource page also includes connections to web sites that we have found helpful, comments by students who have completed the process and offer their thoughts about key insights that helped their learning, and pointers to simulations and hands-on activities developed by others (e.g., Parchman, 1997).

 

            Depending on how comfortable and confident students feel about the material, they can move between resources and perspectives to probe further and learn more about the relevant concepts.  Once the students feel that they have made satisfactory learning progress, they move to test your mettle to test the strength of their knowledge. After students complete the learning cycle for challenge 1, they move to subsequent challenges, which require students to rethink concepts that they have already learnt, and also to deal with new concepts and misconceptions. Subsequent challenges are structured like the first challenge.

 

            This dynamic assessment environment is different from other dynamic assessment models that are automated (Lajoie & Lesgold, 1992) because it keeps the instructor in the loop.  In part this is because it makes it much easier for others to replicate our efforts as compared to the overhead of creating automated or self-contained systems (Bell, 1998; Murray, 1998).   But in part, we have left the instructor in the loop because ADL requires a level of flexibility we cannot reasonably program into a machine.  Our instructors try everything at their disposal to help students learn.  They try to adapt to student needs and to the peculiar demands of the domain. DC-Legacy helps in this endeavor because it provides a flexible but pedagogically sound structure, multiple methods of instruction, and a single gathering of "at the ready" resources.  There are two questions that come to mind now.  One question is what aspects of the domain were generally difficult or impossible to remediate.  A second question is whether certain conceptualizations facilitate the students' subsequent learning. 

 

Discussion

            In this section, we described a theory that can help evaluate misconceptions in the context of instruction.  To this end we proposed a dynamic assessment approach to assessing domain learnability.  In this approach, researchers try their best to teach students.  Those concepts that students still have difficulty with tell us something about the components of the domain that are particularly difficult to learn, at least with respect to the instruction that we can provide.  The results help focus attention on those concepts that are particularly problematic, rather than simply making a list of possible misconceptions in a domain.  Ideally, the important concepts are also identified with respect to their impact on future learning in the domain.

           

            We described several classes of learning difficulties (misconceptions) that others and we have found for the domain of electricity.  We tried to organize these misconceptions according to the way they fit into basic cognitive processes -- differentiation, simplifying assumptions, local reasoning, and the need for framing. Our underlying assumption is that domain learnability is best understood as the interaction of individuals' cognitive tendencies, the demands of the domain, and instruction.   We constructed a DC-Legacy that targeted these different classes of learning difficulties to fulfill the instructional component of our assessment.  Using DC-Legacy, several members of our group were able to add their own expertise to make a rich dynamic assessment environment for learning DC circuits.

 

             We conducted a small study to see if there is merit to the approach.  The preliminary results are promising.  We found that some commonly cited misconceptions, like the difficulty of handling parallel resistors, are not problematic by the time students leave a typical college course in electricity.  We found that other misconceptions, like local reasoning about the movement of current from point to point, are not treated by our courses.  However, they are easily remediated and do not have to serve as blocks to learning the domain.  And, we found that some aspects of the domain are difficult to learn even with special attention.  In particular, it appeared that people have trouble integrating multiple causes and this is exacerbated by faulty intuitions that cause them to focus on singular causes.  We suspect that most instruction does not sufficiently help students construct mental models that incorporate both the empirical reasoning of causal intuition and the helpful structure of mathematics (Schwartz & Moore, 1998).  In electricity it seems particularly important to help students make sense of the mathematical formulas (qualitatively or quantitatively) so they may overcome the tendency towards minimum causality.  In our protocols with college professors and field experts alike, we have found that when they come to an obstacle in their reasoning, they resort to equations to solve difficult conceptual problems.  And, in our discussions of AC circuits, electrical engineering experts rely so heavily on mathematics that they often cannot even generate physical analogies.  They are reasoning about representations, primarily mathematical; the empirical phenomena are far in the background.

 

            The current theorizing, the computer environment that implements our theories, and the empirical results present our beginning efforts at creating dynamic assessment tools that can inform instruction in complex domains.  As such, none of the three are ideal and more work is left to be done.

 

5.         INDUCTOR: DESCRIPTION AND PILOT STUDIES

Inductor was designed as an online assessment tool by Jay Pfaffman to provide “a means to author, administer, grade, and learn from multiple-choice tests” (Pfaffman, Schwartz, & Martin, 2001).  These tests are often criticized for being shallow and for promoting memorization, but they can be transformed into assessments for learning (Bransford, Brown, & Cocking, 1999).  They may be redesigned to specifically target misconceptions and important knowledge principles, and when they are presented in a computer environment, they can be designed to provide immediate and elaborate feedback to students (Hunt & Minstrell, 1994).  By using an online system, we may also provide access to outside resources for learning, such as instructional web sites and simulations or animations (see Appendix B for an annotated listing of some of the resources used in our tests with Inductor).  This creates a learning environment that goes beyond the typical sequestered problem solving context for taking tests in which there is no access to tools or resources (Bransford & Schwartz, 1999).

 

            Inductor is implemented as a web-based tool, through the use of PHP scripting to create a web browser interface that is connected to a MYSQL database backend.  This allows instructors with a web browser to remotely create and author test questions, and students to work on assignments outside of the classroom environment.  Test authors may include graphics, audio, and video with each question, and provide general and specific feedback for each multiple-choice answer (both the right and wrong ones) to a question.

 

            Appendix A visually demonstrates how a student uses the online Inductor tool (see also: http://relax.ltc.vanderbilt.edu/onr/demo.php ). The student selects a test to take, and is presented with detailed instructions. Once they are done with the instructions and the preliminary screens, the student sees a grid on Inductor depicting all of the questions in the test, organized by topic or some other structure that the designer has chosen.  For example, in Inductor we organized the test into four categories of questions: DC resistive circuits, AC resistive circuits, DC circuits with capacitors, and AC RC (filter) circuits. These were laid out row-wise. In addition, we identified three classes of problems: diagnosis, design, and analysis problems. An icon associated with a question indicated the type of question to the student. Inductor is set up to provide continual feedback to students on their performance via the “control your own destiny” (CYOD) grid interface.  Students access a question by clicking on the question’s icon.  They may work on one question at a time, but after answering a question may revisit it any number of times to review its contents.  After answering a question, the student returns to the grid.  At all times, color-coding on the question icons (green for correct answers, red for wrong answers) tells the student how well he or she has performed thus far. At this point, students may review these previously answered questions, review resources,  or move on to a new question.

 

            All questions are described in text with an accompanying schematic or circuit diagram, and a set of multiple choice answers. The student is asked to pick the right answer to the question, and provide additional information, such as explanations in additional boxes that we provide on the screen. Every screen also has pointers to generic resources, as well as specific ones that may contain material relevant to that particular question.  In this study, we required students to develop a problem solving methodology using invariants. Invariants come in two forms. First, they may pertain to laws of the domain that directly apply to the problem solution being sought. Second, they are also linked to the set of variables in the particular problem-solving situation that do not change. Foe example, if one is dealing with a flashlight problem, where the bulb is replace by a higher wattage bulb, once can reason that the source voltage (battery) does not change, but the resistance of the bulb does. Therefore, the source voltage is an invariant. Depending on the question asked, one can then invoke an invariant law, such as Ohm’s Law, the Power laws, or Kirchoff’s Voltage Law to answer the question. The list of invariant laws we have used in the study can be accessed at  http://relax.ltc.vanderbilt.edu/onr/invariants/invariants.html.

 

To promote and teach expert-like reasoning, we made students select the invariants they felt were relevant to solving the given circuit problem. Another unique characteristic of our questions was that there were no numbers associated with the circuit parameters ad variables. Students were required to derive their answers using qualitative reasoning techniques (i.e., current will be higher, the resistor must be shorted, etc.).  This was to get them to reason from first principles as opposed to putting in a lot of effort into numeric calculations. After answering a question, selecting the relevant invariants, and explaining their answer, students received feedback about their answer.  If incorrect, students were shown what invariants an expert chose as relevant to the problem.  In addition, students answering incorrectly received a hint and links to relevant outside resources to explore.