ONR YEAR 2 REPORT

Analysis of Student Understanding of Basic AC Concepts

ONR Research Group

Vanderbilt University

Box 1679, Station B

Nashville, TN 37235

Abstract

This project has progressed in multiple stages. In the first year of the project we studied student understanding of basic concepts related to voltage, current, and power in the domain of DC and AC circuits. Protocol analyses conducted in the context of problem solving tasks demonstrated that electricity is a hard domain to learn and understand, and most beginning student knowledge is “in pieces.’’ Students had difficulty in differentiating key concepts such as voltage and current, lacked the ability to map from physical processes to abstract notation, and experienced problems because they had incomplete mappings for metaphors and analogies. The “invisible nature” of electricity contributed to the complexity of the domain. Students, in general, had very few preconceived notions about electricity, and most of what they learned was gained through instruction.

In year two, our emphasis shifted to a study of how to prepare students to learn DC and AC circuit concepts they had difficulty with, and then to assess how instruction in these topics improved problem solving behavior. This led to the development of our framework for Assessment of Domain Learnability (ADL) and the implementation of a computer environment, STAR-Legacy, that integrates instruction with dynamic assessment. Preliminary studies depict the effectiveness of this approach in improving student problem solving capability in the DC circuit domain.

To further characterize problem solving ability with more advanced AC concepts we developed a set of problems dealing with voltage regulators and filters and performed protocol studies on a set of undergraduate and graduate students at Vanderbilt University. The results of this study are also reported.

I. INTRODUCTION

As a part of a larger project, we have been investigating students' knowledge and understanding of basic concepts in electricity and their application to solving electrical circuit problems. By conducting a set of protocol studies on Vanderbilt University students and Navy trainees in Memphis, we were able to identify and characterize domain concepts that students had difficulty applying correctly to problem solving tasks. The primary finding of this study was that student knowledge was “in pieces,” and their lack of understanding could be broadly classified into four different categories: (i) undifferentiated concepts, (ii) experiential impoverishment, i.e., the inability to link physical processes and parameters to abstract circuit models, (iii) incomplete metaphors, and (iv) simplifying assumptions of minimum causality. We summarize our primary findings for AC and DC concepts in a later section. The Year 1 report presents a more detailed account of our earlier findings.

Reflection on the protocol studies and results brought up a number of interesting issues:

· The “invisible” nature of electricity makes it difficult to comprehend, and beginning students come into the domain with very few preconceptions (and, therefore, misconceptions). Most of what a student knows is picked up from instruction.

· Students have a number of misconceptions but it is not clear how “dangerous” these are in terms of ability for “future learning” and problem solving. Some misconceptions are easily removed by instruction, whereas others are more difficult to deal with.

· The range of misconceptions and student learning styles are best handled by employing different perspectives and instructional resources. Developing learning environments that provide resources for self-assessment along with learning can be very powerful.

The rest of this report is divided into two parts. First, we describe results of our protocol studies and catalog student misconceptions in the analysis of basic AC and DC circuits. Then we discuss results of our preliminary protocol studies of student understanding of more advanced AC phenomena, that involve the use of RLC circuits in real applications like voltage rectifiers and signal filters. In part two we describe our work in developing a software environment that promotes learning and assessment, STAR (Software Technology for Action and Reflection)-Legacy, and results of studies that demonstrate the effectiveness of this approach. The report ends with a summary of the current status of our work, and proposes directions for future research.

Related Work

There are studies that report misconceptions about DC, papers that suggest better DC instruction, and work that concentrates on the models and analogies that students use, or that can be used in instruction

The DC misconception literature lists the erroneous conceptions students have about the domain as well as the omissions of knowledge that they demonstrate. In our previous work (Biswas, et al, 1997; Schwartz, et al, 1998) we categorize and report most of the known misconceptions and omissions that students have about the notion of voltage, current, resistance, power and other electrical circuit concepts.

Cohen, Eylon, and Ganiel (1982) found that students think of current as the primary concept (potential difference is regarded as a consequence of current flow, and not as its cause), and that the battery is often regarded as a source of constant current. They also observed students' "difficulties in analyzing the effect which a change in one component has on the rest of the circuit" and dealing with a simultaneous change of several variables. These misconceptions cause major problems in students' reasoning about electrical circuits.

There have been some studies that have put an effort into finding better ways to teach DC electricity. Several researchers suggest the use of analogies and metaphors in instruction. For example, White, Frederiksen, and Spoehr (1993) compared the use of two different models of electricity, the "particle model" (PM) and the "transport model" (TM). The authors reached the conclusion that true understanding of concepts in electricity can only be achieved by a set of linked models where the "emergent properties at one level become the primitive properties at the next level." White and Frederiksen (1990) designed a progression of qualitative, causal models of electrical circuit behavior that represent a transition from naivete to expertise. The models enable the instructional system to simulate circuit behavior and to generate causal explanations.

Other literature in the field concentrated on student understanding using analogical models. For example, Gentner and Gentner (1993) dealt with two different analogical models: (i) the “flowing water model,” where the flow of current through wires is analogical to the flow of water through pipes and (ii) the “teeming crowds model,” where the analogy was made between current or the flow of charged particles and the movement of crowds through passageways. Magnusson, Temple, and Boyle (1997) discovered eight different students' models of the path of electric current in parallel circuits and adapted six different models of students' conceptions of current from work reported in Osborne (1983), Russell (1980), and Arnold and Millar (1987).

We extend the work on student understanding and misconceptions in the DC domain to the AC and DC domains. Since there is not much work reporting protocol analyses in the AC domain, we briefly review basic concepts in the AC domain before discussing our experimental setup for protocol analyses. Our description of the DC circuit domain can be found in the Year 1 report.

AC Domain Description

Like DC circuits, the fundamentals of the AC domain are represented in terms of voltage, current, and power. In AC circuits, voltage and current values are time varying, and described visually as waveforms, most typically sinusoidal waveforms. When problem solving, students use the mathematical description of the waveforms, i.e., trigonometric functions defined by two parameters, frequency and phase. Typically beginning students are able to reproduce voltage and current values in mathematical and visual form, but do not really understand their link to voltage drop and current flow in a given circuit.

The time-varying nature of voltage and current is the basis for the differences in AC and DC circuit analysis. For purely resistive circuits, this difference is not significant because voltage and current remain in phase, and voltage and current values are computed using simple algebraic relations. Power computations in AC circuits have an equivalent DC expression when voltages and currents are expressed as root mean square (RMS) values.

Capacitor and inductor elements exhibit significantly different behaviors in AC circuits. Their impedance values (the equivalent of resistance) are a function of the frequency of the AC waveform, and this property is exploited in the design of a number of applications. Capacitor and inductor elements also cause a phase difference between voltage and current, and this is used in the design of applications like filters, oscillators, and signal generators.

Our approach to analyzing student understanding of DC and AC concepts is based on the observation that the two domains share a number of fundamental concepts. The first phase of our study on student understanding of AC concepts focused on these basic concepts. The second phase looks at more advanced AC concepts in the context of applications.

The primary applications of AC systems are in power transmission, broadcasting, and communication. AC is still the most effective way for power generation and transmission, but in the present day digital generation, most equipment, such as computers, convert the input AC voltage to DC before use. Communication systems use AC waveforms superimposed on DC signals for their operation. In keeping with our previous protocol studies (Biswas et al., 1998; Schwartz, et al., 1998), where we studied DC concepts in the context of real-world devices, our study of student understanding of advanced AC concepts has been in the context of the applications discussed above.

II. EXPERIMENTAL SETUP

For the protocol analysis studies, we made up a number of AC circuit problems, starting from the simple flashlight circuit used in our DC experiments, replacing the DC source with an AC source. The first set of problems were set up for students to analyze contrasting cases, such as what happens in the flashlight circuit when the DC source is replaced by an AC source, and where would you place fuses to protect a component in identical DC and AC circuits. The students involved in this study were beginning Electrical Engineering (EE) students at Vanderbilt University who had completed their first circuits course. We also interviewed students in the Navy training center at Memphis. For the advanced AC problem set, students were asked to explain how a particular device worked, and especially why it exhibited certain behaviors and functionality. The students involved in the study were more advanced undergraduate and graduate EE students. We also interviewed an electrical technician.

The first set of problems aimed at capturing students' understanding of the basic AC concepts was presented to students in beginning EE courses. Specifically, we were after a set of misconceptions that students had exhibited in an earlier study on DC circuits: (i) the empty pipe and sequential flow misconceptions, (ii) the inability to recognize the differences between voltage and current, and (iii) the belief that current remained constant in a circuit, and what impact these misconceptions may have on their understanding of AC circuits. In addition, there were questions that asked students to analyze the effect of changing source frequency on power consumed in a circuit. In some cases, the students were asked to plot the voltage and current waveforms at different points in a circuit. A list of the questions asked appear below in Figures 1-5.

The second set of problems tested student understanding of capacitors and inductors in AC circuits, and the use of RC and RLC circuits in a number of practical applications. The list of questions are presented in Figs. 6-9. This set of problems were presented to senior undergraduate students and some graduate students. The focus was on whether students could analyze the circuits and produce a qualitative explanation of the observed system functionality.

III. PROTOCOL ANALYSIS

The analysis of student responses provided interesting results. We interviewed a total of 18 people, 12 in the first group (Protocol Set 1) and 6 in the second (Protocol Set 2). All of them were Vanderbilt University students, 12 of them taking the beginning electrical engineering course, and 6 of them were undergraduates in more advanced engineering courses or graduate students. In our protocol analysis we found a variety of erroneous knowledge about basic AC concepts.

AC misconceptions

AC is a difficult domain. Even students who seem to understand basic DC concepts and apply them correctly in problem solving tasks, found it hard to grasp the concepts of alternating current and voltage. In most cases, beginning students seemed to have difficulty in resolving what was alternating. A very high percentage of students did not understand that current changed direction in AC. We heard responses like “current can only go in one direction,” and “voltage and current cannot really be negative, the absolute value is what is really happening, a ‘minus’ appears sometimes in calculations, and you should not worry about it.” Students could draw the sine wave forms for voltage and current in most cases, but could not map it to the circuit, and explain what it meant for them to go negative. In some cases students described it to be “just like a phase shift.”

Another common error was that the sine waveform was perceived to be a spatial rather than a temporal property of the voltage and current. In this form, the sine wave represented the different values of voltage and current at different points in the wire. In other words, the sine wave illustrated how “current flowed at different points in the wire.”

Answers to Protocol Questions Set 1

The first question asked students to explain voltage and current at different points in the flashlight circuit when the DC battery was replaced by an AC source. This brought out a range of misconceptions in the beginning EE student population. As discussed above, some were related to the notion that current had to keep flowing in one direction to enable power delivery to the bulb. Other students could not attribute any physical meaning to negative voltage and current. To check on the empty pipe misconception, we asked students what would happen if the length of the wire connecting the source to the bulb was progressively increased till it became very long. Would the light bulb not light up in this case? Only a few students gave the correct answers and consistent explanations for the set of questions asked, but most students could not comprehend the meaning of negative current and negative voltage. Some students, who knew that change in the sign of current implied a reversal in direction, got confused because they had the “empty pipe” misconception, and wondered what would happen if the electrons reversed direction before they reached the light bulb. In this case, no power would be delivered to the bulb. Students were also asked if change in frequency of the AC source waveform affected the power delivered to the bulb. Again, only students who understood the basic nature of the AC waveforms answered this question correctly.

Figure 1

Question 2 also focused on the “empty pipe” misconception. Students were asked where would they put a fuse in a DC and an AC circuit to protect an expensive light bulb: at the top or at the bottom? A majority of the students incorrectly answered “top” for DC

and “both places” for AC. Only a small percentage of students said that it “did not matter, 'cause the current is the same everywhere.” Graduate students in EE and some of the advanced students who were asked questions 1 and 2 answered these questions correctly, indicating that students gradually overcome their problems with the empty pipe misconception and the inability to differentiate between voltage and current. We will investigate this further in future work.

Figure 2

Question 3 was designed to see if students understood the relationship between AC and DC voltage. Students were asked if an oscilloscope display, which showed a voltage sine wave measurement centered above the zero level, could actually occur or whether it was an error in the oscilloscope settings. This was to check if students understood the concept of DC bias of an AC waveform. Only a few students could explain the concept of DC bias. Others thought that there was something “wrong” or “shifted” in the oscilloscope settings.

Figure 3

Question 4 asked students to sketch the voltage and current waveforms at different points in a series parallel resistive circuit (see Figure 4). Students were also asked to sketch the power waveform on one of the resistances, and how this waveform would change if (a) the source voltage frequency were changed but not amplitude, and (b) source voltage amplitude were changed but not frequency. Students who did not exhibit the empty pipe misconception in the earlier questions had no problem in answering this question. Others, however, talked about “waves canceling out” and “waves crashing into each other,” which again demonstrated that a number of students thought of the sine wave form as a spatial characteristic of current. Therefore, it was not clear what happened at junctions. Depending on their spatial locations, the waveforms may cancel (for example, if one waveform value was positive and the other negative). Other students recognized the fact that the sine wave defined the temporal characteristic of current, and since the current is in phase at all points in a resistive circuit, current values just add at junctions. Students did better in stating that the power delivered in a resistive circuit was a function of the RMS values for voltage and current, which does not depend on frequency.

Figure 4

In Question 5, we pushed further to see if students understood the concepts of RMS values of voltage and current, and how to compute the DC equivalent of effective power delivered by a source. Students were asked to compare two AC circuits, one with a sinusoidal AC source and a second with a square wave AC source, both with the same peak to peak voltage. Approximately 50 percent of students gave the correct answer, and others thought that there was no difference in the power being delivered to the circuit.

Figure 5

Summary

As we had concluded from the DC protocols, we conclude here that student knowledge is “in pieces,” and they attempt to piece together information from different metaphors in explaining phenomena. Beginning students did not understand the mapping of the sinusoidal waveforms to the physical concepts of voltage and current in the circuit.

A number of characteristics picked up during analysis of DC circuits, such as the constancy and fixed directional flow of current were carried over to the analysis of AC circuits. Like before, we can characterize student difficulties in the AC domain into four distinct categories:

1) Incomplete metaphors. As discussed earlier, this arises because students try to explain the flow of electricity using the water flow analogy, i.e., the empty pipe misconception. In the DC domain, this was manifested as “electrons take time to flow from the battery to the light bulb,” and “when you place two light bulbs in series the second will light up after the first one does.” In the AC domain, this problem manifested in different forms:

· “since electrons just stop, turn around and go the other way, they may never reach the light bulb, and the bulb may never light up,”

· “how can current flow from one source terminal to another if it reverses,”, and

· in the fuse problem (Question 2) “in DC you have to place it at the top, and for AC you need it at both places (i.e., top and bottom).”

2) Undifferentiated Key Concepts. In the DC protocols, this had manifested primarily as students not differentiating between the concepts of voltage and current. Students talked about the flow of voltage and voltage drop through a resistor. In the AC domain, students often had difficulty in differentiating between the continuous time varying sinusoidal voltage and current, versus voltage and current pulses. They made statements like:

· “voltage and current switch on and off”, and

· “voltage and current switch between positive and negative.”

In other cases students often attempted to import DC models to explain AC phenomena:

· “increasing voltage implies build up of charge at the terminal; when sufficient charge accumulates, current flows. Current turns on and off,” and

· “alternating current going through a resistor is constant in time.”

3) Relating Physical Concepts to Abstract Relations. In our DC protocols this had manifested as students’ inability to link circuit parameters to variables in mathematical equations (e.g., the link between V = I . R and the voltage drop across a resistor), and the lack of knowledge about components in a circuit (e.g., battery as a source of electrons, therefore, constant current). In the AC domain, students exhibited two primary misconceptions. The first was in considering the sinusoidal waveform as a spatial property of current flow:

· “sinusoidal waveform is a spatial property of current; it describes the current values at different points in the circuit.”

The second misconception was linked to interpreting the meaning of negative current and voltage.

· “voltage or current cannot really be negative; the absolute value is what is really happening. A minus sign appears in some calculations and you should not worry about it,” and

· “it’s o.k. to have something negative. It’ll fix itself, it’s not really a negative value.”

4) Minimum Causality Error. In our DC protocols this manifested itself in the 5 watt versus the 10 watt light bulb problem. Students concluded that a 10 watt bulb must have a greater resistance than a 5 watt bulb. This was attributed to students using one equation to derive a cause-effect relation in a circuit (P = I² . R, therefore an increase in R implies greater power consumed) and ignoring others (V = I . R, therefore, if R increases and V does not change I must decrease). In the AC domain, students had similar problems. For example, they believed that voltage varied sinusoidally, but still flowed in one direction in an AC circuit.

· “in an AC circuit, voltage can vary sinusoidally but current must remain constant to allow electrons to flow from one terminal of the battery to another.”

All of the misconceptions were widely prevalent among beginning EE students, but seemed to decrease significantly as students advanced in their program. We felt it important to study how students applied their knowledge to more real-world AC systems, and what difficulties they had in problem solving and explaining the function of these systems.

Answers to Protocol Questions Set 2

The first question in the second problem set (Figure 6) asked students to compare the function of a capacitor in parallel with a light bulb in a DC circuit and a capacitor in parallel with a light bulb in an AC circuit. The former circuit is common in car doors, where a capacitor in parallel keeps the light bulb on for a short period of time even after the car doors are shut, and the battery is disconnected from the light bulb. About half of the students interviewed answered the DC circuit behavior correctly, i.e., the capacitor charges to the DC voltage, and when the switch is turned off it discharges through the light bulb keeping the bulb glowing for some time. A number of the students also had the correct response for the AC circuit, i.e., the capacitor voltage, and, therefore, the charge on the capacitor follows the AC source, so the resultant behavior depends on what point of the cycle the switch is turned off. A small number of students did not understand how a capacitor functioned: “a capacitor in DC or AC is always an open circuit because the plates are separated.” Some others had the misconception that “a capacitor in an AC circuit is always a short circuit,” therefore, the bulb in the AC circuit would never light up. A few students had ingrained in them the model of a series RC circuit. They reasoned that the capacitor would take time to charge up, and while it was doing so would draw away from the light bulb making it dimmer. One student even said that “the bulb would take longer to light up because the capacitor draws most of the current as it charges up with time constant RC.” When prompted about Ohm’s law, the student said that it does not hold in this situation.

What happens when the switch s is opened in circuit A? What happens when the switch is opened in circuit B? What happens when the frequency is lowered/increased in circuit B?

Figure 6 DC vs. AC capacitor in parallel

Question 2 (Figure 7) focused on properties of time-varying current. Students were asked to contrast two situations: Will a coil wrapped around a nail make a magnet when connected to (a) a DC battery, and (b) an AC source. Only two students (out of six) had the right answer, i.e., the magnetic field generated by a coil is proportional to the rate of change of current. Therefore, the nail would not be magnetized in the DC circuit. The rest of the students saw no difference between the DC and AC circuit. This demonstrated that they were not aware of the properties associated with time-varying voltage and current in AC circuits.

Given a battery and a wire wrapped around a nail, will you be able to pick up a paper clip? What if the source was AC?

Figure 7 A battery an a wire wrapped around a nail

Question 3 (Figure 8a, 8b) required students to reason about the role of a capacitor and an inductor in stabilizing the output voltage in a full wave rectifier circuit. Students were presented with the output waveform of a full-wave rectifier using a four diode configuration. They were informed that the output resistance of this circuit was very high. In part (a) of the problem they were asked to explain how a capacitor placed in parallel with the load helped reduce the fluctuations in output voltage. In part (b) they were asked to explain the role of an inductor placed in series between the capacitor and the load in further stabilizing the V_out. A number of the students came to the conclusion that since the load was in parallel with the input voltage, V_out would equal V_in, and the capacitor would play no role in the circuit. Only a few students could explain that the capacitor charged up during the first part of the cycle till V_in reached its peak value. As the value of V_in fell during the next part of the cycle, the capacitor had to discharge through a very large load resistance, R. The time constant associated with this discharge was very large, and, therefore, the capacitor did not discharge much during the down part of the V_in cycle. In this manner, the capacitor helped stabilize the fluctuations in V_out. In part (b), a number of students attempted to write the differential equations for the circuit. When prompted to think qualitatively, only one student was able to reason using the constituent equation of an inductor, i.e., V_L=L . di/dt. The implication is that as the V_L changes, the inductor resists changing the current value, because of the integral relation between current and voltage. Since the inductor resists changes in current, the output current to the load changes by smaller amounts, and, therefore, the V_out tends to change by smaller amounts.

When the following V_in waveform is applied to the given circuit, what is V_out like? Draw it. What happens when the frequency is increased/decreased?

Figure 8a The rectifier problem

Now, when an inductor is added to the circuit, the DC voltage becomes more stable. Why?

Figure 8b The voltage stabilizer

Question 4 asked about RLC tuners for radio circuits. As preparation for this question, students were first asked to plot resistive, capacitive, and inductive impedance as a function of frequency (Figure 9a). More than half the students interviewed got this right without any prompting or help. Only one student did not seem to have any idea of the relation between impedance and frequency, so he had no clue about how to approach the problem. Students were then asked how a series RLC circuit (i.e., a RLC filter) could be used as a tuner for a radio (Figure 9b). Most students who drew the impedance curves correctly were able to reason that the overall circuit impedance was minimized at a fixed frequency value for a chosen R, L, and C value. A tuner is usually designed by incorporating a variable capacitor or inductor.

Draw the impedance between output points of the circuits as a function of frequency.

Figure 9a Impedance vs. Frequency

What does the circuit do? Explain how.

Figure 9b Radio tuner

Discussion

Our preliminary study of student understanding in the AC domain has proven to be quite revealing. Beginning students seem to have very little understanding of the time-varying nature of AC voltage and current. This can be attributed to a combination of problems they exhibit in their basic understanding of concepts. The empty pipe misconception affects their understanding of current flow, and makes it especially difficult for them to reason about current that reverses direction periodically. The inability to differentiate between voltage and current and the lack of understanding in mapping from physical concepts to abstract circuit parameters compounds students’ problems. They are often stuck with beliefs such as a source provides constant current, and a source cannot deliver power unless the current flows in one direction from one of its terminals to another. These misconceptions and lack of knowledge are not unique to the AC domain; in fact students exhibited the same problems when reasoning in the DC domain.

From the point of view of instruction, these observations can be interpreted in many ways. On the one hand, one can make the argument that since DC instruction traditionally precedes AC instruction, it is very important to ensure that students do not develop misconceptions and omissions described above during DC instruction. Careful contrasts also need to be made when making the transition from the DC to the AC domain. On the other hand, one could say that the similarity of the basic concepts in the two domains imply that the most effective form of teaching should focus on the concepts and their implications in problem solving rather than spend a lot of effort in focusing on the differences. For resistive circuits, the time-varying nature of AC voltage and current has no strong implications on behavior. Students need to understand the concept of power delivered, and how to compute the power delivered. As discussed earlier, the time-varying nature of current and voltage has important implications in circuits with capacitors and inductors, and it may be best to introduce these concepts by demonstrating their use in real applications and devices. The latter approach may be further justified by the observation that a number of the misconceptions of the beginning students seemed to go away as they moved on to more advanced courses.

Another issue of importance that we have observed among students is their reliance on mathematical formulations and solving of equations to derive answers to problems. As discussed earlier, this implies the students lack understanding of the underlying physical phenomena, and therefore, do not develop a deep understanding of the basic concepts in the domain. This problem is even further compounded in the AC domain, especially when students have to deal with the more complex phenomena associated with real world devices and systems. When dealing with the questions in problem set 2, a number of students attempted to convert the given circuit or problem description into mathematical equations. However, the resultant differential equations were hard to analyze, and did not directly provide the information required to solve the problem. The implication here is that students need to develop a better qualitative understanding of phenomena, and how these phenomena combine to produce circuit and system functionality. In our protocol studies on the second problem set, a number of students had to be coached to reason about a problem qualitatively. Only then were they able to analyze the problem, and generate the desired solutions and explanations. Developing qualitative reasoning skills and function-level understanding may also contribute to the development of better troubleshooting skills, a long-term goal of this research.

In the next section of the report, we develop a methodology for instruction that combines learning with assessment. The goal is to exploit computer technology to provide students with an environment for selecting from a set of available resources depending on their self-identified needs.

IV. FROM PROTOCOL ANALYSIS TO INSTRUCTION:

The Assessment of Domain Learnability Framework

Our studies of student understanding in AC and DC circuit problem solving suggest that student misconceptions and difficulties can be linked to instruction as opposed to the preconceived notions of domain concepts. These observations have led us to turn to dynamic assessment approaches (Feurestein, 1979; Campione and Brown, 1985 and 1987; Bransford, et al, 1987) and focus more on how to prepare students to learn through instruction. Our first steps in this direction have been to build computer-based tools that provide resources to help students learn concepts they have found difficult to learn.

Assessing Domain Learnability

It appears that some electricity concepts may be more difficult to learn than others. With respect to the instruction in this domain, we believe that an important research task is to identify features and concepts that influence learnability of concepts that affect problem solving tasks. We will call this task "assessing domain learnability” or ADL for short. By trying to remediate people's misconceptions and missing conceptions, we may determine which are particularly difficult to remediate given our methods of instruction (e.g., Heller & Finley, 1992), and which type of understanding has the greatest impact on subsequent learning. The basic observation is that not all misconceptions are equally strong or equally relevant to future instruction. For example, although we have rarely seen it in the literature (Cooke & Breedin, 1994), it would be interesting to ask people to compare their confidence in answers where they exhibit misconceptions relative to those that they do not. We suspect that for many of the misconceptions that have been documented, people are reasonably aware that they do not know what they are talking about. For those misconceptions that are of low confidence, should we expect that people would be more likely to overcome their misconceptions and learn? Much of the research on misconceptions has no handle on this question. An ADL approach seems more likely to provide an answer.

There are, of course, limitations to ADL as we have conceptualized it so far. One possible weakness of ADL is that it is particularly prone to the ways that we assess whether someone has learned a correct conception or not. For example, if we ask the exact same question that we taught, does this mean that people have learned in any meaningful sense? The problem of assessing and deciding upon ecologically satisfactory understanding, however, is a problem faced by much educational research. ADL actually fairs better than most in this regard. This is because the ultimate test for ADL is whether a given concept has implications for future learning. For example, consider the typical course sequence in electrical engineering where students begin with direct current (DC) circuits and then move to study alternating current (AC) circuits. Students start with many misconceptions about DC circuits. Are all the misconceptions and their correct counterparts equally important in shaping students' ability to learn AC circuits? This is the question that ADL is designed to answer.

A second potential weakness to ADL is that if our instruction fails to teach a correct conception of a domain, we cannot know whether it was a function of the domain's difficulty or a function of our teaching methods. On the one hand, we can never disentangle these two possibilities beyond a reasonable appraisal. On the other hand, it is the interactions of the instruction and the domain that constitute the important parameters of assessing domain learnability. The emphasis of ADL is not on domain learnability in the abstract, but rather domain learnability with respect to the state of the art in instruction. The next section describes a computer environment that captures many of our ideas about the state of the art.

STAR-Legacy: A Framework for a Computer-Based Learning Environment

ADL depends on the instructional techniques used to teach about the domain of interest. A computer-based environment provides an integrated learning-assessment tool for pulling together different instructional techniques and resources that can be applied to a domain. A single instructional technique would be too restrictive for ADL. For example, one might use a dynamic tutoring system to teach the procedural knowledge of a domain, but there are other types of knowledge that are important to assess as well, like, do people have difficulty constructing a mental model of the domain (Lajoie & Lesgold, 1992). Similarly, one might create a system that matches an individual's misconceptions against a known "bug list" and teaches to those bugs directly, but this typically assumes that misconceptions are non-interacting. In the following section, we describe ADL in electricity implemented using the STAR-Legacy framework.

In our previous work (Biswas, et al, 1997) we give the description of the main STAR-Legacy interface. The interface represents a learning cycle where each of the icons reflects an often implicit, yet important, component of most learning events. The interface presents a "learning map" that helps people understand where they should be in their knowledge development, and it helps them see that there are typical activities, like first tries and revisions, involved in learning. The cycle is not meant to imply that Legacy is a rigid sequential environment that locksteps the learner and designer. We expect people to navigate through the system depending on their learning needs. For people to be able to determine their learning needs, we have included multiple opportunities for assessment. This is one of the reasons that STAR-Legacy is appropriate for a dynamic assessment approach. It integrates assessment and instruction into a single design model. In the following paragraphs we describe the components in the context of assessing the learnability of electricity.

ADL in Electricity

Our protocol studies have identified four primary classes of difficulty -- lack of differentiation, simplifying assumptions of minimum causality, incomplete metaphors, and experiential impoverishment caused by the invisible nature of electricity. All these can be attributed to basic cognitive tendencies. The question is how serious the difficulties are with respect to learning electricity. Some of the difficulties may be easily remediated. For example, perhaps experiential impoverishment makes students heavily dependent on instruction to provide surrogate intuitions. Consequently, students’ misconceptions arise from instruction that provides incomplete analogies or that provides mathematics at the expense of the causal explanations that help people construct mental models. In this case, one might expect that appropriate experiences, perhaps provided by simulations, would give students the experiential knowledge needed to help constrain their model building. On the other hand, some of the difficulties, like the simplifying assumption of minimum causality, may be difficult to remediate because the solution requires simultaneous reasoning with multiple equations. The goal of the DC-Legacy is to determine which of these difficulties make the domain particularly difficult to learn and which concepts are particularly important for further understanding in the domain.

DC-Legacy

In this section we briefly describe our software environment, STAR-Legacy, which we created for assessing the learnability of DC concepts. In line with the test-teach-retest model of dynamic assessment, students begin with a question in the Look Ahead problem and end with the same question when they Reflect Back. In this case, the Look Ahead and Reflect Back problem asks students to explain what happens in a simple flashlight circuit when a 5-watt bulb is replaced by a 10-watt bulb. This problem does not equally capture the four learning difficulties; it is simply a sample of the type of problem that one would like students to understand. If useful, the Look Ahead and Reflect Back could include a more comprehensive set of questions.

The three Challenges for the DC-Legacy were chosen on the basis of our protocol research (Biswas, et al, 1997). We found three problem situations that were particularly good at making students' thinking visible. Challenge 1 asked students to reason about the possible causes of a dim bulb (see Fig. 10). This problem was intended to help students differentiate voltage and current, to help them overcome the minimum causality error, and to give them some increased experience in the domain and its analogies. Challenge 2 asked students to design a battery operated drill that could run at different speeds. In this design problem, students progressively deepen their understanding of the topics raised by Challenge 1 while adding the issues of local reasoning and framing (we say more below). Finally, Challenge 3 tried to bring the lessons together into a single problem. In this challenge, students were asked to reason about a flashlight that has two bulbs, one that points forward and one that points to the ground. They are told that somebody wants to change the forward bulb to a higher wattage. How will that effect the flashlight overall? These challenges are intended to bring forward the different classes of misconceptions. At the same time, we expect the interaction of the challenges and instruction to reveal other conceptual hot spots. This is one of the attractive features of ADL -- it can reveal misconceptions in the context of instruction.

In our protocol work, we found these challenges to be revealing, but to avoid "contaminating" our results, we did not try to teach the students. When they exhibited misunderstandings, we simply probed further. As a result, we developed some idea of student difficulties, but we did not develop any understanding for how strong these difficulties were nor how to remediate them. The DC-Legacy captures our movement towards an ADL approach. It includes multiple resources for trying to help the students learn. DC-Legacy was not designed for students to complete on their own (although they could). Rather, DC-Legacy supplements a structured interview format where interviewers do their best to figure out and remediate a student's difficulties. We briefly describe the resources in the DC-Legacy that helped the students and interviewers.

After reading Challenge 1 (Figure 10a), students try to generate their first thoughts about how to prepare for testing the dim light bulb in challenge one (see Fig. 10b). These initial thoughts usually provide the interviewer with a sense of the strengths and weaknesses of the students. This helps the interviewer and student choose which of the multiple perspectives to listen to. Each perspective directly targets key learning difficulties with a 10-15 second comment by an expert. For example, one of the perspectives has an expert explain the minimum causality error, although not in those terms. The expert states, "a common mistake that people make with these problems is that they often do not realize that when the power changes, two other things in the circuit must change." Another perspective tries to tie the perceptual phenomena (a dim bulb) to the electrical concepts by pointing out that a dim bulb means less power is being consumed. A third perspective prepares students to differentiate voltage and current by discussing the importance of using an ampmeter or voltmeter rather than simply swapping components in the circuit. And, a fourth perspective,