Unfortunately, IT ISN'T. At least it isn't, if the focus is on what students
have learned and understand about the fundamental concepts of their discipline.
Teachers in the very common and traditional chemistry (or other science) course
described above usually focus on student responses to examinations that are most
likely to be based mainly on algorithmic problem solving, multiple choice, and/
or short answer (fill-in-the-blank) responses. There is, however, abundant
evidence from studies of students in chemistry (as well as physics and biology)
[Nurrenbern and Pickering, 1987; Pickering, 1990; Sawrey, 1990; Nakhleh and
Mitchell, 1993; Zoller, Lubezky, Nakhleh, Tessier, and Dori, 1995; Smith and
Metz, 1996] that although they are often able to use algorithms with great
facility to solve traditional (end-of-chapter) problems, they demonstrate basic
misunderstandings of the conceptual and atomic/molecular basis of the problems.
What ARE Students Learning?
A very common and relatively easy way to test for conceptual understanding is to
have students draw pictures and diagrams that describe their molecular- level
understanding or to choose (in a multiple choice format) a diagram that
describes the system in question at the molecular level. Gas-law problems have
been a favorite topic for such studies since both the kinetic-molecular
conceptual model and the algebraic relationship PV=nRT are so simple (at least
for the teacher). For example, in their study of students in the introductory
course for chemistry majors, Nakhleh and Mitchell [1993] gave the following two
problems (based, in part, on Nurrenbern and Pickering [1987]:
(1) (algorithmic problem) 0.100 mole of hydrogen gas occupies 600 mL at 25 deg C and 4.08 atm. If the volume is held constant, what will be the pressure of the sample of gas at -5 deg C? The boiling point of hydrogen is -252.8 deg C. (a) 4.54 atm; (b) 3.67 atm; (c) 6.00 atm; (d) 2.98 atm; (e) 4.08 atm.
(2) (conceptual problem) The following diagram [a circle with 20 dots randomly distributed throughout]) represents a cross-sectional area of a rigid sealed steel tank filled with hydrogen gas at 20 deg C and 3 atm pressure. The dots represent the distribution of all the hydrogen molecules in the tank. Which of the following diagrams illustrate the most probable distribution of molecules of hydrogen gas in the sealed steel tank if the temperature is lowered to -5 deg C. The boiling point of hydrogen is -252.8 deg C. (a) circle with 20 dots randomly distributed throughout in a different pattern than the original; (b) circle with the dots clustered near the center; (c) circle with the dots clustered at the bottom; (d) circle with the dots distributed about the inside "wall" of the circle (as though stuck to the wall); (e) smaller circle with the dots clustered into its bottom three-quarters.
Although 85% of the students chose the correct response for the algorithmic problem, only about half of these chose the correct response for the conceptual problem. In all, less than 50% of the students were successful on the conceptual problem, even though, as the authors point out (perhaps indulging in a bit of wishful thinking; see the [Phelps, 1996] paper discussed below), "this sample represents declared chemistry majors, the students most likely to desire to understand these concepts." These also were good students who got a high proportion of honor grades, presumably assigned on the basis of success on the algorithmic problems that made up the bulk of the examinations.
It is sobering to consider that more than half of these chemistry majors (who
are representative of a great many more across the country) leave the freshman
course thinking that when a gas is cooled (far above its boiling point) in a
rigid container, the molecules somehow cluster together. (After all, gases
contract when cooled. Right?) We are all familiar with the old student
refrain, "I understand the concept, but I can't do the problems." These data
suggest that the opposite is true: students can (be taught to) do the problems,
but they don't necessarily understand the concepts.
What AREN'T Students Learning?
The students who can't do the above conceptual problem seem not to understand or
be able to use the particulate model of matter that is so fundamental to the way
chemists think about the world [Gabel, Samuel, and Hunn, 1987]. To put it
another way, these students don't "see" molecules. Johnstone [(1990), cited in
Gabel, 1993] has made a point of delineating three levels on which chemistry is
taught: the molecular/particulate level, the macroscopic/phenomenological/
observational level, and the symbolic/algorithmic level. Our courses tend very
heavily to emphasize the symbolic level, to the neglect of the others, and the
result may be what is showing up in the many studies like the one represented
above.
Gabel [1993] designed a study "to determine whether students' understanding of chemistry would increase if the particulate nature of matter was emphasized." The study was carried out with high school chemistry classes taught by the same teacher. Since high school chemistry courses are virtual clones of introductory college chemistry courses, her results are probably applicable at the collegiate level. For the treatment classes the teacher used a set of overheads that pictured the particulate model and worksheets (prepared by Gabel) that required students to link the particulate nature of matter to physical phenomena (the macroscopic level) and/or to chemical symbols.
Gabel's hypothesis was that emphasizing the particulate level and connecting it to the others would improve achievement on all three levels. Tests were written that had triads of questions designed to test understanding at the three levels on each of the topics covered. The results were that the treatment group (43 students) had mean scores of about 8 out of 20 on each of the three levels of test items while the control group (23 students) had mean scores of about 6. The results are all in the same direction and lend support to the hypothesis, but not enough information is provided to judge the statistical significance of the higher scores. Readers should refer to the original article to judge the credibility of the findings and see what would be required to replicate the study on a larger scale and at the collegiate level.
Two further observations from this study are relevant to the arguments of the
present paper. First, instruction had occurred on only 80% of the topics for
which the materials were prepared. Time ran out. Second, as you see, the means
of the absolute scores on the tests, even for the treatment groups, did not
reach 50% for any of the levels. One explanation Gabel proposes for these
phenomena is that so much content is included in the traditional chemistry
course that the average student simply can't assimilate more than about half of
it (and all of it can't even be "covered" in the time allotted).
Reducing "Coverage" (= Increasing Understanding?)
Are we going to have to cut down on the bulk of ideas that we try to teach, so
that students have time to learn important ideas meaningfully -- and retainably?
This is where we make one connection between instruction in chemistry (at all
levels) and Project 2061, the long-term initiative of the American Association
for the Advancement of Science (AAAS) to reform K-12 education in natural and
social science, mathematics, and technology. If Project 2061 is unfamiliar to
you, a very brief synopsis of the project is provided in a recent Journal of
Chemical Education editorial [Lagowski (1996)] or you can contact us through the
World Wide Web: http://www.aaas.org/project2061/2061main.htm. Although Project 2061 and its
publications [AAAS, 1989; AAAS, 1993] focus on science literacy, the learning
principles upon which it is based are applicable as well to courses for students
who are going to be science practitioners.
The deepest underlying principle of Project 2061 is that what we need to do is "teach less in order to teach it better," by providing students greater opportunities to learn, to understand, and to be able to use their understanding. "By concentrating on fewer topics, teachers can introduce ideas gradually, in a variety of contexts, reinforcing, and extending them as students mature. Students will end up with richer insights and deeper understandings than they could hope to gain from a superficial exposure to more topics than they can assimilate" [AAAS, 1989, pp. xviii-xix]. The first challenge then is in the choice of topics to be taught and learned.
The less-is-more principle does not, however, require merely selection of "topics." It requires reflection on just what it is about any selected topic that is important to know. "Photosynthesis," for example, is an inadequate topic specification if what we are after is the stoichiometric idea that most of earth's dry biomass is derived from a single atmospheric greenhouse gas, carbon dioxide. Such highly-focused goals are necessary to shape instruction and to prevent every conceivable idea related to light and dark reactions or balancing redox equations from being stuffed in under the justification of "photosynthesis" or "stoichiometry."
This photosynthetic/stoichiometric example is chosen because there is striking evidence from one of the studies done by "The Private Universe" project [Schneps, 1995] that graduates of highly selective technical institutes and prestigious universities have about the same level of understanding (misunderstanding and lack of understanding) on this topic as fifth graders. In this study, subjects are videotaped as they are presented an acorn and a two- foot length of a rather substantial oak tree branch and asked where they think all the mass of the branch came from when the tree sprouted initially from a tiny acorn like the one they are holding. Only a small number of the subjects can relate the gain in mass to the plant's assimilation of carbon dioxide from the air to produce the bulk of the material that makes up the mature tree. One subject waxes (pseudo)eloquent about electron pathways and redox, but ends up confessing that he has no idea where the mass comes from.
Perhaps even more distressing are the results when the subjects are asked to respond to the proposition that it is a gas from the air that is used by the plant to create its mass. Many say they simply can't believe this can be the case, since gases are so light (ephemeral) and the tree so heavy. One subject says that the gas would have to be really highly compressed in order to provide the mass and she doesn't believe that could happen. This seems to be yet another example of a lack of understanding of the particulate nature of matter and its phases. and the implications and applications of this concept.
Extrapolating these results to the classroom, we would suggest that it is of little lasting value to be able to balance the reaction of carbon dioxide with water to produce sugar(s) and molecular oxygen, if the balancer does not recognize the implications of the fact that most of the mass of the solid sugar (except for the hydrogens) is derived from the carbon dioxide. Similarly, counting up oxidation numbers to show that, in the photosynthetic process, carbon has gained a larger "share" of the bonding electrons is pretty pointless, if the counter doesn't understand the need for energy (from sunlight) to drive a reaction that "compresses" a gas into solid form.
Driven by the kinds of data briefly sketched above that seem to suggest that coverage often stands in the way of understanding, the chemical education community has, for some time, been exercised about the selection of topics for an introductory chemistry course [Lloyd and Spencer, 1994]. While the qualitative stoichiometry of photosynthesis is certainly not on everyone's list, an understanding of the chemistry of electrolyte solutions (common acids, bases, and salts), including stoichiometry, surely is and is one central feature of almost all general chemistry courses. How are we doing?
Smith and Metz [1996] have obtained some provocative results in a study that involved undergraduates in chemistry courses, chemistry graduate students, and 11 faculty. All the subjects were tested individually and asked to voice their reasoning aloud as well as respond in writing to the test items. The first part of the test involved choosing from among a series of diagrams the one that best represented HCl, a strong acid, and from another series (identical except for the label on the circles representing the anion), the one that best represented HF, a weak acid. The good news is that 90% of the graduate students and faculty got the strong acid correct (and one could argue that the incorrect faculty answer was due to trying to be too subtle). The bad news is that only 80% and 60%, respectively, of the faculty and graduate students got the weak acid item correct.
A lesson for those who design tests of conceptual understanding comes from the second part of the Smith and Metz test where the subjects had to draw their own diagrams, rather than choosing from among ones produced by the tester (instructor). The task was to represent the reaction [with subscripts properly shown on the test]
NiCl2(aq) + 2NaOH(aq) --> Ni(OH)2(s) + 2NaCl(aq)
by diagrams like those they had seen in the previous questions. The test item showed two empty containers labeled "NiCl2(aq)" and "NaOH(aq)" that were combined to yield a third (larger) container labeled "Ni(OH)2(s) + 2NaCl(aq)." Three criteria were used to evaluate the responses to this item: (a) properly dissociated reactants and products, (b) correct bonding and structure of the reactants and products, and (c) conservation of matter.
Less than half the graduate students and faculty produced diagrams that
satisfied all three criteria. (Fewer than 10% of the undergraduates did this
well; 30% missed all three criteria.) 80-90% satisfied two or more of the
criteria, but one faculty member failed to satisfy any of the criteria. Common
errors included not dissociating the ionic species, taking apart the O and H in
hydroxide and binding them separately to the metal, and misinterpreting the
stoichiometric coefficients (2NaCl interpreted as Na2Cl [with subscript]).
Increasing Understanding (Constructivism)
Clearly, as Smith and Metz, point out, students "memorize chemical definitions
and use chemical terms without true comprehension. Misconceptions [regarding
the topics tested] can interfere with subsequent learning and can persist beyond
the undergraduate level." The authors suggest a teaching approach that might be
helpful. "These chemical concepts readily lend themselves to microscopic
representations. Teaching strategies using these visual aids could explain the
concept before applying the mathematics. This might increase comprehension and
retention by allowing students to picture the chemistry."
Again there is an intersection with Project 2061 and its advocacy of an approach to effective teaching and learning of science that is consistent with the nature of scientific inquiry. This approach is largely based on the general notion of "constructivism" arising from cognitive psychologists' research on how people learn. Science for All Americans [AAAS, 1989, Chapter 13] puts it this way: "People have to construct their own meaning regardless of how clearly teachers or books tell them things. Mostly a person does this by connecting new information and concepts to what he or she already believes. Concepts that do not have multiple links with how a student thinks about the world are not likely to be remembered or useful... Concepts are learned best when they are encountered in a variety of contexts and expressed in a variety of ways, for that ensures that there are more opportunities for them to become imbedded in a student's knowledge system" [AAAS, 1989, p. 198].
As an analogy, we might imagine that our minds are like forests with a lot of paths running through them that we have trampled down through a great deal of use and by means of which we can get from one place (piece of information, problem solving technique, etc.) to another. As we learn, we are continually increasing the number and extent of the paths and their crossing points (interconnections). If a new idea comes along that is easily accessible from one or more of these paths, then we can easily incorporate it into our working knowledge and make connections to it.
However, the new idea might be one that fits best in a remote region of the forest where we have made few paths. If we are forced to use previous knowledge to construct our version of this new idea for ourselves, we have to start from well-known paths (perhaps more than one) to get where we need to go. We probably will have to retrace the new paths several times as we struggle with our construction, so the paths will begin to be pretty well worn and the new knowledge, once connected to the rest will be more easily accessible both now and later.
By contrast, we could be given a map (simply told the idea) of how to get from
some familiar path to where the new idea should be located. If we follow the
map diligently, we will get where we need to be and will have trod the path
once. However, this barely detectable path will provide only a tenuous
connection to everything else and little ability to use the knowledge in a new
context. A further analogy might be made to homeopathic medications that are so
dilute they are unlikely to contain a single beneficial molecule -- but the
solvent "remembers" them. Our faith in students remembering ideas they don't
understand may be comparably optimistic.
Teaching Practices Consistent With the Nature of
Scientific Inquiry
Teaching practices that are consistent with the nature of scientific inquiry and
constructivism include these [AAAS, 1989. pp. 200-203]. Begin with questions
about nature and phenomena with which the students are familiar or have
observed. Engage students actively with many opportunities for interacting
directly with the world around them, including many opportunities for
measurement and quantification, especially ones where figuring out what and how
to measure present challenges. Emphasize the collection, evaluation, and use of
evidence as a means of interpreting what they observe and interact with.
(Students have a very difficult time making the distinction between observation
and interpretation and they need a great deal of practice clearly expressing
their understandings of the connections and pathways from the former to the
latter.) Encourage and provide opportunities for collaborative efforts to help
to model the way most scientists work, sharing responsibility and learning from
and with each other. Connect very closely what we know with how we know, so
that neither factoids nor a set of procedures (the scientific method) is
presented in isolation and appropriate habits of mind are engendered. De-
emphasize memorization of technical vocabulary; emphasize understanding that
uses technical terminology only as essential to clarify thinking and promote
effective communication (vocabulary as an aid to thinking and communication, not
as a substitute for them). (As an aside, we point out that the National Academy
of Sciences (NAS) in its National Science Education Standards [NAS, 1996] places
even greater emphasis on scientific inquiry, raising it to the level of an
objective to be evaluated.)
Changing Practice (ConcepTests and Grading)
Is the traditional lecture approach used in most chemistry courses consistent
with these practices that are designed to promote student learning and
conceptual understanding? The evidence suggests that it is not. For example,
[Mazur, 1995, and at URL:http://mazur-www.harvard.edu] was dismayed to learn
that his students in beginning physics, who were doing very well on standard
examinations, did not perform at all well when he tried a conceptual
examination. His response was to develop "ConcepTests," an in-lecture method to
harness the power of peer instruction. Briefly, a conceptual question and
several plausible answers are presented by the lecturer and the class polled for
their responses. Then the students get a few minutes to discuss the problem and
answers among themselves and a second poll is taken. The result of the second
poll is usually a substantial fraction of the students selecting the correct
response; those who have grasped the concept have been able to explain it
effectively to their peers. Mazur reports marked improvement in student
performance on conceptual examinations that previously had given discouraging
results. Ellis [1995, and at http://www.chem.wisc.edu/~concept] and his colleagues are
implementing the ConcepTest approach in chemistry and welcome participation in
their project.
Peer instruction is the key element in the ConcepTest lecture methodology. To
make it work, there has to be a cooperative spirit in the class, but cooperation
is not the working mode for most students in courses graded on a curve.
Therefore, implementing this change in the lecture format also demands a change
in grading to an absolute scale; this puts students in "competition" with the
content of the course, not other students. Any help understanding the content
that one student gives another cannot harm the student giving the help (and
might actually help solidify her/his own understanding). Since absolute grading
systems also have value in their own right and in other situations, they have
been advocated by others as well [Bell, 1991; Herschbach, 1993].
Changing Practice (Demonstrations and Molecular
Representations)
Besides the examples already presented, what other techniques can we use to
increase the opportunities for students to learn, understand, and be able to
apply the concepts we think are important in chemistry? There are examples of
changes we can make in our presentations. Miller [1993] suggests a lecture
approach he calls "demonstration-exploration- discussion" which is highly
interactive and uses demonstrations to start discussion and exploration, rather
than to illustrate what has been presented. [See also Phelps, 1996.]
The studies by Gabel [1993] and Smith & Metz [1996] certainly suggest that greater emphasis on pictorial representations of the particulate nature of matter and its implications will be helpful. The visualization of molecules and molecular interactions that is almost second nature to a chemist is not an innate trait; it has to be learned (and taught). That the majority of our students apparently haven't learned it seems abundantly clear from all the studies that have been done on conceptual learning in chemistry.
The examples we have chosen for this paper focus on the first chemistry course, but it's virtually certain that the problem of "seeing" (or not seeing) molecules goes on at least into organic and biochemistry. Large numbers of students confounded by even the simplest organic reaction mechanism almost certainly see only a collection of alphabetic symbols and connecting lines on the page (or computer screen), not an entity that has actual physical reality, shape, and polarities. The use of physical and computer models can help these students get the answer, but until they automatically see (in their mind's eye) that collection of symbols as a physical reality, they probably do not truly understand the mechanism.
Reaction mechanisms retain their importance in biochemistry, but now a new
complication, multiple reaction pathways, arises. Of course these are present
in simpler systems as well, but usually not emphasized. In biochemical systems,
a product produced at an intermediate stage in one pathway does not necessarily
continue down that pathway, even though we write the pathway in a nice linear
progression with lots of curved arrows showing how other species are involved in
the reactions. Without a good grasp of the particulate model, students have
difficulty visualizing the pool of molecules of an intermediate, how they might
partition themselves among all their possible fates, and how the size of the
pool as well as the presence of other species "directs" these processes. The
better a student's conceptual understanding of the particulate model in the very
earliest courses where the complications are at a minimum, the more likely it is
that s/he will be successful later on.
Changing Practice (Concept-centered Lectures)
Changing the style of instruction to center it explicitly around conceptual
problems (or critical thinking [Kogut, 1996]) in lectures and examinations is an
approach reported by Phelps [1996]. Her article is worth studying for several
reasons. She provides some concrete examples of conceptual problems used to
introduce a new topic, reports on student interviews she conducted (from which
she includes very interesting snippets), and includes examples from notes of her
in-class and out-of-class observations and interactions with students, as
another source of evidence for her conclusions. None of these approaches is
unique, but the combination applied to thinking about the consequences of making
changes in a course is a succinctly presented model for others to follow.
The most interesting aspect of Phelps' study is that she introduced the same kind of changes into moderate size (150-200) introductory courses for science majors and for non-science majors. She compared the reactions of the two groups to an approach that emphasized conceptual problem solving tasks to introduce each new concept. As she says, "The very different responses these tasks elicited from the two groups of students were striking." The science majors basically sat on their hands and did not respond to a question like "Why does salt melt ice on the roads and help homemade ice cream freeze faster?" The non- science majors responded with an extended discussion involving give and take among the students as well as prompts and questions from the teacher.
Throughout the semester, the science majors were reluctant to take a stab at these conceptual problems. As the author concludes, "Science majors are comfortable... hiding behind numbers, and they know the dangers of asking 'Why?'" The science students seemed to have learned from their previous courses that too much time wasted questioning the underlying concepts slows you down and therefore hurts your grade. Although the science majors first resisted the new approach and were never as open as the non-science students, many finally sought help filling in the gaps in their conceptual knowledge, since this knowledge had become required for success (good grades) in the course.
A perceived (by others) liability of her approach that Phelps points out is that
not as much content can be covered. The instructor aiming for conceptual
understanding has to insist on class participation, in order to get a sense what
the students are thinking and getting. The instructor can't wait until the
examination to find out, but must be continually assessing, since s/he shouldn't
go on if a substantial fraction of the class has not yet "gotten it." Since
concepts learned well are generally retained well, students from the author's
course have not been at a disadvantage in succeeding chemistry courses, despite
the apparent loss of content coverage in the conceptually-oriented course.
Changing Practice (Technology)
Finally, it seems only appropriate, in a conference conducted on-line, to
speculate about the role(s) technology might play in creating and enhancing
opportunities for student conceptual learning. AAAS advocates the appropriate
use of technology of all kinds, from the simplest overhead projector to the
newest multimedia computer and Web-searching software, in ways that provide
students routes to increase their depth of understanding and improve their
ability to use that understanding. For example, the molecular modeling tools
that Jones will discuss (paper #3) could be an excellent way for some students
to deepen their grasp of the particulate nature of matter. But "deepening"
implies starting with at least a partial conceptual understanding of this model,
probably attained only after students have confronted simpler situations and
systems and have shown their understanding on problems such as those described
previously. Otherwise, the modeling software may be just a sophisticated way of
manipulating symbols that have no concrete meaning to the manipulator.
Smith and Stovall (paper #7) will describe an instructional network that gives
students and instructors access to instructional materials of all kinds (at
least anything that can appear on a computer screen). It's reasonably obvious
that many materials presented in this way can reinforce algorithmic problem
solving, but what kinds of materials that can be made available in this mode
will help students with conceptual learning? Perhaps Smith and Stovall will
address this question in their discussion.
Project 2061 and Chemistry (reprise)
If we want most of our students to learn chemistry at all, we have to help them
develop a better understanding of each idea along the way, which inevitably
means attempting to cover less. What should that less be? One stratagem that
often proves useful is to imagine encountering a student, say, five years later.
What ideas in chemistry would you feel worst about that student not
understanding? This question should be asked with no constraint from (if
possible, no thought about ) the syllabus or examinations per se. Entertaining
this question, teachers sometimes recognize that the ideas they subjectively
consider most important are not explicitly taught or tested in their course;
they are rather considered obvious or left for the students to infer.
Science for All Americans [AAAS, 1989] was an attempt by the AAAS to identify
thoughtfully and articulate carefully a body of inter-relating, essential ideas
about science and the picture it paints of the world. Although ostensibly about
what high-school graduates should eventually know, for the present its
expectations are beyond most college graduates. It would be a helpful place to
look in thinking about what ideas you would feel worst about your former
students not knowing. When you look there, keep in mind what an American
Chemical Society (ACS) Task Force [ACS, 1993] concluded about how chemistry is
distributed in the Benchmarks for Science Literacy [AAAS, 1993]: "The coverage
of chemistry in the Project 2061 Benchmarks is broader than might at first
appear. Chemical concepts are found mainly in: The Physical Setting--a
reflection of the traditional view of chemistry as a physical science; The
Living Environment--a recognition that chemistry is the basis for understanding
molecular biology; The Designed World--an exploration of the contributions of
chemistry to applied science and technology, as well as of the relevant
principles; and, The Historical Perspective--a review of the pivotal
intellectual input that has led to the development of modern chemistry. This
perspective does justice to the breadth of modern chemistry."
Bibliography
American Association for the Advancement of Science (1989). Science for All
Americans (Oxford University Press, New York).
American Association for the Advancement of Science (1993). Benchmarks for Science Literacy (Oxford University Press, New York).
American Chemical Society (1993). "Chemistry at the K-12 Level: An analysis of the chemistry content of the Project 2061 Benchmarks and the Scope and Sequence Content Core." Report of a Select Task Force of the ACS Society Committee on Education.
Bell, J. A. (1991). "It's Not What You Know, But How You Know," American Journal of Pharmaceutical Education, 55, 383-386.
Ellis, A. B. (1995). "Treating Students and Industry as Customers," Chemtech, 25(3), 15-21.
Gabel, D. L., Samuel, K. V., and Hunn, D. (1987). "Understanding the Particulate Nature of Matter," Journal of Chemical Education, 64(8), 695-697.
Gabel, D. L. (1993). "Use of the Particle Nature of Matter in Developing Conceptual Understanding," Journal of Chemical Education, 70(3), 193-194.
Johnstone, A. H. (1990). "Fashions, Fads and Facts in Chemical Education," abstract #9, Divsion of Chemical Education program, American Chemical Society Meeting, Washington, DC, 26-31 August 1990.
Kogut, L. S. (1996). "Critical Thinking in General Chemistry," Journal of Chemical Education, 73(3), 218-221.
Lagowski, J. J. (1996). "A Decade of Project 2061" Journal of Chemical Education, 73(4), 287.
Lloyd, B. W. and Spencer, J. N. (1994). "New Directions for General Chemistry," Journal of Chemical Education, 71(3), 206-209, and references therein.
Mazur, E. (1995). ConcepTests in Physics (Prentice-Hall, NJ).
Miller, T. L. (1993). "Demonstration-Exploration-Discussion: Teaching Chemistry with Discovery and Creativity," Journal of Chemical Education, 70(3), 187-189.
Nakhleh, M. B. and Mitchell, R. C. (1993). "Concept Learning versus Problem Solving: There Is a Difference," Journal of Chemical Education, 70(3), 190-192.
National Academy of Sciences (1996). National Science Education Standards (National Academy Press, Washington, DC)
Nurrenbern, S. C. and Pickering, M. (1987). "Concept Learning versus Problem Solving: Is There a Difference?" Journal of Chemical Education, 64(6), 508- 510.
Phelps, A. J. (1996). "Teaching to Enhance Problem Solving: It's More than the Numbers," Journal of Chemical Education, 73(4), 301-304.
Pickering, M. (1990), "Further Studies on Concept Learning versus Problem Solving," Journal of Chemical Education, 67(3), 254-255.
Sawrey, B. A. (1990). "Concept Learning versus Problem Solving: Revisited," Journal of Chemical Education, 67(3), 253-254.
Schneps, M. H. (1995), producer/director. "BIOLOGY: Lessons Pulled From Thin Air," Session 2 from The Private Universe Workshop Series, Annenberg Foundation/ Corporation for Public Broadcasting Math and Science Project 1995, President and Fellows of Harvard College.
Smith, K. J. and Metz, P. A. (1996). "Evaluating Student Understanding of Solution Chemistry through Microscopic Representations," Journal of Chemical Education, 73(3), 233-235.
Zoller, U., Lubezky, A., Nakhleh, M. B., Tessier, B., and Dori, Y. J. (1995). "Success on Algorithmic and LOCS vs. Conceptual Chemistry Exam Questions," Journal of Chemical Education, 72(11), 987-989.