APS FEd Spring 2006 Newsletter -- Christensen and Engelhardt: Tapping Physics Education Research for a graduate-level curriculum

Every summer a painful ritual is undertaken by many would-be physicists in classrooms across the country. A comprehensive written examination, although it has been modified or even removed at certain institutions, is still a key measure used by many schools to determine who is qualified to continue on a quest for a Ph.D. in physics. Over the last two years, the authors of this article have created a Ph.D. qualifying exam preparation course that utilizes several research-proven methods.[1] These methods include, but are not limited to, peer-led instruction, training in problem-solving skills, and the use of multiple representations. The teaching of upper-level undergraduate and introductory graduate physics content in this manner was not only engaging and interesting, but also gave students an opportunity to improve their understanding and, as is borne out in our data, an improved chance to pass the exam.

At Iowa State University (ISU), the physics Ph.D. qualifying exam is administered in two four-hour-long exams in a large lecture hall, on a Tuesday and Thursday morning during the same week, approximately 2 weeks before the start of fall classes. The first exam, known as the “Classical Exam,” covers questions on Newtonian mechanics, Lagrangian mechanics, electricity and magnetism, relativity, optics, as well as qualitative questions about experiments or scientific ideas. The “Modern Exam” (the Thursday exam) includes problems on quantum mechanics, condensed matter physics, high energy physics, nuclear physics, and astrophysics, as well as other modern topics.

Problems range in difficulty from introductory level concepts to advanced graduate material, with most of the exam being at the advanced undergraduate and first-year graduate school level. Until recently, students were required to pass both exams in the same year in order to continue on with their studies. Now, however, students can pass the two parts in subsequent years and still continue toward their Ph.D. Graduate students who are new to ISU, entering without a Masters degree, are expected to pass the exam within their first two years, and students entering with a Masters degree are expected to pass after one year. Those students who fail to pass the exams in their allotted number of attempts become ineligible to continue working towards a Ph.D. in physics at ISU. In most cases, these students choose to finish a Masters degree from the department and then transfer to another department on campus or to another university in a similar field of study.

We both entered graduate school having received degrees in physics, but discovered that we were unprepared for the breadth and depth of the exam’s topics. Upon the recommendations of a graduate student who had previously failed his two attempts to pass the qualifying exam, we confined ourselves in a classroom and collaboratively worked problems from old qualifying exams for roughly 20-40 hours a week for multiple months. Initially, we had many questions regarding the material, and we discovered that the answers to our questions could not be efficiently found in textbooks. We were provided solutions to these old exam questions, but they were often limited to algebraic solutions with very little written explanation. Thus, determining how or why we were supposed to apply a certain method, and interpreting details about underlying assumptions or approximations, was nearly impossible. We found that discussing ideas between ourselves was an effective method for studying but lacked efficiency. What we really needed was an “expert” to direct our conversations, leading us not only to correct answers but to correct understanding as well.

Not surprisingly, neither of us was successful in passing the exam on our first attempt, although the material that we encountered in future courses became much more accessible and comprehensible. The following summer, we adopted a different approach that we believed would improve the efficiency of our studying. We focused our attention on the key concepts behind each particular problem and strived to look at a larger number of problems. We met weekly with a larger group of people to discuss specific worked problems, but also did a great deal of independent studying. With these revisions incorporated into our study tactics, we both passed the exam on our second attempt, much to our own delight.

After an external review of the department and several meetings between the department chair and the graduate student body, it was obvious that the qualifying exam and, in particular, the lack of assistance in passing it, was a significant cause of distress among graduate students. The department chair thus determined that instruction directly focusing on the qualifying exam was desired by the students, and he approached us with the idea of creating such a course. Having painstakingly developed our own successful study techniques, and being familiar with proven pedagogical techniques used in physics education research (PER), we enthusiastically agreed.

Our 12-week course covers a different subject each week, alternating between topics in classical and modern physics. In a given week, two class meetings occur. A one-hour introduction to the material takes place early in the week, and later in the week the students spend two hours presenting the solutions to assigned problems. In the first meeting, we introduce the weekly topic in a brief PowerPoint® presentation, lasting no more than 20 minutes. We purposely minimize lecture instruction for the following reasons: 1) Developing lecture instruction at an appropriate level for everyone was impossible due to the diverse background (and content knowledge) of our graduate student population. 2) Although it has not been rigorously tested at the graduate level, the PER community has provided overwhelming evidence that standard lecture instruction is not an effective method of learning physics for the majority of students.[2]

The remainder of the first meeting has students working in small groups, solving problems from old qualifying exams. These specific problems are chosen because they satisfy two criteria: They are relatively straightforward, and they clearly showcase the key aspects of the weekly topic. At the conclusion of this meeting, the students are assigned five problems which are to be presented during the second class of the week. The second class period, which lasts two hours, involves students taking turns working problems out at the board, spending 20-30 minutes per problem. Each student leads a discussion of the solution, responds to questions, and is asked to elaborate on the concepts of a particular problem in various ways that are discussed in the following section.

Another key feature of our course was the administration of full-scale practice exams to students throughout the summer. One of the underlying challenges of passing the exam is the context in which it is taken: A four-hour time limit, an 8 AM start time, and a formal test-taking environment. This makes for a very different experience when compared with a student’s typical problem-solving environment (i.e., casually working problems often with readily available resources). We therefore schedule four different sets of exams that are administered to students in a classroom, at eight o’clock in the morning, on Tuesday and Thursday mornings throughout the summer. As one of our students stated, “They were quite helpful in forcing me to sit through a full exam early in the morning in cramped conditions. The practice exams were also useful in that by the time the real qualifying exam came by, it was old hat and I was quite relaxed, which helps.”

Our primary goal for the course is quite simple: To enable students to pass the qualifying exam. In order to succeed in this goal, there are a number of strategies that we employ. Some of these strategies are aimed at learning physics, by developing both our students’ conceptual understanding and their problem solving abilities. Other methods focus on preparation and test-taking tactics for the specific type of exam for which they are studying. In this section, we describe some of the specific methods that we use, as well as our motivation for choosing them.

Since the exam consists of solving written problems, it seems obvious that the most appropriate means of studying is also to solve problems. However, we found that many students relied primarily on reading physics books to prepare for the exam. We therefore placed an enormous emphasis on working problems, both during the class hours and throughout the rest of the week. Solving problems, however, is quite challenging if one does not already have a firm grasp of the different topics and methods that should (and should not) be employed to solve the myriad of problems. Simply being told “work problems” can lead to hours of painfully inefficient studying as we discovered during our first summer of preparation.

The alternative, reading books, has the advantage that one can easily make progress, but it is a highly ineffective means of studying for this type of exam.[*] The central strategies of our course are therefore to provide the students with summaries of the most relevant physics content (in the form of our 20-minute presentations) and to provide immediate feedback on their progress (in the remaining 160 minutes of weekly class time). If we could focus students’ time on working problems in an open group environment that allowed for immediate feedback, we were confident that we would give them the best chance to succeed.

With this in mind, we set out to create an environment in our class that would support students discussing, critiquing, and assisting one another. We had groups of students work problems under the guidance of experienced exam-takers (i.e. us), with rapid feedback regarding both their solutions and their solution methods. While working problems, student questions arise and are often redirected back to the other class members, asking for volunteers to explain certain techniques or ideas. This not only helps answer the inquisitive student’s question, but it also allows another student the opportunity to explain his or her ideas, thereby benefiting both students. In addition, other members of the class become involved in the process, commenting and asking further questions. Our role as peer-instructors (we are fellow graduate students) further facilitates these discussions in that students do not hesitate to engage us in healthy debate. Unlike the previous alternatives that we described, solving challenging problems in this way is very efficient, since in a class of fifteen graduate students, someone almost always knows the answer or method that should be used to solve the problem.

We strive to build on student understanding primarily via the student-student and student-instructor interactions that occur while students are solving problems. While these interactions are present during the first meeting each week, they truly flourish throughout the second meeting when students are working problems at the board. Instructor-led discussions cover all aspects relevant to the problem, with particular emphasis placed on promoting problem-solving skills. The ability to identify key ideas and plan an efficient solution strategy is imperative for success on the exam. There is a vast research base that supports the notion that use of structured problem-solving strategies is an effective means of developing student conceptual understanding.[3] Additionally, we explore alternative contexts, alternative solution methods, and how slight modifications to the question would affect the solution. The goal is to strengthen the understanding of the student working at the board by challenging them to think on their feet, while also eliciting ideas from the class to paint a complete picture of how each problem fits in with other concepts.

Another pedagogical technique that has been shown to improve student conceptual understanding is the use of graphical and diagrammatic representations,[4] both of which are often required as a part of qualifying exam problems. While initially we felt it was important to practice such skills to be prepared for these types of questions, we subsequently realized that substantial knowledge can be gleaned from a proper sketch, and that improved depth of understanding can result from analyzing it. Once a sketch has been produced, questions concerning limiting cases and points of interest (such as equilibria) are readily tractable. By using graphical representations, peer-led instruction, and a variety of other methods, we continually refocus students' attention on their method of approach to solving problems.

A key feature of our course is the highly focused nature with which we present the material. During our own exam preparation, we spent a great deal of time determining what types of questions are commonly asked in order to improve the efficiency of our studying. To specialize our course, (and to save our students from unnecessarily investing similar time) we meticulously cataloged and analyzed the most common topics and problem-solving methods that have been used in previous years of the exam; we hence determined which topics should be covered, and in which order. We also provided our students with a detailed inventory of all 26 years worth of old exam problems. Sorted primarily by topic, this resource allows students who are looking to practice, for instance, boundary value problems, to instantly locate 19 previously asked qualifying exam questions.

A few weeks into the first summer of teaching the course, we become aware that, at times, students were misinterpreting portions of the questions. This was sometimes as simple as clarifying the distinctions among scientific words (e.g., constant, uniform, invariant). Occasionally confusion also arose when students were trying to interpret the instructions in the question, such as the difference between “Write down …”, “Determine…”, and “Derive…”. Students, particularly those who received undergraduate educations outside the United States, also had difficulties narrowing the scope of particular problems. When discussing problems, we therefore make a pointed effort to address precisely what each question is asking and what is required for the solution. While this may seem trivial to some, considering the timed nature of the exam it is important to focus students on doing the work that will yield the most points. As one student remarked after taking our course, “As a foreign student, language is always a barrier… I need to be familiar with the way they ask questions.”

We also highly recommend the series of books titled “Major American Universities Ph. D. Qualifying Questions and Solutions” (1998)[5] as another resource for problems at the appropriate level. A set of these books was purchased by the department and is on reserve for the students. All other resources are made available to the students online[†], and recently the school produced CDs that contained all of our course material, including PowerPoint® files, the question inventory, and every qualifying exam with its solutions in electronic format going back to 1979.

This past August, 37 Ph.D. hopefuls took at least some portion of the qualifying exam. Of these students, 17 were new arrivals at ISU and, as such, had limited opportunities to attend our summer preparatory course. Typically, these students have little to no chance of passing the exam anyway, so we have removed them from our data set. Furthermore, due to the recent change in the passing requirements, five students taking the exam only had to pass one portion of the exam (all five did pass). By also removing those five students from our data, we are left with 15 students, eight of whom regularly attended our course. To attempt to assess the effectiveness of our course, we have analyzed the performance of those 15 students.[‡]

As shown in figure 1, five of the eight students who attended class regularly passed at least one of the exams, while only two out of seven non-attendees passed. Although a higher percentage of our attendees passed, it was not obvious whether this was as a result of having attended our course, or if the students who attended our course were already better prepared. In an attempt to shed additional light on this issue, we obtained the average scores that these two groups achieved on the GRE Quantitative and GRE Physics Exams which they took prior to entering graduate school. These data, shown in figure 2, suggests that our attendees were unlikely to have had any type of pre-instruction advantage. Given this very small sample of students, it is impossible to claim any statistical significance with these findings. However, we believe that these data suggest that our course is successfully fulfilling its goal, that is, to enable students to pass the qualifying exam.

We have developed a summer-long course whose goal is to prepare graduate students for the comprehensive written qualifying examination that is administered at Iowa State University. This course is taught using pedagogical methods from Physics Education Research that have been proven to be effective at the introductory level, with a particular emphasis on active learning and peer-led instruction. We also teach efficient studying techniques and stress their importance in order to drastically improve our students’ chances of passing the exam in a matter of mere weeks. Data are presented which suggest that this course is indeed effective. We believe that this course could effectively serve as a model, both for qualifying exam preparation at other universities and for GRE exam preparation for advanced undergraduates.

We would like to acknowledge with gratitude the contributions of the late Ngoc-Loan Nguyen, in particular his insight and interest in assisting fellow graduate students in their exam studying. Thanks to Eli Rosenberg, our department chair, for his drive and financial support in improving the opportunities (and the quality of life in general) for graduate students at ISU. Also, thanks to David Meltzer for his invaluable discussions and suggestions, and to Lori Hockett, the graduate secretary, for compiling the data from student records.

[*] Perhaps an analogy might better explain our idea: You and I are going to have a swimming contest in three-month’s time. I am going to spend that time reading all of the best books and articles about proper swimming techniques. Meanwhile, you will go to a pool and swim everyday for three months. Who do you think will win the race?

[†] URL:http://www.public.iastate.edu/~wmchris/qual.html

[‡] Note that the data presented in this section were only given to the authors in summary form in order to protect the confidentiality of the results for those who took the exams.

[1] “Resource Letter: PER-1: Physics Education Research,” L.C. McDermott and E.F. Redish, Am. J. Phys. 67, 755-767 (1999). Surveys a wide scope of research on teaching methods for and student learning of physics at nearly all levels.

[2] “Guest Commentary, How we teach and how students learn-A mismatch?,” L.C. McDermott, Am. J. Phys. 61, 295-298 (1993).

[3] “Resource Letter: RPS-1: Research in problem solving,” L. Hsu, E. Brewe, T.M. Foster, and K.A. Harper, Am. J. Phys. 72 (9), 1147-1156 (2004). Comprehensive review of relevant research on problem solving.

[4] “Relation between students’ problem-solving performance and representational format,” D.E. Meltzer, Am. J. Phys. 73, 463-478 (2005). As part of a discussion of student difficulties with multiple representations, this paper contains a thorough listing of references on research in the use of graphical and other forms of representation in physics instruction.

[5] “Major American Universities Ph. D. Qualifying Questions and Solutions,” Y.-K. Lim, ed., (World Scientific Publishing Co. Pte. Ltd., 1998). There are a total of seven volumes in this series.

Tapping Physics Education Research for a graduate-level curriculum: A novel approach for a Ph.D. qualifying exam preparation course