Using Clickers in a University Physics Course to Improve Student Achievement

Judith C. Stull, David M. Majerich, Andria C. Smythe, Susan Jansen Varnum, Joseph P. Ducette and Tiffany Gilles

University faculty are being charged to augment traditional methods of teaching in the large lecture hall with learner-centered, student-engaged, interactive strategies informed by what is now known about how many students learn [1]. To better preparing students for the skills needed for success in the 21st century [2], using new technologies during instruction that are interactive have shown to assist faculty in creating active learning environments where students can learn by doing, receive feedback during the process, construct new knowledge and improve skills, and continually refine their understandings of course material [3]. One way to create an active learning environment as just described is to integrate “clickers” [4] into instruction.

In brief, clickers are radio-frequency, battery powered, hand-held devices that are part of an electronic polling system. The predominant research about the clicker use has been shown that they promote student discussion, increase engagement and feedback, and improve attitudes toward science [5]. Although several research efforts report positive effects of clicker use on students’ achievement [6,7], the empirical evidence that is needed to corroborate existing results and substantiate any claims for using clickers requires additional studies [8]. The research reported here provides evidence clicker use can improve student achievement in a university physics course.

Science Education Reform and Assessment

In recent years discussions of the role of assessments have taken center stage in the arena of science education reform debates. As generally understood, assessment is used by most instructors to determine what learning has occurred when compared to course expectations and is the basis for the assignment of grades to overall achievement. This type of assessment is summative and is the measurement of achievement at the end of a teaching-learning sequence. Assessment is formative when frequent evidence during the students’ learning process is gathered and analyzed, where the results inform changes needed to instruction in order to meet students’ needs, and provide students with feedback about their learning that they can then use to revise their studying strategies [9].

While the curriculum is already established in many college and university courses, and if assessment and learning are two sides of the same coin [10], it would seem reasonable that administering frequent assessments, analyzing their results, and sharing them with students, could inform changes to instruction needed in order to accommodate students’ needs for continued learning.

Methodology

Learning Environment

This study was conducted to determine the effect of increased feedback from clicker episodes (formative assessment) on students’ physics achievement (summative assessment) for students who used clickers when compared to students who did not. Students who enrolled in this physics course were mostly science and health profession majors and took this course to fulfill either a university core requirement or a major requirement. Taught in the large lecture hall, enrollment numbers generally ranged between 150-250 students per course. While all students were taught together during the lecture by the same instructor, the students were required to register for recitation and laboratory sections which generally have 25-40 students and were taught by other instructors.

Methods and Subjects

This study was conducted at a large, public, urban university in the mid-Atlantic region. Data were obtained from two fifteen-week introductory physics courses that met twice a week for 80 minute periods over two semesters taught by the same instructor. In all 157 and 152 students were enrolled in the two sections of the course, one taught in the fall semester and the other in the spring semester. The fall semester course was traditionally taught while the following spring semester course had clicker episodes (formative assessments) integrated into the instruction. Each learning object episode began with a multiple-choice question associated with a specific course topic, followed by a discussion of the results. The results of the clicker-based questions were collected, tabulated, and results displayed for students at the beginning of the next scheduled class. Problem areas were identified and provided the topic for discussion for the instructor and students. Based on the discussion, the instructor made appropriate adjustments to the instruction and hoped that the students would make the needed adjustments to their studying strategy. In the end, the spring semester students (clicker group) completed a total of seven formative assessments.

Results

Equivalent Groups

Both groups suffered loss of students. The attrition rates for the control and clicker groups were 20.4% and 23.0%, respectively; however the difference of proportions was not significant. It is expected that the more challenged students have a higher probability of withdrawing from the class. Accounting for self-selection bias, it is acknowledged that the groups’ content and skill sets should be better at the end of the course than at the beginning.

Pretests composed of a mathematics and a physics section were administered in both courses. The maximum possible points for the physics and mathematics pretests were 7 and 25 points, respectively. Points for each component of the pretests were summed separately. Results of the pretests revealed the control group’s pretest physics percentage scores (M=31.4%, SD=11.3%) were higher than the clicker group (M=30.7%, SD=11.3%), but the difference was not statistically significant. The clicker group’s pretest mathematics percentage scores (M=57.3%, SD=23.1%) were higher than the control group (M=56.8%, SD=21.5%), but again were not statistically significantly different. Based on these results, the groups were deemed equivalent.

Regression Analyses

Regression analysis was used to control for differences among students and to quantify the effect of clicker use. In the model for predicting the students’ physics achievement, the dependent variable was the student’s final examination score and the independent variables were the physics/mathematics pretest score, the number of clicker quizzes taken, whether the course was a required one, the number of different types of assessments the student had previously experienced, the number of hours per week the student reported working, and whether the student was male. In specifying the model, the percentage of correct answers on any quiz were entered, but never proved significant. They were dropped from the analyses as a result. In entering the pretest score, we were estimating a “value added model” by taking into account the students’ presenting knowledge base. Table 1 presents a summary of regression analysis for variables predicting students’ Physics achievement.

Table 1. Summary of Regression Analysis for Variables Predicting Students’ Physics Achievement

Model	Unstandardized Coefficients	Standardized Coefficients
	B	Beta	t	Sig.
Physics/Mathematics pretest score		-.059	-.952	ns
Number of clicker episodes taken	1.756	.230	3.298	.001
Was this a required course?	-1.799	-.030	-.485	ns
Number of types of assessments student had experienced	-1.881	-.413	-5.895	.000
Number of hours students works at a job per week	-.109	-.036	-.594	ns
Male student dummy	-.624	-.017	-.277	ns
(Constant)	58.142		9.567

The R Square equaled 33 indicating that the included variables explained 33% of the variation in the dependent variable. In the end, two variables proved significant – the number of clicker episodes and the number of different types of assessments the student had experienced. In all, there were seven clicker episodes. The regression results indicate that controlling for all of the entered variables, for every one more clicker episode the student took, the final grade was raised by 1.756 points. Thus, if a student took all seven of the “clicker quizzes,” the final grade would have been 12.3 points higher, a difference of a grade. Interestingly, how well the student did on these “clicker quizzes” never proved significant. The number of different types of assessments the student has experienced is negatively related to how well they did on the final exam. Perhaps what is needed is consistency in assessing learning. One could speculate that students were using the clicker episodes to assess is they had studies enough or if they needed to study more. Thus explain that the number of clickers episodes and not the percentage correct proved to be significant.

Conclusion

While there is an abundance of anecdotal information that advocates the use of clickers to improve student achievement in the science classroom, this study offers results to substantiate the claim. It is apparent that integrating clicker episodes, in this case weekly formative assessments consisting of multiple choice questions with in-class discussion of results, had a significant effect on student achievement. On average, students who used clickers achieved significantly higher scores on the cumulative final examination compared to the other group. The regression results quantified the effect. In sum, using clicker episodes did prove to be positively associated with improved achievement, but this is offered with caution as learning is a complex process and more data are needed on students’ attitudes and behaviors.

The work reported herein was supported in part by the National Science Foundation grant 0455786 to Temple University. The opinions expressed do not necessarily reflect the position of the supporting agencies and no official endorsement should be inferred.

References

[1] John Bransford, Ann Brown, and Rodney Cocking, How People Learn: Brain, Mind, Experience, and School (Washington, DC, National Academy Press, 2000).
[2] Partnership for 21st Century Skills. (2010), “21st Century Learning Environments.”Online at http://www.p21.org/documents/le_white_paper-1.pdf
[3] Joel Mintzes and William Leonard(2006), Handbook of College Science Teaching (Arlington, NSTA Press, 2006).
[4] Douglas Duncan, Clickers in the classroom: How to Enhance Science Teaching using Classroom Response Systems (San Francisco, Pearson/Addison-Wesley, 2005).
[5] Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system. Journal of Computer Assisted Learning, 20(4), 81-94.
[6] Stephen Addison, Adrienne Wright, and Rachel Milner, “Using Clickers to improve Student Engagement and Performance in an Introductory Biochemistry Class.” Biochemistry and Molecular Biology Education, 37(2), 84-91 (2009).
[7] Erica Watkins and Mel Sabella, “Examining the effectiveness of clickers in promoting learning by tracking the evolution of student responses.” Proceedings of the 2008 Physics Education Research Conference, Edmonton, Alberta, CA, 1064, 223-226 (2008).
[8] Richard Mayer, Andrew Stull, Krista DeLeeuw, Kevin Ameroth, Bruce Bimber, Dorothy Chun, and Hangjin Zhang, “Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes,” Contemporary Education Psychology, 34(1) ,51-57 (2009).
[9] Paul Black and Dylan Wiliam, D, “Assessment and classroom learning,” Assessment in Education, 5(1) , 7-74 (1998).
[10] National Research Council). National science education standards. Washington, DC, National Academy Press, 1996).

---

Disclaimer - The articles and opinion pieces found in this issue of the APS Forum on Education Newsletter are not peer refereed and represent solely the views of the authors and not necessarily the views of APS.