by Ashwin Mandakolathur Balu, Public Policy
Teaching Effectiveness Award Essay, 2021
Using experimental methods to establish causal relationships has been the holy grail of research in social sciences. However, it is a challenge to teach these concepts since it requires knowledge about sampling methods, statistics, and probability, as well as how to manipulate data in R or Python. Traditionally instructors introduce the concept using an example, and later give additional questions for students to practice. When I started teaching this topic, it became apparent to me that students were overwhelmed by the different underlying concepts and believed complex methodology was a prerequisite to establish causality.
I wanted to dispel this notion by conducting a simple experiment. During the sections, I introduced a hypothesis: Does giving hints during exams lead to higher learning outcomes among students? To test the hypothesis, I designed a geography quiz using Google Forms and randomly assigned students (using breakout rooms in Zoom) to two groups. The first group (intervention) were provided with hints while the second group (control) solved without any hints.
The following week, I discussed hypothesis testing including two-sample t- and z-tests. Armed with the necessary knowledge pieces, I asked students to apply the concepts of hypothesis testing to the quiz data and establish a causal relationship. To this end, I asked students to follow five steps: (1) identify the null and alternative hypotheses by calculating the mean for each sample group; (2) write the significance level; (3) calculate the t-statistic by calculating the standard deviations of the sample groups; (4) calculate the p-value; and (5) write the conclusion based on the evidence. As a result, students were able to establish a causal relationship between providing hints in a quiz and final scores. This experiment helped students understand the application of random sampling and hypothesis testing to establish causal relationships.
In the following classes I used the same example to teach other concepts, like errors due to sampling bias, and the ethics of research (e.g., de-identified data, transparency). It helped students connect various concepts through the lens of one example. To judge my performance, I looked at the overall evaluation of my sections. Both of my sections were rated favorably compared to those of my peers on the overall effectiveness of the GSI. Further, one of the students commented, “[Ashwin] created activities in class relating to the topics that stick with you along with the material like the geography test experiment.” Now, I am unsure whether students will remember every concept taught through the experiment or otherwise. However, as long as students remember that a simple five-minute experiment is all that is required to establish causality, my job as an instructor is fulfilled.