by Jeff Benca, Integrative Biology
Teaching Effectiveness Award Essay, 2013
One of the more difficult skills for students to learn in the classroom is how to interpret and analyze scientific literature. This includes learning to assess the merits and all-too-common pitfalls in the scientific method. While the virtues of the peer-review process prove logical to students, the ways in which different errors can make it through the system are not so clear-cut. Some flaws in research papers result from honest mistakes, less conservative use of language, failure to evaluate limitations of results, or even incompetence. Other flaws can be more severe, compromising the assets of the work as a whole. The most serious flaws tend to be unwarranted extrapolations of the results and worst of all, scientific fraud (that is, data manipulation and fabrication). Errors at any point within this spectrum can prove highly cryptic, to some extent requiring “expert” knowledge of the subject to recognize. While certainly important, prior expertise is only part of error recognition. In practice, a wide variety of literary “street smarts” are often used to detect different shortcomings within peer-reviewed works.
Exposing potential flaws and helping students develop the street smarts necessary to recognize them as part of a structured course can be risky. Doing so haphazardly can alter students’ trust in the integrity of the scientific method and peer-review process. To avoid this in designing and teaching a new upper-division discussion course on Paleobotany, I used a three-pronged approach with built-in mandatory (graded) checkpoints. The approach consisted of student-led presentations, an in-class debate, and an individual written debate report. I used checkpoints throughout to observe how students’ analytical approaches (street smarts) developed over the course while providing constructive feedback in the process.
Through these checkpoints, I could intercept students whose evaluative tendencies were hypercritical or overly accepting. In such circumstances, I would meet with the student and address the tendency up front. I would then use guiding questions to allow the student to “walk through” my thought process in order to highlight which elements of their criticisms appeared well-grounded and which seemed less so. Furthermore, these interactions provided me with ample feedback for assessing which street smarts I conveyed most effectively, over-emphasized, or could address further in the classroom.
For the presentations, teams of two students designed a 15- to 20-minute mini-lecture on a technically advanced research article they read and assessed (in terms of both strengths and weaknesses). Half of the assignment grade hinged upon the team submitting a draft presentation to me two days before the lecture. Reviewing drafts and meeting with several of the students, I was able to intercept those displaying “trigger-happy” criticism or excessive leniency towards flaws. Following the feedback sessions, both the lecture instructor and I were amazed by how professionally and tactfully the vast majority of our 36 students presented the key merits and shortcomings of their papers.
During the in-class debate, we focused on the question “What caused earth’s greatest mass extinction?” To do this, I split the class up into four groups, assigned each group one of four papers, and then had students talk through their assigned paper while completing a worksheet. The papers covered radically different sets of hypotheses, methodologies, and results, embodying a wide gradient of potential flaws. The worksheet required students to discuss the affiliations of their assigned paper’s authors in relation to the subject-area of the argument; assess the study’s strengths and weaknesses, methodological approach; and finally, advocate for or against the paper. In the following debate session, I “scrambled” the former discussion groups so that each debate group had at least one representative for each of the papers.
It was truly inspiring for me to hear both discussion sections of the class spend 1.5 hours actively comparing notes and debating which arguments held most credence by analyzing the approaches of the papers, considering the expertise of the authors, and applying trends in the fossil record covered in previous lectures. What was extremely encouraging for me to witness was that most discussions lacked overbearing hypercritical or overly lenient tones. I noticed that many of the students I had provided feedback to earlier in the course were applying their evaluative skills more effectively than before.
To ensure students were retaining the street smarts I was trying to convey over the long term, I had one last check-point: a one-page write-up on the debate activity. To my delight, grading, it was clear that students later used the debate handout questions I provided in class as an evaluative framework for paper comparisons on their own. Thus, through a variety of different assignments with graded feedback stages, I found it possible to equip students with street smarts needed to read between the lines and more effectively interpret scientific literature.