March 25th, 2010

Evidence of Effectiveness


The scholarship of teaching movement has made us aware that the effectiveness of those new activities and approaches we implement in the classroom must meet higher standards of evidence. Even though we may be thrilled with the effectiveness of what we tried, it no longer suffices to say that we thought it worked really well and our students loved it. Evidence of effectiveness needs to include more objective measures of success.

In a recent editorial in the Journal of Management Education, a discipline-based pedagogical periodical I particularly admire, Jane Wilk-Schmidt identifies four characteristics of evidence valued by the journal. It’s a great list that offers a criteria for looking at the effectiveness of instructional innovations whether you are thinking about reporting what happened in an article or about assessing impact more objectively for your own information.

1. Develop and test activities through multiple classroom iterations. Try it more than once! See if the same outcome occurs. See if some minor alternations make it even more effective.

2. Collect evidence from multiple sources, such as students and outside observers. Yes, your opinion as to whether and how well something worked counts, but verify what you think happened by collecting information from students. They don’t always experience things the way we think they do. Ask a colleague or a professional from the teaching center to come to class and observe and report what results they’re seeing.

3. Collect evidence using multiple methods. Most of us don’t evaluate what students know by only using multiple-choice methods. So our instructional alternations ought to be assessed with multiple methods—qualitative, quantitative, descriptive, and so on.

4. Tie evidence to learning objectives. Why did you try the new activity? Is it an attempt to better reach one of your learning objectives for the course? Usually changes are, which makes it natural to judge their effectiveness by looking at evidence documenting how well they accomplish the learning objective.

Reference: Schmidt-Wilk, J. (2010). Evidence: Where scholarship meets artistry. Journal of Management Education, 34 (2), 195-199.