Review sessions—Some faculty don’t have in-class review sessions because that means one less period for covering content. The question is whether students benefit more from being exposed to additional material or from having a chance to organize, summarize, distill, and integrate the content they must now learn for the exam. Should students do this summarizing and integrating on their own as they study? Perhaps. Will they learn to do it better if they their efforts are guided by an expert who understands how the content domain is organized? Probably.

Typically in the review session, the teacher goes over important or challenging content. Students are supposed to ask questions and they do, but generally they focus their questions on trying to ferret out what’s going to be on the exam. There are better alternatives. The teacher who already knows (and loves) the content doesn’t need to review it. Students need to review. The period should be structured so that students are doing the work, with the teacher providing guidance. They can be working individually or in groups, but they should be solving problems, answering old exam questions, writing possible test questions, or extrapolating key concepts from assigned readings. Groups could be given different topics, concepts, problem sets, etc., and tasked with preparing review materials/study guides for the rest of the class. They could bring these materials to the review session and present and/or distribute them.

In addition to revisiting the content and seeing more clearly how individual topics relate, review sessions can also be used to help students figure out what’s going to be on the test. That’s a question they shouldn’t need to be asking the teacher. The answer is a function of being about to determine what’s most important and how the content is going to be applied. And that’s a skill students need to develop.

Exams—Regular exams don’t promote deep learning because the questions don’t challenge students to think. Many students memorize well; they forget with the same efficiency.

Questions that challenge students to think are much harder to write, and for that reason you don’t find a lot of them in question banks that accompany textbooks. The problem is not with the multiple-choice format per se. SAT and ACT questions are multiple-choice and many of those are quite challenging. If exams are returned to students, then new questions must be generated for each new class. It is smarter to let students have access to their exams (when they’re returned and subsequently in the prof’s office), but not to let them keep their exams. That way, questions can be recycled and across the years a collection can be developed, revised, and reused.

Exam circumstances rarely change. Students work alone without access to resources and under surveillance so that they don’t cheat or they cheat less. This newsletter does, with some regularity, highlight different kinds of exam experiences—like having the students take the exam individually and then take the same exam with a group. Their grade may be some combination of their individual score and the group score. Or let students prepare a crib sheet (of a specified size) that they are allowed to use during the exam. Preparing a crib sheet forces students to make decisions about what’s going to be on the exam.

Debrief sessions—Typically teachers go over the most missed questions, but that approach may not be the best way to maximize the learning potential that is still present after the exam. Teachers don’t need to correct the answers—students do. Whether in groups or individually students can be given the chance to find the correct answers and fix their mistakes. Maybe that happens during the debrief session, or maybe students do the work at home, completing it before the next class session. Maybe their grade isn’t recorded until they’ve corrected their errors, and maybe it’s a few points higher if they get all their mistakes taken care of.

Debrief sessions can also be designed so that they address some of the decisions students have made about preparing for the exam. Class attendance makes a difference. You can say that, but you should show some evidence. Take the five highest exam scores and list the number of times that group of students missed class. Take the five lowest scores and list the number of class sessions that group missed. Let the facts speak for themselves. Many students aren’t taking enough notes in class. You can say that, or you can demonstrate it. Pick a question that many people missed. Identify the date that material was covered and have everybody look at their notes. Do they have what they need there to answer the question? Were they absent and got notes from somebody else? Do they understand those notes? Quick discussions of topics like these can be concluded with students writing themselves a memo addressing “things I learned taking this exam that I want to remember for the next one.” Collect those memos and return them shortly before the next exam.

Exams motivate students and learning results. They review their notes, read the text, and talk with each other. The question is how much and how well do they learn? How seriously they study determines part of the answer to that question. But it is also answered by the design of the exam experience including what happens before, during, and after the event. Exam experiences can be designed so that more of their potential to promote learning is realized.

Reprinted from The Teaching Professor, 26.4 (2012): 3.

Understand critical thinking as a multidimensional construct – In their discussion of critical thinking in psychology, these authors propose that critical thinking includes skills, dispositions, and metacognition. Critical thinking skills in psychology include argument analysis and evaluation, methodological reason, statistical reasoning, causal reasoning, and skills for focusing and clarifying questions. Dispositions refer to “the willingness to engage in effortful thinking and the tendency to be open- and fair-minded in evaluating claims, yet remain skeptical of unsubstantiated claims.” (p. 6) Metacognition means being aware of one’s thinking and in control of it.

A recent article in The Teaching Professor highlighted the variation in definitions for critical thinking. These authors point out that critical thinking is either thought of generically or as being discipline-specific. They cite research that critical thinking is probably a combination of both. As a multidimensional construct, it contains some general reasoning skills and some skills that are specific to the discipline. The point is that if you want to assess learning outcomes associated with critical thinking, you cannot do that well without understanding how critical thinking is defined in your discipline.

Select important goals, objectives, and outcomes for assessment – What critical thinking skills and knowledge should students be able to demonstrate as a result of being in a course or program? Some faculty have learning goals so general that they are all but impossible to assess. They need further specification. If the assessment is to be scientific, then the goals, objectives, and outcomes must translated into specific hypotheses—ones that can be tested.

Align assessment with instructional focus – “Measures for assessing the impact of instruction must be sensitive to the changes instruction is intended to produce.” (p. 7) If the measures are sensitive, then classroom assessment can be used to look at the techniques being used, compare their effectiveness with other techniques, and conclude which are better.

Take an authentic task-oriented approach to assessment – Taking an authentic task-oriented approach means using a performance to assess how well students are completing a task. In psychology, tasks requiring critical thinking include evaluating the quality of information from the Internet, analyzing and evaluating research literature, using psychological theory to analyze and evaluate behavior, and writing research and case reports, among others. Many of those tasks can be used to evaluate critical thinking in a variety of fields.

Use the best and most appropriate measures – Because critical thinking has multiple dimensions, multiple measures should be used to assess it. The authors point out that standardized tests of critical thinking (the Watson-Glaser Critical Thinking Appraisal and the Cornell Critical Thinking Test are the two examples referenced in this discussion) are “probably better measures of general CT skill.” (p. 9) In many cases, no standardized tests or measures assess the specific type of critical thinking or aspect of critical thinking being developed in a particular course. In situations like this, new instruments may need to be developed.

Conduct assessments that are sensitive to changes over time – “Simply testing seniors once in their capstone courses is not sufficient to infer changes over time because the levels of skill and knowledge of students entering the program are unknown.” (p. 9)

Assess frequently, embedding assessment and feedback into instruction – Students can be assessed too much, especially if the same instrument is being used. They become sensitized to those instruments. The authors recommend a formative approach that embeds assessment in instruction. In this case, the assessment provides the instructor useful feedback and helps students focus on their development of critical thinking. It offers them feedback that can be used to improve their critical thinking skills.

Interpret assessment results cautiously and apply the results appropriately – The quality of the data collected must be considered before decisions to change a course or a program are made. Not considering the quality of the data and not carefully interpreting the results can result in changes that do not improve learning outcomes.

Reference: Bensley, D. A. and Murtagh, M. P. (2012). Guidelines for a scientific approach to critical thinking assessment. Teaching of Psychology, 39 (1), 5-16.

Reprinted from Assessing Critical Thinking Skills, The Teaching Professor, 26.3 (2012): 4.

The university where I teach is accredited by the Higher Learning Commission (HLC) and is on the new review cycle. As part of that initiative, members of the university assessment committee, of which I am a member, are charged with reviewing program assessment plans and reporting to the accreditation committee. This article was prompted by research that I conducted for the committee to develop a new and more inclusive rubric. The information I learned is taking our assessment review from basic to advanced and is allowing us to develop a more in-depth and inclusive rubric for analyzing assessment plans.

There are some excellent resources on program assessment that can drive the right questions when assessing assessment. The Council for Higher Education Accreditation (CHEA) provides multiple resources to guide institutions. Boiling it down to some key areas can help you decide as a faculty if you are on the right track in developing your assessment materials. There are five main areas that assessment reporting should cover.

1. Learning Outcomes. Is there a comprehensive list of program learning outcomes? By that we mean the knowledge, skills, and values students are expected to know at the completion of the program. Depending on whether it is an undergraduate or graduate program, the level of sophistication may be different. There should be no strict rule on the number of outcomes addressed, but the list should be reasonable, well-organized, and take into account any national disciplinary standards that are appropriate.

2. Assessable Outcomes. Are the learning outcomes measurable? The statements should specify what students can do to demonstrate their learning. Criteria for demonstration are usually addressed in rubrics and there should be specific examples of work that doesn’t meet expectations, meets expectations, and exceeds expectations. One of the main points here involves faculty communication – have all faculty agreed on explicit criteria for assessing each outcome? This can be a difficult accomplishment when multiple sections of a course are taught or different adjuncts are teaching. However, faculty should agree on what is assessed and how it is assessed.

3. Assessment Alignment. Is the curriculum (course sequence) aligned to support opportunities for students to develop knowledge for each program outcome? This design is sometimes in the form of a curriculum map, which can be created in something as easy as an Excel spreadsheet. Courses should be examined to see which program outcomes they support, and if the outcome is assessed within the course. After completion, program outcomes should be mapped to multiple courses within the program. If a course doesn’t support a program outcome, why is it being taught?

4. Assessment Planning. Faculty need to have a specific plan in place for assessing each outcome. Is that outcome being taught in the second course of the sequence sufficient to meet the program outcomes assessed in the capstone course? Outcomes don’t need to be assessed every year, but faculty should plan to review the assessment data over a reasonable period of time and develop a course of action if the outcome is not being met.

5. Student Experience. Students who are enrolled in a program should be fully aware of the expectations of the program. Are program outcomes aligned on the syllabus so that students are aware of not only what course objectives they are required to meet, but also how the program outcomes are supported? Do students and other stakeholders participate in review of outcomes, criteria, or related activities? Many outside accreditors also require communicating program outcomes to other relevant parties, including the community, business partners, and advisory councils. Assessment documents should clearly communicate what is being done with the data results and how it is contributing to improvement of the program and curriculum.

Assessment can be a dirty word at the university level. Accountability has become a sore issue, particularly with claims of infringement of academic freedom. With good planning, content experts can continue to teach what they are good at, while at the same time making sure they are providing the best educational experience possible. Assessment doesn’t have to be a dirty word – it just has to be done.

Dr. Vickie Kelly is program director and assistant professor of technology administration at Washburn University.

It may surprise us to think of it like this, but today’s students grew up in a culture of routine assessment and feedback. Yet when they click (or walk) into our courses, the experience is often quite different: there are few high-stakes grades, big exams, or one-shot term papers. Despite critiques of high-stakes testing – Wideen et al. (1997) said such “examinations discouraged teachers from using strategies which promoted enquiry and active student learning […] this impoverishment affected the language of classroom discourse”—teachers often still see “assessment as an index of school success rather than as the cause of that success” (Chappuis and Stiggins, 2002).

Certainly, grades, when misused as what Filene (2005) calls a “pedagogical whip,” can lead to problems: Grading curves pit students against each other, fostering strategic rather than deep learning (Bain, 2004). High-stakes grading may contribute to grade inflation (Rojstaczer and Healy, 2010). Grading pressures may even encourage cheating.

I offer the strategy/philosophy of frequent, low-stakes (FLS) grading: simple course evaluation methods that allow you to provide students with many grades so that an individual grade doesn’t mean much. FLS grading can work in any course but is especially useful online, as it provides grade transparency for students and creates a steady information flow in an environment in which student-teacher communication is crucial to success. FLS grading can have several advantages:

• It creates dialogue. Frequent grades can establish a productive student-teacher conversation, and students have an ongoing answer to the question, “How am I doing?”
• It builds confidence. Students have many opportunities to succeed, and there is a consistent, predictable, open evaluation structure.
• It increases motivation. FLS grading fits into students’ conceptions—and, perhaps, expectations—of assessment and evaluation: This is the culture they grew up in!

Some teachers may have an “allergic” response to the idea of giving lots of grades, but much “classic” pedagogical thinking (and writing) about grading predates both this culture of assessment and feedback and the teaching technologies now available, especially to online instructors. While some may resist grade-centric approaches, remember, in ideal teaching, perhaps everything is formative and you have small ratio, even one-on-one, interactions with students. Maybe there are even no grades at all. But such ideal environments are rare. We must give grades, so the issue is how we grade to the benefit of students.

The growth of online courses provides additional exigency for FLS grading. I’m always skeptical about those who privilege teacher-student interactions in onsite courses – how often do students talk to the instructor of their 200-student onsite lecture course? – but no doubt a key to effective online pedagogy is making sure you are present for students as their teacher. All students benefit from having a clear idea of their overall course standing, but we need strategies to provide online students with meaningful communications about the course, and what is more meaningful to students than clear grade data?

Frequent grade information also provides motivation, another especially important factor in online student success (i.e., see Schrum & Hong [2002]). Frequent, immediate grade data should help students overcome the inertia of procrastination far better than that delayed reward of the grade far off in week 12.

FLS grading does mean that you will re-conceptualize the grading function in your course, and while FLS grading has a summative micro structure—sure, you give grades—the overall structure is formative. You can remove unproductive grading pressure, encourage intellectual risk-taking, and discourage plagiarism/cheating. And especially online, your overall response strategy will include this grade-based dialogue with your students.

You can still have your major papers and exams, but with FLS grading, a series of low-stakes assignments helps uncover points of intervention long before any high-stakes evaluation. Teachers are busy, but FLS grading can actually result in less work overall if done right, as dialogue occurs through the grades. For FLS grading, you will shift your course requirements, like this:

FLS is about feedback. Really, a high-stakes evaluation structure often precludes a feedback plan: You basically just provide summative evaluation. The meaning of “frequent” will vary based on your teaching style. At one time, I provided as many as five grades per week. I have shifted my approach, clumping various small assignments into one weekly grade so, each week students get one status grade, although I can break that down to individual assignments for them if asked.

I’ll focus on two particular assignment methods: informal writing and quizzes.

Frequent short, informal writing assignments can take many forms:

• Responses to readings or focused content questions
• End-of-unit notes on important or confusing points, questions
• Journals
• Brief annotations or notes about calculations, charts, tables
• Metacognition: Have students think through/reflect on reasoning, thinking, writing processes

The technological environment of online learning is a major asset in using short, informal writing. Technology reduces the paper shuffle, easing logistics, and digital writing forums and tools allow students to write to one another, making open dialogue a fundamental course component. Message boards are an easy-to-use and readily available dialogic technology for online courses, and blogs or even wikis can be used to replace notebook-based response journals.

Rubrics provide structure for responding to writing and demystify evaluation – for you as well as the students. A simple rubric for brief informal writing could involve two simple criteria, on a scale of 1 to 5:

• Demonstration of understanding of a key idea.
• Writing quality (judged loosely, maybe even as your readerly response to the piece).

When developing a rubric, remember what you want the assignment to accomplish. This is your decision based on your course goals. Don’t outsmart yourself. In line with writing across the curriculum approaches, remember what you’re trying to accomplish when you assign informal writing, and remember what you don’t want to worry about. You do not need to evaluate everything. For instance, if you want to evaluate their understanding of a main idea about a chapter but end up pegging them for dangling modifiers, you will likely become frustrated and may give up on using informal writing at all. Think about simple, specific, often content-oriented goals you want to assess. Rubric performance language/levels can be simple, excellent to poor, and reflect a range of responses. You can use rubric creation tools like Waypoint Outcomes or Rubistar.

Quizzes (as I’ve written about previously in The Teaching Professor [2004]) need not be a pedagogical stick. Quizzes should be easy to create, take, and grade. They should have a specific objective. For instance, I always give straightforward, weekly online reading quizzes, almost at this level: “What large sea mammal is featured in Moby Dick?” I just want them to read.

Technology again simplifies logistics, easing both assignment submission and grading. Course management system (CMS) assessment tools allow for simple quiz features like question sets so not all students receive the same questions, and I use the basic simplicity, frequency, and low-stakes aspects of my quizzes to discourage cheating.

The primary question most teachers have is this: How do I give lots of grades without breaking my back? Again, use a simple grading scale for individual assignments: 1 to 3, 1 to 5, 1 to 10, or even a check/check plus system. You can share/display grades in a CMS grade book. Remember, the object is creating grade-centric feedback, and the time payback comes when students do not constantly have to reach out to you about class performance; they already know, and when they do raise questions, the conversation is more focused than, “So, how am I doing in this class?”

Filene (2005) said, “For better or worse, grades matter; the challenge is how to make them work for your purposes.” FLS grading can demystify course assessment, letting your online students know how they are doing. Done right, it can result in less work/stress for teachers, helping identify struggling students early. Communicating meaningfully with every student is a teaching challenge, but a stream of FLS grades allows student to know where they stand so they can better reach their goals in our courses.

References
Chappuis, S. and R. Stiggins. (2002). Classroom assessment for learning. Educational Leadership. September: 40-43.

Bain, K. (2004). What the Best College Teachers Do. Cambridge: Harvard UP.

Filene, P. (2005). The Joy of Teaching. Chapel Hill, NC: University of North Carolina Press.

Rojstaczer, S. & C. Healy. (2010). Grading in American colleges and universities. Teachers College Record. March 04, 2010. http://www.tcrecord.org.

Schrum L. & S. Hong. (2002). Dimensions and strategies for online success: Voices from experienced educators. Journal of Asynchronous Learning Networks 6.1: 57-67.

Warnock, S. (2004). Quizzes boost comprehension, confidence. The Teaching Professor. 5.

Wideen, M.F., T. O’Shea, I. Pye & G. Ivany. (1997). High-stakes testing and the teaching of science. Canadian Journal of Education 22.4: 428-44.

Dr. Scott Warnock is an associate professor of English and Director of the Writing Center and Writing Across the Curriculum at Drexel University.

Reprinted from Online Classroom, 12.3 (2012): 5,7.

Let me give you an example. At my college, each professor is encouraged to pick one assignment or class activity per semester for formative assessment and to write a report on how he/she will better help the students achieve a desired outcome next time. Formative assessment is undoubtedly useful, however, the professor now feels pressured to find strategies for ensuring that students perform to his/her new standards. Invariably, this involves temptation to provide the students with a grading rubric and/or guidelines that are so comprehensive that the student can be left in no doubt as to what is expected. As a result the students do better at said assignment and their grades improve. The administrators are delighted and use it to exemplify how assessment is improving learning at their institute, but the professor is left with the uncomfortable feeling that he/she has encouraged over-reliance on guidelines that stifle creativity and independent thought and that his/her students will not be able to accomplish future projects without the same explicit instructions.

In his book The Global Achievement Gap, Tony Wagner, co-director of the Change Leadership Group, Harvard School of Education, identified the top four skills needed to get a good job as: critical thinking and problem solving, collaboration and leadership, agility and adaptability, and initiative and entrepreneurialism (Wagner, 2010). Most faculty understand the need to help students develop these abilities, but feel that they are being pushed into coming up with short-term strategies (such as grading rubrics) to improve student performance in the classroom at the expense of developing sustainable, higher order thinking skills.

At my college, we use capstone courses to assess students’ independent thinking proficiency. Recently however, we have experienced that, despite doing reading, writing, and analytical thinking exercises in their classes for three years, many of our students are still unable to independently perform these tasks in their capstone course and repeatedly ask for more guidelines. So how do we help students meet our outcomes expectations without compromising their independence or future success in the workplace?

One way to do it is to hand the task of formative assessment over to the students. Rather than simply telling them what we expect from them, we can challenge them to determine for themselves what the characteristics of a well done assignment are. I have done this in my classes and have found that when incited, most of my students had quite a good understanding of what a quality assignment should look like, but were afraid to use their intuition. In several of my classes, I have encouraged students to design presentation review worksheets to help them understand what a good presentation should resemble. They used the worksheets to prepare their own presentations and to review presentations by peers. The worksheets were essentially rubrics disguised as guidelines that the students came up with themselves (Robertson, 2012).

I have also asked students to write their own guidelines for writing a lab report. I frequently have students edit and grade their own, first drafts of papers, and I sometimes ask them to write short essays critiquing their own work and describing the improvements they think they should make, as well as the resources they will use to help them. The quality of assignments has improved in my classes, and the students also experienced the process of independently planning, writing, and editing their own work. More importantly, they started to let go of their dependence on highly detailed guidelines from me.

Assessment is probably not going to go away any time soon, and my examples above are simply some suggested techniques that I have used in an attempt to not allow assessment to compromise my teaching while helping students develop important new skills. Self-evaluation is a big part of tenure and promotion for many faculty and most of us appreciate its value for our own professional development. Required assessment has become, for me, an opportunity to teach self-evaluation to my students.

References:
Katz, S. N. (2010) Beyond Crude Measurement and Consumerism. Academe, 96 (5). Available online at www.aaup.org

Wagner, T. The Global Achievement Gap: Why Even Our Best Schools Don’t Teach the New Survival Skills Our Children Need &ndash and What We Can Do About It. New York: Basic Books, 2010.

Weimer, M. (2013) The Assessment Movement: Revisiting Faculty Resistance. Faculty Focus, January 18th. Available online at http://www.facultyfocus.com/articles/educational-assessment/the-assessment-movement-revisiting-faculty-resistance/

Robertson, K. (2012) A Journal Club Workshop that Teaches Undergraduates a Systematic Method for Reading, Interpreting and Presenting Primary Literature. Journal of College Science Teaching, 41 (6) 20-26.

Dr. Katherine Robertson is an associate professor of biology at Westminster College, New Wilmington, Pa.

Why let students work on exams collaboratively? There are a number of reasons, most of which have been explored empirically. When students discuss questions and possible answers, they intensely engage with the content, which increases the learning potential of an exam experience. The activity develops cooperation and communication skills. But the reason most often given is that working with other students decreases exam anxiety, particularly for those students whose levels of anxiety compromise their ability to perform on exams.

When asked why they don’t use collaborative testing, most faculty report being afraid that students who have not prepared for the exam may inappropriately benefit from the knowledge of students who have studied. Grades should be measures of individual learning.

Based on a review of the literature on collaborative tests, these authors decided to explore four questions in their study: 1) How does collaborative testing affect test scores? 2) Is anxiety related to the effectiveness of collaborative testing? 3) Which students benefit from collaborative testing? and 4) How does the quality of interaction within the groups affect test performance? (p. 165)

The study included some unique design features. To deal with the potential problem of students coming to the exam unprepared and thinking the group would pull them through, students were told to prepare for the exam as if they were taking it individually. Those who would be taking the exam collaboratively would be randomly selected at the beginning of the period. On test days, those selected to take the exam collaboratively were moved to another room. Their group interactions were observed by a proctor who evaluated the quantity of interaction, the level of enthusiasm, and the degree of give-and-take displayed by the participants. (p. 166) The groups did not have to agree on answers, although they could change their answers based on discussions that occurred within the group. Tests were still submitted and graded individually.

As for how the collaboration affected test scores, the results were positive. “Collaborative testing was more successful for a significant majority of students than was traditional, individual testing, although the advantage (3.83%) was smaller than found in some previous studies.” (p. 172) Based on standardized measures of test anxiety, “students with higher initial test anxiety scores were most likely to benefit from the collaborative testing procedure and to show a decrease in test anxiety when taking tests collaboratively.” (p. 172) And finally, “high interaction scores, both proctor and student-rated, were related to better performance under the collaborative condition.” (p. 172)

Consistent in the research on collaborative testing mechanisms is the favorable response they generate from students. Students regularly report enjoying the experience and indicate they would choose it in the future, if given the opportunity. In this study, students said the testing mechanism was beneficial because it helped them develop good working relationships with classmates and helped them remember material they had forgotten. There were students in this study who did perform better on the tests they took individually, and a third of those students thought they did better on the individual exams because they found the discussions with other students confusing.

The researchers conclude that collaborative testing is a “worthwhile technique.” “For, even if overall test performance is not greatly improved by collaborative testing, the positive experiences of increased camaraderie and anxiety reduction it engenders could constitute substantial long-term benefits in the form of increased confidence, motivation, and willingness to continue one’s education.” (p. 173)

Reference: Pandey, C., and Kapitanoff, S. (2011). The influence of anxiety and quality of interaction on collaborative text performance. Active Learning in Higher Education, 12 (3), 163-174.

Reprinted from The Effects of Collaborative Testing, The Teaching Professor, 26.1 (2012): 3, 4.

Early on he observes, “A great deal of ink has been spilled in recent years by a small number of professors and a much larger number of educational administrators arguing for assessment and pleading for greater faculty support of institutional assessment efforts.” Shortly after, he admits that faculty have been an impediment to the adoption of institutional assessment measures. Why? Because faculty fear the worst. They imagine (and with some justification, based on US Department of Education efforts in the previous administration) the generation of cross-institutional data that could be used for comparison shopping, thereby promoting even more unhealthy educational consumerism. After all, it was former Secretary of Education Margaret Spellings who regularly pointed out that Americans could get more information when they bought a used car than when they selected an educational institution.

Katz admits that these fears of externally imposed assessment have some legitimacy, but he thinks they are a worst-case scenario. “Faculty members should be capable of contemplating more benign, educationally helpful uses for sophisticated measurement of student learning.” Later in the essay, Katz makes a strong case for formative assessment—using the “evaluation of learning outcomes to improve teaching and learning on an ongoing, continuous basis.” Faculty should want to know whether students are learning what they are being taught, and the end-of-course ratings provide little feedback relevant to this fundamental question. Moreover, Katz wonders, “Should we not worry whether students can relate what they learn in one course to other courses, and to the multiple other learning experiences of their undergraduate years?”

But faculty resistance is not all that has slowed the progress of assessment. It’s hard to create reliable measures of learning outcomes. And once created, those instruments must be purchased. Do instruments, even well-developed ones such as the National Survey of Student Engagement (NSSE) or the Collegiate Learning Assessment (CLA) effectively measure student learning? It’s an unanswered question because the higher education community (including faculty) is not in agreement about which aspects of learning should be assessed. Student learning is the goal, but learning what and learning how? Katz believes that good assessment depends on faculty having specified learning benchmarks, and he points out that in many fields faculty don’t have clear notions of precise learning outcomes. Finally, should assessment be the same across the diversity of institutional types? Every institution shares the learning goal, but learning at a research 1 university is not necessarily the same learning as in a professional program offered by a community college.

Still Katz doesn’t let faculty off the hook. He says that faculty attitudes about assessment must change if “we are to take seriously the emerging conception of institutional responsibility.” Mostly faculty see assessment as an individual matter between teacher and student. Teachers assess student learning via grades, which they assign carefully and fairly. Maybe the overall impact of a departmental program or major might be assessed in a capstone course, but faculty don’t really see that they have a larger assessment responsibility. It’s a responsibility Katz describes this way: “Those of us who want to take ownership of the evaluation of undergraduate education must devote considerably more time, effort and ingenuity to the assessment of student learning over the life course of undergraduate education than we have been doing.”

Reference: Katz, S. N. (2010). Beyond crude measurement and consumerism. Academe, 96 (5). Available online at www.aaup.org.

Reprinted from The Assessment Movement: Revisiting Faculty Resistance, The Teaching Professor, 25.7 (2011): 5.

Critical thinking is assessed in a variety of ways by individual teachers, but unlike many other college-level learning skills, it is also regularly assessed via a battery of standardized tests such as ACT’s Collegiate Assessment of Academic Proficiency (CAAP), the Collegiate Learning Assessment (CLA), ETS’ Proficiency Profile (PP), and a set of scoring rubrics known as the Valid Assessment of Learning in Undergraduate Education (VALUE).

Stassen, Herrington, and Henderson report on an interesting activity undertaken to answer several questions regarding critical thinking definitions. They wondered what dimensions of critical thinking were emphasized by these standardized tests and measures and whether those dimensions reflected how faculty at their institution defined critical thinking. “This exploratory analysis was intended to help us understand the relevance (or fit) of each of these tools to our faculty’s priorities for students’ critical thinking development.” (p. 135)

They began by having a group of general education instructors generate an operational definition of critical thinking. The definition grew out of faculty responses to the following question and prompt: “What learning behaviors (skills, values, attitudes) do students exhibit that reflect critical thinking? Students demonstrate critical thinking when they …” (p. 128) Analysis of the instructors’ responses resulted in 12 dimensions of critical thinking: judgment/argument, synthesizing, problem solving, evidence-based thinking, drawing inferences, perspective taking, suspend judgment, application, metacognition, questioning/skepticism, knowledge/understanding, and discipline-based thinking.

Next they looked at how the four standardized tests defined critical thinking. “To understand the commonalities between the four external sources and our campus’s own critical thinking definition, we used our internal definition as the anchor definition and coded the external sources in relation to the categories present in that internal definition.” (p. 130) A table in the article presents this comparison.

Their analysis shows that “judgment/argument is the predominant component of critical thinking reflected in all of the external assessment options (accounting for between one-half to over three-quarters of all the descriptors associated with critical thinking).” (p. 133) They found “substantial emphasis” on drawing inferences and evidence-based thinking and lesser emphasis on synthesizing, problem solving, and perspective taking. But some aspects of their definition of critical thinking, such as application, suspending judgment, metacognition, and questioning/skepticism, received no emphasis in the standardized assessments. “The results suggest that all three standardized tests address a narrow set of constructs present in the campus definition, with the primary focus on judgment/argument, evidence-based thinking, and drawing inferences.” (p. 135)

This analysis was not a study of the validity of the items on the standardized assessments, but rather an exploration of how the basic construct of critical thinking was defined by the assessment tool. Furthermore, their campus definition was not assumed to be the “correct” definition. The authors note that it wasn’t systematically vetted or compared with the responses of other groups of faculty on their campus or elsewhere, although the list of dimensions identified by these general education instructors is not notably unusual.

Despite these limitations, other benefits derive from this kind analysis. Most notably it generates rich conversations about critical thinking. It helps individual faculty, collections of faculty teaching related courses (in this case general education), and institutions clarify what they mean when they say they are teaching critical-thinking skills.

Reference: Stassen, M. L., Herrington, A., and Henderson, L. Defining Thinking in Higher Education. In Miller, J. E. and Groccia, J. E., eds. To Improve the Academy, 30. San Francisco: Jossey-Bass, 2011.

Reprinted from The Teaching Professor, 25.10 (2011): 8.

Using survey research, a faculty team explored these questions. Their study builds on earlier work by J.B. Adams, and some of their findings replicated those reported earlier. Faculty respondents in the Adams study reported that in a situation where student effort was high but performance was low, approximately 17 percent of the grade should be ascribed to effort. Students in that study thought that in those circumstances, 38 percent of the grade should be based on effort. In this study, the percentages were similar. Faculty thought 13 percent of the grade should be given for effort and students thought 39 percent was appropriate. A bit surprising, though, was the fact that students were more willing than faculty to penalize students when their effort was low but the performance was high.

These overall averages were looked at more specifically in terms of the type of course: Should effort count more or less in a required general-education course, a general-education elective, a required course in the major, an elective in the major, a minor requirement, or a medical course? Faculty thought the kind of course did not make a difference. Effort should count the same regardless of the course. Students reported that course type should make a difference.

The survey also included some interesting questions that pertained to study habits, such as “For a 3-credit course, how many hours of studying a week would indicate outstanding performance?” Students said 6.73 hours (SD 4.41) and faculty said 8.53 hours (SD 3.72). They also asked about typical study patterns—cramming, weekly study and review, or daily studying. Faculty estimated that 68 percent of the students studied by mostly cramming for exams. Students chose cramming as the typical study pattern 53 percent of the time. That difference is significant. Perhaps more important is that cramming remains a common approach to study for many students much of the time. Finally, students estimated that on average students spend 14.10 hours a week studying. Faculty estimated weekly student study time at 19.10 hours. That’s another significant difference.

As for measuring effort, both students and faculty agreed that it is a difficult construct to measure. Can it be measured by class attendance? Regular participation in class? Self-reports? Both faculty and students agreed that effort is best measured by performance on assigned work. If the student is working hard, that will be seen in the work they produce and on their exam performance. Both agreed that the least accurate measure of effort are those self-reports.

“These results suggest that students and faculty may benefit from communication about grading procedures and policies, as well as [from] a frank discussion regarding what faculty consider to be ‘outstanding effort’ in a class. Students likely do not know what workload is appropriate for college-level courses and often struggle because they do not know how to direct their effort.” (p. 15)

Reference: Zinn, T.E., Magnotti, J.F., Marchuk, K., Schultz, B.S., Luther, A., and Varfolomeeva, V. (2011). Does effort still count? More on what makes the grade. Teaching of Psychology, 38 (1), 10-15.

Reprinted from The Teaching Professor, 25.9 (2011): 4.

Most of us hold deeply rooted presuppositions about grading that have rarely been confronted, and this makes sense. We became specialists in our fields without having learned a variety of grading strategies, purposes, and theories. We never had to interrogate our presuppositions about grading nor have our institutions supported us doing so. At our college, for example, we have a grading percentage chart, suggesting a range of grades might be used for a class, and a line that appears on all official course outlines stating, “Evaluation and assignment of grades will be based upon the quality of work produced relative to the objectives of the course.” This, of course, is vague enough to confound students and to allow the use of just about any grading strategy.

I began confronting my own grading presuppositions with a somewhat radical idea that I’ve now tested over three semesters, with the full approval of our Dean. During the first class of an upper-level course, I go over the course outline and grading rubric with the students. Afterward, I ask them if they will do all the work. They usually give me perplexed looks while agreeing. In response, I inform them that they will each receive an A for their final course grade. Some students loudly proclaim, for the sake of peer witnesses, they are writing it down.

In undertaking this pilot-project, basically a form of contract grading, I was required to deeply reconsider the way I understand student attainment of outcomes and my role in their meeting such outcomes. In each class that I have tried this approach, I have observed that students’ attendance, energy, intrinsic motivation, and level of work are equal to or higher than that of classes where I have used typical grading strategies. (I have had less luck with contract grading variants at lower levels.) Students are quick to provide feedback. They state, in ongoing feedback forms and in their end-of-year course evaluations, that the lack of stress regarding grades and not having to figure out ways to please the teacher really allowed them to engage, express themselves, immerse themselves in complex material without always worrying about the correct answer, and, most importantly, to learn.

This pilot-project makes some of my colleagues uncomfortable. They suggest that an average grade, a C, is good and they speak of grade inflation. Or they say I have the luxury of this project because I teach in the humanities field and this grading strategy can never work for the hard sciences. There is lots of room for discussion. The main point of objection is to bring up the hypothetical student who won’t do the work. So far, there has been no such student in these classes. If there were, I would intervene early and often, and if that failed, the contract is based upon the student’s agreeing to do the work.

Generally the arguments against giving all students an A seems to stem from a main presupposition: that all students cannot succeed at a high level, that the purpose of grading is a process of selection. The idea, when pressed, seems to be both vague and deeply held and is usually exposed by phrases like, “That’s just the way it is,” or, “All students do not have equal abilities.” The philosopher R.G. Collingwood would call this an absolute presupposition of which he wrote, “people are apt to be ticklish in their absolute presuppositions” meaning they don’t enjoy being confronted about them. Imagine if teachers were called into the Dean’s office and the conversation went something like this: The fact that many of your students are only reaching an average level of work and comprehension is a reflection of your ability to facilitate learning — what can we do to improve it? (I told you at the beginning of the article that I would be provocative.)

I am suggesting that, regardless of whether one agrees with my position or not, we all hold presuppositions about grading that affect the way we use grading to support learning. If our job is to deliver content, facilitate learning, to scaffold difficult material, and to assist all students in achieving the outcomes of our courses, then from my point of view something is wrong with what we are doing if most of our students are not achieving the top levels of comprehension. I think it’s worth thinking about, deeply.

Reference: Collingwood, R.J., (1939). Essay on metaphysics. Chicago: Henry Regnery, (p. 31).

Christopher Willard teaches at the Alberta College of Art + Design. He is currently working on his PhD in Educational Research at the University of Calgary.

A project being piloted this year at the University of Louisville School of Medicine in the Pediatric Clerkship (where third-year medical student are introduced to clinical practice) introduced students to the Four C’s of Cultural Competence in the context of an actual case study of an Amish family dealing with tetanus in their 6-year-old son. The module included diagnosis, treatment, and prevention of tetanus; and recognition of cultural issues that affected how the family viewed modern medical practice. Because the module had multiple teaching/learning objectives, we used a brief reflective writing task to assess learners’ mastery of the content. Our assessment strategy needed to balance learners’ writing skills, critical thinking, and ability to apply basic principles.

Our team of two physicians and two medical educators developed a descriptive matrix rubric that outlined five areas of writing competence: (1) organization, (2) professional presentation, (3) depth, (4) content, and (5) point of view. Each element was graded at one of three performance levels: (1) honors, (2) acceptable, or (3) needs additional training. We followed Dyrud’s (2003) line of reasoning that simplifying the grading process creates greater equity and increased student satisfaction. In addition, since we were concerned with both medical practice and cultural competence, we worked in pairs (a clinician paired with an educator) to come to consensus on the grade for each student. Each team graded 50% of the reflection papers from the six clerkships (cohorts). We shared the rubric in advance of the assignment so students had a clear description of how their work would be assessed.

In regard to the rubric, all team members agreed that it set clear expectations and put the responsibility on learners for the quality of their performance. There were very few challenges to grades, and in each case, the student was concerned that the grading was “subjective”. A face-to-face meeting between the student and the course director resolved these challenges based on the rubric notes. The downside to using a rubric, however, was that it may have stifled the creativity of some students who preferred a different style of writing and processing the material. In this situation, where respecting individual differences was a teaching point, the rubric may have worked against the content by allowing a few learners to follow the “letter of the law” to achieve high marks but miss the implications of the content.

In terms of team grading, the collaboration definitely allowed a checks and balances approach. If one member was indecisive, the other weighed in with another perspective. There was no problem with inter-rater reliability as both teams had essentially the same score distribution.

One of the unexpected outcomes of the reflective writing exercise was that faculty gained new insights into learners’ thought processes. One team member expressed it this way: “I found myself forming an opinion about each student based on my own bias about the subject matter, applauding when it was incorporated in the writing and saddened when it was lacking. I was appreciative of the rubric that was developed to allow as much objectivity in grading as possible.”

So the solution may be in having a well defined rubric but being able to apply it with discretion and sensitivity to individual learner differences. The team grading strategy was a great help in this regard because gave each of us a little more security in assessing learners who deviated from the rubrics but “made up for it” with creativity and imagination.

References:
Chretien, K.C., Chheda, S.G., Torra, D., and Papp, K.K. (2012). Reflective writing in the internal medicine clerkship: a national survey of clerkship directors in internal medicine. Teaching and Learning in Medicine, 24(1), 42-48.

Dyrud, M.A. (2003). Preserving sanity by simplifying grading. Business Communication Quarterly, 66(1), 78-85.

Kennison, M.M., and Misselwitz, S. (2002). Evaluating reflective writing for appropriateness, fairness, and consistency. Nursing Education Perspectives, 23(5) 238-242.

Slavin, S., Galanti, G.A., and Kuo, A. The 4 C’s of Culture: A Mnemonic for Health Care Professionals; available online at http://gagalanti.com/articles/The4CsofCulture.pdf

Karen Hughes Miller, PhD, V. Faye Jones, MD, PhD, Pradip Patel MD, and Michael Rowland, PhD all teach at the University of Louisville School of Medicine.

In the recent online video seminar Grading Strategies to Promote Student & Faculty Success, Anderson talked about some of the misconceptions that faculty and students have about grades and how faculty can establish a fair and supportive grading environment. Anderson, a professor of biological sciences at Towson University and co-author of Effective Grading: A Tool for Learning and Assessment in College, also acknowledged that grading typically isn’t a teacher’s favorite part of the job.

To illustrate, she told the story of a faculty member from New York whose department chair called out of retirement. It seems the institution had been unable to find someone to teach a specific course that seniors needed in order to graduate, so the department chair asked whether the retired professor would be willing to teach one more semester on a part-time basis.

The professor agreed to teach the course. Then the department chair said, rather apologetically, “Well, now I hate to bring this up, but you do know this is going to be at part-time pay. What’s the least you would take for teaching this course?”

And the professor looked right at him, and said, “Oh, I’ll teach for free,” before adding, “but you’re going to have to pay me to grade.”

“I think that’s the way that most of us feel about this process,” Anderson said.

However, given the importance of grading to the teaching and learning process, faculty need to be more strategic in how grading and assignments are used. Here are a few of the strategies Anderson shared during the seminar.

1. Rethink the big question. Ask yourself “What will students be able to know and do at the end of this course?” rather than “What am I going to cover?”

2. Stop using the word “understand.” If you list as a learning objective that, for example, “Students will understand the concept of photosynthesis,” students may gain a very surface level of knowledge that it’s something plants do and then will feel they’ve met the requirement. Anderson recommends using words from Bloom’s Taxonomy that require higher levels of cognition, such as apply, analyze, synthesize, and evaluate.

3. Identify, model and engage students in the kinds of thinking that are important to your discipline.

4. Construct a welcoming, thorough and explicit syllabus, and refer to it often. Anderson recommends printing your syllabus on colored paper so it stands out and students can find it easily throughout the semester. The tone you use is equally important.

“When I read syllabi … I have to remind myself that I’m not working for the state penal institution because they start off with threats of how students will fail, and how they will be thrown out. Those [consequences] have to be there. But first say, ‘Welcome to Biology’ or ‘Welcome to English 101.’ ‘Here are the kinds of things that you’re going to be able to do when you successfully complete this course.’”

Finding a lack of literature on the topic, two faculty researchers in marketing decided to seek answers to several interesting questions, starting with how makeup exams are typically handled by faculty. To answer that question, they collected 146 syllabi from 57 faculty members. Almost 87 percent of those syllabi listed some sort of makeup policy for assignments, primarily exams. Nearly 77 percent of faculty required students to contact them beforehand, indicating that they would be missing the exam; 76 percent required written documentation for excuses; and almost 79 percent let students miss exams only for university-stipulated reasons. Fewer than 30 percent indicated the period within which the makeup needed to be completed, and about the same percentage specified a day of week for makeup exams, with some seeking to deter them by scheduling the exams early Friday morning or late Friday afternoon.

Also of interest is how often students missed exams. Here the researchers looked at several semesters of a large course with four exams and a final. During one semester, the miss rate was 3.34 percent, and the second semester it was 2.37 percent. Almost all the absences involved medical reasons, with a few for deaths in the family, out-of-town interviews, and court appearances. The most often missed test was the fourth exam, and the least missed was the final. A bit tongue in cheek, the authors write, “Interestingly, we observed a situation where relatively large numbers of students were sick, grieving or taking job interviews at the time of the fourth exam. However, the following week, there were no makeup finals because the class was apparently filled with healthy students who postponed job interviews and were not grieving the loss of a loved one.” (p. 110)

Most faculty have suspected the legitimacy of a student-offered excuse, and most have these suspicions more or less regularly. The authors note that “for unethical students, makeup exams represent an opportunity to cheat.” (p. 105) They may discuss the exam with other students, not letting on that they have not yet taken the exam, or they may skip the exam debrief session but again ask specifically about test questions and answers, or if it’s a very large class, they may even attend the debrief hoping they won’t be noticed.

The authors also reference other survey results where 72 percent of the student sample surveyed indicated they had asked to be excused under false pretences while in college. Thirty-five percent said they had used a fraudulent excuse this semester. In this cited survey, 62 percent of the students said that fewer than 25 percent of their professors required any kind of proof for excuses. Does this encourage students to fabricate excuses?

Faculty can end up expending a good deal of time and energy trying to separate legitimate excuses from those that are not. Some students make it easy by regularly asking for makeup and extended deadlines. Other times, it’s not always easy to differentiate a real excuse from one that is fake. A good policy for missed exams needs to find a balance between making it so easy that students regularly miss exam dates and assignment deadlines and making it so draconian that those with legitimate reasons to miss are punished for circumstances beyond their control.

What’s your policy for makeup exams? Please share in the comment box below.

Reference: Abernethy, A. M. and Padgett, D. (2010). Grandma never dies during finals: A study of makeup exams. Marketing Education Review, 20 (2), 103-113.

Excerpted from A Look at Makeup Exams. The Teaching Professor, 25.7 (2011): 6.

“Assessment is not a fad that is going to be going away. Every accreditor expects not only that you push to meet the commission’s standards or the accreditor’s standards when your institution comes under review, but that assessment becomes part of the ongoing fabric of the institution. If you only work on assessment when your institution comes up for review, you’re constantly reinventing the machine. If you wait [five, six, seven, or eight years], no one will be able to find what you did before. The [lead] person will have retired, the files will have vanished, and you’ll pretty much need to start over again. But if you keep things sustainable, it’s actually much easier to get through your next accreditation review,” Suskie says.

If your institution has had some experience with assessment, it’s a good idea to ask questions such as the following to determine the current status and future of your assessment efforts:

• Is assessment an ongoing priority? Why or why not?
• Where do you want to be with assessment five or 10 years from now?
• In what ways are things going well?
• In what ways are things lagging?
• What is the institutional community doing to foster an ongoing culture of assessment?

Suskie recommends simplicity as the best approach to make an assessment program sustainable. “The more complex you make an assessment process, the more time it takes, and people get burned out. People won’t have time to work on their day jobs. You’ll want to do all you can to minimize the burden of assessment. Look for processes on your campus that can sort of do double duty. For instance, if you have a cycle of academic program review, synchronize those expectations with your assessment expectations so faculty aren’t preparing two different reports for two different ends. Being flexible is really important as is continually reviewing what you’re doing and asking, ‘Is this really helping us?’ If it’s not helping, don’t keep doing it.”

Institutional culture also plays an important role in assessment, Suskie says. “You need to understand why there’s not a pervasive culture and take appropriate steps. For example, at some institutions I’ve worked with, the problem is that a lot of faculty still don’t really understand what assessment is or why it’s useful or how to do it. And so professional development is the major obstacle. Sometimes there’s an issue with an institutional leader. A key leader hasn’t bought into this personally. The institutional leaders set the priorities, climate, and tone. And so if the institutional leader doesn’t think this is very important, then it’s hard to get people on board. Sometimes it’s a matter of time. We have schools today that have faced horrific budget cuts. Everybody is trying to do more than they ever with fewer resources.”

The HEOA requires that institutions document student achievement. Suskie offers the following advice on meeting this requirement:

• Course assessment should demonstrate that your students meet your learning outcomes.
• Make sure that learning outcomes are of appropriate rigor. “This is something that external agencies and constituents are interested in,” Suskie says. “If you’re teaching in a bachelor’s-level program, are the learning goals truly college-level goals? If it’s a graduate program, are they really graduate-level outcomes?”
• Ensure that students succeed in the pursuit of their own goals. “We want students to graduate, but you can’t focus solely on retention and graduation rates. You still need to make sure the students are learning what they’re supposed to learn and that it’s of appropriate rigor,” Suskie says.

Excerpted from Academic Leader, 27.6 (2011): 8.

The course is designed to assess the acquisition of three skills—critical thinking, written communication, and oral communication—across the 10 courses that make up the political science major. The course also aspires to “expose students to a holistic review of political science as a discipline, reviewing the broader themes that link the various subfields together; and allow students to reflect on their experience in the major and consider future applications of the major’s themes and skills to a variety of civic and professional contexts.” (p. 524)

A variety of innovative assignments are used to accomplish these goals. The primary activity is a simulated academic conference. Students select a paper they have written for one of their political science courses and prepare it for presentation in this course. The instructor organizes the papers into panels that are presented during four of the eight weeks of the course. The instructor chairs these panels and facilitates a wide-ranging discussion of the papers. The goal of the discussion is to raise questions that pertain to central issues within the field such as power, citizenship, accountability, and legitimacy. The papers are also used to assess critical thinking and writing skills. Each paper is assessed by the instructor and two randomly selected students. All three reviewers use a detailed rubric contained in the article. The instructor uses another rubric (also contained in the article) to assess the oral communication skills displayed in this presentation and in the learning-through-teaching activity described below.

Three other activities contribute more assessment data. In a course mapping exercise students rate each of the 10 major courses in terms of how well they enhanced the four key learning goals expressed in the departmental mission statement: critical thinking, written and oral communication, and understanding the discipline. Students also complete an open-ended exit survey “that asks them to anonymously and candidly evaluate the strengths and weaknesses of the program and faculty, and to make recommendations for future development.” (p. 525)

Finally, students complete a learning-through-teaching activity. For this activity, a pair of students makes a 30-minute presentation and facilitates a discussion of it with groups of about 10 beginning students enrolled in a large 100-level American government course. Students may do the presentation on a topic of their choosing, but it must contain substantive content and engage students in discussion.

Besides describing the course design, the authors also share the assessment results produced and in doing so demonstrate what valuable assessment data a course like this can produce. For example, when assessing students’ critical thinking, writing and oral communication skills, the communication skills were consistently lower than departmental expectations. In reviewing course mapping data, they discovered that students perceived only two of their major courses as enhancing their communication skills.

The authors write candidly: “The results from assessment through the capstone have illuminated both programmatic strengths and weaknesses. Maintaining the status quo on strengths is an easy task. However, taking action to address the weaknesses is a more significant undertaking.” (p. 527) To redress the oral communication deficiency, faculty members agreed to include more oral exercises in their courses, although content and class size make this difficult. It was this feedback that encouraged the development and implementation of the learning-through-teaching activity in the capstone. Departmental faculty also decided to piggyback onto a recent university general education requirement for a public speaking course.

There has been considerable faculty resistance to programmatic assessment. How will the data be collected and how will it be used? Those concerns are legitimate, but an article like this shows that collection of data can be done using viable processes and the data collected can be used to benefit students, faculty, and the program. “Using results generated by the capstone, our department is building a culture of assessment that facilitates across-the-board programmatic enhancement and boosts student learning opportunities.” (p. 528) As the authors note in the opening quote, this is an adaptable and adoptable course design model—and, we would add, not just for political science degree programs.

Reference: Sum, P. E., and Light, S. A. (2010). Assessing student learning outcomes and documenting success through a capstone course. PS: Political Science and Politics, 43 (3), 523-531.

Excerpted from The Teaching Professor, 25.6 (2011): 7.

Three-color group quiz — Students prepare for a quiz on a specified topic. Groups of four or five students assemble in class and first take the short-answer quiz individually. They write their answers in black with their books closed. Then the group collaborates by discussing questions they haven’t answered or answers about which they have doubts or need more details. After that discussion they may revise what they have written, only that information is written in green ink. Finally, the group is allowed to access textbooks, notes taken in class, and other resources. That material is added to their responses in blue ink. This approach allows students to gauge the level of their knowledge against the knowledge of others in their group and the content contained in course materials. The teacher can comment on these proportions when providing feedback on the quiz. Students reported an overwhelming preference for this approach over the traditional quiz. Most noted that they never looked up material they did not know after taking a traditional quiz. This strategy is designed so that they must.

Midterm student conferencing — The goal of these conferences is to connect with individual students, provide descriptive feedback, and review student performance so far in the course. The unique characteristic of these conferences is that students lead the conference, as in students are doing most of the talking. Several weeks before the conference, the teacher gives students the conference format and criteria. This allows them time to collect materials, reflect on their class performance, and think about what they will say. The teacher takes notes, answers questions, offers suggestions, and gives his/her perspective on the student’s performance. These midterm conferences mean that there is time for students to make changes. They also develop rapport between the teacher and students, making it more likely that students will approach the teacher with questions and concerns.

Assignment blogs — Designed to encourage communication, collaboration, and dissemination of feedback, assignment blogs can be used to “receive questions and provide feedback about certain aspects of an assignment.” (p. 140) Because they are open-access, if a student asks a good question, all students can benefit from reading the teacher’s response, and if many students have the same question, rather than repeating the answer, the teacher can give it once. Teachers can also use the assignment blogs to identify general areas of concern based on previous student work or to offer feedback to the class as a whole, thereby allowing students the chance to self-assess.

One of the authors notes that activities like these improve students’ critical-thinking skills. She writes, “You can’t just say, ‘Think critically’ and expect students to understand how to do it. The word critically often creates a negative perception of what critical thinking is all about. Instead, by thoughtfully trying to improve each other’s products, students naturally engage in the analytic and generative processes we call critical thinking. As a result, not only are student products better, but students improve in thinking and communication skills as well.” (p. 139)

Reference: Fluckiger, J., Tixier y Virgil, Y., Pasco, R., and Danielson, K. (2010). Formative Feedback: Involving Students as Partners in Assessment to Enhance Learning. College Teaching, 58, 136-140.

Reprinted from The Teaching Professor, 25.5 (2011): 3.

In the online courses at Graceland University Gleazer School of Education M.Ed program, we have a structure in place called “Critical Friends.” It works like this: Students are organized into groups of three or four students, and must submit their weekly assignments to their group prior to submitting them to the instructor. As the excerpt from the weekly participation rubric below outlines, students are expected to give critical feedback to each other as it relates to both content and structure:

Next, students submit their assignment to the instructor for further feedback, which includes both affirmation of meeting the objectives of the assignment and/or suggestions that will assist students to improve their papers.

One aspect of our program is that within each eight-week course, students create a portfolio entry comprised of individual assignments that are developed as the class progresses. In week seven, the instructor assesses students’ consistency in incorporating both peer and instructor feedback:

Assessment in each course is standardized. Fifty percent of the course grade is drawn from weekly participation and the other 50 percent is applied to the assessment of the final portfolio entry.

A student who ignores peer and instructor feedback throughout the course will find their grade greatly reduced. We have observed that the skill of giving peer feedback requires instructor scaffolding and cultivation. At the beginning of each course, instructors post an announcement that addresses participation; explaining what it looks and sounds like. With regard to giving and receiving feedback, the instructor addresses each criterion referenced on the weekly participation rubric. For example:

Giving critical feedback that is personal, relevant, and challenging — both on a student level and instructor level — has a positive impact on learning outcomes. One of the key questions on the student course evaluation inquires whether the instructor’s feedback provided assistance in improving the quality of assignments. Hence, improving our own feedback skills is a topic that is frequently re-visited in faculty development. Modeling effective feedback provides good examples for our students to observe and enhances their own peer feedback skills.
We are reminded of John Hattie’s words that the most valuable single innovation that enhances student achievement is that students receive “dollops of feedback.” Cultivating a learning environment where receiving critical feedback is invited and well received enhances the level of learning that is experienced in a class.

Reference
Hattie, J, & Timperley, H.,(2007). The power of feedback. Review of Educational Research. 77:81-112.

Debbi Leialoha, Ph.D. is the associate dean of Graduate Studies in the Gleazer School of Education at Graceland University. Shelly Leialoha, Ph.D. is an associate professor in the Gleazer School of Education at Graceland University. Sherry Leialoha-Waipa, Ph.D. is an assistant professor in the Gleazer School of Education at Graceland University.

The author proposing this idea points out how rubrics have expedited the grading process for many faculty and also clarified expectations for students, but when the paper is returned, the student gets the rubric with a check next to quality level attained and maybe a few brief remarks squeezed into a small space provided for comments. What this assumes is that students will look at their paper and see why it merited that particular quality rating. That assumption is questionable, based on student levels of skill and their motivation to attend to feedback.

What the author has done is create a large collection of detailed comments that he imports into the grading rubric. He doesn’t show students all the levels—they see those when the rubric is distributed at the time the assignment is made. They see the level their assignment has been given and then a detailed set of comments that explain why that level was earned and how the student can improve for a higher level on the next assignment.

It may take a while to develop the collection of comments, but you can start using them before the collection is complete. The quality of these comments can be significantly higher than those we dash off after a full day of teaching, cleaning up the kitchen, and helping the kids with homework. They can be prepared and revised when we aren’t tired. Once the collection gets large enough, comments can be categorized, and any given comment may exist in several different versions. The author categorizes according the levels that appear on the rubric. So, if the assignment meets the top criteria, he has a collection of top-criteria comments he can make. The author recommends storing comments in an Excel spreadsheet.

What if students figure out they are getting “canned” feedback? Many are already inclined not to pay much attention to our careful comments. Wouldn’t the fact the comments aren’t written exclusively for them give them an excuse to ignore the feedback even more thoroughly? Technology makes it easy to personalize any comment. You can use the student’s name, insert an example pulled from their assignment, or think of the comment as a canned shell that you can slightly revise as you use it. All of a sudden, the feedback is personal. The author maintains his students never figured out they were getting “canned” comments.

This approach may not be for everyone, but with so much on our plates, we need to be open to time-saving possibilities. The author of the article referenced below was able to document some positive impacts on student work and attitudes with the system of automated comments he developed.

Reference: Czaplewski, A. J. (2009). Computer-assisted grading rubrics: Automating the process of providing comments and student feedback. Marketing Education Review, 19 (1), 29-36.

Reprinted from Expediting Feedback to Students. The Teaching Professor, 25.4 (2011): 4.

Smith decided to use five “tactical strategies” (p. 72) in his junior-level financial management class. First he gave students access to previous exams. He put two semesters’ worth of exams on reserve in the library. They were exam copies minus any answers. Part of this was a fairness issue. Greeks on his campus collected previous exams—it didn’t seem fair that non-Greek students had no access to those exams. More important, having access to the exams relieved a lot of anxiety students had over the format, style, and difficulty of the exam. The downside of this strategy is that it forces the instructor to write new questions every semester. That is easier in some content areas than in others.

Next, Smith conducted a review session prior to each exam. He scheduled the two-hour sessions the evening before the exam. Students could come and go at their leisure—between 80 and 90 percent of the students attended the session. The decision to schedule the session the night before the exam was based on the assumption that students would have already devoted time to study. Smith provided correct answers to the exam questions during the session. Most of the students had already tried to work the problems, and so they came with questions. “The review session provides a wonderful teaching opportunity in that students are very attentive. In other words, it is a prime learning opportunity.” (p. 74)
The sessions did not take place in the regular classroom, and Smith found that made for more open dialogue.

Students were allowed to use a cheat sheet during the exam. Specifically, it was a 5×7 hand-written card. Most students filled their cards with definitions, formulas, and instructions for solving particular kinds of problems. Being able to use a cheat sheet got the message across to students: they didn’t need to memorize the material. Smith says that the stress-relieving effects of the cheat sheets were “one of the most gratifying unforeseen consequences.” (p. 76) Coupled with having access to prior exams, this allowed students to come to the exam much more focused on the material. Interestingly, Smith observed many of the students rarely looked at their cheat sheets. When they did, it was to quickly check something. The process of preparing the cheat sheet seemed to have helped students organize and remember the material.

Smith’s exam questions require an answer with justification. The exams contained 25 to 30 multiple-choice questions. Students selected the correct answer, but then they had to provide a written justification for their choice, and it was that written justification that was evaluated, not what the students had circled. The practice virtually eliminated guessing because students who had the correct answer marked but provided an erroneous or irrelevant justification could get only partial or no credit for the answer. Likewise, if an incorrect response was supported with a reasonable justification, it could earn partial or full credit.

And finally and most innovatively, Smith individually graded each exam with the student present. This took place in a 15-minute appointment scheduled during the week after the exam. The instructor and the student sat down together at a table and proceeded through the exam. The discussion was easy when the answer and justification were correct. The discussion was not as easy if the answer was correct but not the justification. Smith reported that he spent time listening carefully as students re-explained their thought processes. He learned much about students’ thinking processes and could more easily identify the problematic assumptions they had made. Clearly, this was a time-consuming process. Because he didn’t spend any class time talking about the exam, Smith canceled one class session and used the time for individual appointments. He used this technique in classes with 25 students.

Smith thinks the impact of each individual strategy is enhanced when all of them are used. “Each strategy is a cog in a larger system, and there are many interdependencies among the various tactics.” (p. 81) When Smith uses all five strategies, “the first noticeable systematic effect is that students are willing to work harder.” (p. 82) The strategies also help build trust between the instructor and students. It’s a way of using exams that makes students more accountable and lets them experience how much learning an exam can promote.

Reference: Smith, T. “Exams as learning experiences: One nutty idea after another.” In R. J. Mezeske and B. A. Mezeske, eds., Beyond Tests and Quizzes: Creative Assessments in the College Classroom. San Francisco: Jossey-Bass, 2007.

Reprinted from The Teaching Professor, 25.2 (2011): 5.

A biology professor reports on his experience using research articles and asking test questions about them in an undergraduate course for students majoring in life sciences. Students were assigned a research article to read—the article was relevant to content being covered in class. It was posted on an accessible website. Sometimes the article was discussed during the lectures and sometimes it was the topic of a tutorial session (these were large classes that included tutorial sections). Either way the students had access to the articles before and during the assessment activity.

The students were then given test questions on these articles. The questions were set at three levels on the Bloom taxonomy. Questions at level one were straightforward, testing students’ scientific literacy and conceptual understanding. Questions at level two focused on students’ abilities to link prior knowledge or textbook content to material in the research article. The goal was to see whether students could correlate different components and understand scientific reasoning. At level three, the questions asked students to link the research content to daily life—integrating it with their current knowledge and applying it in a creative way. Three different research articles were assigned and the test on each counted 10 percent, for a total of 30 percent of the course grade.

An elaborate system used to evaluate student responses revealed that students had read, understood, and were able to write about the research articles. A majority of the students were even able to correctly answer level three type questions. And students responded favorably to this approach. They felt it positively affected their motivation in the course and showed them interesting and relevant aspects of the content. “The results showed that the approach strongly motivated students to step out of their comfort zone (textbook) and to develop high-order cognitive skills, including correlation, application, and synthesis.” (p. 289)

A good deal of the success of this approach can be attributed to the criteria used to select the research articles. The author notes that finding suitable articles was a “major challenge” in developing this approach (p. 284). But the criteria used help to explain the success of this assessment strategy. Articles had to meet four criteria. First the article had to be relevant. It had to link with the content being covered in the course. Second, it had to be interesting. It had to address some topic that would capture students’ curiosity. It could be addressing a question so far unanswered or on some controversial issue. Third, the article needed to be comprehensible. Students had to be able to understand it, or at least most of it. It could not be overly complicated as these students had little (if any) previous experience reading research material. And finally, it had to be heart stirring—the author’s way of saying it had to be an impressive piece of work, something that would inspire students and set high expectations for their future work as scientists.

Reference: Wu, J. (2009). Linking assessment questions to a research article to stimulate self-directed learning and develop high-order cognitive skills in an undergraduate module of molecular genetics. Cell Biology Education—Life Sciences Education, 8 (Winter), 283-290.

Reprinted from “An Assessment Technique Using Research Articles.” The Teaching Professor, 24.10 (2010): 8.