Summarizing and Using Assessment Results

Accreditation agencies and others are increasingly requiring institutions not only to assess student learning and institutional effectiveness but to then use the results to improve teaching, learning, programs, and services. This seminar discusses how to share assessment results in meaningful ways and use the results to improve programs and services.

You put a lot of hard work into creating student assessments. And then what? Are you sharing your results? Celebrating your successes? Using your findings to improve methods, programs and services?

With all the time spent developing and administering assessments, it’s a shame if you don’t actually do something with all that valuable data you’ve collected. Even disappointing outcomes can lead to improvements if they cause you to tweak the curriculum, or explore new teaching methods.

View a brief clip from the seminar:

In Summarizing and Using Assessment Results, Linda Suskie, VP of the Middle States Commission on Higher Education, provides a wealth of ideas on how to share your assessment results in meaningful ways, and use them to continuously improve programs and services.

Order the CD + transcript package

During this presentation, you’ll learn how to:

• Identify the true purpose of your assessments.
• Create and share assessment summaries that are useful to you and your colleagues.
• Find the story in your assessment results, and tell that story effectively.
• Focus assessment summaries on decisions that the results can inform.
• Celebrate and publicize good assessment results.
• Analyze the possible causes of disappointing results: goals, programs, pedagogies, or assessments.
• And much more.

In short, you’ll discover smart, practical ways to ensure that the time and effort you put into assessments translate into meaningful opportunities for growth, change and improvement.

When you order the recording of this seminar on CD, you’ll also receive a PDF of the transcript, all for the same price as attending the seminar live.

An optional Campus Access License is available for an additional $200. It allows the purchasing institution to upload the CD of the seminar onto the institution’s password-protected internal web site for unlimited access by members of the campus community.

Who will benefit:

• Chief academic officers
• Deans
• Assistant/associate provosts/vice presidents for academic affairs
• Assessment coordinators/directors
• Assessment committee members
• Institutional research directors
• Student development staff
• Academic administrators
• Faculty governance leaders
• Department chairs

You work hard to create assessments…make sure they work hard in return. Discover how to use them to benefit your institution, your students and you!

Order the CD + transcript package

All seminars include a discussion guide for facilitators
Participating in a Magna Online Seminar as a team can help leverage unique insights, foster collaboration, and build momentum for change. Each seminar now includes a Discussion Guide for Facilitators which provides step-by-step instructions for generating productive discussions and thoughtful reflection. You’ll also get guidelines for continuing the conversation after the event, implementing the strategies discussed, and creating a feedback loop for sharing best practices and challenges.

I wondered whether allowing multiple test attempts was an educational folly. But for 13 percent of the course points, I decided it was worth investigating patterns of student usage and performance before dismissing a new idea.

Of the 20 students enrolled in the course, multiple trials (at least two) were utilized by 13 (65 percent) in the first quiz, 12 (60 percent) in the second quiz, eight (40 percent) in the third quiz, and seven (35 percent) in the fourth quiz. When comparing the average first attempt to the average multiple trial score, I found variation in student performance. Students improved their scores by an average of .71 points (close to a full letter grade) with multiple attempts on the first quiz versus a negative result (-.03) on the third exam, and two smaller gains for quizzes 2 (+.23) and 3 (+.24).

The syllabus also gave students an option of stopping if they liked their scores on the first attempt. Using a 10 or 9 (the lowest A performance for the course) as a cutoff score, I found that nine (45 percent) students stopped with a “liked” score in the fourth quiz compared with three (15 percent) students in the first quiz. In the first quiz I glimpsed a gamesmanship strategy I found fascinating: a student who earned 8 points in the first attempt and scored the maximum in the second attempt. I thought the student would have “held” with an average score of 9. But the student used the third trial to score 10 points again, raising the average to 9.33. In the second and fourth quizzes, I saw a magnification of this effect: multiple 9/10/10 patterns. Rather than holding with a 9 “A-hand,” a student decided to draw two other quiz cards to move to a 9.67, a stronger A.

The final exam had been designed along a more traditional format: a onetime shot during a time limit of 90 minutes within an 11-hour window. After several polite student requests for an extended time because of multiple exams, I decided to use the same multiple-attempt format, allow an additional 30 minutes to take the exam, and lengthen the time window to 25 hours. I was pleased to find that students who requested additional time available took the exam within the extended testing window. As for multiple final exam attempts, only three (15 percent) students took advantage of the opportunity; they earned an average improvement of 6 points (half a letter grade).

I ultimately decided to allow multiple final exam trials when my inquiry toward the end of the course showed a spread in student performance. After the final, student performance variation continued. Five (25 percent) students who tested well (scored high and/or took advantage of multiple attempts) and submitted strong individual projects (55 percent of the grade) earned top grades. Of the other 15 students, I estimate 13 (87 percent) could have earned at least a half grade higher by taking advantage of multiple test trials. I was amazed at the many missed opportunities.

An interesting benefit came after the semester ended when a “grovel” email arrived from a student who requested rounding up her percentage to the next grade. In showing I was unable to replicate her higher percentage, I pointed out that of five testing situations in which she had multiple attempts available, she had taken advantage only two times. As I looked more closely at the student’s online pattern, I found an irregularity: the student had an open final exam first attempt with scores for second and third trials. I felt comfortable indicating that the student could have earned a better grade, but there were many lost opportunities. The biggest benefit of multiple testing attempts came when the course evaluation report showed high student ratings. The goodwill generated by the security of having additional testing opportunities available was viewed positively by students.

Despite my initial skepticism, my one-course investigation of student usage and performance provides compelling evidence for applying multiple attempts to some testing situations in the future. A final exam might not be the best option in some courses, but using multiple-trial testing worked well as an instructor-pacing method. Even if students decide not to take tests more than once, multiple test trials provide a learning opportunity.

Aimee J. Luebben, EdD, OTR, FAOTA is a professor of occupational therapy at the University of Southern Indiana.

Excerpted from Offering Multiple Test Trials: Educational Folly or Learning Opportunity? November 2008, Online Classroom.

At the same time, however, using an electronic grade book took something that had previously been my private domain and opened it up for every student to inspect. This aspect of the technology challenged me to rethink some of my teaching policies. If I wanted to give credit for attendance and participation (and I did), I needed to make sure that I was calculating that in a way that could stand up to direct student scrutiny. Instead of recording a series of cryptic checks, dashes, and check “pluses” into a black book, I had to create a more quantifiable and defensible system for determining these kinds of grades. I soon found that the transparency of an open grade book fostered a healthy level of instructor accountability in my classes.

For students, the open grade book made it easier to follow their performance over the course of the semester, and it opened a new avenue for teacher-student dialogue. At midterm, for example, when students received their grades, they could compare daily and weekly scores with a letter grade summary to see how it all added up. They could ask questions or contact me if they thought a mistake had been made. Being able to review and correct entries seemed to increase my students’ confidence in the grading process. They recognized my good-faith effort to handle grading in a consistent and reliable way.

Online grade books appeal to students
After a couple of years of using an online grade book, I decided to survey students about their perspectives. Over two semesters, I collected anonymous responses from 71 students in three different sections of the same course. Only 28.1 percent of these students were using an online grade book for the first time in my class, so the majority had taken other courses in which teachers used this technology. I was curious how frequently they checked their grades online. I found that 58 percent of the students surveyed checked their grades a couple of times a month, and 20 percent answered that they checked them a couple of times a week. Only one student reported ignoring the grade book completely.

When asked about grade accuracy, 76 percent said they never found any mistakes in the recording of their grades, but 7 percent found mistakes without reporting them and another 15.5 percent said that they reported mistakes and I subsequently corrected them. I do record attendance and daily homework assignments, so there are many opportunities for mistakes or misunderstandings (students who come to class after I check attendance or hand in homework without their names on it, for example). Learning that 16 students out of 71 experienced some kind of error in the grade recording process was somewhat alarming, but it also confirmed the value of the transparency that an electronic grade book provides.

I was encouraged that 85.9 percent of my students agreed or strongly agreed that “[t]he online grade book gave me a clearer understanding of my status in the course this semester.” In a section left open for additional comments, students reported that they appreciated being able to track their progress in the course and having a clearer picture of their performance throughout the semester.

Based on these survey responses and my own experiences with the online grade book, I’ve come to believe that this electronic tool is more than a convenience. It has created a new level of accountability and student-instructor communication in my classes. The technology is easy and it offers many advantages while requiring no more effort than does a traditional paper grade book. This transparent approach to grade recording seems to be one digital innovation that can be recommended to faculty wholeheartedly.

Tom Schrand, PhD., is the associate dean of the School of Liberal Arts and the head of the Environmental Sustainability program at Philadelphia University.

Excerpted from Online Grade Books: Surprising Accomplishments, April 2008, The Teaching Professor.

The end of the spring semester correlates with increased absences and assignment apathy. The weather is beautiful, my classes are in the afternoon, and student attendance drops. In addition, students in my classes are preservice teachers who must do a minimum number of field observations in area schools before the end of the semester. Those who have procrastinated start feeling the crunch and begin to miss class in order to complete the required number of hours. Those attending class often arrive unprepared. Clearly, this is not the easiest time of the year for teaching.

In a mathematics class for prospective elementary teachers, we had been working on a particular section for several class sessions, so students had more time than usual to complete the homework assignment. On the day this homework was to be discussed, I decided to offer a bonus activity. I created a sheet with 11 problems that applied many of the concepts we had covered in previous class sessions.

Students could earn one point for each problem solved correctly. The problems had to be worked out during the allotted class time, and students could not begin working until a trade had occurred—the bonus sheet in exchange for completed homework. This trade made the students accountable for previously assigned work and removed my fear of giving them something for nothing. Students who had not completed the assignment had less time for the bonus opportunity because they had homework to finish up first.

An interesting classroom dynamic occurred after I explained how this bonus opportunity worked. Many of the students with their homework done began helping students who had not been able to work through all the homework problems. Students who had not even started the homework began to work diligently in order to have even a little bonus time. As I walked around the room, I heard not only the buzz of mathematics but also comments like “I told Julie she shouldn’t miss class” and “I knew I should’ve done my homework!”

I want students to be successful in and out of the classroom. This means learning the mathematics we’re covering in the course. But I also want students to realize they are ultimately responsible for their own learning and accountable for their actions. The bonus problems reviewed concepts that the students needed to know and understand. By design, the activity reinforced the responsibility of students to complete assigned homework. Since the only students who received few or no points were the students who missed class or had not completed the homework assignment, the lack of bonus points earned was not the fault of the teacher (e.g., test too hard, too long) but rather the consequence of a personal decision.

The bonus activity was a success and is a practice I’ll repeat. My students were delighted with the opportunity, and I was guilt-free. The activity let students know that I am sensitive to their needs and ideas, but it also showed how a missed class is a missed opportunity—and that doing your homework pays off!

Tena Long Golding, is an associate professor of mathematics at Southeastern Louisiana University.

Excerpted from Bonuses of a Bonus Assignment! The Teaching Professor, June-July 2008.

Typically, this means that institutions file reports that show the institution to be in compliance with certain standards as demonstrated by statistical assessments. (Remember that the art of statistics is the ability to draw a perfectly straight line from a faulty assumption to a fallacious conclusion.) Who could be opposed to “accountability,” a term as revered as “Mom” and “apple pie”? But dig deeper and some might wonder: Who is accountable to whom for what?

It might be helpful for academic leaders to reflect on such questions. It seems to me that the concept of educational accountability has morphed through several murky, even mysterious, stages, in less than a straight line from assumptions to conclusions:

Stage #1—The Parent is accountable. One of the earliest educational laws in colonial America was a statute in 1642 in Massachusetts, declaring that “the great neglect of parents … in training up their children in learning” could result in the court assessing fines to said parents. Churches also took on a major role in ensuring that students were accountable for certain values in the young. Only after the development of schools in the 18th century did accountability for student learning shift elsewhere.

Stage #2—The Student is accountable. Once students became the charges of public schools, the expectation was that they would follow the established curriculum, standards of behavior, and evaluation processes. Parents would get report cards showing progress, and if a student was punished in school, the parents would follow suit at home. Such accountability concepts assumed that teachers, administrators, and school board members would make the rules and issue the judgments.

Stage #3—The Teacher is accountable. By the middle of the 20th century, testing of students was joined by testing of teachers. Certification of teachers by states often mandated successful passing scores on such standardized tests as the National Teacher’s Exam (now itself morphed into the PRAXIS exams developed by the Educational Testing Service). By this time, state departments of education, other governmental agencies, and accrediting bodies were requiring teachers to be accountable for their professional and academic knowledge and performance. State dismissal laws specified “just cause” for firing “incompetent” or “unfit” teachers.

Stage #4—The Institution is accountable. This is our current stage, and it extends from the pre-K level through higher education. Today the entire educational system is being held accountable, not merely for the quality of its faculty, curriculum, and facilities (so-called inputs) but also student learning, behavior, and success—in the school and beyond. To enforce this latest form of educational accountability, institutions have developed elaborate planning and assessment mechanisms and face increasing levels of inspection to see if “student learning outcomes” have been achieved. If not, accreditation, funding, and reputation are in jeopardy. And now the federal government is itching to make colleges accountable to it.

So, academic leaders, what is next? I have no idea. Dealing with the responsibilities of institutional accountability for learning outcomes keeps me too busy to speculate on the future of this “conundrum” for administrators.

What future do YOU see for accountability in higher education? Please leave your comments below.

Thomas R. McDaniel is a professor of education, senior vice president, and acting dean of graduate studies at Converse College in Spartanburg, S.C.

Excerpted from Parting Shot: The Accountability Conundrum, Academic Leader, Feb. 2007.