Summarizing and Using Assessment Results

Accreditation agencies and others are increasingly requiring institutions not only to assess student learning and institutional effectiveness but to then use the results to improve teaching, learning, programs, and services. This seminar discusses how to share assessment results in meaningful ways and use the results to improve programs and services.

You put a lot of hard work into creating student assessments. And then what? Are you sharing your results? Celebrating your successes? Using your findings to improve methods, programs and services?

With all the time spent developing and administering assessments, it’s a shame if you don’t actually do something with all that valuable data you’ve collected. Even disappointing outcomes can lead to improvements if they cause you to tweak the curriculum, or explore new teaching methods.

View a brief clip from the seminar:

In Summarizing and Using Assessment Results, Linda Suskie, VP of the Middle States Commission on Higher Education, provides a wealth of ideas on how to share your assessment results in meaningful ways, and use them to continuously improve programs and services.

Order the CD + transcript package

During this presentation, you’ll learn how to:

• Identify the true purpose of your assessments.
• Create and share assessment summaries that are useful to you and your colleagues.
• Find the story in your assessment results, and tell that story effectively.
• Focus assessment summaries on decisions that the results can inform.
• Celebrate and publicize good assessment results.
• Analyze the possible causes of disappointing results: goals, programs, pedagogies, or assessments.
• And much more.

In short, you’ll discover smart, practical ways to ensure that the time and effort you put into assessments translate into meaningful opportunities for growth, change and improvement.

When you order the recording of this seminar on CD, you’ll also receive a PDF of the transcript, all for the same price as attending the seminar live.

An optional Campus Access License is available for an additional $200. It allows the purchasing institution to upload the CD of the seminar onto the institution’s password-protected internal web site for unlimited access by members of the campus community.

Who will benefit:

• Chief academic officers
• Deans
• Assistant/associate provosts/vice presidents for academic affairs
• Assessment coordinators/directors
• Assessment committee members
• Institutional research directors
• Student development staff
• Academic administrators
• Faculty governance leaders
• Department chairs

You work hard to create assessments…make sure they work hard in return. Discover how to use them to benefit your institution, your students and you!

Order the CD + transcript package

All seminars include a discussion guide for facilitators
Participating in a Magna Online Seminar as a team can help leverage unique insights, foster collaboration, and build momentum for change. Each seminar now includes a Discussion Guide for Facilitators which provides step-by-step instructions for generating productive discussions and thoughtful reflection. You’ll also get guidelines for continuing the conversation after the event, implementing the strategies discussed, and creating a feedback loop for sharing best practices and challenges.

Using that definition as a common starting point, Shibley led seminar participants through a five-step process for renewing program-level learning outcomes assessment.

Step 1: Create a team – Determining who to involve in the assessment process is a critical first step, and Shibley recommends creating a team that includes those who care about learning, faculty who need scholarship, both junior and senior faculty, and possibly students.

Once the team is assembled, you need to assign a point person to lead the team. You should also provide training, so that each member understands the process. Finally, given that participation on an assessment team is an additional responsibility to an already full faculty plate, it’s always a good idea to find a way to recognize and reward team members for their contributions.

Step 2: Identify issues, problems and needs – This is where you begin the conversation about assessment, clarifying assumptions along the way. Find out how faculty in your program define student learning outcomes assessment, and what they’re currently doing. Take the time to revisit the key elements of your assessment process, review prior reports, and determine specific strengths and challenges.

Step 3: Select opportunities – During this stage you want to prioritize issues for each phase of the assessment cycle. The cycle includes defining outcomes, establishing criteria, collecting evidence, interpreting results, and taking action.

Step 4: Develop solutions – Shibley uses a learning opportunities matrix to provide an action-plan framework for addressing the deficiencies and priorities identified through the previous steps. As you begin to develop solutions, it’s important to communicate to those outside of the team what you’ve discovered and how you plan to tackle each issue.

Step 5: Update the process – The final step of the assessment renewal process is the implementation phase, where you take everything you’ve learned about what needs to be improved in your assessment processes, and make it happen.

“It’s important to think about program-level learning outcomes assessment as much more than simply meeting the demands of accountability,” says Shibley. “It’s really about quality, and about focusing on student learning, and looking for strategies to help you improve student learning.”

Many institutions have made incremental progress in moving away from an instructional model that views learning as a passive, receptive act on the part of the student, a model that favors competition over cooperation, individual achievement over collaboration, and divisiveness and control over individual differences and choice. We talk about developing learner-centeredness at our institutions that is characterized by a new focus on active learning, collaboration, and engagement. The focus, however, has been almost exclusively on what the faculty need to achieve. Little has been said in regard to the role that academic leaders need to play to foster a true, comprehensive, systemic shift in paradigms.

The term paradigm shift was originally used by Thomas Kuhn in The Structure of Scientific Revolutions (1962) to refer exclusively to scientific theory. Since then its use has become more generic, referring to radical changes in thought that require individuals to completely re-envision systems or organizations. We tend to use the term so generically now that we lose sight of the magnitude of the concept. To us, the word shift makes the challenge of radical change seem too easy, like shifting gears on a bicycle. Shifting gears on bicycles allows riders to maintain their cadence uninterrupted as the terrain becomes more difficult. This is most definitely not how shifting paradigms works. Our cadences will be interrupted. Shifting paradigms is unbalancing and unsettling because it is about shifting thinking and attitudes. It is an organizational metamorphosis requiring all parties to change, to alter our cadences in response to the new landscape or else fall off our bikes. The shift is not exclusively about classroom practices, and academic leaders have an important role to play in bringing it about.

There are two fundamental concerns for leadership in this enterprise. First is the need to transform administrative approaches to be consistent with the values of the new paradigm. Second is for leadership to lead the way by encouraging, promoting, and supporting the learner-centered agenda, ensuring that policies and practices do not impede progress in order that a true learner-centered institutional culture becomes a reality. This process will be a personal challenge as well as an institutional one. It will call for leaders who can envision the goal while implementing practices that will drive the change in very practical, identifiable ways.

In our roles as academic leaders, we need to take steps to foster and even push the shift toward learner-centeredness, guiding the efforts of faculty who are making attempts to transform their practices and providing support to encourage change. But to do so we need mechanisms to assess our current academic environment in order to have a clear understanding of where we are and the steps that will be involved in making progress toward the ultimate goal.

Great strides have already been made in assessing features such as student engagement, one of the features of the new paradigm. The National Survey of Student Engagement is now used by 610 campuses to help them assess good practices in undergraduate education. Other features of the learning-centered paradigm do not have large-scale assessment mechanisms readily available. Until that time, individual efforts at developing assessment of learning-centeredness are necessary. Toward that end, we have developed a mechanism for assessing the degree of learner-centeredness in a unit/department using course syllabi and a rubric that we developed for this purpose.

Right now, if asked about the state of learner-centeredness in a department or unit, we can usually point to individual faculty members who are making significant changes in teaching practices and experimenting with innovative strategies. We may also be able to point to new technology or new policies that show progress toward making the shift, but we rarely have data that clearly delineates department/unitwide the areas of success or areas of need when it comes to the distinctive features of learner-centered pedagogy.

Excerpted from Assessing the Degree of Learner-Centeredness in a Department or Unit, Academic Leader, April 2009.

This is very unfortunate for a number of reasons. First, the most proficient practitioners of learning outcome assessment are the faculty themselves. Second, faculty members are the designers and developers of curricula, courses, and the core of higher education itself. Third, faculty members are the ones who will innovate and develop new methods of teaching and learning, and will implement any changes based on assessment results developed through the program’s assessment methods.

It’s fashionable lately to try to motivate faculty through statements like “If we don’t do assessment ourselves, someone else will do it for us.” This gets attention at times, but it misses the whole point; we do assessment already, because it is useful to us. If we do it better, it will be more useful and valuable.

Step one is to recognize what is already happening, build a structure for it, and make that structure transparent. Step two is to build on the work that already is being done and focus it into the areas of most importance to the faculty in assuring that their students know, value, and do what the faculty intend. Getting faculty and departments to realize this, and act on it, is of utmost importance. Assessment structures can be imposed, mandates can be made, and penalties for noncompliance assessed from outside the program, but this is a dead end. If program assessment is ever to be taken seriously and used effectively, it must be systematic and built into the very structure of the program or department by the faculty themselves.

How does a department chair motivate faculty to participate in the construction of a systematic and transparent program assessment structure? First, by making the point that assessments already are being conducted, but in a way that is not as useful as it could be. Second, by showing that a systematic approach to program assessment has value to the department, and not just because it helps the students, but because it helps faculty have more valuable, meaningful, and successful teaching experiences.

This second step requires departmental leadership—sustainable leadership—and the role of the chair is the largest obstacle to the endurance of useful program assessment. In many universities, the chair’s position is short lived. A motivated chair must build institutional structures within the department that will outlive her or him. These involve the following:

• A clear message on the purpose of assessment
• Making assessment methods simple and useful
• Making assessment collaborative, collegial, and cooperative
• Having an incentive structure that rewards useful assessments and the scholarship of teaching and learning
• Making connections with alumni
• Making connections with the community

Thinking seriously about what students should know, value, and do allows faculty members the opportunity to think about their own practices, their own work. This not only can lead to a renewed interest in their own department and its curriculum, but can help them refocus their own attention on things that really matter to them in their research and their service. Most important of all, it can reignite the passion for teaching in many by treating research on teaching and learning as valuable and meaningful for the department’s health.

Gary A. Gigliotti is associate vice president of academic affairs for teaching and assessment and director of the Center for Teaching and Advancement and Assessment Research at Rutgers University-New Brunswick.

Excerpted from The Faculty and Program-Wide Learning Outcome Assessment, February 2009, Academic Leader.