Moving Ahead with Learning Assessment

Assessments draw a lot of criticism—and sometimes ire—from just about every constituency. There are always some students, professors, and administrators who think assessments are at best a nuisance and at worst a total waste of time.

If your campus treats assessments merely as a necessary evil, you are missing out on a valuable opportunity to improve student learning, make better budgeting decisions, refine goals and strategic operations, and demonstrate accountability. Assessments can do all that and more. Really, they can.

Find out how to get the most out of your assessments during the online seminar, Moving Ahead with Learning Assessment, a 90-minute online audio seminar that will help you review your assessment efforts to identify what has worked, what hasn’t, and why. You’ll also learn what you can do to get better results next time.

The seminar is led by national assessment expert Linda Suskie, a vice president at the Middle States Commission on Higher Education and author of Assessment of Student Learning: A Common Sense Guide. She understands assessment resistance, and yet she knows that smart, efficient, and effective assessments can convert even the most ardent critics.

Here’s a brief clip from the seminar:

In Moving Ahead with Learning Assessment you will learn to:

• Simplify assessment tools and strategies;
• Realistically assess “fuzzy” and “sticky” goals;
• Determine true costs and benefits of assessment activities;
• Sync assessments at various levels to maximize efficiency and minimize the intrusion;
• Review and improve how assessments are used;
• Identify and eliminate ineffective assessments;
• Identify audiences for assessment results;
• Identify tools to keep assessments sustainable; and
• Understand the requirements of the 2008 Higher Education Opportunities Act regarding learning assessment.

Order the CD + Transcript Package

Suskie knows that assessments can stagnate, which causes resistance to build. She helps you assess your assessments and make the changes necessary to keep the tests and the results relevant to your school, your students, and your goals. Getting more usable data doesn’t always require doing more assessments. It usually requires doing more with the assessments you already use. Suskie will show you how.

When you order the recording of this seminar on CD, you’ll also receive a PDF of the transcript, all for the same price as attending the seminar live.

An optional Campus Access License is available for an additional $200. It allows the purchasing institution to upload the CD of the seminar onto the institution’s password-protected internal web site for unlimited access by members of the campus community.

You don’t have a choice about assessments. You have to do them. But set your expectations too low, and you’ll miss valuable opportunities to demonstrate accountability, to celebrate successes, and to make vast improvements in student learning. You don’t want your students to slide by doing the bare minimum that is required. You shouldn’t tolerate that of your assessments, either. You can get more out of them and this seminar will show you how.

Who will benefit:
As with all other Magna seminars, this results-oriented presentation focuses on an issue important to higher education officials. You will finish with ideas you can immediately implement within existing programs.

This seminar is recommended for chief academic officers, deans, assistant/associate provosts or vice presidents for academic affairs, assessment coordinators/directors, institutional research directors, student development staff, academic administrators, faculty governance leaders, and department chairs.

Order the CD + Transcript Package

All seminars include a discussion guide for facilitators
Participating in a Magna Online Seminar as a team can help leverage unique insights, foster collaboration, and build momentum for change. Each seminar now includes a Discussion Guide for Facilitators which provides step-by-step instructions for generating productive discussions and thoughtful reflection. You’ll also get guidelines for continuing the conversation after the event, implementing the strategies discussed, and creating a feedback loop for sharing best practices and challenges.

Many institutions have made incremental progress in moving away from an instructional model that views learning as a passive, receptive act on the part of the student, a model that favors competition over cooperation, individual achievement over collaboration, and divisiveness and control over individual differences and choice. We talk about developing learner-centeredness at our institutions that is characterized by a new focus on active learning, collaboration, and engagement. The focus, however, has been almost exclusively on what the faculty need to achieve. Little has been said in regard to the role that academic leaders need to play to foster a true, comprehensive, systemic shift in paradigms.

The term paradigm shift was originally used by Thomas Kuhn in The Structure of Scientific Revolutions (1962) to refer exclusively to scientific theory. Since then its use has become more generic, referring to radical changes in thought that require individuals to completely re-envision systems or organizations. We tend to use the term so generically now that we lose sight of the magnitude of the concept. To us, the word shift makes the challenge of radical change seem too easy, like shifting gears on a bicycle. Shifting gears on bicycles allows riders to maintain their cadence uninterrupted as the terrain becomes more difficult. This is most definitely not how shifting paradigms works. Our cadences will be interrupted. Shifting paradigms is unbalancing and unsettling because it is about shifting thinking and attitudes. It is an organizational metamorphosis requiring all parties to change, to alter our cadences in response to the new landscape or else fall off our bikes. The shift is not exclusively about classroom practices, and academic leaders have an important role to play in bringing it about.

There are two fundamental concerns for leadership in this enterprise. First is the need to transform administrative approaches to be consistent with the values of the new paradigm. Second is for leadership to lead the way by encouraging, promoting, and supporting the learner-centered agenda, ensuring that policies and practices do not impede progress in order that a true learner-centered institutional culture becomes a reality. This process will be a personal challenge as well as an institutional one. It will call for leaders who can envision the goal while implementing practices that will drive the change in very practical, identifiable ways.

In our roles as academic leaders, we need to take steps to foster and even push the shift toward learner-centeredness, guiding the efforts of faculty who are making attempts to transform their practices and providing support to encourage change. But to do so we need mechanisms to assess our current academic environment in order to have a clear understanding of where we are and the steps that will be involved in making progress toward the ultimate goal.

Great strides have already been made in assessing features such as student engagement, one of the features of the new paradigm. The National Survey of Student Engagement is now used by 610 campuses to help them assess good practices in undergraduate education. Other features of the learning-centered paradigm do not have large-scale assessment mechanisms readily available. Until that time, individual efforts at developing assessment of learning-centeredness are necessary. Toward that end, we have developed a mechanism for assessing the degree of learner-centeredness in a unit/department using course syllabi and a rubric that we developed for this purpose.

Right now, if asked about the state of learner-centeredness in a department or unit, we can usually point to individual faculty members who are making significant changes in teaching practices and experimenting with innovative strategies. We may also be able to point to new technology or new policies that show progress toward making the shift, but we rarely have data that clearly delineates department/unitwide the areas of success or areas of need when it comes to the distinctive features of learner-centered pedagogy.

Excerpted from Assessing the Degree of Learner-Centeredness in a Department or Unit, Academic Leader, April 2009.

Professor Heidi Andrade and doctoral student Ying Du suggest that teachers frame self-assessment as an opportunity for students to reflect on their own work with the goal of learning more, making the work better, and thereby improving the chances for a good grade. In this paradigm, self-assessment is not the same as self-grading. Rather, students are looking at their work and judging the degree to which it reflects the goals of the assignment and the assessment criteria the teacher will be using to evaluate the work.

“Put simply, we see self-assessment as feedback for oneself from oneself.” (p. 160) Students in Andrade’s educational psychology course were given rubrics and checklists for each assignment. Prior to submitting the assignment, students used these assessment tools to judge their work. They were required to submit their self-assessments with the completed work, but their assessments were not graded.

Andrade and Du then conducted a series of interviews with students to explore their reactions to self-assessment conducted this way. They chose their sample purposefully “for its potential to illuminate areas in need of further study, not to represent a larger population.” (p. 163) This limits the generalizability of their findings, but their interviews offer insights into the experience of a population who responded thoughtfully to the experience.

Students in this sample reported that their attitudes toward self-assessment became more positive as their experiences with the process accumulated. Noteworthy was the fact that none of this sample reported having any previous experiences with academic self-assessment. Not surprisingly, they didn’t value their opinions about their work and saw self-assessment as a vehicle for figuring out the teacher’s expectations.

“The difference between self-assessment and giving the teacher what he or she wants was a recurring theme. A few students referred to self-assessment in terms of their own expectations. More often, however, students spoke of the tension between their own and the teacher’s expectations. … Over and over again, students rejected their own judgments of their work in favor of guessing how their teacher or professor would grade it.” (p. 168)

These students reported that their ability to self-assess depended on knowing what the teacher expected. In an appendix Andrade and Du share some of the rubrics and checklists used in this educational psychology course. They are very clear and explicit.

When they self-assessed, these students reported that they checked their work, revised it, and reflected on it more generally. Before this class their self-assessment efforts were “relatively mindless.” (p. 65) But from this experience they learned that careful self-assessment could improve their work to the degree that they did get better grades. Most did not see the larger value of the skill they were developing. Most did not use self-assessment in their other courses. They did see potential value in doing so. “They cited a lack of motivation and a lack of support for self-assessment among the reasons that ‘we slip.’” (p. 166)

Based on their experience and analysis of the interviews, Andrade and Du offer some tentative implications. First, “student self-assessment is feasible and likely to be beneficial if it is employed as a process of having students critically review their own work with an eye for improvement.” (p. 172) They also recommend that teachers share expectations for assignments and define quality. Showing students examples of effective and ineffective pieces of work can help to make those definitions real and relevant.

As this work illustrates, self-assessment need not necessarily be about self-grading. There are ways of framing and then using self-assessment that can help students develop that all-important ability of looking objectively at their work and then making changes that improve its quality.

Reference: Andrade, H., and Du, Y. (2007). Student responses to criteria-referenced self-assessment. Assessment & Evaluation in Higher Education, 32 (2), 159–181.

Reprinted from The Teaching Professor, April 2009.

Although there are commercially available electronic portfolios, the university decided to design its own system. “We spent four months looking at commercially available products three years ago, but none of them met our needs. The products would have had to drive the process, and we were so adamantly against that. We wanted the process to drive the product, so we developed our own,” says Neal Topp, professor of education at UNO.

The myMAPP system is intended to serve as a self-reflection as well as a means of documenting and assessing performance. For faculty, this means that they will use these portfolios to collect academic/professional activities, practices and achievements related to teaching, research, and service to reflect on their performance as well as generate reports for promotion and tenure documentation and annual reviews.

Because the myMAPP system is database driven, information needs to be entered only once and can be used for different purposes. For example, a faculty member could include an example of student work (from the student’s portfolio) to be used as a basis for the faculty member’s personal reflection on his or her progress in an aspect of teaching, as evidence of learning outcomes to support the faculty member’s case for tenure, as evidence at the department level to indicate progress in teaching, and to illustrate progress toward college and/or university teaching goals.

One of the strengths of myMAPP is the ability to incorporate artifacts—documents, multimedia, projects, etc.—things that do not necessarily come across on paper into the system as a way of cataloging one’s work. “The system has really caught on with the faculty since they realized it’s a good personal archive system. If they have a document associated with an achievement that they want to put in their own portfolio, then they don’t have to worry about finding it a few years from now, because it’s saved in the system,” Shorb says.

Excerpted from Academic Leader, February 2007.

]]> http://www.facultyfocus.com/articles/educational-assessment/portfolio-system-provides-integrated-assessment-across-the-institution/feed/ 0 http://www.facultyfocus.com/seminars/developing-tools-and-strategies-to-assess-student-learning/ http://www.facultyfocus.com/seminars/developing-tools-and-strategies-to-assess-student-learning/#comments Tue, 13 Jan 2009 16:52:04 +0000 Mary Bart http://www.facultyfocus.com/?p=2228 How to create your own student learning assessment tools

Developing Tools and Strategies to Assess Student Learning

Educators have at their disposal a wide variety of educational assessment tools to measure student learning outcomes. From published instruments to locally developed assessments, each has its place in higher education and in your assessment toolbox. This seminar will teach you how to develop your own tools and strategies to assess learning.

If all of that sounds like a tall order, Developing Tools and Strategies to Assess Student Learning delivers a common sense approach for making sense of it all.

Order this seminar

Led by Linda Suskie, vice president of the Middle States Commission on Higher Education, this seminar looks at the strengths and weaknesses of assessment tools, and how you can create your own tools to measure student learning. You’ll learn:

• The role of assessment in the teaching/learning process.
• The four key characteristics of effective assessment tools and strategies.
• The difference between “direct” and “indirect” evidence of student learning.
• Which tools return the most information in proportion to the time spent developing and using them.

Suskie also provides tips on how to:

• Identify the assessment information that’s already available at your institution.
• Develop different types of rubrics to evaluate student work.
• Create multiple choice tests that assess more than basic understanding.
• Use reflective writing to assess attitudes, values, and dispositions.
• Write effective “prompts” for essays and other assignments.

Who will benefit from this seminar:

• Faculty
• Faculty Development Personnel
• Academic Affairs Personnel
• Academic Administrators
• Department Chairs
• Academic Deans
• Continuing/Extension Education Personnel

Order this seminar