This question was explored in a study that involved a 4,000-student population and 848 undergraduate courses. The students have a two-week window during which they can electronically submit their anonymous course evaluations, one for each course in which they are registered. During that two-week period, they receive three email reminders.

The collected data enabled the faculty researcher to identify several characteristics that differentiated students who completed the course evaluations from those who did not. First-term beginning students respond more, as do students who are evaluating a course that is a requirement for their major. The author suggests that new students may be more enthusiastic about participating in university life. More seasoned students may think that the evaluations are not taken seriously by the instructor or institution and therefore are less motivated to complete them. It makes sense that students would consider courses in their major more important than other courses. Interestingly, course size was not a variable that reliably predicted who would complete the surveys.

The data also revealed that men, students with light course loads, and students with low cumulative GPAs and low course grades were less likely to evaluate the course. Why are women more likely to evaluate their courses? The researcher refers to this result as “puzzling.” (p. 22) The course load variable “appears to be a measure of student attachment to the university.” (p. 23); those taking fewer courses tend to be less committed to the institution.

Certainly the most interesting finding is the data indicating that students doing poorly in the course are less likely to complete the course evaluations: “A matched pairs test that completely controls for class- and instructor-invariant student characteristics confirms the finding that students who do better in the course are more likely to participate in SET (student evaluation of teaching).” (p.28) Add to this another finding documenting that students who are more likely to have strong opinions about the course (indicated by how quickly within the two-week window they submitted their evaluations) had, on average, positive views about the course and instructor. The author concludes that based on this data, online course evaluations do not attract disproportionately more dissatisfied students. In fact, they do the opposite, giving credence to the contention that the results may be biased in favor of the instructor and course as opposed to against them.

Reference: Kherfi, S. (2011). Whose opinion is it anyway? Determinants of participation in student evaluation of teaching. Journal of Economic Education, 42 (1), 19-30.

Reprinted from “Who Participates in End-of-Course Ratings?”The Teaching Professor, 25.9 (2011): 4,5.

For more on Faculty Evaluation, download a FREE copy of Effective Strategies for Improving College Teaching and Learning!

Sign In Create an Account

When those feelings are positive, they impact levels of motivation and cognitive learning in direct and measurable ways. What’s being explored here empirically makes a lot of intuitive sense as well. Most of us have experienced first-hand the “good vibes” generated when students in a class are reacting positively.

How are positive feelings about a course and instructor generated? Communication plays a central role. It matters what the instructor says in front of the whole class and in individual interactions—such was when students request information, ask about course content, exchange greetings, or ask a personal question.

As in other research reports, the instrument developed and used in these studies is included in the article referenced below. It contains 24 items in its long version and eight in a shorter version.

Here are some sample items from the instrument:

• I usually feel positive about my conversations with my teacher.
• My teacher makes an effort to satisfy questions I have.
• I feel comfortable talking with my teacher.
• I wish my teacher was better at communicating with me.
• My teacher genuinely listens to me when I talk.
• My teacher makes time for me when I want to talk to him/her.

It is interesting to speculate what students might have to say about their communication with you, but with this instrument, and others like it, you can anticipate student responses by completing the form at the same time students do. Comparing a self-assessment with student feedback is a great way to learn more about the impact of teaching efforts on students—in this case the impact of communication exchanges.

Because the instrument is specific and detailed, it enables identification of areas of strength and weakness. As important is whether or not you can trust your assessments. If what you believe about your communication with students is verified by their feedback, that knowledge enables teaching with greater confidence. If not, the feedback offers insights that can lead to more accurate assessments of student responses. This applies not just to student-teacher interactions but to whatever aspect of teaching or student learning you and your students are assessing.

Instruments like this one can be used by practitioners to gain individual feedback. Obviously they cannot be used to collect research data without permission of the researchers.

Reference: Goodboy, A. K., Martin, M. M., and Bolkan, S. (2009). The development and validation of the student communication satisfaction scale. Communication Education, 58 (3), 372-396.

Excerpted from “Communication Satisfaction Scale,” The Teaching Professor, 24.10 (2010): 6, 7.

Yet, some areas of higher education have undergone very little change. In particular, the methods used to evaluate and ensure that quality teaching is occurring in traditional and online classes. Yes, there are those required course evaluations dutifully and anonymously completed by students at the end of every term, but these are typically viewed quantitatively and do little to transform instruction. The quality of teaching is rarely given serious attention. Basically, if a professor simply shows up to teach class (and sometimes even if she doesn’t) she receives a satisfactory rating without consideration of how well she did her job. In other words, the methods used to evaluate teaching in higher education are as outdated as cassette players and floppy disks.

Routinely taking for granted that satisfactory teaching at the college level is acceptable requires us to ignore a significant body of research on this subject. Scholars agree that good teaching requires time, effort, commitment, knowledge, presence, and ingenuity (Weimer, 2010). Good teaching does not just happen. So how do we invoke change in a system that has been in place since dissertations were written on manual typewriters?

Faculty can bring attention to the need for a better system of evaluating teaching by demonstrating excellence even when it is not required or even expected. Documenting and sharing evidence of quality teaching serves as a reminder to administrators (as well as colleagues) that there is more to teaching than showing up. Submitting a report with additional teaching-related information is a good first step. For example, faculty members can report on course load, number of students enrolled in each class, grade distribution, and the number of tests and other graded assignments. Further documentation may provide information on less quantifiable items such as the type and quality of feedback given to students, use of technology to teach content, and creative delivery methods.

Faculty members can take the voluntary evaluation process one step further by implementing supplemental evaluations of their teaching. The following section offers five suggestions for supplemental evaluations that provide meaningful feedback that can be used to improve teaching.

Midterm Course Evaluation – Student learning and experience with the class can be gauged halfway through the course through an anonymous evaluation. The instructor can review the feedback and immediately make adjustments if there is a general consensus about an issue in the class. The midterm course evaluation may also provide information about the type of lessons and assignments that students feel are beneficial. Surveys may be formal with specific questions answered on a Likert scale or it may ask open-ended questions which allow students to elaborate in more detail. This type of evaluation has obvious advantages over those required at the end of the term because students can benefit directly from the feedback provided.

Questionnaire Following Exam – Feedback from students about the format, content, and level of difficulty for specific questions can be useful in making changes to an exam or the lessons that were given in preparation for the exam. This same technique can also be used at the conclusion of a major assignment or project. Ask students: “1) Were instructions clear? If not, what additional information would have been helpful? 2) Was the exam or assignment beneficial to learning and retaining course content? 3) Please share other constructive comments.” Faculty can then use the information to make changes prior to the next exam or project.

Interview Students – Invite the highest achieving students to participate in an exit interview at the end of the semester. Why the highest achieving? These students will provide the most accurate and honest perspective of how to improve the course. Prepare questions in advance and be specific. For example: “If one assignment had to be removed, which should it be? Which assignment should definitely be kept in the course? Why?” Reflect on their responses and decide how information can be used to improve the course.

Peer Evaluations – Peer feedback typically takes the form of classroom observations but can also include reviewing course materials and assignments with a colleague who can give suggestions. Faculty should choose a colleague whom they trust and respect as a teacher to conduct the peer evaluation.

Self-Evaluation – When examining one’s own teaching behavior it is imperative that the faculty member realistically assess areas of strength and weakness. Videotaping a class to view and evaluate later is an excellent tool. The purpose of the self-evaluation is not just to show teaching ability in a positive light, but to demonstrate how feedback can be used to improve instruction.

There are many options for documenting teaching through records and supplemental evaluations. Hopefully, these suggestions have sparked interest in developing a voluntary evaluation system.

Now it’s your turn. What are some of the ways you evaluate your teaching effectiveness? Please share in the comment box.

Reference: Weimer, M. (2010). Inspired college teaching: A career-long resource for professional growth. Hoboken, NJ: Wiley, John & Sons, Incorporated.

Dr. Karyn W. Tunks is an associate professor in College of Education at the University of South Alabama.

1. Understand and accept today’s college students. First and foremost, students want us to know who they are. They want us to know their names and to know about their world. Today’s students are busy, technologically savvy, and multitaskers. Some are prepared for college work; others are not. Regardless of their backgrounds, all students have lofty ambitions and want to succeed. To help them, we can provide background knowledge in our subject areas. We also need to share the rationale behind what we do and ask students to do. I recommend making invisible expectations explicit. I regularly start class by saying, “We are learning this because …” When students understand why and how the material is relevant to them, they find more motivation to study and end up rating the course more highly.

2. Establish clear criteria for grading. All students want good grades, and they want to know exactly how to get those grades. College students today have experienced criteria sheets and rubrics since elementary school, and they want the same in college. They want to know where they stand on any given day in the semester.

After 20 years of college teaching, I have learned that telling students that their grades are based on percentages (20 percent homework, 25 percent quizzes, etc.) does not have meaning for them. They cannot figure their grades with a percentage system. A total point grading scale provides that clarity. Each assignment, quiz, lab paper, project, or exam has a certain number of points. These point values can be listed in the syllabus as well as the total points needed for the final grade. To help students keep track of their points, I give them a worksheet and explain that if they record their points, they will know exactly how many they have and how many they still need to earn.

I get rave reviews about my “no-mystery” approach to grading on course evaluations, and I believe that the good reviews are due to the clarity and ease of the total point system. I re-explain the grading system after the first assignments/quizzes/tests are returned. On the first day of classes, students are overloaded with information. It’s not until after an assignment has been graded that they are ready to understand how the grading system works.

3. Get formative feedback early. The end-of-course evaluation is a summative one. Although it aims to help us improve future courses, it does not enable us to respond to the needs of the students currently enrolled in the course. Formative feedback collected early in the course accomplishes that goal.

The first major paper or exam is a great time to collect formative feedback. I recommend attaching a page to the back of the exam, or asking students to respond to questions like these on the day papers are due:

1. How long did you study for this exam or work on preparing this paper?
2. How/where did you study/write?
3. Which class activities (lectures, discussions, reviews, online notes) helped you the most in learning this material? Why?
4. Which class activities helped you the least? Why?
5. Which topics remain the most difficult for you?
6. What has a professor done in the past that helped you learn?

If I have students answer these questions on a page attached to the exam, I let them know they can tear that page off and submit it anonymously when they turn in the exam.

You can also ask for feedback once that first exam or paper has been graded and returned. I like to ask questions then about improvement goals—what the student wants to do better and what else could be done in class to support their efforts to improve. If you don’t want to deal with open-ended questions, students can rate declarative statements such as “I would prefer more discussion of assigned readings.”

If you ask for their input, students will want to see that you listened to them. They may expect some changes. A short report back to them can be used to explain what you are willing and not willing to change, and why.

Students aren’t qualified to comment about all aspects of instruction, but they can rate how they feel about the classroom climate. They are good evaluators of what helps them learn and what confuses them. Getting their feedback early in the semester enables you to tailor the course to their learning needs.

Finally, it helps to talk about your evaluation results with a trusted colleague. Sharing student responses can help us see patterns in evaluations and sort out the “outliers” or just plain wacky comments. Knowing what our students need helps us teach in ways that promote their learning, and that means better results on the end-of-course evaluations.

Dr. Mary C. Clement is the director of the Center for Teaching Excellence at Berry College.

Excerpted from The Teaching Professor, 25.4 (2011): 1,3.

From our discussions with colleagues in various disciplines and at different institutions of higher education across the country, we learned that few faculty have navigated the dossier process with ease or without some angst. Many faculty members commonly report on the lack of precise directions about the preparation and documentation process, and, further often find the dossier process ambiguous and overwhelming. For those who look to the literature for direction, few books and articles exist; and even fewer books and articles direct junior faculty toward specific annual dossier preparation, even though this is a critical step toward earning tenure.

During the past four years, each of us has earned tenure, presented on the topic of preparing the annual tenure and promotion dossier, and recently published a book, Tools for dossier success: A guide for promotion and tenure, (Burnham, Hooper, & Wright, 2010) related to dossier preparation and documentation process. From our own experiences, we learned several ways to reframe this process to help make the overall tenure and promotion process more feasible, manageable, and controllable—including a focus on what faculty members might do before, during, and after preparing the required annual dossier.

During the dossier preparation process, we often asked ourselves and others such questions as: “How do I present this representative collection of materials related to my teaching, research, and service?”, “How can the dossier preparation process be managed efficiently this year and next year?”, and “How can I simplify this process, yet at the same time, present accurate, reflective evidence that underscores my accomplishments?”

To assist those still in the tenure-earning trenches, we offer 10 dossier strategies that helped us along the way. We believe if followed, these strategies can assist junior faculty prepare an annual tenure and promotion dossier that best demonstrates and documents competencies in teaching, research, and service.

1. Begin on day one. Each of us realized that the tenure clock started the first day we were on campus. In fact, we began to establish our research, teaching, and service programs even before our arrival. Think in terms of starting your teaching, research, and service activities immediately; start collecting evidence of your teaching, research, and service activities. We suggest keeping a file for all your evidence so it can be easily retrieved and organized later. Initially keep everything; as the dossier develops, you can determine what documentation best illustrates your accomplishments in teaching, research, and service.

2. Read your college and university guidelines carefully. Find the section regarding dossier expectations to determine exactly what should be included in the dossier. If further clarification is needed, consult with others (e.g., department chair, senior faculty) on specific guidelines and unique expectations across all levels at your institution. This strategy simplifies the dossier process and provides a framework for compiling your evidence correctly, which is an important part of having a successful review.

3. Listen to the recommendations of senior faculty. Senior faculty members serve on the committees that review dossier evidence and thus inform the process. Senior faculty members have knowledge, perspective, and wisdom that can benefit you greatly. Listen, and most importantly, consider how to respond to their recommendations put forward in previous reviews.

4. Follow your college/university’s specific guidelines and rules for the items that should and should not be included in the promotion and tenure dossier. Know what the guidelines are and follow through with accuracy, diligence, and precision. Although junior faculty members may be tempted to be creative, this is not the time to divert from the prescribed guidelines.

5. Work persistently and consistently while moving through the tenure process. Specifically, working hard during the academic year, rather than waiting until the last minute, will make the move from dossier preparation to dossier assembly much easier. Of the strategies we offer, working hard is an area in which you have the most control. When you work hard, and do quality work, you are likely on the road to success and on the way to achieving tenure.

6. Seek out mentors. No one on the tenure-earning track should be on an island. Find people who will assist you. Seek multiple mentors, such as fellow tenure-earning peers who are in the same boat with you. Also, consider seeking out associate professors who still have a perspective of and proximity to what the complete dossier preparation process is like (e.g., activities in which to engage before, during, and after developing and preparing the annual dossier). And, of course, seek out mentors who have years of experience and wisdom (e.g., ask to see their exemplary dossiers). All in all, having mentors is important and, in our opinion, worthy of the time it takes to cultivate the relationships.

7. Demonstrate transparency in your dossier. Deception is easily detected in dossiers. Hiding problem areas in your dossier instead of noting them will not assist you and, instead, can be detrimental to your success. Similarly, embellishing aspects of your dossier is almost always viewed unfavorably. Transparency, and honesty, in representing all aspects of your teaching, research, and service is important in all cases.

8. Show consistency and accuracy across the different sections and parts of the dossier. Know that the dossier represents you in the review process (at many colleges or universities you are not present during the review to defend yourself). Make sure your curriculum vita and your dossier are aligned and consistent, be exact with the information you are expected to provide, ensure that you do not have misspelled words or careless errors, and place your evidence in the correct section, often noted by your university’s guidelines. We suggest asking a peer to review your work to ensure that careless blunders and inaccuracies are avoided.

9. Follow the suggestions of the reviewers. Reviewers (e.g., tenure and promotion committee, department head, Dean) often return to previous year’s recommendations they put forward in review letters; and, they expect you to follow through with their recommendations. No doubt, it is imperative that you follow the recommendations outlined from your committee(s) and discuss, how you followed their previous suggestions (you can point these out in your dossier narratives and opening statements). Knowing that you addressed and followed through with recommendations carries significant weight with dossier reviewers and can be beneficial to your tenure-earning review.

10. Consult at all stages of your dossier preparation (i.e., before, during, and after). We see the importance of consultation within your department, across your college/university, and outside your college/university. When you have questions, consult. This is an important step to ensure that you are doing what you need to do to have an excellent dossier.

Clearly, there are other considerations in preparing the annual tenure and promotion dossier, however, we believe these 10 established and tested strategies provide an excellent start for junior faculty members preparing their first dossier. Further, we hope these strategies will assist you in navigating the complexities you may face as you decide how to best represent your work in the triad of teaching, research, and service.

References:
Burnham, J. J., Hooper, L. M., & Wright, V. H. (2010). Tools for dossier success: A guide for promotion and tenure. New York, NY: Routledge.

Hooper, L. M., Wright, V. H., Burnham, J. (in press). Acculturating to the role of tenure-track assistant professor a family systems approach to joining the academy. Contemporary Family Therapy.

Joy J. Burnham is an Associate Professor in Counselor Education at the University of Alabama, Lisa M. Hooper is an Associate Professor in Counselor Education at the University of Alabama, and Vivian H. Wright is an Associate Professor in Instructional Technology at the University of Alabama.

Supervisors frequently use language of this sort when they are trying to have it both ways: they want to be seen as strong supporters of employees at the same time they are giving constructive advice on how the employees’ performance could be improved. But attributing concern to a “perception by others” rather than to clear and documented behavior is a cop-out at best, and has potential for a serious legal challenge at worst.

In other words, when the supervisor says, “I’m not saying that’s my opinion or that it’s even justified,” he or she is essentially saying, “Other people here are wrong about you, but I’m expecting you to fix this problem. In fact, you may well suffer negative consequences if you don’t correct the false impressions of others.” The supervisor can thus be regarded as setting the employee up for failure by obliquely threatening punishment for circumstances beyond the employee’s own control, indeed for what appear to be the mistaken impressions of others.

So, what should academic leaders do about “the perception problem”? First, be sure to eliminate use of this expression from your own vocabulary. Supervisors may believe that it’s an effective but face-saving way to alert an employee about potential concerns, but, in fact, it’s simply a poor management practice. An employee will inevitably see through this transparent ruse, and rather than feeling grateful to the supervisor for having offered collegial support, the employee will probably resent the supervisor because he or she didn’t have enough respect to be candid about the issue. Although many performance problems cannot easily be quantified, it’s the duty of a supervisor to base his or her advice and evaluation on specific behaviors, not on vague impressions or subterfuges.

Second, academic leaders should be proactive in eliminating “the perception problem” from the practice of others. If the individual who attributes a problem to “other people’s perception” happens to have a reporting relationship with you, your solution is fairly straightforward. Both as a mentor and as that person’s supervisor, you should advise that person on how he or she can be more effective in evaluating the staff and thus avoid potential grievances. But if the individual is one of your peers or your own boss, the situation can be a bit more challenging. You may need to use all your diplomatic skills and break through the facade of “perception” to arrive at the observable behaviors that the person believes to be causing the problem. If, in the end, he or she can’t offer any specific instances of problematic behavior but remains adamant that the “perception” of other people is the real problem, then you will need to alter your approach. Tactfully suggest that the appropriate way of handling this challenge, then, is not with the employee himself or herself but with those people whose perceptions appear to be flawed.

Third, each institution ought to review its procedures for evaluating and mentoring employees to make certain that they’re based on legally defensible means of achieving the institution’s most important goals. There’s absolutely no reason why even senior employees or tenured faculty members cannot be reprimanded or dismissed for poor interpersonal skills, a lack of collegiality, or simply an inappropriate “fit” with the evolving needs and environment of the institution. That conclusion must, however, be based on documented instances of behaviors that cause problems, not on something as intangible as a vague “perception.”

In conclusion, therefore, while it’s tempting to avoid responsibility for negative assessments by attributing them to the perceptions of others rather than to one’s own judgment, sound academic leadership can never be based on the avoidance of responsibility. The “perception problem” often arises from a misguided desire to spare an employee’s feelings, which ends up causing far more problems than it prevents. If you can’t replace the phrase “There’s an impression out there that …” with “It’s my professional judgment that…,” you’re probably on the verge of making an observation you shouldn’t really be making anyway.

Dr. Jeffrey L. Buller is dean of the Harriet L. Wilkes Honors College at Florida Atlantic University. His latest book is Academic Leadership Day by Day: Small Steps That Lead to Great Success (Jossey-Bass, 2011).

Excerpted from “The Perception Problem.” Academic Leader, 26.4 (2010): 1-2.

Like all large-scale transitions, the process included some bumps. However, we now have a paperless process that allows for work to be reviewed from any secure Internet site.

With the benefit of hindsight, here are some recommendations regarding the transition.

Assemble a cross-disciplinary team. We developed a cross-disciplinary team to research the options for a vendor, and another cross-disciplinary team to develop and roll out the project once the vendor was chosen. In addition to representation from each of the three academic colleges, we actively solicited faculty from the fine arts department to ensure the cross-disciplinary utility of the process and outcome.

Get information technology on board early. Although the information that faculty enter regarding their teaching, scholarship, and service is stored via the vendor, uploaded documents (such as syllabi and reprints) are stored on a local server. The vast majority of technical issues we encountered involved the efficacy of the hyperlinks to the documents, so we would recommend involvement with a specific IT liaison on campus. In addition, whoever is most knowledgeable about workload data such as teaching schedules and student credit hours must be an integral component of the process and the decisions.

Rollout speed. It took approximately a year to develop the project and make most of the faculty aware that change was coming. It took another year for the process to become the standard practice. In some ways, it would have been less jarring to convert one college first to discover the bumps that deployment would uncover. But the decision to convert the entire campus allowed us two benefits: First, when problems were discovered, it was easier to request changes all at one time. Second, the wholesale approach clearly communicated the fact that the change would occur.

Leadership matters. Senior administrative and faculty leadership was crucial to the process. The provost’s office worked carefully with the three academic deans to decide on the basic format of the portfolios that would come forward for review for retention.

A faculty liaison is necessary. A faculty member with knowledge of personnel processes makes the best liaison with the vendor. She or he can best understand the concerns of the faculty users and the role of the product in the personnel processes. Although we are now in a “maintenance” phase, we still estimate that at busy times of personnel review, the liaison is allotting at least 25 percent of his or her time to the process. During the development and rollout, the time committed was closer to 50 percent of overall responsibilities. The liaison currently responds to all troubleshooting and facilitates the annual rollover of information and the upload of course information (number, name, and enrollment) each semester.

Mind the gap! The digital divide is real, and like most colleges and universities, we still have a few faculty members who do not utilize email and resist technology. Consequently, we had users who had never attached or uploaded a document, and for whom “browse” meant a stroll down a library stack. We also had faculty members who didn’t understand why changing a typo on the report drawn from the database did not fix the typo in the database. As we rolled out the product, we offered individualized hands-on workshops for each department, during which each faculty member worked at a computer.

Continue the training. We developed a substantial Web page devoted to the use of electronic portfolios including short video-capture films that show how to use of the software. We have annotated portfolio samples and, with the permission of the faculty involved, we post samples of successfully promoted portfolios. Most importantly, faculty members within each department now serve as informal liaisons to new faculty regarding the use of the system.

Security issues. We continue to face security issues regarding the integrity of the documents. Users quickly found issues such as the need to remove Social Security numbers from grants before uploading them into the system. The fundamental problem remains, in that we have to balance the ability to make the materials accessible to appropriate readers with the need to keep materials somewhat secure. One security solution is the use of our classroom management system (D2L) as the home for the portfolios. Both departments and committee set up sites where they can control access and upload the documents.

Factors ensuring a less painful process. The single greatest benefit our university had going into the process was the existence of a “page limit” on paper portfolios for promotion. The university-wide committee had a one-inch binder rule. Faculty submitted a three-page narrative on teaching and two pages each on scholarship and service. We kept the page limit on narratives, but replaced the one-inch binder with the standard of 10 hyperlinks per workload area—teaching, scholarship, and service (syllabi were excluded from the limit). We limit the size of a document or upload to 10MB, but we do not have a page limit. Department chairpersons load information regarding student evaluations, merit reviews, and letters of review.

Technology issues. Our campus is approximately 75 percent PC and 25 percent Mac. We continue to find oddities in terms of processes that work well on one and not on the other. PDF documents save space and are more easily read across platforms. Access to a fast scanner is necessary and the ability of individual faculty to convert documents to PDF aids the process.

Final benefits and drawbacks. The primary drawbacks are the cost of the vendor, the fact that the campus supports a person who devotes a portion of his or her job solely to process and the frustrations with technology when it doesn’t work, and the number of sites and junctures at which problems can arise. Like all technology, when it works, it is a time-saver, and when it isn’t working, it eats up copious amounts of time and patience.

As expected, the process benefits new faculty members the most. They have a single interface with a program that allows them to enter information and upload documents once, and that can then be compiled over time and drawn on for a variety of personnel-related reports.

Administrators benefit from the ability to pose queries of the data (such as the number of peer-reviewed publications in the college) without laboriously asking chairpersons and faculty. Committee members appreciate the fact that they can review files from the comfort of offices or homes. The promotion committee was uniformly positive about the use of electronic portfolios the first year in terms of accessibility.

However, the most lauded aspect was the existence of a single format for organizing portfolio materials. Committee members appreciated the fact that for each candidate they could systematically find the information they needed. Overall, the process of converting to electronic portfolios prompted a series of decisions that helped clarify some of the muddier points of personnel review at our university, and made the review of portfolios more effective and accessible.

Dr. Betsy L. Morgan is a professor of psychology and department chair at the University of Wisconsin-La Crosse. She led the conversion to electronic portfolios when serving as the faculty assistant to the provost.

Excerpted from “Tips for Converting to E-portfolios for Faculty Personnel Review.” Academic Leader, 26.1 (2010): 4-5.

In an interview with Academic Leader, Debi Moon, assistant vice president of educational affair at Georgia Perimeter College, and Rob Jenkins, associate professor of English and director of The Writers Institute at Georgia Perimeter College, discussed some common mistakes administrators make regarding promotion and tenure and ways to avoid them.

“I think the most common mistake administrators make is not giving clear standards to faculty of what’s expected and not backing that up with evaluations that give faculty guidance on how they can achieve promotion and tenure,” Moon says.

Mixed messages
One issue among academic leaders is that often they come up through the faculty ranks and have developed close relationships with colleagues and often view themselves as advocates for these colleagues rather than administrators. “You have to draw the line. You represent the university when you’re an administrator. As much as you want to be a friend to your colleagues, there has to be a line,” Moon says.

As an academic leader, it’s important that you watch what you communicate to faculty members when it comes to promotion and tenure. When communicating with colleagues, one’s instinct may be to be reassuring, but this can backfire. Casual reassuring comments to faculty members such as, “You’re on the right track” or “Tenure is not going to be a problem for you” can lead to lawsuits when the faculty members’ expectations of earning promotion or tenure are not realized, Moon says.

Instead of these types of comments, academic leaders should “give a clear path for their faculty to go down in order to get promoted or get tenure,” Moon says.

‘Grade inflation’
Another issue is being overly generous in formal faculty evaluations. As with student grading, there seems to be a trend toward inflated evaluations, Jenkins says. This type of grade inflation can lead to faculty members who perform marginally appearing to have met requirements in formal evaluations. “It’s hard to deny tenure or promotion when you’ve been telling the faculty member that he or she has been meeting expectations all this time.”

Ambiguous policies
When dealing with individual faculty members, specific situations, if not clearly articulated in tenure and promotion policies, can open the door to legal problems. For example, what do you do when a faculty member submits his tenure portfolio, which notes that he is waiting to hear back from a publisher about whether his manuscript has been accepted? Can his tenure committee delay action until the publisher decides on his manuscript?

What about when a person who is not on the tenure committee comes forward with information that she believes should be considered in the committee’s decision?

Lacking clear guidelines on such situations, administrators sometimes decide these issues on a case-by-case basis, which also can be risky. These and other situations should be addressed in the tenure policies, and any policy changes must be communicated to the faculty.

Actually, the institution should do more than inform faculty about policy changes. It also should provide training to ensure that faculty are clear about tenure and promotion policies.

Reprinted from Avoiding Tenure and Promotion Controversies, Academic Leader, 25.12 (2009): 2,7.

The researchers surveyed this group, asking the faculty members to respond to several questions, including this most important one: “Your student ratings have increased for at least three consecutive semesters during the last three years in your [Course Name] class. What factors led to this change in your teaching performance?”

The slightly more than 200 respondents most frequently attributed the increase in ratings to changes made in one or several of these five areas:

1. more active/practical learning, including efforts to make the content’s relevance apparent to students;
2. better teacher/student interactions, exemplified by learning students’ names and having individual conferences with them;
3. making expectations for learning outcomes clearer while still maintaining high standards;
4. being better prepared for class; and
5. revising the evaluation policies and procedures used to assess student work.

The first three of these categories accounted for almost 50 percent of the faculty responses. A bit surprisingly, 5 percent of the respondents whose scores had improved didn’t list anything they’d done or they indicated that they were not aware of having implemented any changes.

This cohort of faculty included full-time tenured faculty (actually this was the largest group, 56 percent), full-time nontenured faculty (12 percent), and part-time appointees (35 percent). The researchers note that this indicates how faculty in all kinds of positions can improve. That so many in the already-tenured and part-time categories did so is especially noteworthy and encouraging.

In addition to the survey, 30 faculty from 10 of 12 colleges at the institution were interviewed “to gain a better understanding of the change process.” (p. 167) Several interesting findings emerged from the interviews. For many faculty members, the most difficult part of the process was being willing to admit that they needed to change. “Humbling” was an adjective used to describe the feeling. Often there was some sort of triggering event—frequently it involved end-of-course ratings results. After teaching a course seven times, one faculty member received his lowest-ever overall course rating. He was shocked but reported that he decided to find out why. Others talked about an overall lack of excitement in the course and their own motivation to change and do better.

In the interviews, almost 80 percent of the faculty indicated that the effort required to implement the changes was minimal. It seemed that for most it was more a matter of fine-tuning their teaching. The researchers conclude, “The results of this study should be encouraging to faculty members who feel they cannot improve.” (p. 171)

Reference: McGowan, W. R., and Graham, C. R. (2009). Factors contributing to improved teaching performance. Innovative Higher Education, 34, 161-171.

Reprinted from “Teachers Who Improved.” The Teaching Professor, 23.10 (2009): 2.

Here’s a condensed version of the seven “overall guidelines” she offers for classroom observation by peers.

1. “It cannot be assumed that peer reviewers are skilled classroom observers.” (p. 99) Faculty need to be trained for the task. If they are, the reliability of their observations increases.

2. “A single classroom observation by one rater is not a reliable indicator of teaching quality.” (p. 99) How many observations are needed? Some researchers recommend three; others, two different reviewers each doing two observations; still others, three or four reviewers observing between eight and 10 of the instructor’s classes.

3. Pre-observation information is needed to provide context for what is to be observed. Observers need details about the course, the instructor, and the students.

4. When in class, the observation needs to be focused. Checklists are a great way of helping the observer look at specific aspects of the instruction. Questions and other more general guidelines can be used. Multiple exams are included in the book.

5. “The observer should try to be as unobtrusive as possible.” (p. 99) This means the peer is an observer, not a participant in the class. Once the observer starts participating, the focus is no longer exclusively on observing the teaching and students’ responses to it. Moreover, those observer contributions affect responses of both the teacher and the students.

6. Observing for a substantial amount of time is necessary. If the class is an hour long, peers should observe for the entire hour. It takes time for the instructor and the class to relax and move into teaching behaviors that are typical.

7. Notes, forms, or letters should be completed promptly after the observation. The information gleaned from being in the class remains fresh for a limited amount of time. Details become increasingly difficult to remember when time lapses between observation and preparation of the feedback.

Reference: Chism, N.V.N. (2007). Peer Review of Teaching: A Sourcebook. 2nd Ed. Bolton, Mass.: Anker. Note: Anker Publishing now belongs to Jossey-Bass.

Excerpted from Classroom Observation: Guidelines, The Teaching Professor, May 2008.

At lot of institutions now students do the rating forms online, and as a consequence the number of students who complete them has plummeted. Why should they when they see that their feedback has little impact, most especially on those teachers who need to improve? (You’re detecting more cynicism?) I did discover an effective solution to the problem of low response rates, though. The authors of a research article noted in passing that at their institution students who completed the online course ratings instrument got earlier access to their grades than those who did not. Worried that that might bias the sample with those grade-hungry students being the only ones who completed the course evaluation? That wasn’t the case at this place: the overall student response was almost 80 percent—a sample size large enough to be representative.

I have written a lengthy article for the newsletter on the research undertaken in this study. It offers the first viable model I’ve seen for dealing more systematically and objectively with the student comments collected on most of these end-of-course forms. They start out by asking students good open-ended questions: “What are the teacher’s strengths?” and “What improvements would you suggest to the teacher?” I wouldn’t give those questions an A—the best questions ask students to connect instructional approaches, assignments, and other aspects of the course to learning experiences: “How did the instructional approaches used in this class affect your efforts to learn?” But the questions used in the research were far superior to those questions that inquire about what students “liked” most and least about a course and/or its instructor.

One of the best ways to overcome the cyncism so many of us feel about end-of-course summative evaluations is to see them as part of the picture—one window offering a view of the teaching. To make the picture more complete and accurate, teachers need to cut out many windows seeking feedback from students in different ways, at different times, and on different topics. It also helps to remember that these ratings are assessments of experiences in a course—they do not measure our worth as human beings.

Look for the article summarizing the research mentioned here in the June-July issue of the newsletter, or check out the article for yourself.

Pan, D., Tan, G. S. H., Ragupathi, K., Booluck, K., Roop, R. and Ip, Y. P. ( 2009). Profiling teacher/teaching using descriptors derived from qualitative feedback: Formative and summative applications. Research in Higher Education, 50 (1), 73-100.

Classroom observation—either for the purpose of promotion and tenure or through the assistance of a teaching and learning center for those faculty seeking self-improvement in instruction—is not that common of a methodology for helping higher education instructors with information that they might use to improve their teaching. This is due, in part, to the infrequency of classroom visits by either colleagues or supervisors, the general unfamiliarity with what measures could be used to evaluate effective face-to-face instruction, and the time involved in documenting an observation.

However, one classroom observation tool, Flanders’ Interaction Analysis, can minimize the potential drawbacks and objections because it focuses on one aspect of a classroom: the interaction between students and their instructor. The focus on interaction as one aspect of an effective classroom and the identification of a set of predefined variables to measure interactions makes the Flanders model a viable method for analyzing teaching.

Hilda Taba, Edmund Amidon, and J.B. Hough were among the first proponents of examining classroom interactions as a way of describing what occurred during the teaching and learning process. Working with Amidon, Ned Flanders developed a system of interaction analysis to give a picture of what occurred during opportunities to observe classroom activities. This system was presented in The Role of the Teacher in the Classroom (Amidon & Flanders, 1967).

While the work of Flanders and others was designed for the precollegiate arena, it has applicability for the collegiate classroom. One is able to observe the behavior (Flanders calls it “talk”) of the instructor as being direct (lecturing, giving directions, or criticizing or justifying authority) or indirect (accepting feelings, praising or encouraging, accepting or using ideas of students, or asking questions). Student “talk” is categorized as either response (usually a convergent answer to a question posed by the teacher) or initiation (a divergent question or statement that departs from the flow established by the instructor). Flanders also allowed that some activity may be seen as confusion, with different communication occurring simultaneously, or silence.

To make use of these observations, codes are recorded every three to five seconds by the observer. The juncture between two coded responses is considered the interaction. For example, if a teacher asks a question (4) and a student responds appropriately (8), this would be called a 4-8 interaction. When the observation period (a full or partial teaching session) is completed, the interactions are entered into a matrix. This matrix can be examined for how much of the time the teacher talked (with direct or indirect influence) and how much the time students talked (either convergently or divergently). This data can then be used for a conference between the instructor and the observer.

In addition to providing department chairs and directors of centers for teaching and learning with an objective evaluation tool for assessing teaching, research conducted by Flanders and others indicates that the use of the tool by instructors for self-improvement purposes results in improved teaching. Many articles concerning the Flanders Interaction Analysis focus on the usefulness of the tool to provide immediate feedback to instructors regarding their interactions, but also report that it is easy to use, understand, and implement.

Recently, a pilot study was conducted at Central Michigan University to assess the feasibility of using Flanders’ Interaction Analysis system in providing faculty and administrators with better feedback regarding teaching effectiveness. After approximately 10 hours of training on use of the tool, the pilot study was conducted in the observation of university faculty. Three independent observers reviewed video recordings of several classroom sessions to verify consistent coding. The inter-rater reliability was very high (>.90) between the video analysis as well as between the video analysis and the data collected during the live classroom sessions. The next phase of this study includes establishing a debriefing format and obtaining usefulness and satisfaction feedback from instructors.

Michael B. Gilbert is professor and chairperson of the Department of Educational Leadership at Central Michigan University. Alicia Haley is a doctoral student and graduate research assistant in the same department.

Excerpted from Filling a Need: Classroom Observations for Higher Education Faculty, January 2009, Academic Leader.

That’s not to say there weren’t some bumps along the way as the school transitioned from the traditional 1-inch binder approach to the e-portfolio platform. In the online seminar Developing Effective Faculty Activity Electronic Portfolios, Betsy L. Morgan, a professor of psychology at the UW-La Crosse, and Bruce May, an associate dean and professor of management at the school, shared their experiences in getting the system up and running, including factors to consider when evaluating solutions and implementation pitfalls to avoid.

To get started, May recommends looking at the Three U’s – users, uses, and usability – and then creating primary and secondary categories for each based on what’s important for your institution. The potential “users” could include department chairs, faculty, department and college committees, deans and associate deans, and promotion, tenure and retention committees. The potential “uses” go beyond promotion and retention and might include annual reports, faculty vitae, accreditation reports and more. The potential “usability” requirements might include customizable templates, the ability to create customizable reports, and a robust search functionality.

UW-La Crosse used a cross-disciplinary team to research vendors, and second cross-disciplinary team to develop and roll out the project once the vendor was chosen, says Morgan, who led the conversion to electronic portfolios when serving as the faculty assistant to the provost. She noted the importance of gaining support from senior administrative and faculty leaders, and involving the IT department early-on in the process.

Of course there was some opposition, especially from those faculty members who largely resist technology, fear a security breach, or who’ve already achieved tenure and feel it’s a waste of time to upload past information. To address this last concern UW-La Crosse required senior faculty to enter only the last three years of work into the system; with the rest of the work
contained in an uploaded CV.

“The process certainly benefits new faculty members the most,” says Morgan. “They have a single interface with a program that allows them to enter information and upload documents once, and that can then be compiled over time and drawn on for a variety of personnel-related reports.”

Misconception: Outcomes are the only things worth measuring.

Reality: Outcomes, such as numbers of new courses developed, enrollments, retention, and satisfaction levels, are important and they should be measured. But it’s also important to evaluate critical processes, such as support for faculty course development, relationships between course designers and developers and faculty, and student ability to get help as needed.

The processes that are involved in producing and delivering online courses and instruction should be evaluated alongside the outcomes of these processes so it’s possible to see what changes would allow for better outcomes.

I recently worked with an institution that had an adversarial relationship between faculty and the online course development team, and both spent time pointing fingers to explain why the results weren’t optimal. What they didn’t see was that this adversarial relationship created bottlenecks and course development problems. Obvious solution? Build a better process and fix the damage caused by the old one.

If the process of producing and delivering online courses and instruction is problematic, courses and instruction are also likely to be problematic—and these problems are unlikely to improve without improving the process. So, while evaluating outcomes, it’s also important to evaluate the processes that impact those outcomes. You will find inefficiencies, poor relationships, rework, contention, and more that are making better outcomes difficult or impossible.

Misconception: Evaluation is a CYA activity to be endured.

Reality: The purpose of evaluation should be to continuously improve, not to check off boxes on a checklist and then breathe a sigh of relief until evaluation needs to be done again.

Most higher education institutions conduct end-of-course evaluations, but this kind of evaluation often doesn’t result in significant improvements to courses and cannot impact courses in progress.

Because end-of-course evaluations may be required but often aren’t sufficient, some online instructors have begun to implement weekly or bimonthly anonymous evaluations by students so they can make changes to the course and the process in the here and now.

Bottom line? The purpose of evaluating online courses and instruction should be improvement, not pain. And improvement efforts are most successful when they are valuable to all concerned. So analyze whether the misconceptions described in this article apply to your institution—and if some do, consider how to change them for the better.

Course and Instructor Evaluation: If It’s So Good, Why Does It Feel So Bad? excerpted from Online Classroom, January 2008.

In the recent online seminar, Avoid the Top Seven Mistakes in Hiring, Promotion and Tenure, Debi Moon, assistant vice president of educational affairs at Georgia Perimeter College, and Rob Jenkins, associate professor of English at Georgia Perimeter College, outlined the most common personnel missteps colleges and universities make, and how to correct them.

One of the big new challenges for higher education institutions today with regards to hiring is the emergence of social media, says Moon. Many people have an active presence on blogs, Facebook, Twitter, or other social networks, making it relatively easy for hiring committees to learn more about candidates than what they include in their hiring materials. Unfortunately, due to its newness, few institutions have established a policy for how social media fits within the overall hiring procedures.

Moon also talked about the importance of staying on script when interviewing candidates and only asking the approved questions, even if you’re just making small talk and the “interview” hasn’t officially begun. It’s during these rapport-building times that slip-ups can happen, which is why many institutions have stopped taking candidates to lunch or dinner.

While the hiring process is a perilous journey, promotion and tenure decisions have their share of pitfalls as well – ranging from hurt feelings to litigation. Oftentimes department chairs will give faculty evaluations that only mention what the instructor is doing well – in very glowing but general terms. But it’s just as important to provide constructive criticism and identify areas for improvement. Otherwise, says Jenkins, “you’re telling them year after year how well they’re doing and then all of a sudden they’re not good enough for a promotion or tenure. They’re completely blindsided.”

According to Moon and Jenkins, the seven mistakes in hiring, promotion and tenure are:

1. Not providing adequate Equal Employment Opportunity (EEO) training for all faculty and administrators involved in the hiring process.
2. Not providing training or assistance in the writing of job descriptions and interview questions.
3. Ignoring social networking sites when reviewing materials for new hires and/or not having a policy on what materials to consider for candidates.
4. Treating adjuncts as if they are less valuable than full-time faculty by not providing adequate professional development.
5. Not writing candid evaluations with constructive criticism.
6. Not understanding the parameters of academic freedom in the classroom and campus life.
7. Failure to provide proactive audits of potential pitfalls in the system, update policies, or require regular training for all administrators.

1. Ask students to write a letter to a future student about the course. This technique helps students to reflect on the course with students’ needs in mind rather than the instructor’s.

Here are some prompts that Hall uses to get students to produce this letter:

• What did you find interesting about this course?
• What did you not like in this course?
• Provide some insight about problems you had in the course.
• What does it take to succeed in this course?

2. Seek feedback from a few select students. Hall used to ask the entire class for critical feedback, but he has since recruited just one or two people two or three times per course to reflect on the negatives.

“I ask them to skip the praise and ask, ‘What’s one negative thing you can find about the course?’ They hate that, but sometimes they’re able to spot something,” Hall says.

Excerpted from Tips From the Pros: Two Creative Means of Eliciting Student Feedback. Online Classroom, January 2007.

In an interview with Academic Leader, Marietta Del Favero, professor of educational leadership, counseling, and foundations at the University of New Orleans, talked about faculty investment in their institutions and the factors that encourage and inhibit it.

Del Favero noted that the literature does not have a consistent definition for faculty investment. She refers to “investment” in this context as “the extent to which faculty are engaged in, and proactively contributing to, the collective concerns of the institutional community… [which] involves social activity related to institution-building, which may be related to, but is often apart from, one’s individual scholarship.”

New Faculty Investment Expectations
The accountability movement has placed increasing pressure on faculty to do things beyond the teaching, research, and service faculty were expected to do 20 years ago. “It used to be that the faculty were responsible for what goes on in their classrooms, Del Favero says. “Now there is pressure … to involve the faculty in various kinds of programs and activities important to the development of the whole student.”

The growth of distance learning and interdisciplinary programs has also gotten faculty to look beyond their disciplines and make more contributions to the institution. In many cases, involvement in these programs is expected; however, faculty members sometimes get mixed messages when it comes to the value of participating in activities outside their disciplines.

Obstacles to Faculty Investment
Contributing to these mixed messages are faculty reward systems and the socialization process. As institutions seek more investment by their faculty members, one of the obstacles they will face is the ambiguity of their faculty reward systems. “The reward system is very problematic because depending on how faculty members’ contracts read—and often such contracts are not very specific—they don’t really spell out what is expected, particularly in the service role” Del Favero says.

A faculty contract might call for 50 percent research, 40 percent teaching, and 10 percent service. “But they’re still pretty vague,” Del Favero says. “Junior faculty come in and say, ‘Gee, what does this mean? I’ve got a 10 percent service obligation, but then senior faculty are telling me to try to limit my service and get my research going.’”

Getting faculty to invest in their institutions cannot be a top-down endeavor. Faculty need to take the lead in changing the reward system and culture to reflect the importance of faculty investment, Del Favero says.

Faculty Rewards
The faculty reward system “needs to be totally revamped,” Del Favero says. One solution would be to incorporate Boyer’s matrix of talent, which consists of four types of scholarship:

• Scholarship of discovery—the traditional type of research involving the pursuit and discovery of new knowledge
• Scholarship of integration—interpreting and connecting original research across various domains
• Scholarship of application—applying research to practical problems
• Scholarship of teaching—includes pedagogical research and the advancement of teaching

By creating a reward system based on this matrix, faculty could emphasize one or more types of scholarship more than the other types, resulting in faculty doing the things they are best at and have the most interest in. “It’s the notion that not everybody has to be invested equally in each type of scholarship, but that a matrix of talent will ensure that institutions are better served. Such appreciation for a broader valuing of faculty contributions opens the door for valuing of faculty investment in their institution,” Del Favero says.

Excerpted from Understanding Faculty Investment in their Institutions, Academic Leader, April 2007.

“Non-tenure-track faculty tend to be people with a variety of skills that don’t fit into the logical slot of a tenure-track faculty position. These people tend to have a broader variety of experience, and we’re trying to take advantage of that,” says Dave Haney, associate vice chancellor for academic affairs at Appalachian State University.

In some cases, positions have been created specifically for individuals. “We’re trying to be more flexible. We just hired a guy for a full-time position that’s half-time co-curricular and honors advising, one-quarter-time admissions, and one-quarter-time teaching, so there are three funding sources for this position,” Haney says.

In addition to the changes in the status of non-tenure-track faculty, the university is in the process of revising its curriculum, moving toward an outcomes-based undergraduate experience. “We find ourselves reevaluating what some of these non-tenure positions can do to enhance how we’re trying to change the way undergraduates experience their four years here,” says Cindy Wallace, vice chancellor for student development.

“It’s given us a way to think about this pool of people and give them jobs that straddle divisions—sometimes within academic affairs and sometimes between academic affairs and student affairs. I think a lot of the non-tenure-track faculty, because they tend to be student oriented rather than discipline oriented, are very well suited to these sorts of positions where they’re working with students outside the classroom,” Haney says.

Creating non-tenure-track full-time faculty positions has been somewhat controversial among tenured and tenure-track faculty members. Throughout discussions on the non-tenure-track faculty positions and general education curriculum changes, Haney and Wallace have relied on data to support their views and have tried to hold others accountable for the same. One critical piece of data in the discussion was the teaching assignments of tenured and tenure-track versus non-tenure-track faculty teaching assignments and how these impact departments. Since many non-tenure-track faculty members teach lower-level courses, tenured and tenure-track faculty are able to teach upper-level courses for which they have a real passion, Wallace says.

“There was a real deliberate effort out of our provost’s office on countless occasions where we shared very important information, and I think it’s starting to matter,” Wallace says.

Haney encourages department chairs to hold faculty accountable for supporting their statements with data. “Think about how you run a meeting and at what point you call people on statements that have no basis in fact,” he says.

Excerpted from Non-Tenure-Track Faculty Contracts, Academic Leader, August 2007.

Q: Why is faculty collegiality important?

Cipriano: Some departments are enthusiastic, collaborative, and intellectual (faculty enjoy coming to work) while others are isolated, deadening, and depressing (faculty stay away as much as possible). The difference may be that one department member treats his fellow faculty in demeaning, degrading, and uncivil ways. Lack of civility and collegiality can deleteriously affect the department, its students, and faculty.

Q: Is collegiality more important now than in the past?

Cipriano: Collegiality is more important now than at any other time in the history of the academy. Early career faculty report that climate, culture, and collegiality are more important to their satisfaction than compensation, tenure clarity, workload, and policy effectiveness.

Q: How much influence does a department chair have in creating a collegial environment?

Cipriano: If there is one area that chairs most directly impact, it is the climate in their department. The chair sets the tone by building an atmosphere of trust, respect, and collegiality. In the best departments, the environment is one that invites expression, exploration, inquiry, and mutual trust.

Q: If a department lacks collegiality, what is a good first step that a department chair can take to promote collegiality?

Cipriano: A chair promotes collegiality by being trustworthy and an excellent communicator. Also, an effective chair fosters collegiality by:

1. emphasizing consensus
2. sharing power
3. consulting with all faculty
4. de-emphasizing status differences
5. being constructive and informative
6. clarifying performance expectations