December 2, 2013

7 Assessment Challenges of Moving Your Course Online (and a Dozen+ Solutions)

By: in Online Education

Add Comment

Not all online courses are created from scratch. Many—if not most—are online versions of courses that have previously been taught face-to-face. In these cases, where an instructor or instructional designer is adapting an existing face-to-face course for online delivery, assessments already exist.

But to be effective in an online environment, the assessments that worked perfectly fine in a face-to-face classroom may need to be tweaked or even replaced. Why? Because the online teaching and learning environment presents the following seven challenges to traditional assessment implementations:

1. Cheating is easier to do (and harder to detect) online. While it’s not clear whether online students do, in fact, cheat more than face-to-face students (Watson & Sottile, 2010), the truth is that it is more difficult to monitor who’s taking a test and how they’re taking it online than it is in a classroom. Strategies for adapting assessments for online delivery include:

  • Timed/open book tests. Online, every test is an open-book test (except those that are proctored). To minimize read-as-you-go test-taking, reduce the amount of time students have to take the test so that only those students familiar with the material can answer the questions in the time allotted. Alternatively, replace selected response tests (such as multiple choice and T/F) with short-answer or essay questions that require students to apply textbook facts to novel scenarios.
  • Shuffled/randomized test questions. Shuffling questions helps reduce the likelihood that two students sitting in adjacent library carrels can take the same test together, one answering the “odds” and the other answering the “evens.” Selecting questions randomly from a large test bank takes this idea one step further, providing each student with a similar (but not identical) assessment. (Most learning management systems provide both shuffling and randomization capability.)
  • Plagiarism detection software. Having students run their essays through a for-fee plagiarism detection service such as SafeAssign or TurnItIn can potentially deter cut-and-paste plagiarism. At the very least, it can start a conversation about how to cite sources properly.
  • Frequent low-stakes tests, such as short quizzes or self-check activities worth no more than a few points each, help make cheating more trouble than it’s worth.
  • Performance assessments. Assignments that require students to write, speak, or present to the class are harder to fake—especially if they occur regularly throughout the course.
  • Coordinated tests. Instructors who teach multiple sections of the same class may want to coordinate tests so that all students take the same test at the same time. (Staggering tests increases the likelihood that the first students to take the test can pass on question details to their colleagues.)
  • Proctoring. Requiring students to take proctored exams takes cheating off the table—or, at least, returns it to the same level as a face-to-face class.

2. Online courses need more student-to-student interactions “built in” than do face-to-face courses. In a classroom setting, students interact—socially (chatting before class starts) and as part of common classroom activities (asking questions for clarification, weighing in on impromptu discussions, etc.) Online, these opportunities to feel connected and learn from each other do not occur naturally; they must be carefully planned and managed. Assessments that incorporate student-to-student interactions—while not appropriate for every course—can play a powerful part in a course’s overall communication strategy:

  • Peer-review. Asking students to review their classmates’ work (and grading them on their reviews) can help motivate best efforts as well as help students learn from each other.
  • Group projects. Well-designed group projects help students master both course content and team participation skills.

3. Online students need more student-to-instructor interaction than their face-to-face counterparts. While instructor interaction and feedback is important to all students, it’s critical to the success of online learners and—like student-to-student feedback—must be carefully planned and cultivated in an online setting. Fortunately, online tools make incorporating student-to-instructor interaction into assessments relatively achievable via:

  • Frequent, low-stakes testing (“self-check” quizzes and activities). Options range from short selected response quizzes and watch-and-discuss questions to complex games and activities accessed through textbook publishers’ add-on course cartridges.
  • Rich, detailed feedback. Strategies for providing rich, detailed feedback vary based on the activities you’ve selected for your online course. The most practical include building detailed feedback into selected response quizzes; actively managing discussion boards; and administering weekly surveys asking students to identify the concept(s) they’re struggling with and then addressing the most-identified concept(s) by using a product such as Jing to create and post quick video clarifications.

4. Online students need more planned structure—that is, more help in staying on time and on task—than their face-to-face counterparts. The structure that occurs naturally when students and instructor congregate in the same place at the same time (seeing “Test next Tuesday” written on the white board, for example, or overhearing classmates a row over discussing an upcoming assignment) doesn’t just “happen” in an online course; it must be planned and managed. These assessment-related techniques can help:

  • Frequent, low-stakes tests (ungraded or low-point-value “self-checks”) help students gauge for themselves how well they’re mastering the material.
  • Graded milestones. Breaking up large projects into smaller graded milestones helps students (and you) identify problems areas early enough to address them.
  • Graded participation. Using a rubric to grade discussion board participation is time-intensive, as is asking students to review each other’s work. However, the benefits in terms of being able to gauge and guide students’ understanding can often be worth the time investment.

5. Performance assessments (such as presentations and demonstrations) can be more challenging to administer online. Putting students “on the spot” in a classroom setting (by assigning them to demonstrate a process or deliver a speech to the entire class, for example) can strengthen their communication skills and help them learn from each other. Online, however, students need to be able to package their presentations so that their instructors and fellow students can view and provide feedback on them. Accomplishing this requires:

  • Low-cost/free hardware and software. Webcams and microphones are relatively affordable, and—given clear, step-by-step instructions—students can use free software (such as Jing, Screencast-o-matic, or VoiceThread) to capture and distribute presentations. Students with access to iPads can use free/low-cost online services such as Educreations and Explain Everything to create and submit handwritten assignments (useful for math proofs, for example).
  • Free Web conferencing services (optional). With Google Hangouts, students can conduct synchronous presentations—useful for hosting live post-presentation Q&A sessions.

6. Students expect more visually rich and interactive materials delivered via screen than they do from an in-class experience. Face-to-face classes are inherently visual and interactive. In an online class, however, the amount and quality of visual and interactive materials can vary widely. To avoid turning an online class into an old-fashioned correspondence course (“read the textbook, take a multiple-choice quiz, and contact the instructor if you have questions”), online courses should incorporate as much rich, relevant multimedia as possible—including, where appropriate, interactive multimedia assessments such as:

  • Drag-and-drop activities. Consider replacing one or two multiple-choice quizzes with a drag-and-drop quiz that requires students to order or categorize concepts visually.
  • Image-based activities. For highly visual subjects such as art appreciation or biology, replace one or two multiple-choice quizzes with a series of images and require students to “click” their answers.
  • Audio-based activities. Consider requiring students to take assessments by recording their answers in an audio-only mp3 file format.

7. Because an online course typically takes more time to teach than the same course taught face-to-face, containing instructor workload with regard to administering online assessments is important. When students and instructor are in the same place at the same time, giving verbal instructions and feedback is easy and natural. Online, much of the feedback instructors give must either be written (via discussion boards, for example), which takes longer to compose; or it must be mediated through technology. Rather than hand-grade a math assignment and simply hand it back in class, for example, an instructor must either locate and learn how to use tools to mark up the digital version of the assignment, or mark it up by hand, scan and digitize it, and send it back to the student. None of these steps is particularly difficult—but the extra minute or two, multiplied by the number of students and number of assessments, can add up quickly (to say nothing of the additional time the instructor must spend teaching students how to hand in such assessments and dealing with the technical glitches that inevitably arise).

Strategies for containing the time required to provide feedback on assessments include:

  • Group projects. Having small groups of students work together to write a paper, create a presentation, or produce some other product assessment reduces the number of assessments you’ll need to grade.
  • Peer-reviewed activities. Students post their work to a discussion board or a blog and then review two of their classmates’ submissions. This strategy ensures all students receive valuable feedback while helping reduce the time the instructor must spend reviewing each submission.
  • Student-led discussions. Artfully guiding multiple discussion boards containing dozens of student participants is time-consuming. For selected discussions, organize students into small groups and assign one student to manage and summarize each small-group discussion.
  • Representative submissions. Using a multimedia tool such as Jing or VoiceThread to dissect, just as you would in the classroom, a few representative submissions (such as a research paper containing an ineffective conclusion and one that cites sources incorrectly) lets you address the most common problems efficiently, saving your remaining grading time for more personalized, in-depth student-to-student communications.

It’s important to note that while implementation strategies differ in an online environment, the pedagogical strategies underlying assessments shouldn’t change when you move a face-to-face course online. Online or off, assessments need to align with course objectives and provide a consistent measurement of student learning. If having students hand-write math proofs so that you can assess understanding and grant partial credit makes sense in a classroom, for example, chances are it makes sense online, too. The pedagogical strategy (hand-written homework) stays the same; only the implementation strategy (having students create and sending handwritten work as a digital file) needs to change.

Reference:
Watson, G. & Sottile, J. (2010). Cheating in the Digital Age: Do Students Cheat More in Online Courses? Online Journal of Distance Learning Administration, Volume XIII, Number 1. Retrieved online October 1, 2013 from http://www.westga.edu/~distance/ojdla/spring131/watson131.html

Emily Moore is an instructional design consultant at Boise State University, eCampus Center – Extended Studies.

email
Add Comment

Tags: , , , ,


Comments

CyberProf | December 2, 2013

Excellent points as well as strategies! I currently use Skype for webconferencing discussions in my online courses instead of the traditional discussion thread, and am in the process of preparing a proposal for my institution to adopt Adobe Connect for this purpose. Once students adapt to this synchronous approach, they find that this method is more beneficial for a number of reasons including immediate feedback/clarification from professor and peers and a real sense of community. I also use this format for student presentations, and students use Skype to work on group projects together. Since the synchronous method closely mimics a classroom setting, I am no longer burdened by all the extra time it takes to participate in and evaluate the discussion threads. I also assign students as discussion facilitators which they relate is a wonderful opportunity to build leadership skills. I believe that synchronous online courses may be the future of virtual learning as this method affords students and faculty the ability to experience face-to-face interaction from the comfort of their own homes.

Emily Moore | December 2, 2013

CyberProf,

You're absolutely right–synchronous is fabulous for all those reasons you mention. (I especially like the idea of assigning students as discussion facilitators.) The only drawbacks I've encountered are the fact that synchronous meetings tie students to specific times (which can be an issue for adult learners with full-time jobs and families) and also the fact that some synchronous tools can be problematic in terms of stability/ease of use. But if students are able to commit to occasional synchronous meetings and are provided with step-by-step instructions on how to use the synchronous tool–Skype, in this case–the drawbacks are effectively minimized.

Thank you for your comment!

Shana | December 2, 2013

Is there an article somewhere that describes step-by-step how to create a drag and drop quiz?

Susan | December 2, 2013

SoftChalk is one authoring tool for those type of quizzes. Studymate, Articulate, and other online authoring tools offer similar quiz or self-feedback interactive activities.

Emily Moore | December 2, 2013

Shana,

As Susan mentions, the step-by-step instructions you'll follow will depend on the software you're using to create the drag-and-drop quiz. I've worked with several different software products that do this, and my current favorite is Articulate Storyline (although in addition to the products that Susan lists, Raptivity and Adobe Captivate can also be used).

Hope this helps!

Jim Whitman | December 2, 2013

Excellent article clarifying some of the confusing aspects of beginning to teach online courses. I teach online courses from our Ecuador branch campus of Broward College to other Broward branch campuses in Peru and Bolivia, and next year in other countries. I use Desire2Learn and Moodle (preferably-it's easier for everyone). I create 10 to 20 minute Keynote slide voiced over presentations-as I would do for a f2f course. Then, once a week, I conduct a 45 minute to 1 hour f2f course using Go2Meeting (Skype isn't up to what I need in consistency and synchronous communications with a class of 25 students. But, as we all know, the real secret, and greatest challenge, is being in touch so students don't feel out of touch.
Jim Whitman, swhitman@broward.edu

Emily Moore | December 3, 2013

Jim,

Working from Ecuador sounds exciting! I've had good experience with GoToMeeting, too; it's a great option for institutions that have purchased it. I love hearing how keyed in you are to the need for online teachers to take the lead on synchronous meetings in particular and interactions in general. I agree that this is vital. Without frequent teacher-student interactions (however that happens), an online course is nothing more or less than a distance course.

Continued good luck to you!

charles s. | December 3, 2013

You might want to thoroughly test, If you haven't already, any product that you are going to buy. My experience with Adobe Connect was not good. But there were all kinds of reasons for that and we simply did not do enough homework/testing before we got it. Controlling "our" end is one thing, but controlling the student end is another thing all together; unless you control the hardware they use as well as the connections they are supposed to use. IT should be sure to include instructional staff and students in the testing process, including those who are not early tech adopters. My experience is the problems associated with many of technologies can end up frustrating users to the point that they simply opt out, loudly complain or both. Usually leaving the instructor to fix it.

I am not suggesting that anyone not proceed with any type of web conferencing (or any kind of) app, but I often wonder if we put enough time into understanding these systems before "buy" them, then get annoyed when they give the results we expected or unanticipated problems crop up. Just my musings…

CyberProf | December 3, 2013

Charles S.
Thank you so much for the heads up about Adobe Connect. Does anyone have a recommendation for webconferencing software that includes video for 25-30 people? I am open to any and all ideas!

PS: Emily, thank you so much for allowing me to use this thread to learn more about synchronous options!


Trackbacks

  1. 7 Assessment Challenges of Moving Your Course Online (Plus Solutions) | Faculty Focus | Patric Lougheed
  2. 7 Assessment Challenges of Moving your Course Online | CeL Community of Practice
  3. 7 Assessment Challenges of Moving Your Course Online (and a Dozen+ Solutions) | Tech Tips and New Technologies for Instructors
  4. 7 Assessment Challenges of Moving Your Course Online | USFSP Distance Learning Blog
  5. 7 Assessment Challenges of Moving Your Course Online (and a Dozen+ Solutions) | My Educational Technology Blog: A Place of Resources and Tools for Educators
  6. Teaching Tip: Assessing Students in Your On-Line OM Course | Jay and Barry's OM Blog
  7. Seven Assessment Challenges | Our English Class
  8. 7 Assessment Challenges of Moving your Course Online | FAU's eLearning Community of Practice
  9. Finals Week Teaching Tips | MCTC Center for Teaching & Learning
  10. Module 6: The Challenges of Online Assessment | To Teach One Needs To Learn