The Absent Professor: Grading Assignments for MOOCs

As Massive Open Online Courses continue to gain popularity and media coverage, much of the focus has been on learning online vs. learning in a classroom. However, just as important as learning itself, is how to evaluate the learning. For most MOOCs, this comes in the form of tests and quizzes. The problem presents itself mainly in humanities courses, where Professors are simply unable to grade written assignments for all the students enrolled. As a result, MOOC providers such as Coursera and EdX have developed new techniques to help solve this problem.

Getting Creative

In an article written in The New York Times on November 2, 2012, writer Laura Pappano gives a general overview on MOOCs, and the questions surrounding their validity as educational classes. One of the biggest concerns that she addresses in her article titled “The Year of the MOOC” is how assignments are graded. While multiple choice and true/false questions are standard for computer grading, written assignments force professors to “get creative.” She continues to explain how Coursera has developed the process of “peer grading” in order meet these needs. This works by making an individual grade the paper of five peers, before seeing feedback for their own work (which will have been graded by five people). Pappano adds that Coursera is also in the process of developing software to help eliminate the “bad graders”. By comparing the scores of all five peers, Coursera will be able to flag people who grade inaccurately, and will therefore “give their assessment less weight.”

Colorado State University Professor (and former MOOC student) Jonathan Rees wrote an article on March 5, 2013 in which he expressed his disapproval. He says in his article titled “Peer Grading Can’t Work“, that while peers often grade accurately, it is the comments that truly matter, and are unfortunately often sub-par. He stresses how learning from mistakes on tests are fundamental in college education, and is unsure whether MOOCs can deliver helpful responses. Rees states that this will not impact those taking MOOCs simply for expanded knowledge, but “its victims will be the future students who take MOOCs to earn college credit.” To him, these students will be deprived of the necessary instruction on writing, argument forming, and the appreciation of literature that are necessary for success.

Instant Feedback

On April 4, 2013, John Markoff wrote an article for The New York Times titled “Essay-Grading Software Offers Professors a Break“, which explains how MOOC provider EdX has created new artificial intelligence software that grades written assignments instantly. According to Markoff’s report on research done in this area, the quality of these grades are “similar to the variation you find from instructor to instructor.” However, Markoff also explains how many people, such as MIT researcher Les Perelman, criticize the effectiveness of the software, and can often fool it with “nonsense essays” that somehow receive “high marks.” Perelman also started a petition against the program (now up to 2,000 signatures), in which they claim that computers are unable to detect many of the essentials that compose good written communication. Despite this, Markoff ends his article optimistically, referencing University of Akron in Ohio professor Mark D. Shermis and saying “In his view, the technology — though imperfect — has a place in educational settings.”

Both of these articles show that while the ability of MOOCs to accurately grade papers is still developing, it may be on the rise. Although they are likely to remain imperfect, will MOOC grading nonetheless one day become effective? Or, will the absence of Professors in this vital area create a hole that will eventually sink the entire ship?

This post is part of a series on “Making History Online” that involves an examination of open online learning in the field of history funded by the Mellon Digital Humanties grant.  Students and faculty at the House Divided Project at Dickinson College  are collaborating this summer on a new open, online course called, “Understanding Lincoln,” taught by Prof. Matthew Pinsker and covering ways to teach Abraham Lincoln’s legacy using close readings of his most important writings.  This new type of online course represents a unique partnership between Dickinson College and the Gilder Lehrman Institute of American History.  The course is available for both graduate credit and free participation.  Registration for the course closes on Friday, July 19, 2013.  For more information, go to https://www.gilderlehrman.org/programs-exhibitions/understanding-lincoln-graduate-course