Loveland-area resident Alice Culbertson, graduate student in Education at Xavier University, is conducting a research study that asks the following question: “Do Ohio parents believe that humans or computers should grade their children’s written essays on standardized tests?” Below is some general information about the topic of human vs. computerized scoring, followed by a short survey for parents to share their opinions on this issue.
If you are an Ohio parent who has had at least one child take a standardized test in an Ohio school any time during grades 3-12, you are encouraged to complete a short survey to share your opinions on this issue.
Every year, millions of school children in America take state-mandated standardized tests. To manage the cost of such large-scale testing, many of the questions follow a multiple-choice format because computers can score the answers quickly and cost-effectively. Some questions, however, require written essay responses that are graded by humans, a slower and more expensive process. In an effort to reduce the time and cost associated with human scoring, several testing firms are working with technology companies to produce automated essay scoring (AES) software that can score essays as well as humans can.
One of the leading proponents of AES is Mark D. Shermis, Dean and Professor, School of Education, University of Houston–Clear Lake, who has conducted several studies in which computers were shown to score essays with comparable results to human scoring. Les Perelman, research affiliate at the Massachusetts Institute of Technology (MIT), former professor of writing and composition, and a vocal opponent of AES, has also conducted research in which computer scoring proved to be less accurate than human scoring.
Shermis, AES software developers, and testing companies claim that the Artificial Intelligence (AI) has the capability to grade essays correctly and consistently. Perelman and primary, secondary, and post-secondary educators maintain that computers are incapable of evaluating a number of important elements of writing such as logic, tone, organization, and creativity, and should therefore not be used to grade essays.
Pros of Computerized Scoring cited by Shermis and other Proponents of AES
- Several studies have shown that computers gave virtually identical scores to those given by humans on the same set of essays.
- Computerized scoring can save time and money spent on testing, and test results can be returned to families and schools more quickly.
- Humans who score essays are temporary employees who only need a bachelor’s degree in any subject, and the ability to adhere to a rubric (a chart that shows them how to score the essays).
Pros of Human Scoring cited by Perelman and other Opponents of AES
- Several studies have shown that computers can be fooled into giving high scores to poorly written essays based solely on the length and key words contained within the essay.
- Human scorers are able to evaluate important elements of writing that computers cannot, such as tone, logic, organization, and creativity.
- Computers can only score simplistic essays with scores comparable to those given by humans, meaning that test questions would have to be “dumbed down” in order to be accurately scored by a computer.
There is virtually no research documenting how parents feel about the prospect of computers grading their children’s written essays.
The question of whether computers should score essays is a growing area of contention among educators, software firms, and testing companies across the country. Largely absent in this debate, though, is a sector of the population that has a personal stake in 
standardized testing scoring: the parents of those students who take the tests, whose educational paths may be affected by the scoring outcomes. While research has shown that parents have had differences of opinion as to whether their children benefit from standardized tests in general, there is virtually no research documenting how parents feel about the prospect of computers grading their children’s written essays.
If you are an Ohio parent who has had at least one child take a standardized test in an Ohio school any time during grades 3-12, you are encouraged to complete a short survey to share your opinions on this issue. Simply click the link below to access the survey. Please feel free to share this link with any other Ohio parents. Thank you!
https://kwiksurveys.com/s/VJ8A5d3V#/0
All survey respondents will remain confidential, and the results of this survey are for classroom purposes only and will not be published. Results of the study will be available after January 2, 2018, and may be obtained by emailing a request for results to [email protected].
Here is what Anton Batey of the Mises Institute has to say about America’s over-reliance on standardized testing:
https://mises.org/library/trouble-no-child-left-behind
“At first glance, the concept of standardized tests seems reasonable. Children should be tested, and the tests are clear indicators either of how intelligent they are or of how much the school is teaching them. But what is the school “teaching” them, exactly? The answer is simple but unfortunate: they’re teaching them how to take the test.
Linda Valli, Maryland associate professor of education, conducted a long study on the federal program and determined that standardized testing “actually undermined the quality of teaching in reading and math” and that the decline in teacher quality and tangible information being taught to the students is because of “the pressure teachers were feeling to ‘teach to the test.'”
Alfie Kohn, author of over a dozen books on education, parenting, and anthropology, decries NCLB’s “overemphasis on standardized testing and punitive sanctions.” He generally disparages the program, saying that the “law is not about narrowing the achievement gap; its main effect has been to sentence poor children to an endless regimen of test-preparation drills.” And furthermore, “even if the scores do rise, it’s at the expense of a quality education.” According to a 50-state survey by Teachers Network, a nonprofit education organization, only 3% of teachers think No Child Left Behind helps them teach more efficiently.
One infamous criticism that English teachers gave concerned the time spent on the proper use of a comma as opposed to on developmental writing skills. As Richard and JoAnne Vacca noted in their book, Content Area Reading, “good readers are often good writers,” and “wide reading improves writing.” However, since the federal, standardized tests place more emphasis on grammatical correctness than on reading comprehension, in the class, reading is sacrificed to punctuation precision. Virtually no person, however, would seriously argue that in the real world, reading comprehension is less important than knowing where to put a comma or knowing what verbs and nouns are. This is especially true in the real world of contracts, newspapers, etc.
NCLB is simply a way for the federal government to tighten its grip on schools by threatening them with punishment. Those who control the schools control the future. The tests and regulations indirectly control what children learn in school (and what they do not learn in school).
“Those who control the schools control the future.”
More importantly, what are the results of the program? One should keep in mind, however, what Kohn said regarding the scores: the higher test scores may come at the cost of learning. However, in 2006, for example, math and reading test scores dropped significantly, showing that only 32% of high-school students were proficient in math.”
HC,
Thank you for your interest in this topic. As you noted, the debate as to the value and necessity of standardized testing continues to rage on, with no foreseeable end in sight. Another debate in this area is brewing, though, which is the prospect of computers being used to score essays, as a less time-consuming and less-costly alternative to having humans score essays. This particular study does not intend to debate the merits of standardized testing; rather, it simply seeks to survey Ohio parents as to their feelings about the issue of computerized vs. human scoring on their own children’s standardized tests. Thank you again for your interest.
When humans score, essays are scored two times. If the two scorers don’t agree, the essay is scored by a third scorer. Some schools use holistic scoring, which is more concerned with intent. A computer can’t determine that aspect.
Kgarbar, thank you for your interest in this topic. It is true that some states require having 100% of their standardized tests scored by humans, with a third scorer (typically a supervisor) reviewing an essay on which the first two scorers disagree. However, some states require only a portion of the essays to be read twice, likely due to the time and cost required for having two readers for all essays. Each state that engages in standardized testing has its own contract with various test scoring firms around the country, and each state provides its own unique scoring guidelines to which the testing firms must adhere. Thank you again for your interest.