Tag: Xavier University

Loveland-area resident Alice Culbertson, graduate student in Education at Xavier University, is conducting a research study that asks the following question: “Do Ohio parents believe that humans or computers should grade their children’s written essays on standardized tests?” Below is some general information about the topic of human vs. computerized scoring, followed by a short survey for parents to share their opinions on this issue. 

If you are an Ohio parent who has had at least one child take a standardized test in an Ohio school any time during grades 3-12, you are encouraged to complete a short survey to share your opinions on this issue.

Every year, millions of school children in America take state-mandated standardized tests. To manage the cost of such large-scale testing, many of the questions follow a multiple-choice format because computers can score the answers quickly and cost-effectively. Some questions, however, require written essay responses that are graded by humans, a slower and more expensive process. In an effort to reduce the time and cost associated with human scoring, several testing firms are working with technology companies to produce automated essay scoring (AES) software that can score essays as well as humans can.

One of the leading proponents of AES is Mark D. Shermis, Dean and Professor, School of Education, University of Houston–Clear Lake, who has conducted several studies in which computers were shown to score essays with comparable results to human scoring. Les Perelman, research affiliate at the Massachusetts Institute of Technology (MIT), former professor of writing and composition, and a vocal opponent of AES, has also conducted research in which computer scoring proved to be less accurate than human scoring. 

Shermis, AES software developers, and testing companies claim that the Artificial Intelligence (AI) has the capability to grade essays correctly and consistently. Perelman and primary, secondary, and post-secondary educators maintain that computers are incapable of evaluating a number of important elements of writing such as logic, tone, organization, and creativity, and should therefore not be used to grade essays. 

Pros of Computerized Scoring cited by Shermis and other Proponents of AES

• Several studies have shown that computers gave virtually identical scores to those given by humans on the same set of essays.
• Computerized scoring can save time and money spent on testing, and test results can be returned to families and schools more quickly.
• Humans who score essays are temporary employees who only need a bachelor’s degree in any subject, and the ability to adhere to a rubric (a chart that shows them how to score the essays).

Pros of Human Scoring cited by Perelman and other Opponents of AES

• Several studies have shown that computers can be fooled into giving high scores to poorly written essays based solely on the length and key words contained within the essay.
• Human scorers are able to evaluate important elements of writing that computers cannot, such as tone, logic, organization, and creativity.
• Computers can only score simplistic essays with scores comparable to those given by humans, meaning that test questions would have to be “dumbed down” in order to be accurately scored by a computer. 

There is virtually no research documenting how parents feel about the prospect of computers grading their children’s written essays.

The question of whether computers should score essays is a growing area of contention among educators, software firms, and testing companies across the country. Largely absent in this debate, though, is a sector of the population that has a personal stake in standardized testing scoring: the parents of those students who take the tests, whose educational paths may be affected by the scoring outcomes. While research has shown that parents have had differences of opinion as to whether their children benefit from standardized tests in general, there is virtually no research documenting how parents feel about the prospect of computers grading their children’s written essays.

If you are an Ohio parent who has had at least one child take a standardized test in an Ohio school any time during grades 3-12, you are encouraged to complete a short survey to share your opinions on this issue. Simply click the link below to access the survey. Please feel free to share this link with any other Ohio parents. Thank you!

https://kwiksurveys.com/s/VJ8A5d3V#/0   

All survey respondents will remain confidential, and the results of this survey are for classroom purposes only and will not be published. Results of the study will be available after January 2, 2018, and may be obtained by emailing a request for results to culbertsona@xavier.edu.