Can software spot a great essay?

Computers advance as writing assessment tool that saves teachers time
By: | Issue: June, 2015
May 26, 2015

Three times each year, middle school students in Birmingham, Michigan, take a 30-minute, timed writing assessment online.

The test is done through Criterion, an ETS online writing evaluation service. Student writers receive immediate feedback on their grammar and mechanics, as well as links to exemplary writing that displays techniques the test-takers need to work on.

Remember, a computer tool, not the teacher, is doing this.

As Common Core standards require students to write extensively across the curriculum, such automatic assessment tools can help ease the grading burden for teachers. More districts are using online writing assessment tools to save teachers time and to give students writing practice that includes immediate analysis.

These programs bring the futuristic intelligence of machines to today’s classrooms, helping to achieve rigorous educational standards with high technology.

“Students learn best when they have immediate feedback on their writing,” says Deborah Gollnitz, curriculum coordinator at Birmingham Public Schools, who views Criterion as a learning tool more than a scoring tool. “It is a learning tool for students and a teaching tool for teachers.”

For instance, when students are regularly reminded to use active voice or avoid sentence fragments, they will eventually internalize those techniques. And teachers can see where students are tripping up on grammar and can create a lesson on problem areas for the very next day. Teachers also might pull small groups of students out to work on certain skills.

Students nationwide need more effective, consistent training in writing. In 2012, the National Assessment of Educational Progress (NAEP) administered the Writing Computer-Based Assessment to about 13,000 fourth graders.

The assessment measured students’ abilities to develop, organize and express ideas to achieve a purpose and address a specific audience. About 14 percent of the responses demonstrated competent or effective writing skillsÑmeaning the writing was fairly balanced, it reflected some awareness of the audience, and demonstrated proper grammar and mechanics.

In addition to more and longer writing assignments, the Common Core increases focus on writing on computers and includes computer-based writing assessments. While technology is no substitute for real, live teachers, the consistent use of online grading tools in the classroom is helping to improve students’ writing, says Luci Willits, deputy executive director at Smarter Balanced.

Following are case studies from three districts that use online writing assessment tools to build students’ writing skills.

Placentia Yorba Linda USD, Orange County, California

About 15 years ago, Kraemer Middle School in Placentia Yorba Linda USD began using ETS’ Criterion online writing evaluation as a pilot program with honors classes. Because of the program’s ability to individualize assignments to include various levels of studentsÑas well as its ability to give users more writing chances with timely feedbackÑthe school has a subscription for every student, says Shane Twamley, language arts teacher.

This feedback is based on natural language processing research that is specifically tailored to analyzing student responses.

With online assessment, students can resubmit their essays several times, allowing them to practice peer-editing and revision.

“Student writing can tend to be a one-shot deal, but now our students enjoy the editing process, as they see how it can improve their writing and scores,” Twamley says. “Editing and revision simplifies the task of the teacher so they can do a better job of reading for the students’ completion of ideas.”

Not only is grading time reduced, but teacher comments about content, organization and style are saved in an online feedback library so students can refer to them throughout the year.

“Many times, I have seen my hours of correcting be thrown in the wastebasket as soon as a student gets a grade,” Twamley says. “Instead of taking a week to grade 180 essays, my turnaround time is sometimes two days.”

And the tool provides students and teachers with graphs showing the frequency of common mistakes, and this data can guide a teacher’s instruction. Teachers can set the assessments to focus on specific skills, such as spelling, time and whether students are allowed to use a thesaurus on the assignment.

Twamley doesn’t believe technology will replace teachers. But he does find automatic online assessments valuable because they work with the student to self-correct most mistakes.

And it allows teachers to focus on “the logic behind the writing,” he says. “These assessments take advantage of students’ enthusiasm at the time of writing rather than making them wait until they’ve forgotten the assignment before receiving comments.”

Cherokee County School District, Canton, Georgia

The Cherokee County School District uses Pearson’s WriteToLearn in some fourth- and fifth-grade classrooms and in grades 6 through 8. The program provides detailed analysis of the student’s writing skills, and provides feedback on strengths and weaknesses, says Bobby Blount, assistant superintendent for accountability, technology and strategic planning at Cherokee County schools.

The web-based program helps students practice essay- and summary-writing skills and expand vocabulary by instantly assessing work. It evaluates the meaning of text, not just grammar and spelling.

As students work through writing assignments, they receive personalized feedback, hints and tips to encourage, instruct and reward progress. Teachers use the reports to help understand the students’ strengths and weaknesses, enhancing discussion about effective improvement strategies.

As more standardized tests are administered via computer, using a tool like WriteToLearn also helps students get accustomed to writing with keyboards rather than with pencil and paper, Blount says. The regular practice helps improve test scores over the long run.

The program is “a huge time saver for teachers” because it can automatically assess errors in spelling, grammar and punctuation, and it can flag suspected plagiarism, allowing students to self-correct, Blount says. “Teachers are able to help parents understand and better support those identified weaker areas in the students’ writing development, and monitor the students’ growth over time.”

Still, Blount prefers a teacher because “writing is subjective when it comes to content,” he says. “Teachers can better help a young writer explore, identify and refine his or her style of writing expression,” he says. “[They] can help students understand style, idea creation, content, good word selection, organizing a story so it makes logical sense to the reader, sentence fluency, and capturing the expression or emotion behind what is written.”

Portland Public Schools, Portland, Connecticut

Before Common Core standards, Connecticut used Measurement Inc. to conduct annual state testing, and districts had free access to the company’s online writing assessment tool. Now that Common Core assessments have replaced state testing, the tool, now known as PEG Writing, is not free.

But Portland Public Schools continues to use it. “It offers a good first glance and takes a lot of work off the teacher,” says Donna Mingrone, Portland’s director of curriculum, instruction and technology. “It helps students clean up the writing so the teacher can focus on content.”

Unique assessment tools make PEG almost like a living, breathing teacher. It instantly scores student writing in multiple categories, and provides immediate evaluation by generating feedback for each essay within seconds.

The automatic feedback in each score report provides students with a good foundation to improve their writing skills, but a teacher’s analysis is still crucial. With PEG Writing, teachers can score essays for textual evidence and content accuracy using a 0-to-3-star scoring system.

Teachers use PEG Writing in grades 3 through 8, and can write their own writing promptsÑthat is, statements or questions that inspire a response in the form of an essay. They can also use prompts from the company’s bank, which uses data from thousands of student writing samples to score writing effectively.

As students create writing samples, PEG highlights grammatical and spelling errors. Writing is measured by six traitsÑideas, organization, voice, word choice, sentence fluency, conventions and presentationÑwith five possible points in each trait, for a total of 30 points.

When students get their scores, the tool also asks leading questions to help them improve writing. For instance, it may ask, “Have you used specific details for your argument?” or “Have you supported each idea?”

And the tool is used to help improve students’ writing across the curriculum. For example, sixth-grade science teachers are responsible for teaching expository writing, which is similar to what should be in a lab report. Teachers have students write research papers in PEG. They score papers for science content but PEG helps in scoring grammar and other items they don’t usually teach, Mingrone explains.

Similarly, social studies teachers teach argumentative writing, which fits in with current events and social studies content standards. And math teachers teach writing that includes problem solving. In these cases, Mingrone says, they use PEG for structure and support in grading the mechanics of writing, which frees the content area teachers to focus on content. 

Nancy Mann Jackson is a freelance writer based in Alabama.

Interested in edtech? Keep up with DA's Future of Education Technology Conference®.