TESTING: IT’S ALL ABOUT HOW YOU HANDLE IT
by Corey Ford

When I went down to Charlotte I was skeptical. I had heard that standardized testing drove the turnaround of its public school system – and I wasn't buying it: Weren't teachers dumbing down the curriculum? Weren't they “teaching to the test?” And what's the point of so much testing anyway?

But what I learned in Charlotte is that testing improves teaching. The key is the data: how you collect it and how you use it.

Every nine weeks, bubble sheets and No. 2 pencils are handed out to all of Charlotte's students. Teachers unlock their drawers and pull out stacks of pink packets about 10 pages thick. They are the quarterly exams – documents that will collect the data that drives everything that happens in the Charlotte-Mecklenburg school district from that day forward.

But that data is only as good as the content that's covered and the questions that are asked. The purpose is to find out if students have learned the skills they need to know. The test is pointless without a clear definition of those skills.

In North Carolina, the state determines the curriculum. It publishes a book called the Standard Course of Study that lays out what skills North Carolina's public school children need to master in each grade. It then tests them on those skills in the annual End of Grade Test.

To prepare for that test, Charlotte's district office takes this unwieldy book and translates it into specific, clearly defined skills called curriculum objectives. A curriculum objective in math may be multiplying fractions, while in reading it may be identifying the main idea in a story. All of these objectives, when put together, add up to the skills a student needs to master by year's end.

The district office then takes each curriculum objective and gathers the best teaching minds around it. These experts use the best research available to develop a lesson plan to teach that objective.

The lesson plan has very specific steps and instructions, sometimes scripted down to what the teacher says, word for word. An inexperienced or substitute teacher may follow the script, while a veteran may embellish the lesson. Either way, the most effective lesson plan is in place for each curriculum objective.

Finally, the lesson plans are laid out in sequence and aligned with the school calendar in a pacing guide that insures teachers stay on track to cover all of the curriculum objectives before the End of Grade Test.

At the end of every nine weeks, a quarter of that year's objectives have been covered. It's then time to test how well the students have mastered them and how effectively the teachers have taught them.

But a test is only as good as its questions, which must challenge students to demonstrate mastery. “These are not rote memorization type questions,” says Susan Agruso, who oversees Charlotte's testing system. “They are thinking questions – students have to understand information and put pieces of information together to come up with the correct answer to a question.”

“For example, when a student reads a passage, he or she will be asked questions about the meaning of the passage, the author's purpose, the main idea,” explains Agruso. “A student has to read the whole passage, understand it and look at this whole piece of literature and say, ‘Oh… That's what the author wants me to understand. That's the tone he's using with it.' You have to understand the whole piece, comprehend it, put all those pieces together.”

But even with a challenging question, it's not good enough to ask it only once. “You can't determine mastery with one question, so we will ask a main idea question in several different ways for several different passages,” explains Agruso.

Over a range of questions, the student will demonstrate whether he or she has mastered, partially mastered, or failed to master the objective – useful data to a teacher. So when the clock stops at the end of the quarterly test, students will have been challenged multiple times on each objective that needed to be covered by that time of the year.

Immediately after the quarterly tests, students' bubble sheets are taken to district headquarters where assistant principals race to feed them through scoring machines. Within hours, all tests have been processed, creating four valuable reports – an individual student report, a class report, a school report and a district report.

It's what Charlotte does with these reports that makes all the difference.

Teachers wait anxiously to find out how their students performed, how well they taught the material, and how they need to teach differently tomorrow. Each teacher receives an Item Analysis Report that lists every student in her class and their answers to each question, so she knows exactly which items each student missed.

Even more critical is the Group Mastery Report, which lists every child in the class and describes all students' level of mastery on each objective. In a glance, the teacher sees the weak spots in her class.

“She gets a view of her classroom,” says Agruso. “Do I have a group of students who are struggling with fractions? I need to work with this group. Do I have a child who doesn't understand fractions? I have to figure out some ways to give him some extra help. Or is my whole class missing this concept? I have to build in some tutorial work for them so that they can do better on fractions.”

Weak spots in scores reveal the weak spots in teaching. “It allows me to see whether or not what I'm doing is working,” says Nicole Barrow, a fourth grade teacher at Highland Renaissance Academy. Her colleague Shonja Alexander agrees, “The test results have been an excellent resource to just go back and say, ‘Okay, maybe that little instruction I did didn't work for this group. Maybe I need to go back and…find something else that's going to reach them.'” Teachers compare scores among their team and figure out which techniques worked best.

The team uses the report to group students by their level of mastery. Students are shuffled out of their normal class arrangements into groups based on ability on a particular skill. Students who need intense intervention receive one-on-one attention from the teacher who teaches the concept best. Children who have mastered the skill explore higher-level work. A separate time period is dedicated to this re-teaching, allowing the regular class to stay on pace.

The principal receives a similar report, broken down by classroom rather than student, giving her an instant read-out on how her teachers are teaching. She uses this data to identify the best way to target the school's limited resources. The principal can focus teaching coaches, intensive training, and tutoring where they are needed most needed. The data enables the principal to be a better instructional problem-solver.

At the district level, all the data is brought together and crunched in many ways to help Charlotte's children succeed. At central headquarters, Susan Agruso can sit at her computer and compare one school versus another, or with the district as a whole. She can “drill down” to minute detail. She's just a few clicks away from telling you how a Hispanic eighth grader in Ms. Smith's class at Bishop Spaugh Academy is doing on factoring quadratic equations.

Real-time data means that district resources are constantly in play. Regional superintendents and instructional specialists gather around reports that list how each school is doing. They see which schools need help, devise a plan and send in rapid response teams the next day.

The experts designing lesson plans look at the data, too. It's a great measure of which lessons are working. They look at which curriculum objectives are giving teachers and students the most trouble and they re-tackle the problem.

“We always struggle with the same things year after year after year,” says Agruso. “The challenge is to break the pattern… so we can figure out how to teach it so they all learn.”

It's a system of constant improvement tailored to the immediate needs of student, teacher and school. None of them can hide from objective data. A child can't hide his weaknesses from the teacher, a teacher can't hide her weaknesses from the principal and a principal can't hide a school's weaknesses from the superintendent.

But it's not about blame. It's about targeting the weak spots in each child, teacher and school and combating them with best practices and resources.

When I first heard that Charlotte did a lot of standardized testing, my gut reaction was extremely negative. I thought Charlotte teachers must be dumbing down the curriculum by “teaching to the test.” But I was wrong.

“I think it's important to understand the difference between ‘teaching a test' and ‘teaching to the test.' And I would say the latter is a perfectly acceptable and desirable outcome,” says Agruso. “Teaching the test questions is inappropriate. And for the teacher to spend the bulk of the year just reviewing test questions is inappropriate. That's not teaching content.”

When the testing system was first established in Charlotte, some teachers tried to sequentially teach the test questions, but the data rooted them out.

“The sense of urgency at the schoolhouse led to some pretty bad practices. Teachers and principals with a lot of workbooks… a test prep kind of approach, which they learned very quickly had a negative impact on students and achievement,” explains Eric Smith, the superintendent who brought all this testing to Charlotte-Mecklenburg Schools. “In order to get the kind of outcomes we needed, you had to have a substantive understanding of the content by children.”

Smith found that the assessments actually raised the quality of instruction. “Teachers had to see the kind of questions, the complexity that would be demanded of children so that when they taught they would be teaching at a high enough level for kids to be able to be successful. So [frequent assessments] helped to drive us – not only what needed to be taught, but at what level it needed to be taught.”

“Our assessments are used to measure if the students have learned what it is you're supposed to be teaching,” explains Agruso. “When you teach the curriculum you are preparing children to be successful on the end of year test, which is a way of saying ‘teaching to the test.'”

Testing works in Charlotte because it is an integral part of a well-designed system that is dead serious about its educational goals. They determined the skills children need to learn, made a clear plan of how to teach those skills, designed a way to test mastery of those skills and created a system that immediately reacted to problems, identified solutions and created a virtuous cycle of constant improvement.

We need more testing like that.