TESTING: IT’S ALL ABOUT HOW YOU HANDLE IT
by Corey Ford
When I went down to Charlotte I was skeptical. I had heard that standardized
testing drove the turnaround of its public school system – and I wasn't buying
it: Weren't teachers dumbing down the curriculum? Weren't they “teaching to
the test?” And what's the point of so much testing anyway?
But what I learned in Charlotte is that testing improves teaching. The key
is the data: how you collect it and how you use it.
Every nine weeks, bubble sheets and No. 2 pencils are handed out to all of
Charlotte's students. Teachers unlock their drawers and pull out stacks of pink
packets about 10 pages thick. They are the quarterly exams – documents that will
collect the data that drives everything that happens in the Charlotte-Mecklenburg
school district from that day forward.
But that data is only as good as the content that's covered and the questions
that are asked. The purpose is to find out if students have learned the skills
they need to know. The test is pointless without a clear definition of those
skills.
In North Carolina, the state determines the curriculum. It publishes a book
called the Standard Course of Study that lays out what skills North Carolina's
public school children need to master in each grade. It then tests them on
those skills in the annual End of Grade Test.
To prepare for that test, Charlotte's district office takes this unwieldy
book and translates it into specific, clearly defined skills called curriculum
objectives. A curriculum objective in math may be multiplying fractions, while
in reading it may be identifying the main idea in a story. All of these objectives,
when put together, add up to the skills a student needs to master by year's end.
The district office then takes each curriculum objective and gathers the best
teaching minds around it. These experts use the best research available to develop
a lesson plan to teach that objective.
The lesson plan has very specific steps and instructions, sometimes scripted
down to what the teacher says, word for word. An inexperienced or substitute
teacher may follow the script, while a veteran may embellish the lesson. Either
way, the most effective lesson plan is in place for each curriculum objective.
Finally, the lesson plans are laid out in sequence and aligned with the school
calendar in a pacing guide that insures teachers stay on track to cover all of
the curriculum objectives before the End of Grade Test.
At the end of every nine weeks, a quarter of that year's objectives have been
covered. It's then time to test how well the students have mastered them and
how effectively the teachers have taught them.
But a test is only as good as its questions, which must challenge students
to demonstrate mastery. “These are not rote memorization type questions,” says
Susan Agruso, who oversees Charlotte's testing system. “They are thinking questions – students
have to understand information and put pieces of information together to come
up with the correct answer to a question.”
“For example, when a student reads a passage, he or she will be asked questions
about the meaning of the passage, the author's purpose, the main idea,” explains
Agruso. “A student has to read the whole passage, understand it and look at this
whole piece of literature and say, ‘Oh… That's what the author wants me to understand.
That's the tone he's using with it.' You have to understand the whole piece,
comprehend it, put all those pieces together.”
But even with a challenging question, it's not good enough to ask it only once. “You
can't determine mastery with one question, so we will ask a main idea question
in several different ways for several different passages,” explains Agruso.
Over a range of questions, the student will demonstrate whether he or she has
mastered, partially mastered, or failed to master the objective – useful data
to a teacher. So when the clock stops at the end of the quarterly test, students
will have been challenged multiple times on each objective that needed to be
covered by that time of the year.
Immediately after the quarterly tests, students' bubble sheets are taken to
district headquarters where assistant principals race to feed them through scoring
machines. Within hours, all tests have been processed, creating four valuable
reports – an individual student report, a class report, a school report and a
district report.
It's what Charlotte does with these reports that makes all the difference.
Teachers wait anxiously to find out how their students performed, how well
they taught the material, and how they need to teach differently tomorrow. Each
teacher receives an Item Analysis Report that lists every student in her class
and their answers to each question, so she knows exactly which items each student
missed.
Even more critical is the Group Mastery Report, which lists every child in
the class and describes all students' level of mastery on each objective. In
a glance, the teacher sees the weak spots in her class.
“She gets a view of her classroom,” says Agruso. “Do I have a group of students
who are struggling with fractions? I need to work with this group. Do I have
a child who doesn't understand fractions? I have to figure out some ways to give
him some extra help. Or is my whole class missing this concept? I have to build
in some tutorial work for them so that they can do better on fractions.”
Weak spots in scores reveal the weak spots in teaching. “It allows me to see
whether or not what I'm doing is working,” says Nicole Barrow, a fourth grade
teacher at Highland Renaissance Academy. Her colleague Shonja Alexander agrees, “The
test results have been an excellent resource to just go back and say, ‘Okay,
maybe that little instruction I did didn't work for this group. Maybe I need
to go back and…find something else that's going to reach them.'” Teachers compare
scores among their team and figure out which techniques worked best.
The team uses the report to group students by their level of mastery. Students
are shuffled out of their normal class arrangements into groups based on ability
on a particular skill. Students who need intense intervention receive one-on-one
attention from the teacher who teaches the concept best. Children who have mastered
the skill explore higher-level work. A separate time period is dedicated to this
re-teaching, allowing the regular class to stay on pace.
The principal receives a similar report, broken down by classroom rather than
student, giving her an instant read-out on how her teachers are teaching. She
uses this data to identify the best way to target the school's limited resources.
The principal can focus teaching coaches, intensive training, and tutoring where
they are needed most needed. The data enables the principal to be a better instructional
problem-solver.
At the district level, all the data is brought together and crunched in many
ways to help Charlotte's children succeed. At central headquarters, Susan Agruso
can sit at her computer and compare one school versus another, or with the district
as a whole. She can “drill down” to minute detail. She's just a few clicks away
from telling you how a Hispanic eighth grader in Ms. Smith's class at Bishop
Spaugh Academy is doing on factoring quadratic equations.
Real-time data means that district resources are constantly in play. Regional
superintendents and instructional specialists gather around reports that list
how each school is doing. They see which schools need help, devise a plan and
send in rapid response teams the next day.
The experts designing lesson plans look at the data, too. It's a great measure
of which lessons are working. They look at which curriculum objectives are giving
teachers and students the most trouble and they re-tackle the problem.
“We always struggle with the same things year after year after year,” says
Agruso. “The challenge is to break the pattern… so we can figure out how to teach
it so they all learn.”
It's a system of constant improvement tailored to the immediate needs of student,
teacher and school. None of them can hide from objective data. A child can't
hide his weaknesses from the teacher, a teacher can't hide her weaknesses from
the principal and a principal can't hide a school's weaknesses from the superintendent.
But it's not about blame. It's about targeting the weak spots in each child,
teacher and school and combating them with best practices and resources.
When I first heard that Charlotte did a lot of standardized testing, my gut
reaction was extremely negative. I thought Charlotte teachers must be dumbing
down the curriculum by “teaching to the test.” But I was wrong.
“I think it's important to understand the difference between ‘teaching a test'
and ‘teaching to the test.' And I would say the latter is a perfectly acceptable
and desirable outcome,” says Agruso. “Teaching the test questions is inappropriate.
And for the teacher to spend the bulk of the year just reviewing test questions
is inappropriate. That's not teaching content.”
When the testing system was first established in Charlotte, some teachers
tried to sequentially teach the test questions, but the data rooted them out.
“The sense of urgency at the schoolhouse led to some pretty bad practices.
Teachers and principals with a lot of workbooks… a test prep kind of approach,
which they learned very quickly had a negative impact on students and achievement,” explains
Eric Smith, the superintendent who brought all this testing to Charlotte-Mecklenburg
Schools. “In order to get the kind of outcomes we needed, you had to have a substantive
understanding of the content by children.”
Smith found that the assessments actually raised the quality of instruction. “Teachers
had to see the kind of questions, the complexity that would be demanded of children
so that when they taught they would be teaching at a high enough level for kids
to be able to be successful. So [frequent assessments] helped to drive us – not
only what needed to be taught, but at what level it needed to be taught.”
“Our assessments are used to measure if the students have learned what it is
you're supposed to be teaching,” explains Agruso. “When you teach the curriculum
you are preparing children to be successful on the end of year test, which is
a way of saying ‘teaching to the test.'”
Testing works in Charlotte because it is an integral part of a well-designed
system that is dead serious about its educational goals. They determined the
skills children need to learn, made a clear plan of how to teach those skills,
designed a way to test mastery of those skills and created a system that immediately
reacted to problems, identified solutions and created a virtuous cycle of constant
improvement.
We need more testing like that.
|