Is Washington willing to pay for all this?
Well, Washington is willing to help as we've helped before. ... But this is a
part of the teaching responsibility that each state has. ... Washington has
offered some assistance. In the legislation we have ... some support to pay
for the development of tests. But even if that should be looked at as a gift,
it is the state's responsibility to do this.
I asked a test maker about the Bush-Paige plan ... and she said, "It's not
going to happen. ... The resources aren't there." So if someone in testing,
who's committed to testing, says that -- it's a terrifying idea.
You want to know what I have for her? Watch. That's it. Watch. It's going to
happen. ... It's going to get done, because it is the right thing.
See, I make this observation. Why is there such a strong defense of the status
quo when the status quo is failing us? ... What solution are they bringing to
the table, these detractors? Are they satisfied with the progress we're making
One of the phrases that keeps coming up, in terms of doing it right, is
"disaggregating the data." Would you explain what that means?
Most of what has happened in the past has happened as a result of our dealing
with averages. Let's take a school, and we're going to say the average score
at this school is X. That doesn't tell us anything about the individual child.
First, we want to disaggregate the data by the different groups in the school.
Maybe we want to look at the disadvantaged population, the Anglo population,
African-American population, Hispanic population, Asian population. ... And
then we want to disaggregate it even deeper ... all the way down to the
Because we're teaching children, we're not teaching schools. You know, for the
school to be making progress doesn't tell us enough about what the individual
children are doing. ... I'll just give you an example. ... Let's take this
school in a major city -- and this is a real example. ... It has 40 or 50
national merit scholars every year. It's a school of the first order no matter
how you measure it. ... But when you disaggregate the data, you find out that
the Hispanic students in the school are not doing well. So they are, in
effect, left behind. ... You won't know this until you disaggregate the data,
past the average. ... Now that you know this, you have a responsibility to do
something about this. ...
Are the tests we use an adequate measure of whether kids are really
Well, yes. ... That speaks to the quality of the tests. ... And we know there
are poor tests out there. We know that there's improper use of tests out
there. But those are things that we're going to work against and try to get
fixed. The bottom line is, we have to have some system that gives us the
information that we need, which is the degree to which the goals, the
standards, are being achieved by the students. ...
Why has it taken us so long to say, "Hey, we need standards"?
... Let's think about that for just a minute. We're saying that in schools we
should have standards. And that turns out to be a controversial discussion.
Would that be the case in the hospital district, or in the public highways or
in the building industry? Everybody else takes standards as the ordinary. And
we're taking standards as something that is extraordinary? I think that gives
you some clue on how much we need to change. ...
Some states have lowered the bar. Do you worry that states may be wavering?
Well, what we know about this from experience in Texas is that it is tough
stuff, and it's not tidy at all. A lot of blood letting, a lot pressure
points. ... I'm reasonably certain some states are [wavering], because it is
politically tricky, you know, and requires great political will. ... So we are
going to have states that are going to waver. And what we're going to try to
do is support those and try to help as much as we can. ...
So you're saying, "Stay the course." ...
Stay the course. Absolutely, take the pain, stay the course. ...
He is the president of Achieve Inc., a
nonprofit organization created by business leaders and the nation's governors
to promote standards-based education reform.
[What is] your reaction to the Bush proposal for testing all kids in grades
3 through 8, in math and reading. Is that a good thing?
It has clearly worked for Texas. [But] this is a big, varied country and I'm a
little cautious about wanting to take a model that has worked in one place, and
think it can automatically produce the same kinds of results elsewhere. The
cautions have a lot to do with my anxieties about the quality of tests and the
way in which tests will be used.
If we're not careful the states may take the sort of easy way out, and go to
the least expensive, off-the-shelf tests to fill in the gaps in the current
testing programs. Again, most states, as you probably know, are not currently
testing every grade in reading and mathematics, grades 3 through 8, so a lot of
states are going to be in the business of having to develop or purchase tests
to fill in the gaps. ... So one concern really has to do with making sure that
there aren't disincentives to really invest in quality tests that really are
aligned with standards. ...
Are tests driving the system?
... This is supposed to be a movement about ... "standards-based" reform, and
in some places, unfortunately, it's becoming test-based reform, or
accountability-based reform. [We're] trying to help people get back to the
centrality of the standards themselves, that is, asking some pretty tough
questions: Do we have the standards right? Are they clear? Are they measurable?
Are they reasonably economical? That is, are the standards focused only on what
really are the most essential things that kids need to know? ...
In some places what we've seen is people sort of moving right past the
standards and simply focusing on the tests themselves. And there you do get
some distortions in the system. If you've got high-quality tests, tests that in
some measure one could describe as worth teaching to, then I worry less about
the role that tests have played. ...
I should say, no education program should be focused solely on preparing kids
for tests. I mean, even the best tests are not worth teaching to exclusively.
... The common denominator in good schools is that, yes, people are mindful of
the tests, they know there are tests out there, but that's not what they're
focused on. They're focused on high-quality instruction for all kids, based on
a sort of rich, challenging curriculum, ideally aligned with standards. ...
What's your biggest fear?
... My biggest fear is that we won't have the patience to really stay with this
strategy long enough to work through a lot of these very difficult, thorny
issues: getting the standards right; getting the tests right; really figuring
out what high-quality sustained support for teachers looks like; getting the
curriculum supports in place.
And how long will this take?
Some of our critics like to say, "Look, the 'Nation at Risk' report ... was
1983. [It's] almost two decades later. Haven't we given this movement time
enough?" If you ask the question, even in the states that have been furthest
along and had really good leadership, how many years have there really been
changes in classrooms? ... You're talking about three or four years. And my own
state, Massachusetts, is a good example. A law was passed in 1993, the first
set of consequences for kids [would be in] 2003. Ten years looked like ample
time. The reality is, changes in classrooms have really only kicked in in the
last two or three years. ...
I think it's probably going to take a generation before we really can make the
judgment. ... It took us a long time to get into the condition that many of
our schools are in. This is not something that is going to happen in a year, or
two, or three.
She is a principal partner at The Education Trust, an independent nonprofit organization devoted to
reform in K-12 education.
Why did we need standards?
... Well, we still do really have two school systems in America: one school
system that educates more fortunate kids, white kids, and another system that
educates lower-income kids and minority kids. And the expectations of those two
systems were, prior to standards, very different expectations. What standards
do is say that every kid in this state, no matter what color she is, no matter
where her father works, will know the same sorts of information, and have the
same sorts of skills. ...
A lot of us have expectations based on race and class. And what you see, and
what you saw in Texas -- prior to when Texas said, "We're going to hold you
accountable for the achievement of black kids, Latino kids, poor kids" -- you
had teachers in Texas who really thought that Mexican-American kids could not
learn at the same level as white kids. ... Standards said to those teachers
that all of the kids in your classroom -- whether they are white, whether they
are Mexican, whether they are affluent, whether they are poor -- are going to
master the same sets of skills. And when the teachers had that imperative, when
it became their job to teach all of the kids to the same level, lo and behold,
you saw much higher achievement levels from the kids. So one of the things that
standards teach us is that how well kids do has as much to do with the
expectations that we hold for them as the kids' own ability. Kids give us about
what we expect. And for black kids and poor kids, we expect far too little.
Tell me about "disaggregating the data." What does that mean?
For a long time, what schools and states have done is test kids, and then the
test scores are reported and evaluated, based on an average of the school. ...
What the average score does is hide the low performance in an integrated
school. It hides the low performance of minority kids and poor kids. And what
the Education Trust believes is that if you begin to hold schools accountable
for test scores that are not averaged, but are broken out -- so you look
separately at the test scores of low-income kids, black kids, Latino kids,
Asian kids -- you begin to set a common expectation for all of these kids, that
all of these kids will reach the same level. And you can drive bigger change in
schools than if you simply look at the average. ...
We have lots of schools that, on average, look OK. But if you take apart the
test scores, and you look at how minority kids are doing, and if you look at
how poor kids are doing, you find that a lot of schools have ugly little
secrets that they didn't want to talk about. ...
When President Bush was governor of Texas, he would use the expression, "The
soft bigotry of low expectations."
It's not very soft. It's pretty harsh, actually, if you were a low-income kid,
or a minority kid who's faced daily with low expectations. There's nothing soft
about it. And as you get older and older and move through school, and finally
get out of school, the harshness of low academic expectations increases.
Because what we do constantly to low-income and minority kids in this country
is say, "Because of her circumstances, because of his circumstances, because
their neighborhood is violent, because their family is disorganized, because of
the color of their skin" -- and we may not know anything about their family --
"we really can't expect anything from him or her. Can this kid really be
expected to learn?" ... And the kid gets out into a world that doesn't make
allowances for them. And the world becomes very, very harsh. ...
You've said [that "the Texas miracle"] is not a miracle. What
I think the disaggregation explains it. But there are two reasons I don't think
you want to call it a miracle. One is that Texas hasn't worked out all of its
problems yet. Texas still has a lot of work to do. ... The other reason I think
it's terribly damaging to call Texas a miracle, is it suggests that high
achievement for low-income and minority kids is somehow miraculous. It's not.
Low-income and minority kids can and do achieve. Their teachers are mere
mortals. ... The schools that get high achievement from low-income and minority
kids are doing the same things that we've done for white and affluent kids for
a very long time. It's good teachers, it's rigorous curriculum, and it's high
standards. And to call it a miracle makes people think that it's precious, that
it's special, that it can't happen again. It can happen again, and we see it
happening in schools all over the country, every day. ...
The Bush administration has a proposal, which has actually gone through, for
testing all kids in third grade through eighth grade in reading and math. ...
Is that a good idea?
The president's proposal ... is probably a good thing, because it will provide
more feedback to teachers in schools about student achievement. Right now what
a lot of states are doing is testing once in fourth grade, and then not again
until, say, eighth grade. There's an awful lot can happen. If your daughter
scored OK on her fourth-grade reading test, and then she gets to eighth grade
... and she's having a problem with reading, you've wasted a lot of time. You
didn't pinpoint the year where that disintegration started to happen. And so I
think that annual testing will help us get help to kids sooner who need it. ...
What makes you think we have the capability [and the resources] to produce
all these increased tests [required by the Bush plan]?
Well, first of all, it's absolutely imperative that the Bush administration put
the money on the table to support this proposal. We can't do testing on the
cheap. We can't do anything in education on the cheap. I mean, we can do it,
and we'll get results that pretty much show that we did it on the cheap. ...
It's not like the psychometricians don't have the know-how to put a good test
together. ... But we have to put the resources there to buy those tests and to
score those tests. And those are questions of political will, not the
sophistication of the psychometricians. ...
What are your fears? What could go wrong?
... We're asking for a very big change in the way schools do business. We're
asking schools to succeed with kids that they never even tried to succeed with
before, and we're saying that you've got to do a better job for these kids. And
my big fear is that it's going to take time, it's going to cost money, and it's
going to be uncomfortable for a lot of adults. And the question is, do we have
the courage, the political will, and the far-sightedness to do what's necessary
for kids who are unpopular and unpowerful? ... That's true whether we stay the
standards course, or run away from the standards course. But that's the biggest
fear -- that we won't have the stuff inside of us to do what's right for kids
we don't like.
He is a professor emeritus at UCLA and a nationally recognized authority on
The most profound misuse of educational tests these days is to employ a
traditionally constructed standardized achievement test ... and use those
scores as a reflection of school quality. These tests should not be used to
evaluate school quality. And many citizens think that should be done and many
educators can't disabuse them of that notion, because they don't know better.
If the tests aren't measuring what's being taught in school, what are they
Traditionally constructed standardized achievement tests measure a bit of
what's taught in school. But, by and large, they measure what children bring to
school, not what they learn there. They measure the kinds of native smarts that
kids walk through the door with. They measure the kinds of experiences the kids
have had with their parents. They do not measure, in the main, what is taught
Do you think the politicians know this? They're the ones who sign off on
Most educational policymakers, state board members, members of legislatures,
are well intentioned, and install accountability measures involving these kinds
of tests in the belief that good things will happen to children. But most of
these policymakers are dirt-ignorant regarding what these tests should and
should not be used for. And the tragedy is that they set up a system in which
the primary indicator of educational quality is simply wrong. ...
A lot of states are moving toward writing their own tests, so-called
criterion-referenced tests, and the feeling is that these tests will reflect
more of what's going on in classrooms. Is that how you see it?
Many states are currently abandoning off-the-shelf standardized achievement
tests and developing customized versions of those tests that supposedly relate
better to the state's curriculum content and what's taught in schools. But the
reality is these tests are typically created by the very same companies that
generated the original traditional standardized achievement test. And in many
instances, there's no reason to believe they function any differently than a
standardized achievement test. Just because a state says it has a so-called
criterion-referenced customized test does not automatically mean that that is a
What could be wrong with that test?
The customized tests that are being built for many states now have the same
kinds of items in them that you'll find in a traditionally standardized
achievement test. They're created by the same companies who have the same item
developers who create the same kinds of items, and they simply try to make it a
little more related to the state's curriculum. The fact is they function
identically to traditionally constructed standardized achievement tests. ...
The people in state departments of education frequently do not know how to
demand the creation of an alternative kind of test. You can have a test that
simply indicates what a student knows and doesn't know. But when these
customized tests are developed, there has to be a new vision of a different
kind of test, and many times that new vision simply doesn't sit there in the
state capital. ...
I'm going to guess that many people out there hear the president speak about
tests, and they hear everyone saying, "I want tough accountability in schools,"
and they think, "Oh, what's the big deal? We write a test." How simple is it to
do this and come up with a good test?
It's hard to write a test that does an accurate job in reflecting what students
have learned, and simultaneously give teachers and students guidance as to what
they should be promoting instructionally. It's very difficult to do that, and
there's an underestimate of that difficulty. ...
So what do you think when you hear President Bush calling for expanded
testing, every year in grades 3 through 8? Are we ready for that?
My concern about the president's call for more testing is that he and his
advisers may not recognize that if we have more of the same kinds of tests
we're currently using, good things will not happen in American education. I'm
not opposed to high-stakes testing. I think the proper kinds of high-stakes
test could be very useful for not only accountability, but for instruction. But
if we have same old same old, in this instance, we'll be harming the kids, not
helping them. ...
[What is at stake here?]
The public disenchantment with American schooling is profound. And many people
are looking for alternative solutions, whether they're charter schools or
vouchers or something else. I believe in the public schools. And I believe
those public schools can be made effective if they are not judged with the
wrong assessment tools, and they're given assessment tools that help them do a
better job. I want to see our public schools persist. But I think you have to
start focusing on a different way of measuring their performance. ...
We have to create tests that really do reflect how well teachers have been
teaching. Those kinds of tests will allow, I think, public education to
survive. The kind of tests that we're using now is setting up public educators
for absolute failure.
He is a professor at Boston College's Graduate School of Education who
specializes in the uses of educational testing in public policy.
[How do you respond to the argument that we need these tests?]
... We don't need these tests to tell us who is having a hard time and who's in
trouble in school. ... You can ask any classroom teacher and they can tell you
... who the kids are that are having trouble in math, reading, you name the
subject. We know that certain populations are poorly served, that there aren't
schools that aren't doing a good job for these. We know that. But now we've
added this test as a quasi-documentary of those problems that we've known for
years exist. And you're not going to test your way out of those problems.
What do you mean by that?
Well, just giving a test and getting the results back, that's not going to
necessarily change things for the better. You've got to do other things. You've
got to have "opportunity to learn" standards. You've got to have better
funding. ... You don't have level playing fields. ... We're not going to solve
the problem by pulling up the tree and looking at the roots every year and then
planting it back again. ...
"Opportunity to learn" standards. What does that mean?
... People [have] raised the issue that it isn't fair to impose a punitive
testing system without first making sure kids had the opportunity to learn
whatever the test was going to teach. ... [If you're going to test in
chemistry, then every kid ought to have] access to a lab and to a qualified
chemistry teacher. ... [But] starting right down in the kindergarten, a kid
[should have] a good meal before the kid comes to school. Politically, [the
"opportunity to learn" standards] went down in flames because a lot of the
governors didn't want to get into that whole area. ...
The test companies, essentially, put a warning on these things. [They] say,
"Don't use these for high stakes." But states do. How do you explain
It's a political question. And education has become a political issue. One of
the things that legislators or governors can do is they can impose tests. And
they don't have to worry about what goes on in classrooms, they don't have to
get into the messy details. They get numbers out that are quantifiable. So,
it's very attractive and it's cheap, relatively speaking. ... [G]iven the
overall education budget, testing is a very small part. It's getting bigger,
but it's a relatively small part. It's not nearly as expensive as equalizing
funding, putting money into in-service training of teachers, a whole series of
things that would cost a lot more money. As I say, you get quantitative results
that can go in newspapers and you can appear to be addressing the problem. And
that's why I say you're not going to test your way out of the problem. ...
So, the test companies, when they say, "Don't use this for high stakes
decisions," is that just being disingenuous? They're taking the money.
Sure they're taking the money. ... This is the other thing that people don't
fully understand. [In] other industrialized countries, testing is not a big
business run by publishing companies. It's run by departments of education or
by examination boards. But here it's a commercial enterprise.
Whoever the person paying the bill is. ...
There's a proposal [by] the president of the United States for expanded
testing. Is this a good idea?
I don't think it is. I think that we have enough testing now. We know who the
kids are that need help. We know the kids that aren't doing well. Putting another
layer of testing on top of all that we have is, if nothing else, going to
take away from instructional time. ...
Some say [this movement] goes way back to "A Nation At Risk" in 1983.
Well, I can go back to the 15th century in Italy where the schoolmaster's
salary depended on how kids did on a viva voce examination on the curriculum,
which at that time was pretty much rhetoric, and up until the 19th century
where you had payment by results in Australia, Jamaica, Great Britain, Ireland,
almost everywhere where the Brits went except Scotland. ... You had a
performance contract here in the United States in the '70s. You had minimum
competency movement in the '70s. This is not a new thing. And in every single
case, we know what the effects of those are. We know that teachers teach to the
exam. ... Now, people say [that's fine] if you can have tests worth teaching
for. Well, no test should replace a curriculum. ...
And it's predictable. We know that scores are low in the first few years of a
testing program and then they gradually go up as people catch on. Like in any
public policy thing, you can corrupt the social indicator. Whether it's
ambulance response time, on-time flight for airlines, arrest rates, these
indicators are corruptible and tests are corruptible. And you can have these
scores go up and not have the underlying learning that you seek to improve go
And I would predict five, six years down the road, there will be modifications.
As we see the corruption of the indicators, the teaching to the test, the kinds
of burdens that are being put on the few contractors that can handle these big
contracts ... you're going to see things change.
There are also reports of teachers saying, "I don't want to teach in this
[environment]," and leaving.
Well, that's that's one of the unintended consequences that needs to be
documented. That may be an urban myth, but we need to know. ... You can have a
very good goal [in] mind, but the unintended consequences of those goals very
often come back and bite you.
A professor at the College of Education at Michigan State University, he is the
national research coordinator and executive director of the U.S. National
Research Center, which oversees the United States' participation in the
Third International Mathematics and Science Study (TIMSS).
People talk about "content standards" and "performance standards," and we
pass quickly over the subject of "opportunity standards." What do we mean when
we say opportunity standards?
... Opportunity standards are the notion that if you're going to have
standards, content standards, and if you're going to have assessments to
measure those, then there ought to be somebody concerned about making sure that
all students will have the opportunity to have what's in those standards. ...
... We might have some children who simply have never ever heard of the concept
of congruence in geometry. And all of a sudden, that's on the test, and they
have no opportunity to actually study that, whereas another set of children in
another school spend two weeks on it. Guess who's going to do well on it? Does
that mean that the children who didn't do well are stupid, which is often the
label associated with failure in these things? No, it means that they never had
the opportunity to learn this.
Is opportunity standards the same as the money we spend?
Oh, I don't think so. No, no. I mean, it's certainly correlated with money. ...
The wealth of the community, in effect, speaks to other issues, so that in fact
wealthier communities have parents who, by their social class and their
interest, push harder on the school. So they make sure that the standards ...
the local standards ... what's done in the schools ... are at a higher level.
You go to a large urban district, where you don't necessarily have parents
pushing in that way, their opportunities may be radically different. They may
be spending eighth grade doing nothing much more than whole number arithmetic
and fractions. Whereas in the suburbs, those kids are studying the algebra and
In the recently released TIMSS data, we found
evidence to this effect, ever so strongly. When you looked at the districts
that participated, you had the suburban districts doing very well and the big
city districts doing poorly. ... The wealthier and the higher the social class
of the parents, the better the performance. But when you looked at the data
more subtly, what you found was there's another strong relationship almost
falling in line, which is ... what the curriculum opportunities were. So the
kids in the districts where the parents were wealthier and higher social class
had more demanding curricula than did those kids that attended schools [in]
districts in which the parents were of a lower social class.
So if you set coherent rigorous standards, then you can say, "OK, how do we
make sure everyone has the opportunity to meet those standards?" Or, "What does
it take to meet those standards?" So it becomes a rational process.
It does. Now it raises a whole host of other issues ... What is currently the
case is that the wealthiest districts have the most money put into schooling,
so that those that have more, get more. Maybe that model would have to be
modified, so that more money from the state or the federal government would
supplement those districts where kids had a longer way to go to begin with.
How would you make that political argument: "Let's spend more money on poor
I think the notion of equality, making it an equal playing field, is sort of an
American ideal. I'm not so sure that that couldn't be sold to people and to
parents. I do know that the state of Michigan did this. They removed the
property formula, the local property taxes, as the main vehicle for supporting
schools. It went to the state level, supported through state sales tax. So
there's much more of an equalization now in the state of Michigan across the
districts. So there are ways to do this. And I've heard of other states that
are doing similar sorts of things. ...
For years, the idea has been, if we could just equalize the spending, if we could just spend as much money on
the kids in poor areas as they do in the suburban areas, things would be fine.
That's not the answer?
I don't think that's the only answer. I think that's a step in the right
direction. But without dealing with the standards, the issue of what it is
that's in the curriculum, ... even if you equalize the funding, there wouldn't
be anything pushing the kids in the urban centers to higher expectations. That,
I believe, would only come from a concerted set of national standards that
would make it the case that all children no matter where they live should be
learning the same basic things. ...
You're working hard on this. Are you optimistic?
You know, I am. I've just been very fortunate, since the TIMSS results came
out, to have the opportunity to address and speak with the president, members
of the House and Senate, with the governors and state legislators and others in
the business community. And I find a lot of interest right now in trying to
grapple with these issues, and to make progress.
I see the real solution lying at the level of the state and the governors. And
I see a real opportunity here. I've spoken to many of the governors in various
contexts, and I see a strong commitment. One of the visible signs of that is
the organization called Achieve, which is a non-profit organization of
governors and business leaders. They're really trying to move in this
direction. They have, in fact, developed a set of standards that would be world
class -- by my definition, comparable to the rest of the world -- in
mathematics, for the middle grades, 6, 7, and 8.
And there's a draft of those now developed, in some 10 or 15 states that are
really seriously thinking about adopting those, and putting in place
professional development and an assessment that lines up with those standards,
and would push in the direction of giving this kind of first-class education to
our children. So yes, I am optimistic.
home · no child left behind · challenge of standards · testing. teaching. learning?
introduction · in your state · parents' guide · producer's chat · interviews
video excerpts · discussion · tapes & transcripts · press
some photographs ©2002 getty images all rights reserved
web site copyright 1995-2014
WGBH educational foundation