secrets of the sat
photo of claude steele
CLAUDE STEELE: He is a professor of social psychology at Stanford University. Steele and his colleagues have described what they call
navigation, see text below
Can you paint a picture of the situation in terms of black/white test performance in the country today and how it's been for a while?
If the leadership of the university expresses a value in diversity-- that people from different backgrounds bring things to campuses that are of value to everybody here...that helps people feel more comfortable there. Sometimes diversity is just seen as an empty piece of rhetoric. But it can send a very important signal to people who are otherwise under the threat of stereotypes in a situation. There is a reliable test score gap between white and black Americans. On IQ tests, it's about what people call one standard deviation, about 15 points on an IQ test. I think on the SAT exam, which is another exam people are commonly familiar with, it's about maybe 100 points on each of the subtests, the verbal and the quantitative sections of the SAT. Those are sizable differences, enough to affect a person's opportunities when tests are used to base admissions decisions (on). So the gap is a serious one, and it has been around for a long time. And people have been and are still struggling to get ahold of all the things that determine it, that go into it. Some hypotheses about where that gap comes from are some of the most controversial parts of psychology.

There is a range of hypotheses about where this gap comes from. I suppose the most severe one is the one that gets the most attention. The Bell Curve, for example, at least alludes to the idea that genetic differences between the groups to some degree account for the difference. That's on one pole. On another pole are many more environmental factors that reflect differences in the experience, in particular the schooling experience that white and black Americans have as the source of this gap. Access to schooling, differences in treatment within schooling, differences in socio-economic status that go along with racial status in the United States. All these things, by affecting the kind of access to schooling and cognitive skill-building, can affect standardized test performance and it can be at the root of the differences. Most social scientists put a great deal more weight on that than for example the genetic difference.

Why are the socio-economic model and the black cultural explanation not really sufficient to explain it in your opinion, or are they?

I think they are. I think it's difficult to quantify the extent to which a difference is cultural or socioeconomic in a given sample, or differences in the way in which students are treated in schools and so forth. That's what's hard. And it's hard to definitively assign a degree of causality to one of those causes or another. And that is what makes the debate go on and on with a lot of attention focused on the different causes of this gap.

What prompted you to originally start down this road that you've chosen to pursue for your research?

It's interesting. I don't think I was prompted by trying to explain the black/white test score gap. Our work emerged more from an interest in trying to understand the nature of race and how race might affect academic performance in general; the student's relationship to schooling and to school achievement. It's out of that general interest that we began to uncover some things that we didn't expect really that seemed to be new candidates for explanations of where this gap in test performance is coming from.

What are the tests that you perform? You came up with a novel approach to finding a solution. What was that?

Well, I think the novel approach begins with just a somewhat different idea about a set of factors that could affect test performance, in particular test performance of African Americans although we've also looked at test performance among women in mathematics. And the general term for the set of processes that we identified is stereotype threat. Should I go on and explain that because that could take a bit of time?

Explain it in terms of what it means to a black kid growing up in America. Why this and why did you use the word threat? What is it?

By the term "stereotype threat" what we have in mind is simply being in a situation where a negative stereotype about your group could apply. As soon as that's the case, you know that you could be judged in terms of that stereotype or treated in terms of it or you might inadvertently do something that would confirm the stereotype. And if you care very much about doing well in that situation, the prospect of being treated stereotypically there is going to be upsetting and disturbing to you. And if you're a member of a group whose intellectual abilities are negatively stereotyped, this threat might occur. That negative stereotype will be applicable to you right in the middle of an important standardized test. And our general reasoning was that this threat, this prospect of confirming a stereotype or of being seen that way would be distracting enough, upsetting enough, to undermine a person's performance right there in the middle of a test.

It's important to stress that everybody experiences stereotype threat in some way or another because we're all members of one group or another that is negatively stereotyped in society. And whenever we're in a situation where those stereotypes apply, we too can experience this threat. From Methodists to white males. You could imagine a group of males talking to women for example about pay equity. The men might experience a sense of being threatened by the male stereotype, that they could be judged in terms of that stereotype or what they say could be interpreted that way. And in particular, for those who care about being seen equitably, that's going to be upsetting and disturbing. It might cause them to avoid that kind of situation or maybe to make slips of the tongue that would be embarrassing in that situation. That's an example of how it can affect, and does affect, everybody. The point that we're looking at in our research is stereotypes about groups for whom the stereotypes impugns their abilities. Could this kind of threat, this kind of distraction come to bear on their test performance, their standardized test performance? That's the heart of it.

What were we observing given two different sets of instructions? What results are you finding from that?

What you're seeing there is us trying to manipulate, as the term is often used, whether a person is taking this test under stereotype threat or not under stereotype threat. So in the case of African American students taking a difficult test that is diagnostic of ability, that is presented to them as being diagnostic of ability, they're under stereotype threat. That's all it takes. Because the stereotype about African Americans impugns their ability, their intellectual ability. So all you have to do to make the stereotype relevant to their performance is to present the test as a test of ability. Then they know that they're at risk of being seen through the lens of that stereotype.

So we give them, what you saw this afternoon is people taking a particularly frustrating test. It's a very difficult test taken from a section of the Graduate Record Examination in literature. We know it's going to cause frustration and that is going to trigger the relevance of the stereotype. When they experience that frustration, they'll sense, oh boy, I could be seen stereotypically here. I could be confirming the stereotype. And for the students you saw who are very strong students, very committed to succeeding in school, that prospect of being seen stereotypically is disturbing. And it can undermine their performance right there. And that's generally what happens. Compared to white students in that situation, they in that situation are not subject to that kind of a stereotype. And so they may be haunted by all kinds of things with regard to performing on standardized tests, but they're not haunted by the prospect of confirming this stereotype. So you get two groups of students, white, black, who are equally prepared. Equal skills, everything. You give them this very difficult test that is presented as diagnostic of ability. The black student has this extra pressure on performance. And that is in our research invariably reflected in lower performance.

Then you shift conditions just with the touch of a change of the instructions, you present the same test as a test that is something we use to study problem solving in the laboratory and is not diagnostic of ability. That turns the stereotype off for the black student. Now as the black student experiences frustration on this test, it has nothing to do with the prospect of confirming a stereotype or being seen from the standpoint of the stereotype. And if that pressure of being seen stereotypically is enough to depress their performance, then taking off that pressure should increase their performance. And that's what happens in this research. Presenting the same test as non-diagnostic of ability, black students perform just as well as equally prepared white students in that situation.

Why was it necessary to say that the test itself has been tested, and it is positively race neutral? What were you trying to get inside the black student's heads by that set of instructions?

That set of instructions is trying to do the same thing we do with the non-diagnostically instructions for a big word. What he's doing there is essentially telling the black students that the stereotype about your abilities is not relevant to your performance on this test. This is a racially fair test; a racially neutral test. You are not at risk when you get frustrated on this test of confirming anything racial or allowing people to see you in some way racially. This is a racially neutral test. That turns the stereotype spotlight off, so to speak. And when that happens, their performance again goes up. So really in our research through a variety of treatments like that with a number of groups, we've been able to establish the same kind of thing. When you turn off the stereotype that could threaten them with the prospect of being seen stereotypically or confirming it, when you turn that off just with an instruction, up goes their performance. And that indicates that under normal circumstances, that weight and that pressure is on their performance. It is to some degree probably depressing their performance and is contributing to this black/white test score gap that we started the conversation with.

How big of an effect is this thing having?

A 12 would be the score that white students or black students who are not under stereotype threat get. This is a 25-item test; they get 12 correct. Under stereotype threat, they get about 7 or 8 correct. So that's a very substantial difference. And that's just in a 25-minute section of the Graduate Record Examination. If you played that out over the six sections that usually comprise that exam, that could cause a substantial difference in score.

What are the implications of black students' vulnerability in the society at large?

There are many implications because stereotype threat is not something that happens just on standardized test performance. It happens whenever these students are in the domain where the stereotype is applicable. So [with] any kind of intellectual performance or interacting with professors or teaching assistants or other students in a classroom, this stereotype is relevant and constitutes a pressure on those behaviors. As I say, we've looked at the same kind of thing with regard to women and mathematics. For that group, it's particularly rife with stereotype as they get into advance mathematics work in college. Then, fewer and fewer women are present. The world of mathematics and science becomes a more male world. And the threat of the sort we're describing here for women gets more intense.

So, the first implication is that this is probably something more general than something that affects standardized test performance, as important as that is. It's something that's a pervasive element of experience in society in general. Also I think it's important to realize what it says about interpreting standardized test scores for different groups. We tend to think of standardized test scores, almost the term, standardized test, gives the image that this is a test that is standardized. It's fair for everybody. It is a fair and impartial measure of school potential, potential to succeed in school. And this research raises at least one source of concern about that, that different groups in the midst of those tests may be under different degrees of pressure, and their performance is likely to reflect that. And the gaps that we see are interpretable in those terms as opposed to differences of preparation or differences of ability.

It sounds like what you were saying was that it was the black students who considered themselves good students who actually are affected the most? Did that surprise you?

That is the thing our research has revealed to us, something that we really didn't think of when we went into this. It has become vividly clear to us and other people who do this research that the effects of the stereotype are poignantly most powerful for the students who are the strongest and the most motivated. For them, functioning at the frontier of their skills with this prospect of being seen, stereotypically, [to be] over their heads, they're the ones who experience a disruption and so forth. Students who are less identified with that kind of success in school for whom it isn't so important. Let's take for example a woman who is performing math but really isn't identified with math. If you give her a very difficult math test, she'll start to do it, she'll do her best, as soon as she gets frustrated though she'll probably start to say, "Well, this is not me. This is not important to me. So I'll sort of withdraw effort," so to speak. It doesn't make any difference whether she's at risk of being seen stereotypically or not. With regard to the whole domain, it's not something that's important to her. So, poignantly, the problem and the stereotype threat that we're talking about, it hits the women math students who do want to do well in math and who are very identified and for whom doing well is very important to them. It's for that kind of student that, again, operating at the frontier of their skills, having a great deal invested in this domain, the prospect of being stereotyped in it, of doing something that would increase the likelihood of them being stereotyped in that domain, is disturbing. In some of our research we find it elevates blood pressure for example. For those people, it's disturbing, distracting, and likely to interfere with their test performance and with their other interactions in the domain, their other behaviors in the domain.

You used a term...

Yes, that is a term that we've sort of started to use to capture this part of the phenomena, that it's the most invested students who feel it the most. That it's the academic vanguard of the group as opposed to the rear guard of the group that is experiencing the threat we're talking about. And experiencing that threat can be thought of as having to pay a sort of pioneer tax: the woman who is really advanced in math, the black student who is really doing well and is sort of in a top flight, high pressure college or graduate program or professional program or something. That person, because they're having to deal with this threat and the prospect of it is paying a certain tax to be there that other students are not having to pay. This is not to make the journey of other students seem trivial or light because it isn't. It's just to argue that in addition, for students who are negatively stereotyped in this domain, there's this other tax to pay.

You alluded to a further price that's paid in high blood pressure because a certain cadre or type of black student actually has maybe an almost too rigid faith in hard work.

Yes, I think one of the reactions we sometimes get to our research on the part of women who are good in math or blacks who are good in academics for example is, "That's very interesting Claude, but look, can't they just bear down and beat the stereotype? Isn't this stereotype something that should just stand as a call to greater effort in the domain rather than something that should be as troubling as you're describing it?" And the African American community is strongly committed-handling this kind of pressure is something that is very familiar to them and they're very experienced at exercising [it]. So when they hear this kind of argument, that can be a reaction. What we have found out is that it is the attempt to overcome the stereotype, to struggle past it under this added pressure that is in some sense causing the under performance. It's causing them to, for example in our research, they're re-reading the items, re-checking their answers, going back and forth, double-guessing. And on standardized tests, that's not the most effective way to take them. You're supposed to kind of move through them in as relaxed a way as you can. Keep paying attention, but be relaxed. And the attempt to be extra careful, to not make a mistake, is in part what's undermining their performance. It' s also associated with elevated blood pressure in that same situation. So there's a good deal of arousal. These things tell us that these kids are not in these situations giving up. They're in these situations trying hard. Maybe they're trying too hard and it's out of that trying too hard that the under performance and the stress is coming.

What's John Henry-ism?

John Henry-ism is a term that a colleague of mine, Sherman James at the University of Michigan, has used to describe a syndrome of high blood pressure among American blacks. He uses a sample of blacks in North Carolina who he has given this name to. And you remember John Henry-ism is an old fable in which John Henry is a steel driving man and he competes with the steam-driven pile driver to see who can drive the most stakes in this railroad construction. And they go at it, the steam-driven machine and John Henry in another track. And they go at it for days and days and days and finally, John Henry wins. He drives one more stake before this machine sputters to a stop. But as he drives that stake he drops dead.

Part of being black is having to deal with an extra burden. Part of the heroism of African Americans is struggling against that extra burden. But there's a price to pay. There's often a price to pay. And I think that's what that term captures. To function in a society where you have to contend with the prospects of being stereotyped and negatively treated in very important domains of life, that's an extra burden and there is going to be--you can and should struggle as valiantly as you can against it--a price to pay and it certainly isn't to say that that's fair.

What's disidentification?

Disidentification refers to a reaction that can happen as a person has to deal with the stereotype threat for a long time for example.

In the context of a student who feels the possibility of stereotype threat, what would disidentification on his part or her part be? What would they do?

It would be essentially divesting oneself from the domain. That is, making it less a basis of one's self esteem and self regard. By caring less about it and seeing it as something that my own self regard is less accountable to, then I stop worrying so much about what goes on in that domain. I described a little earlier a woman who might not care so much about mathematics. Another way of putting that she isn't as identified with mathematics and so her performance isn't as important to her self-esteem as it would be for someone who does care a great deal about mathematics.

I often use the example of having disidentified with the baritone horn in the 8th grade when my band instructor told me, as we went into the concert that night that, "I could hold the horn but I didn't have to play." So I kind of came to the decision that the baritone horn is not really for me. Well, that's an act of disidentification in a sense. It's, "I'm going to find some other domain to take on as a personal identity to hold myself accountable to. Maybe it will be sports, maybe it will be academics where I will really care to the degree that if I don't do well at it, I'll feel not so good and if I do do well at it, I'll feel really proud of myself." When you have those kind of contingent feelings about a domain or an activity, you're identified with it. Anybody can think of activities that they just don't care about that they know are important to other people but for them, their own feelings about themselves are not tied to that. Well, that's not being identified with something. And the sort of tragedy that we're pointing to in our research is that in reaction to a very systematic pressure in society like stereotype threat in a school domain, it can cause some groups to disidentify with very important domains of life like school achievement or to identify less with those domains because of this added threat in the domain.

You mentioned that if you're white, you're right. Are you saying that that is the sort of lesson that black kids growing up just kind of imbibe on a day-to-day basis, and in turn carry into that test-taking situation?

Well, that's very interesting, that word imbibe that you used because we might have had that view initially; that's a classic view in the social sciences throughout much of the 20th century beginning with the work of Mead that there's a set of stereotypes out there and when you're a member of a negatively stereotyped group, you eventually inevitably imbibe those stereotypes. And then when you're in situations where the stereotype is relevant, for example you kind of fulfill them in a self-fulfilling prophesy kind of way or you're fulfilling self-expectations, lower expectations; it's a standard view. And what we're finding isn't quite... that isn't right in the sense that remember, this effect we talk about happens for the students that have the very strongest identification with the domain. They're students who at the beginning of these tests and so on tell us that they have a great deal of confidence in their ability. And they've long had a lot of success in the domain. So the understanding that this is revealing to us is that this is like a situational pressure. You can take a person who is very confident and put them under a certain pressure and disrupt their performance. So it is their confidence that is luring them into the domain and having them identify with the domain and put themselves in situations where they are taking these tests. This is the poignancy of it, that it's this group that is then experiencing a situational pressure of stereotype threat. So it isn't so much of an internalized thing.

Another piece of research that really deals with this directly: Can you produce these effects in groups who are not subject to any negative stereotypes about something? We came up--Josh Arenson, Steve Spencer, Joseph Brown, a number of students and I--came up with a study that looks at stereotypes in white males on math performance. Now here's a group that is not going around in society negatively stereotyped. So there's no stereotype to internalize. However, if you take white males who are very invested in math again, graduate engineering students as we did in one case, or undergraduate honor students in math, these are very good math students. You sit them down just as you saw today in these experiments and tell them just before you take the test that, "Gee, this is a test on which Asians tend to do better than whites." Now, they're under the comparative stereotype threat. Their group is not negatively stereotyped about math, but relative to Asians, there may be some possibility of a difference. So, they're in a negative light because of the positive stereotype about Asians in that situation. And lo and behold, those students under-perform there. Evidence like that and the evidence I just described earlier, makes it clearer to us that this is not something that is coming from imbibing a self-believing and a self-fulfilling of negative stereotype; it's coming from an intense situational pressure that happens to people who really care about doing well in the domain.

You're actually finding poor performance because of wanting the very same things that all of the other students want.

Yes, that's interesting. That's a good way to put it, that what we're finding is poor performance among people who almost want it too much, or not want it too much, that's the wrong word, but are trying too hard. In an effort to succeed [they] are almost trying too hard because they're dealing with this extra pressure. That's the important thing to recognize. However, we wouldn't go so far as to say that this process is the cause of all of the under-performance or explains all of the black/white test score gap. Other factors play a significant role too. And those other factors I think are factors that people are more familiar with: disadvantages connected to race. For example you can imagine the experience of a black student going through school. They're much more likely to be assigned in lower school or to educateably retarded tracks. They're much more likely to go to school in schools that are less well funded. They're much more likely to have teachers that are not as well trained. They're much less likely to have access to coaching classes, the kind of thing you're looking at in this program. And the list goes on. They're much more likely to have their work be undervalued and the like. Well, that is a day-to-day, cumulative kind of experience that is going to be manifest on test score performance and on grade performance in general. Those factors are the larger backdrop to the problem. Our research enters the picture when you ask the following question: suppose a kid survives all that and gets to Stanford and is a really good, highly motivated student and you put that student in a high pressure situation where they're operating at the frontier of their skills, then this phenomena of stereotype threat emerges. This is not to say that it doesn't emerge earlier, because it does. Even in the 5th grade, you have very strong students who are operating at the frontier of their skills and they're very dedicated students and they too will experience stereotype threat. So, I don't want to say it's something that happens just on elite campuses; it happens to the vanguard of the group at every level of schooling.

You said that wise schooling may be the key that opens the school house door. What did you mean? Do you see that as a solution to this problem?

There are lots of ways of making a student feel secure from being stereotyped, or relatively secure from being stereotyped. I don't think any single thing is going to wipe society clean of these stereotypes at this point. This is part of the American landscape; it's the legacy of our history. And to hope that they would be wiped away is too much in an instant. Maybe in some years they will be. So, the aim has been to create learning situations, schooling situations, where people can feel secure. And there I would start enumerating a variety of strategies. One I think is institutional leadership.

Are your challenge workshops an example of this sort of thing that you're talking about?

Let's say you bring a black kid and a white kid onto a college campus. The American presumption is that if you treat them the same, they'll be okay. The complication with that though is that the black kid has this racial stereotype to deal with [and] doesn't really know how he or she is going to be perceived in that situation. [He or she] doesn't really know whether the feedback they're getting is coming from that stereotype and may be racial prejudice or whether it's just plain constructive feedback that they should pay attention to. They're in what Jennifer Crocker and Brenda Major, two social psychologists, call an attributional ambiguity. It's hard to know. The white student isn't quite in that situation. They can take feedback at face value. For a black student that's more complicated. And you can imagine how this plays out in daily life. Almost every interaction can have that ambiguity to it and the threat to it, the threat that perhaps I'm being treated through that stereotype, so that students, even though they're standing there on the same campus, in the same room with the same teacher, they're really in very different environments. And that's what's been difficult for American educators to appreciate, the difference in those environments.

And what we mean by wise schooling and the thrust of the research to examine wise schooling has been how can you level this playing field for this student? What do universities, what do schools have to do to allow that black student to feel comfortable in that situation? And there are some very hopeful things. If you challenge that student, then you are sending that student a signal that we have high expectations for you and we're not viewing you through the lens of the stereotype but we really do value your ability. That's a very important signal.

If the leadership of the university expresses a value in diversity, that people from different backgrounds bring things to campuses that are of value to everybody here, that sends a clear message that the things about my group that are distinctive about my group will not result in negative judgments but will be valued in this environment. And that helps people feel more comfortable there. Sometimes diversity is just seen as an empty piece of rhetoric. But it can send a very important signal to people who are otherwise under the threat of stereotypes in a situation.

I think relationships are very important. Jim Comer, who is someone who has done a lot of research of this sort at the elementary school level puts a lot of weight on positive relationships because if we have a relationship, I will almost by definition trust that you're not going to see me stereotypically. So when college students come in and they have a variety of relationships with people in their environment, then they become confident, relaxed that in this environment, although I know those stereotypes exist in society in general, in this environment I'm kind of not affected by them, they're not a problem to me. You can contrast that with coming onto a campus and having a very segregated social existence. In that situation, I'm more likely to see things in racial terms because it makes sense. Here's my group, there's that group, and it puts up a boundary that enhances the tendency to see things racially and think then that I'm seen racially.

So these are examples. There are others that we have worked on. Jeff Cohen, a student of mine, and Lee Ross, a colleague, and I have done some work that I'm particularly excited about with regard to how to give feedback. How do you give feedback across the racial divide so to speak? How does a white teacher or professor give critical feedback to a black student? Jeff came up with some very intriguing hypotheses about this, about how to do this. And certain standard things that most of us tend to do are not the right things to do.

Like what?

Some people think you should just give the feedback unalloyed, don't soften it. That doesn't work. Black students are in this ambiguous situation where they can't trust that. It has the alternative hypothesis for them that that negative feedback is coming from a stereotype and is not really true about their research or their work. And they have to consider that. A rational person has to consider that. So down goes their trust in the feedback and the utility of the feedback to them. It's intriguing how race in that way creates a bubble that can isolate a person from feedback. Does it help to give a positive bromide first? "Well, I really like you a lot, but here's the negative feedback." No. Everybody can read that for what it is and that isn't very effective. What Jeff found was that when the person had actually looked at the work and said, "Look, we're using high standards in evaluating this work. But, I have looked at your work, and I think you can meet those standards." That combination of things, high standards plus affirmation of that person's potential to meet those standards just deeply inspired the students. Black students were more motivated in that situation than white students were. And this is a rare thing if one looks at the educational literature to find black students more motivated academically than other groups. But with that kind of feedback, that's what happened. So you can see finding the key, the solution, the secrets of wise schooling, is we think a research process. But they do come to light. And there are a lot of talented practitioners out there who have a great intuition about how to do this. There's no shortage of demonstrational programs that show tremendous gains on the part of black students under otherwise very competitive circumstances. They have great insights. We're trying to take their insights and to systematize them into a kind of theory of how to do it.

What does the SAT measure?

The classic phrase is that these tests measure what they test. And the SAT is no exception to that. The way items get on that test is the way items get on most tests of mental ability which is that they are items that correlate with performance in school. So an item that you would give to a norming sample that doesn't correlate very well with school success gets dropped off the test. Items that do correlate get put on the test. That's how tests get made up. They're just empirical creations, creations of American and European pragmatism. If you want to find out what actual mental capacity they measure you have to work backwards. You have to use statistical techniques to classify the kinds of performances that they're measuring and work backwards to "Well, if it measures this cluster of performances, maybe it measures this kind of capacity." And then there have developed big arguments about which performance this cluster measures and what performance that cluster measures and which are central to performance. So it's a very complicated game trying to work backwards and figure out what these tests actually measure.

But is this SAT an IQ test?

It is in a sense an IQ test. The SAT and IQ test correlate very highly. Between tests, between the SAT and the IQ, they correlate almost as much as the SAT correlates with a second administration of the SAT, as much as it correlates with itself. So they're very similar tests in content.

Give me the little history lesson.

Okay, yes. The methodology for standardized tests of the kind that we use today was developed in the 19th century by Francis Galton who was as many say, the jealous cousin of Charles Darwin. And he was trying to get a test that would test his kind of evolutionary, social Darwinist hypothesis that intelligence ran in families. Of course all kinds of other things ran in families like wealth, advantage, and so on, but that didn't bother him. He wanted a test that would discriminate between basically upper class and lower class Brits. He developed this technology of finding items and seeing how much they would correlate with other performances as a criteria for whether the item would be put on the test or not. So he had this situation in the British museum I guess where he would have people come in and perform tasks: reaction time tasks, visual acuity tasks, a whole variety of kind of physiologically rooted tasks that he thought would tap into intelligence, sort of innate, physical intelligence. His presumption was that upper class Brits would do better on these things than the lower class Brits and he would therefore have a set of items that he could give to people that were a measure of intelligence that would discriminate. People who would score high on this would be more likely to be the upper class Brits. People who scored low on this would be the lower class Brits. So, he died a failed scientist never finding a set of items that worked like that.

Alfred Binet in Paris at the turn of the century, beginning of the 20th century, was given a practical task of coming up with a test that would help identify kids who were retarded and wouldn't do well in school. So he simply used Galton's technology. He said, "Well, I'll make up a bunch of items. And the items that kids who do well in school get right, I'll put on the test. And items that kids who don't do so well at school get right, I'll put those off the test because they couldn't be measuring something relevant to school success." So he gets a subset of items that kids who do well in school can perform well on, and now he's got a test that when given to people will tend to identify those who are not going to do well in school. And he can do what the Paris school board asked him to do: screen out kids who are going to have real trouble with school.

Well, as everybody knows, that became the basis of the IQ test. It was transported into the United States, the Stanford-Binet test. That same technology of using success in school as a criteria for whether an item gets put on a test or taken off of a test. And that is how essentially, roughly speaking, all standardized tests are constructed. The SAT, the GRE, the mini-IQ test all have that inherent methodology to them.

The man who developed the SAT, Carl Brigham, was an outright racist. Do you even mull that fact?

As I say, that fact has not been wasted on me. And the area of standardized testing and intelligence testing has always been one of the most controversial areas of psychology for precisely that reason. It has often been used as a way of implementing racist intent, most recently with regard to blacks. But in the post-World War I wave of immigration it was used to screen out Southern Europeans, Jews, and other groups who did not score well on tests at that particular time. So it has, as a tool, a very, very racist past.

What do you think of the SAT, personally?

I think it is an exam that can tell you something. I've used a metaphor if you can indulge that, that I think captures the basic argument I would use. If you had to select a basketball team by the number of 10 free throws that a player could hit, the first thing you'd worry about is selecting a basketball player based on how they shoot free throws and you know you'd never pick Shaquille O'Neal because he's terrible at free throws even though he's a magnificent basketball player. That's what a standardized test is, compared to the domain of real school performance. Real school performance out there--it's like having to select a basketball player based on how well they shoot free throws. That's the first problem with standardized tests.

And the SAT reflects that. The predictive statistics reflect that. The SAT measures only about 18%, [an] estimate range from 7 to 25%, but of the things that it takes to do well in school. This is something that people should realize about the test. People think of it as capturing a very large proportion of things that are important to school success. The people that make these tests tell us, "No, that is not true. They don't capture a large portion of the things--about 18%." In many of the samples I've done research on, much smaller than that, sometimes 4% of the things that are predicting success in college for example. So it's not great, just like a free throw is to selecting a basketball team. And SAT is not going to get you very far with predicting who's going to do well in college. And certainly not far with regards to who is going to do well in society or contribute to society. It's just not that good a tool and that's the first thing to realize about it.

The second set of problems have to do with interpreting the scores on SAT tests. And again, the free throw example is useful. If a kid comes in and he shoots 10 out of 10 or zero out of 10, you might take note of that kind of performance with regard to selecting him on the basketball team. If he hits 10 out of 10, you say, "Well, okay, he's probably pretty good and that probably reflects something about his basketball playing. I'll put him on the team. Zero out of 10, that probably reflects something about his playing, he's off the team." Same with SAT tests I think. When you get really strong scores one way or the other, even though they're not as reliable, they often can bring to light talent that would not otherwise be seen.

And so I am not one who thinks they should be done away with entirely. They can be useful in that regard as long as we understand how to interpret them and how little to use them. And I think many college admissions committees are very sophisticated about this. They are closer to this issue of how predictive tests are, and they can get a feel for it. So, that's the second thing.

Middling scores on the test are very difficult to interpret because you don't know. If the kid practiced a little bit more, maybe he would have hit 9 free throws. Maybe he hit only 4 and he's been practicing for 10 years. It's just hard to interpret the meaning of middling scores and the same is true with the SAT. A kid who gets anywhere from 10 to 1200, maybe he got those scores because of coaching or maybe he got those scores because he didn't have enough coaching or maybe he got those scores because he went to Europe every summer and got a great vocabulary about cathedrals and that happened to be on the test that day. All kinds of things can contribute to performance and it muddies up the diagnosticity of the test.

Is the strict application of tests and grades the standard that we should use? And is it fair?

That's easy for me to answer and that is, absolutely not. And that is the very nub of the problem. That's the argument that many who oppose affirmative action come to. Look, the assumption that they're making is that we have a perfectly interpretable, objective measure of academic potential and contribution to society potential. That we have that in our hands, and that affirmative action is causing us to ignore that or downplay that or make exceptions to that. And the first point that that side has to recognize is that we don't have that kind of measure in our hands. We don't have that. The SAT and no standardized test is that kind [of] measure, can bear the burden of fairly assessing academic potential for all groups in society or all people for that matter in society. We just don't have that.

Is there some way we can prevent the era of the great multicultural universities from being wiped out?

Yes. I certainly hope there is some way that we can stop the resegregation of America's foremost universities. My first appeal [is] based on our research and here I'm speaking I suppose more as a person than as a scientist. As a scientist, you try to let the data reveal to you what they reveal to you. But based on what our data has revealed to us over the years, I feel that these prescriptions against the use of race and gender in especially college admissions are going to be potentially very damaging to society unless we figure other ways of leveling the playing field. I've always been disappointed that many opponents of affirmative action, while opposing the use of racial preferences at that particular moment of admissions to college, have not been very interested in the racial preferences that precede that moment in K through 12 education for example throughout the United States. Where is the commitment there? Because the people who study that--the role of race in education at that level--come to understand that there are tremendous racial preferences against certain groups that just happen on a day-to-day basis and have a cumulative effect over the course of an individual's experience in life and are huge in terms of the... and racially coded. Not just class-coded, but racially coded.

So at the point of college admissions, to get very pure about getting rid of racial preferences when nothing has been done to address the reverse racial preferences leading up to that point, that's not the American way. I almost hope that Americans will just see the unfairness of this at some level. As an American you have to have faith to some degree in the intelligence and good spirit of one's citizens. And I think I'd like to believe that if people begin to see the full scope of this issue, that they'll have a more sensible view of things, not to take this kind of narrow, hyper logical view, decontextualized view of the college admissions decision as the only place where we're supposed to be concerned about racial preferences.

NOTE: The following are peer review articles documenting Claude Steele's findings. They include over a dozen replications of the basic effect of stereotype threat depressing test performance.

Aronson, J., Lustina, M.J., Good, C., Keough, K., Steele, C.M., & Brown, J. (1999). When white men can't do math: Necessary and sufficient factors in stereotype threat. Journal of Experimental and Social Psychology, 35, 29-46.

Croizet. J.C., & Claire, T. (1998). Extending the concept of stereotype threat to social class: The intellectual underperformance of students from low socioeconomic backgrounds. Personality and Social Psychology Bulletin, 24, 588-594.

Shih, M., Pittinsky, T.L., Ambady, N. (1999). Stereotype susceptibility: Identity salience and shifts in quantitative performance. Psychological Science, 10, 80-83.

Spencer, S.J., Steele, C.M. & Quinn, D.M. (1999). Stereotype threat and women's math performance. Journal of Experimental and Social Psychology, 35, 4-28.

Steele, C.M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629.

Steele, C.M. & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797-811.

home | discussion | who got in? | interviews | the race issue | sat & test prep | history of the sat
the screening process | test score gap | getting in to berkeley | bibliography | links | tapes & transcripts | press | synopsis

FRONTLINE | pbs online | wgbh

web site copyright 1995-2014 WGBH educational foundation

SUPPORT PROVIDED BY

NEXT ON FRONTLINE

Losing IraqJuly 29th

FRONTLINE on

ShopPBS
../test/ ../race/ ../interviews/ ../who/ ../talk/ ../