People would like to think that it's a multiple choice test that's scored
by a computer. So it's objective. It was written by a computer. It's measuring
something that is important and objective. It's not. This test was written by
people. Mostly failed educators. Right? It wasn't by like Nobel physicists. It
was written by a bunch of teachers who got together and said, well, you know,
why don't we ask it this way? There's a certain idiom to the test. There's a
certain way they ask questions. There's a certain way to create wrong answers.
And if you're good at that idiom, then you're going to do well on testing in
general. Not just the SAT but all of these tests, are kind written with the
same tools and with the same types of people.|
And if you're bad at test taking--you always think that's sort of, you
know--a parent says, my kid just can't take tests. And in the back of your mind
is, well what you're saying is, your kid is an idiot. But it's really
true--some people can't take tests, because they don't speak that language.
The tricks we teach are common sense to a good test taker. It's what you
do. We're just saying, this is what kids who do well on tests do. They might
not know they're doing it. But they're doing it.
Does it really just boil down to test taking strategies?
A kid's coming in to get higher scores. All right? We're not there to teach
Shakespeare because the test doesn't measure Shakespeare. We're not there to
teach trigonometry because it's not on the SAT. We're there to teach you to
score better on this exam because this exam is going to determine where you go
to college and how much financial aid you get. It's also going to determine how
you feel about yourself in some large measure. You still remember your SAT
scores. And everybody else does too. Everybody's forgotten everything about
themselves, everything else about high school. They remember their SAT scores.
What were yours?
I remember. They were about 1500.
Princeton Review's model is Joe Bloggs, who is Joe Bloggs?
Joe Bloggs is the average American student. He's not stupid and he's not
smart. He's average. Right? He scores a 500 in math and a 500 in English. And
what you've got to do when you take the SAT is realize--people will tell you
it's in order of difficulty. What they really mean or what's really true is
that, they gave each question to a couple thousand kids. Right? Some of the
questions--everybody got them right. How did they know if a question's easy or
tough? Everybody got it right, it must be easy. Everybody got it wrong, it must
be tough. Joe Bloggs got the easy ones right and got the tough ones wrong. So
there you are on an easy question and you know everyone in the country got it
right. Whatever you want to do, whatever you gut answer is, do that. Don't
second guess yourself. Just do it. Because it'll be right. When you're on a
tough question though, whatever you want to do--whatever Joe Bloggs would
do--don't do it. Because it for sure is a wrong answer. If it were right, how
come this question's at the end of the test? Why is, why did everybody manage
to get this question wrong and somehow it came to you in a flash? It didn't
come to you in a flash. The wrong answer came to you in a flash. It's a trick
answer. The--give me the first word that comes to your mind. Cat.
Okay. There's certain words that come to everybody's mind. There's certain
things everybody does. That's Joe Bloggs. It's knowing that my gut answer is
the same as the gut answer of everybody else in this room. And on an easy
question, go with the gut answer because it works. And on a tough question,
eliminate it because for sure it's wrong.
But it sounds so cynical--forgive the expression.
No. It's common sense. It's the way you live your life, your whole life.
Honestly. Like, all the time in real life, you know, you handle the question in
context. You're at a parking meter and you've just parked. And your wife says
do you have a quarter? Right? The only thing that runs through you mind is,
well, do I have a quarter? You know, and you look. Now, you're in the middle of
the park and it's dead of night. And someone big is heading towards you. And he
sort of leans and he says, hey, do you have a quarter? And the first thought in
your mind is not, well, do I have a quarter? And start looking. It's, well, do
I have problem here? It's common sense.
On a tough question--you're in the middle of the park and it's dead of
night. You know, and you just go to-- well, why are they putting that question?
Why am I putting this answer?
When we were in one of the classes, the instructor sat there and said,
you can eliminate this Joe Bloggs answer. Now you're left with three. Don't you
think if we were to put that much instruction into real instruction, the kid
could get the right answer?
If you were put that much time into writing a good test, then we would
spend the time teaching it. If this were an essay test measuring how well you
understood current events, we would teach current events. Like, don't shoot us.
Like, we'll go where ever they want us to go. If you write a test that has
little trap answers all over the place, then we're going to teach kids how to
get around the traps. Because that's what we're paid to do. Our job isn't to
teach you Shakespeare because that's not going to help you get into college.
How is your business? Is it growing? Is it booming? Would you safely
characterize America's view of the SAT as being obsessed with it?
There are a couple different things going on. Number one, college
admissions is much more difficult, much more selective, than it was 20 years
ago. So these tests are more important. Number two, college is a lot more
expensive. And more kids are going to state schools. They're not going to spend
a lot of time reading your essays. It's your scores and your grades. And so the
test is even more important at a large state institution than it is at a small
school. And more kids are going to large state schools. Number three, people
still have an emotional attachment to the SAT. It's still your first kind of
mono-a-mono experience. Like you against the whole country. But back then they
thought, this is an IQ test. I can't do anything about it. So you sort of had a
little nervousness about it. But it was no big deal. It was like, it was like a
blood test. You know? It was something you had to go in and do. It made, made
you a little nervous but it is what it is. Now you realize that this is a very
coachable test. And so it's no longer just a little bit of nervousness. Now
it's like three months of preparation.
ETS says it's not a coachable test...
ETS has refused easily 50 times to do an independent study that we both get
involved in, because it would be so easy, right? We say, here's a list of our
kids. Here's exactly when they took the course from and to. Go put it in a
computer. ETS, you supply all the numbers. And let's look at the prior and the
post and we'll know. Right? We know that as a country, kids who start where my
kids start--which is around 1100 ah, better than the average--ah, go up about
20 points. And these kids go up this many points. And the difference is us.
Right? It's an easy study. And they don't want to do it. The only time they do
a study is by themselves, not asking for any help, not showing anybody their
methodology or showing anybody the raw data.
Why won't they participate?
Let's say that they acknowledged, after 50 years of lying, that the SAT was
very coachable. Number one, they've got to account for 50 years of lying. And
number two, they've got to deal with, with the implications of successful
coaching to race and gender bias issues. Okay. Rich kids all coach. Poor kids
don't. There's a widening gap between rich kids and poor kids. We can no longer
say it's because of the schools or because of the unfairness of society. Now we
have to admit that we're part of the problem. There's a gap between men and
women that's substantial--that they can just sort of laugh off. And you'll
watch as you interview them and they do a dance. Why do men out perform women
on the SAT? The SAT's supposed to predict college grades. Women do better in
high school and they do better in college. What's the problem here? Ah, the
more you use, the more you start accepting that the SAT's coachable, the more
problems you have with it.
Do you think that it's fair to have so many kids actually paying to
bring their scores up and other kids who can't afford to?
First of all, I'll tell you--I mean, there's an appropriate amount of
studying to do for the SAT. Like kids in our course are spending hundreds of
dollars and a lot of time. We also have the best selling books and software.
And those are like $20. But somewhere in between the books or the software and
the course, that's a reasonable amount of money to spend for a pretty important
test. There are also kids spending a year and 20 or 30 thousand dollars
prepping for the SAT. So the problem sort of magnifies as a different strata of
society--no coaching, a little coaching, more coaching, a ridiculous amount of
coaching. Scores just get pulled apart. And then you set social policy on
people's scores and you've got a big problem.
Second--I have a big problem with these laws. These are politicians getting
involved in decisions that they have no business being involved in. Being an
admissions officer is like being a casting director. It's about choosing a
class that's going to work well together. You want men and women. You want
blacks and whites. You want a diverse class. You want people who are great
athletes to be on your teams. People who are great journalists to be on your
papers. And people who are great actors to be in your acting department. You're
building a team. To say to them, look, all you can do is use this score, this
measure of performance--that's all that's important--is making their job into a
joke. These are very serious professionals. And you're saying, look you can be
replaced by--hell, not a computer--like a calculator. Right? Just punch it in
and, and you get a number. And I think that's ridiculous.
Your competitor says the test is flawed but it's the best we have and we
should keep it. Do you think we should keep it?
We should get rid of the SAT as fast as we can. Look, there are bigger
problems in society. This is not the biggest problem we have. But it's so easy
to get rid of it. Right? Just pull the plug.
Can we return to the correlation between the SAT and socioeconomic
status. The higher the income the higher the score. Is this fair that this is
what it's measuring?
Well, it is very fair that rich people have nicer clothes than poor people.
If you believe in capitalism, you got to go with that. The problem with the SAT
is that it pretends to be a meritocracy. Right? Again, if it's in the back of a
magazine and rich people do better than poor people, who cares? But this is a
test where everybody's saying, look, we're just being an incredibly fair
society here. Everybody takes this test. And the better kids go to the better
schools. And it's just bullshit. You know, the better kids hire me.
Tell me about it.
I couldn't agree more with their findings. The SAT is a bad test. It is
biased. It measures nothing. And we should get rid of it. We do need a common
yardstick. We do need a good test of some sort or several tests. The Princeton
Review can prep kids for just about anything. We have kids working with us for
the medical boards on a test of, like, anatomy. It's a real serious content
test. And we teach real serious content. I think we're pretty good teachers. If
you give us a really good test, I think we can be really good teachers for that
test. The SAT will never be a really good test. It is a scam.
You say it doesn't measure intelligence. What does it
The SAT is said to predict freshman year grades in college, a little. And
it does. It measures it a little. Almost anything you do, including family
income, will measure freshman year grades a little. But the point is that it
doesn't measure intelligence. It doesn't measure anything that's worth a 100
million dollars a year prepping for it.
But the man who started ETS, started it with a noble
Yes. To prove that white northern Europeans were smarter than everybody
else. That was his men were smarter than everybody else. That was his goal.
Every problem starts out as a solution to a problem. And the SAT is no
exception. The world before the SAT--it was kids from Andover and Exeter got
into these top colleges. And the rest of us didn't.This was a leveling. But it
was a leveling among upper middle class white men. Because that was the
population who took it. And as the rest of society started taking it, the SAT
became more and more irrelevant. And the tragedy about the SAT is not its
invention. It's as the world changed, the SAT didn't.
You look even ten years ago--before the Internet, you know, before any
number of things--the SAT looks exactly the same. You look 50 years ago and the
SAT is still the same. This test is a dinosaur.
They're saying that the early bird gets the worm. If it's a coachable
test, then let's start coaching yearsahead.
I got to tell you--we strongly discourage students from working with us
before the end of sophomore year. Like strongly. If you call up the Princeton
Review office and say, I'm a freshman and I want to start working for the SAT.
We will turn you away, because we think that you shouldn't spend your entire
high school life thinking about the SAT. That there is a limit to the neurosis
that even we can tolerate.
But the problem with these tests--and the state exams as well as the
SAT--is that as you ratchet up the stakes, this becomes more and more
important. Can you really blame people for taking it very seriously? And you
can't look at the people who are spending a ridiculous amount of money and time
obsessing about the SAT. You have to look to the test and say, why are setting
the stakes like this? Why are making a test of something that has nothing to do
with high school?
Can you walk us through a question from the SAT?
P, Q, R and S are four towns. P is farther north than Q and R. S is farther
south than P. Q is farther south than R. Which town is the farthest south? P,
Q, R, S or it cannot be determined. Like, is it, would you consider that an IQ
A logic question.
Is it a logic question? I think it's....
A trick question?
The first time you see it, you might be confused or you might have seen
something enough like it or you might be good with maps. Once someone has
explained this question to you, you will never get it wrong again. Right? I
mean, there's a simple way to do it.
What kind of a question does it strike you as?
A, very coachable. B, not really math, certainly nothing you learn in high
school and nothing that will be useful in college. Plausibly a little bit of
logic, I'll accept that. But I think more just a little trick. I mean, what
they're trying to see is that just because two things are south of you, you
can't really determine if one is further south than the other just because it's
Q instead of R. It's the kind of question that they do a lot. There are four
towns. A, B, C, D.
Let me start.
It's the kind of thing they do a lot. There are three towns--A, B and C.
The distance from A to B is four. The distance from B to C is three. What's the
distance from A to C? And of course if you're kind of linear, you would say,
well it's seven. It's the two together. But of course, they don't have to be in
order. So it could be one. Does it measure what you learned in high school? No.
Is that useful in college? No. It's a little trick thing. You'll get it wrong
once. Someone will explain it. And then you'll be ready for it.
Why develop a question like that if it's only measuring a test-taking
Because the point of writing an SAT question is not measuring what you
learn in high school or how well you'll do in college. It's separating out
kids. There are kids who will get that right. And they're generally the kids
who have been in math courses where they play with this kind of stuff. Which is
to say, upper income. And there are kids who will get it wrong because they
don't play with this stuff.
So the question is very good at separating kids. And that's why they have
Let's do some more.
Money is to bank, as food is to basket, park is to city, cash is to store,
book is library and article is to magazine. So what you're supposed to do is
say, I put money in a bank. And I guess the, ah--or money is kept in a bank in
the same way that a book is kept in a library. Maybe it's a vocab question. Do
you know the word bank or the word money. I can't believe that it's a logic
question. And I can't believe the kid who gets that right is going to be better
in college than the kid who gets it wrong. Ah, especially since there are some
wrong answers that are pretty attractive here. You know, you sort of think of
banks and money. And there's an answer with cash. And you sort of--you know,
you're stressed. This is important. You're in a rush. You're in the middle of
the SAT, which is a pretty important test. It's nine in the morning on a
Saturday. You're probably hung over. And a lot of kids who get that wrong--it's
not because they don't speak English. And it's not because they won't do well
in college. Our experience is, a kid who doesn't do well on the SAT--it's not
because he gets the toughest questions wrong. It's because they make lots of
careless mistakes on easy questions. They get sucked into trap answers a lot.
Because they don't have their footing. They don't understand the question. It's
not that they can't do it.
So a lot of the course isn't focusing on the toughest questions. It's
focusing on making sure you don't get that question wrong.
How did you come up with the Joe Bloggs technique?
I met a fellow I started the country named Adam Robinson. Who was working
with this girl, and every time she would get it down to two, and every time she
would get it wrong. And she was going nuts. And finally he said, get it down to
two. Guess something. And then answer the other thing. Because whatever you do
is wrong. And then he realized that's actually not random. A bad test taker
isn't bad randomly. He's bad because he believes that there's an easy answer
there. He moves towards the attractive answer on a difficult question.
Let's find some questions.
Okay. I got a question that I pulled, that I, that I sort of think is
useful. Did they do the doughnut question in class? It's sort of a
quintessential SAT question.
What about the doughnut question?
You got a doughnut here. And the question reads, in the figure above,
what's the greatest number of non overlapping regions into which the shaded
region--the doughnut--can be cut with two straight lines? In other words, how
many pieces can you cut the doughnut into with two straight lines?
This is the last question on the test. It's Saturday morning. You're very
stressed. You're very tired. And this is really important. So what Joe Bloggs
does is, he'll just cross lines. Right? The easy answer there is four. I can
make four pieces pretty easily. What are the odds that on the toughest question
on the SAT that you've done enough work? Right? They're zero. No way. And
again, a good test taker's sitting there. And he answers four and goes, god,
there must be tougher than this. And he's right. So you cancel four. And of
course, since you're able to get four, you cancel three and two also. That's
not the greatest number you're able to cut it into. You're able to cut it into
at least four. The answer's got to be five or six. And then you sit back for a
second and you say, what else would Joe Bloggs do? He'd say, well, they want
the greatest number possible. So maybe it's six. Maybe it's the greatest
number. So it's not that either. The answer's got to be five.
On the one hand you might way, well that's kind of a goofy way to take the
test. That's all testmanship. On the other hand you might say, what is this
question telling you about a kids math skill or his ability to do college level
work? This is just a good question. And goofy questions deserve goofy
And another one...
There are three roads from Plattsville to Ocean
Heights. And 4 roads from Ocean Heights to Bay Cove. If Martina drives from
Plattsville to Bay Cove and back, passes through Ocean Heights in both
directions and does travel any road twice, how many different routes for the
trip are possible? 72, 36, 24, 18 and 12?
Like, what is this telling you about your son. Like, is it
telling you he's stupid that he got it wrong? Is it telling you he shouldn't
go to college or he should? What is it telling you? And I would claim it
tells you almost nothing. A great question for Games Magazine but a lousy
question for Harvard.
who got in? |
the race issue |
sat & test prep |
history of the sat
the screening process |
test score gap |
getting in to berkeley |
tapes & transcripts |
pbs online |