Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Charles Murray on the SAT
For about six years, I tutored high-school students in the Boston area, mostly for standardized tests like the SAT and ACT. In most cases, these were high-achieving students who were looking squeeze an extra percentile or two out of their scores. As the instruction was one-on-one, this usually meant traveling to the family’s house, and working with the student at his or her kitchen table. These were generally very nice tables, in very nice kitchens, in very nice houses, themselves in very nice neighborhoods. Instructions as to how to find to the bathroom were often shockingly involved. Suffice to say, most — though not quite all — of these kids were stinking rich.
To left-leaning critics of the test prep-industry — and to many of those within the industry itself — this sort of observation often leads to worries that the tests are merely testing the students’ economic privilege, rather than their actual abilities and knowledge. It seems all the more so when most of your students are sharp-and-rich and those who aren’t are far more likely to be dull-but-rich than sharp-but-poor.
As Charles Murray points out, that’s not an illogical conclusion so much as one based on poorly-chosen assumptions. Yes, rich kids do disproportionately well on standard tests, but that’s less because their parents are rich than because their parents are disproportionately sharp, and sharp people are disproportionately wealthy and have disproportionately sharp kids:
Let’s throw parental education into the analysis so that we can examine the classic indictment of the SAT: the advantage a child of a well-educated and wealthy family (Sebastian, I will call him) has over the child of a modestly educated working-class family (Jane). Sebastian’s parents are part of the fabled 1%, with $400,000 in income, and his mother has a college degree. But her IQ is only average. Jane’s family has an income of just $40,000 and mom has only a high-school diploma. But mom’s IQ is 135, putting her in the top 1% of the IQ distribution.
Which child is likely to test higher? Sebastian is predicted to be at the 68th percentile on the PIAT. Jane is predicted to be at the 78th percentile. If you want high test scores, “choose” a smart but poor mother over a rich but dumb one—or over a rich and merely ungifted one.
In other words, intelligence is no more a guarantor of wealth than poverty is a guarantor of stupidity, however predictive they might be statistically. Moreover, the observation that dull-but-rich kids asking for private test-prep outnumbered the sharp-but-poor ones likely said more about the prices we charged than anything else. And, as the tutors all learned by experience, there was only so much you could raise dull kid’s scores, regardless of how many Lexuses were parked in his folks’ garage.
Interestingly, our company also ran classroom programs at high schools. As you might imagine, these classes were much less expensive than the private lessons and the students came from less wealthy backgrounds than those we tutored one-on-one. There was a wide range of ability in those classrooms, but there was always a couple of sharp kids in each one, and they generally weren’t the ones dressed in name brand clothes.
As Murray’s research shows, those kids will likely be hiring private tutors 25 years from now for their children.
Published in Education
I never liked test prep companies. The cost seemed astronomical and awfully unnecessary. I had more than enough materials and information to help me through both the SAT and ACT, and I spent a grand total of $35 (for the Official SAT Study Guide). If you have an internet connection, and a good bit of determination, you’ve got all you need. If education is “the great equalizer” in life, then the internet is “the great equalizer” in education. In the end, both are merely resources that require effort to harness their potential.
How much of the test-prep time is spent teaching test-taking strategy vs. critical reasoning vs. imparting knowledge? Does it change depending which test you are targeting?
This reminds me a lot of my situation. I came from a fairly modest middle (maybe at times lower middle) class parents who are both quite intelligent, but neither of them finished college. (My birth may have had something to do with that.)
Now both my sister and I have graduate degrees. I took one of the ACT class-type preps and it boosted my score by 2 points overall, which is significant. I don’t think it was too expensive. It mainly taught me some “tricks” to solve the questions more quickly.
I never took a prep course, but I knew a lot of people who did. The majority of companies like Kaplan only get people to a competitive score, so that it doesn’t spoil an otherwise satisfactory application. When I took the SAT it was out of 2400, and these companies would help increase a score from <1500 to 1800. There are some specialized companies that shoot for the upper echelon, but I imagine those are expensive and better-run. Those I don’t have a problem with, since it’s very hard to get the last 300 or so points, but the increase from 1500 to 1800 could be accomplished merely by a few practice tests. Familiarity is a huge advantage that is incomprehensibly overlooked.
Which is a very smart way to go, and certainly the biggest bang for your buck.
My recommendation would be that everyone start there — or with the new program on Khan Academy — and then consider paid-help after doing some studying on their own.
Generally I think these standardized test create a false sense of accomplishment, because they can be easily gamed as demonstrated by the numerous foreign Asian students who manage to get high scores on the English portions of tests like the GRE (which is just the less loved twin of the SAT) but who could not put together a sentence of comprehensible English in person.
These tests are just games. You can draw some conclusion for them, but the more value you place on them as diagnostic or tools the more you open yourself up to erroneous conclusions.
I think you got that second part backwards…
It depends not only on the test, but also on the person.
Bright but nervous test-takers do well focusing on strategy: for them, the key to not screwing themselves over on a test is to become familiar enough with how that particular testing “game” is played so that they can keep their heads cool enough to avoid self-sabotage. Duller but calmer test-takers might do better polishing the skills the test allegedly tests for, like critical thinking or knowledge content.
I have some experience tutoring underclass kids for standardized tests. For the most part, they simply haven’t believed that taking standardized tests would help them in any way. Their reasoning might run as follows, “If I could get a perfect score, then yeah, maybe that would open doors for me. But realistically, that’s never going to happen, so why bother?” Those who did care about the test were usually terribly anxious and fearful, often to their own detriment. Taking the approach that the test is a game, a game that with enough practice, even fairly ordinary people can play to win, was the only thing that seemed to help.
These students didn’t need outstanding scores in order to open doors for themselves. Even a score that would be mediocre by upper-middle-class standards would have distinguished them from among their peers, attracting opportunities that otherwise would have passed them by. I’d tell them, “Since you come from School X, nobody expects you to score above Y (the school’s typically dismal average) on this test. But I know that you’re smart enough to score well above Y if you practice. Don’t pass up this opportunity to pleasantly surprise people. Practice for the test and see what happens. It’s a gamble worth taking.” (Telling them they were smart enough wasn’t a lie, incidentally: our mission was to help the best students in the worst public schools.)
I’m not saying my advice helped much. Just that it seemed to help more than the other things we tried.
Nope, that’s what I meant: it isn’t causative in either direction.
Provided it’s not marked up or hopelessly out of date, I’d say self-study of pre-owned Princeton Review materials is the best bang for your buck, unless you’re facing a subject GRE, where review material of all kinds is sparse and rather unreliable.
But then, I’m the nervous sort of test-taker, and stupid as it might sound, Princeton Review’s irreverent “it’s all just a game” attitude is just what I need to internalize in order to stay calm and not screw myself over.
Seconded.
Sometimes “My kid is really sharp, but he doesn’t test well” is just a cover for saying “I want my kid to score higher” but sometimes it genuinely is true.
They are just games. But games that open up valuable opportunities. That’s why we should apply ourselves to playing them. Asian students get this.
Non-Asians are more likely to see the test as a measure of who they are (a characteristic that no amount of effort can change) rather than a measure of whether they’ve applied themselves. And this attitude, even if it’s realistic in the larger picture, is detrimental to making whatever incremental improvements you’re capable of making.
There is a seldom-noted factor in why kids from culturally rich homes do as well in school as kids from money-rich homes, and it’s something I noticed working with a lot families in my parent-volunteer life.
The kids who are in wealthy homes are busy. All those ski trips and vacations and so on. Granted, they are more sophisticated than the other kids. But sometimes there’s less depth in their knowledge.
In contrast, the kids in relatively lower income homes often excel despite the lack of exposure. The reason: the kids in the lesser-income homes have one thing the wealthier kids don’t have: time.
Time to read, time to practice, time to get to the know the neighbors, sing in the church choir or the school chorus, time to get to know their teachers, their grandparents, time to become an Eagle Scout. All intellectually enriching activities.
I saw it over and over. That’s why I am so glad there are ways for these kids to get college educations. They will make good use of them.
Definitely.
In my upper-middle-class public school system, we took at least one standardized test every year from the age of six on, and plenty of in-class tests modeled on standardized tests. Standardized test-taking was part of our classroom routine. It wasn’t part of the classroom routine in the failing schools I worked at.
I’ve edited a great many standardized and other multiple-choice tests, and my concern is that I have too often found that the second-best and plausible choice is actually correct. In other words, I’m seeing two right answers. I’ve seen it so many times that it makes me worry for test takers generally.
And just for fun, I reviewed by daughter’s PSATs with my editor’s eye, and I found many errors, both content and editorial. It was enlightening and discouraging. I would tell anyone to take multiple-choice test results with a grain of salt.
The article compares the test results of children with sharp and wealthy parents versus those with sharp and poor parents and the difference is small once the income of the poor parents reaches $125k regardless of how much higher the income of the wealthy parents is.. That is strong evidence that greater wealth does not provide greater test results across the board.
Good point. The PSAT I took at the beginning of my junior year had me at the equivalent of a 720 on the SAT for reading comprehension, but when I took the SAT for real in the middle of my junior year, I got a 640. Big swing. That’s the subjective portion of the test, and I did much better on the math and writing (basically sentence correction) sections. Didn’t know if I just had an off day or what.
I took the ACT at the end of junior year and scored much better on the reading comprehension, even though I thought it was harder than the SAT. I didn’t put a whole lot of stock into it though. It’s a game to be mastered.
The problem isn’t so much in the tests, although they are flawed in numerous ways, or in the ability to game the test, which I think is limited, or even in the ability prepare for the test, which is also of limited value. Rather, the problem is in drawing big conclusions from small differences in the outcome. Ceteris paribus, virtually any kid with a 1200 on the SAT can do the work at any university, and any kid with a score below 700 can’t. Other than that there is not a whole lot of predictive power in these results.
I say this as someone whose path through life was made so much easier by high scores on standardized tests that made up for weak academic performance in the eyes of admissions committees at every level.
I have always wished I could spend a day in a locked room with Charles Murray, a stack of current basic-level textbooks, some old basic-level textbooks, and some dictionaries and other references including grammar books, and the two of us could review each question one by one. I think some of his faith in standardized testing might come out a little less secure.
Very little is spent imparting knowledge from my experience near the year 2000.
Critical reasoning is very important. One of the strategies for example is to eliminate the obviously wrong answers then guess, rather than calculate the right answer first. If you can make an educated guess based on ballpark estimate or logic, do so. Calculate only if you absolutely must.
Test taking strategy is also pretty important when you are time limited. For example, if the test penalizes you for wrong answers, you don’t guess, but if there is no penalty for wrong answers then you must make sure to bubble in every multiple choice answer even if you have no clue.
How much of what is taught pertains to content vs test taking skill does change depending on the test. SATII tests used to be the “subject matter” tests which had a lot more content required. Similarly, the AP tests tend to have a lot more content mastery required than the plain SAT/ACT.
Oh yes, I’d agree with most of this. The SAT is probably the least based on critical reasoning. The math questions are middle-school level, the sentence correction falls under about seven different rules which can be learned and mastered in about a week, and the reading comprehension is the only section that takes “critical thinking” skills. The vocabulary section is also more advanced than the other parts of the test.
I always liked the guessing penalty aspect of tests, but they’ve gotten rid of it for the AP tests and I think just recently for the SAT. The SAT IIs are pretty simple tests compared to the AP tests, which are much more comprehensive and impossible to pass unless you actually studied.
Correct. I think that was a bad call on their part.
The big change that I think is a real mistake is removing the essay. For all the criticisms one might make of it, its great benefit was that it’s the one essay colleges could access that they could be sure the parents hadn’t written for them.
It’s been a little while since I did anything for the APs, but multiple choice sections are reasonably comparable so far as I recall. The main difference is that the AP tests require written responses, which are much harder to (effectively) BS your way through.
Also, I think they have different intents: an AP test regards a very specific curriculum and is meant to approximate an entry-level college course (to the extent that it can actually count toward credits). SAT IIs are — in my experience — basically a way for a student to demonstrate some academic competency/knowledge in a subject.
My roommate had 1590 out of 1600 on the SAT, whereas I only had 1200. I am able to write this, he can’t read this because he is dead. He is dead because he wasn’t smart enough to quit smoking. He died of lung cancer 7 years ago and I still miss him all the time.
I agree. The written responses are what separates the men from the boys. My senior year English Lit class was pretty rigorous compared to the other classes at my public high school, and I could tell by peer reviewing others’ essays that a lot of us were unprepared. I don’t think you can throw Jane Austen at someone for the first time when they’re 18 and expect them to be able to write an essay on her literary technique.
AP tests, although tailored to a specific curriculum, represented what I wanted my entire education to be like: incredibly challenging, but perfectly attainable if I wanted it. Instead, my school’s average on any given AP exam was a woeful 1.5 (out of 5, where a 5 constitutes getting fewer than 70% of the answers correct).
Instead, the environment I received was one in which people could not place the Civil War in the right century, let alone the correct years. The total number of U.S. senators was unknown, and the initialism “LBJ” was nothing more than an arbitrary amalgam of letters. I wish the Jaywalking segments on Leno’s Tonight Show were staged or selectively edited, but I’m not so sure.
Fun fact: the other day a girl in one of my classes had to show me a U.S. map and ask me, “Which one is Michigan?” God help us all.
I agree that eliminating the essay was a big mistake, but it was preceded by an even bigger one – failure to teach spelling, punctuation and grammar in the grades leading up to the big tests. And along with them (inseparable from them?), critical thought. Setting aside the cost of grading something like an essay, the testers were testing something that was never taught in the first place. I often have to read reports written by people – college grads or not – who could not pass tests I was given in 6th grade.
And another casualty in the present “you’re special” environment is objective testing and grading – and consequences of failing – is the test-taking skills, also noted above.
That latter issue isn’t exactly new. I’ll never forget a bright young man in my college dorm, with a GPA not to write home about. Many of us tried to help him prepare for tests, and it would be obvious that he knew the stuff. Then he would come back with a bad grade, having missed questions where we (and he) knew that he knew the answer. “I thought it was a trick question.” Aaahhhhggg!
In addition to the cost of grading essays, it is a very difficult process to standardize. I haven’t ever graded essays on the SAT/AP/ACT, but have graded hundreds of them on standardized, individually administered achievement tests (Woodcock-Johnson, Weschsler). It is incredibly difficult to standardize across graders and to maintain the check needed for inter-rate reliability.