Editor’s Note: A few weeks ago, we asked you to take the bubble quiz, based on Charles Murray’s 2012 book, “Coming Apart: The State of White America, 1960-2010,” meant to test how well you know mainstream white American. Today, Charles Murray continues his exploration of the scores and ZIP codes from the bubble quiz in a third post. You can find his first post analyzing the results here, his second here and his third here.
Surprise, surprise. The bubble quiz turns out to be a good test psychometrics wise.
I can’t claim it was part of the plan. The bubble quiz was supposed to be a heuristic device. I wanted members of the new upper class who read “Coming Apart” to come to grips with the degree of their isolation from mainstream white America, and I hoped the 25 items in the quiz would help them to do that. I tried out a draft of the test on the American Enterprise Institute’s interns and research assistants to get their reactions and tweak the wording of the questions, but I didn’t try to construct a psychometrically sound test. And yet that’s what the bubble quiz appears to be.
In my latest download of the numbers from the NewsHour, I have 66,647 cases with complete and interpretable data, which I have defined as cases with participants’ current ZIP codes and ZIP codes at age 10; an age of at least 20; and a bubble score greater than zero and less than 100 (given the nature of the items, the 37 scores of either zero or 100 are awfully suspicious). The distribution of these 66,647 cases is extremely close to a bell curve.
The mean for the NewsHour sample is 40.6 points with a standard deviation of 16.0. Based on supplementary analyses, I estimate the mean for a nationally representative sample to be 44.2.
The first post about the bubble quiz had a graph showing separate lines for the relationship between the bubble score and the socioeconomic status of the ZIP code where respondents currently live and where they lived at age 10. By combining the two ZIP code scores, I get a better measure of the total socioeconomic status experience of people over their life spans. The next graph shows what happens when the bubble score is plotted against that combined socioeconomic status score.
The correlation between the bubble score and the combined socioeconomic status index is .43. For the social sciences, that’s high. For example, the correlation between educational attainment and income is usually around .4. The strongest single predictor of job productivity is an IQ score, and that correlation averages about .4 across studies. If someone took the effort to refine the items on the bubble quiz and added a dozen additional questions, it should be possible to jack up the correlation with the combined socioeconomic status index to around .5 — the average correlation of a child’s IQ with the midpoint of parental IQ.
It’s not just the size of the correlation that counts. The absolute change in score from the bottom percentile to the top is 25.5 points, amounting to 1.7 standard deviations. One’s score on the bubble quiz is, by the standards of the social sciences, a useful and powerful measure of something — exactly what we can argue about.
The graph is based on a sample consisting of people who go online and take a 25-item test. It is not nationally representative. The mean socioeconomic status percentile of the ZIP codes where the respondents lived at age 10 was 61.0. For the ZIP codes where they live now, it was 71.5. But the difference in means in the bubble score between the people who took the test (mean of 40.6) and the estimated mean for a nationally representative sample (44.2) is less a quarter of a standard deviation. Simulations based on a difference of this size indicate that the unrepresentativeness has modest effect on the results.
The question that has consumed many hours of data analysis since my previous post in this series is whether elite ZIP codes in certain metropolitan areas are systematically and reliably associated with larger reductions in the bubble score than other elite ZIP codes. It’s a complicated question. On first inspection, five metropolitan areas stand out: New York, Washington, Los Angeles, the corridor running from San Francisco to San Jose and Boston. But maybe those results can be explained away by population size or the general economic condition of the area. The quantitative techniques for exploring these issues are complex, and I will refrain from presenting any of my results to date for fear I will discover a week from now that there’s a more powerful way of modeling what’s going on.
I’ll leave with you with a pair of maps (not professionally prepared — my apologies) that illustrate why I think there’s something worth investigating.
The first map is of ZIP codes in Minneapolis-St. Paul and its surrounding suburbs. The dark red ZIP codes are in the 98th and 99th percentiles, the lighter red ones are in the 95th to 97th percentiles, and the pink ones are in the 90th to 94th percentiles. In other words, all the shades of red are the top decile. The blue ones are in the 80th to 90th centiles. The yellow ones are below the 80th percentile.
Minneapolis and St. Paul are big cities, with 1.8 million people within their joint city limits. They have a few clusters of affluent neighborhoods, with the largest being on the west side of Minneapolis. But just two of those ZIP codes are dark red, and they contain just 19,000 people. Each is bordered by at least one ZIP code below the 90th percentile. Almost all of the other red ZIP codes in the cluster are also bordered by ZIP codes not in the top decile.
Now, look at the map of Washington and its surrounding suburbs to the west and north.
Washington itself has a much smaller population than Minneapolis-St. Paul — 833,000 people within the District’s borders — but the rest of the ZIP codes shown in the map give it a population that is roughly similar to the population in the Minneapolis-St. Paul map. Instead of just two ZIP codes in the elite 98th and 99th percentiles, however, the Washington metropolitan area contains 42 such ZIP codes with an aggregate population of 776,000 people. Not all of them are on the map, but the key ones are shown: those in Georgetown, the rest of Northwest Washington, Bethesda, Chevy Chase, McLean, Great Falls and Potomac, which together constitute the neighborhoods where the powerful elite in Washington is concentrated. Those ZIP codes form a contiguous blob. Furthermore, with a few scattered exceptions, they are bordered by ZIP codes in the 95th to 97th percentiles, along with others that are merely in the 90th to 94th percentiles. In the District itself, the only place where the dark red blob is bordered by blue or yellow ZIP codes runs along Rock Creek Park — a broad, heavily forested buffer between the homes of Northwest Washington and those of the hoi polloi.
For the 16 respondents from the two elite ZIP codes of Minneapolis-St. Paul, the mean bubble score was 40. For the 991 respondents from the elite ZIP codes in the Washington blob, the mean was 32.6. I’m not treating that discrepancy as proof of anything. But it is suggestive that there’s something worth exploring, just as the maps themselves illustrate how very differently cities have organized themselves into elite enclaves.
And so now I will burrow into the data and see what else is there. I have no idea when I will surface, nor what, if anything, I shall be clutching when I do.