Editor’s Note: If it’s the first Friday of the month, it’s jobs day. But what happens before the Bureau of Labor Statistics releases that data at 8:30 a.m.? In other words, how is the data collected and calculated and whose job is that? Making Sense spoke with two BLS economists for an inside view of the monthly report.
For all the attention the report gets, the data can be notoriously scattered each month – “noisy,” as Justin Wolfers told us. Economists take a stab at predicting the monthly numbers, but they’re the first to admit that any one month’s figures shouldn’t be given too much fanfare.
Zachary Karabell agrees — and goes further. Author of “The Leading Indicators: A Short History of the Numbers That Rule Our World,” Karabell argues that we obsess over statistics that don’t give a complete and nuanced picture of our economy. Watch Paul Solman’s conversation with him below:
As we’ve long noted, the headline numbers that emerge from the BLS report (the unemployment rate and the number of jobs added) come from two different surveys, which sometimes deliver wildly different employment outlooks.
In January, for example, the unemployment rate ticked down to 6.6 percent, its lowest since October 2008. But only 113,000 jobs were added. In February, the unemployment rate went up to 6.7 percent, but the number of jobs added was healthier, at 175,000.
The percentage of unemployed people comes from the household survey, while the number of jobs added comes from the payroll or establishment survey — literally a survey of establishments (businesses) that employ people.
BLS economists Karen Kosanovich, who works on the household survey, and Julie Hatch Maxfield, who works on the establishment survey, spoke with us about how they collect their data and the isolation they face in the days before the data is released.
—Simone Pathe, Making Sense Editor
How do you come up with the unemployment rate?
Karen Kosanovich: What we actually do is survey people all across America and ask them questions that allow us to categorize whether they’re employed, unemployed or not in the labor force. And so it’s the survey data that form the basis of our unemployment measure.
The data collection is actually done by our partners at the Census Bureau; they’re the experts at knocking on doors and talking to individuals. They talk to about 60,000 households and collect data from about 108,000 people each month. We ask these people questions based on their activity in the prior week.
So we will interview people in the first week, asking, “Last week, did you do any work for pay or profit?” We ask questions about work activity first, and then if they didn’t work, we’ll ask if they’ve looked for work in the last four weeks or if they’re on lay-off, waiting to be recalled from a job.
So if you’re working, you’re employed. If you’ve looked for work, you’re unemployed. If you’re on lay-off from a job and waiting to be recalled, you’re unemployed. If you’re not employed or unemployed, we put you in a category called “Not in the Labor Force.” That “Not in the Labor Force” category would include people like my father, who’s retired at 85, or maybe someone who’s staying at home to take care of their children full time, or going to school.
What are the limitations of the data?
Karen Kosanovich: There are strengths and limitations. One of the strengths of the labor force survey is that it’s optimized to produce a measure of unemployment — and the change from one month to the next. But the statistical data are based on a sample, and so every sample survey that we have has a certain amount of error associated with it. … We need about two-tenths of a percentage point change for it to be considered statistically relevant. So you’ll notice in our reports that our language is fairly precise to account for that imprecision that’s part of the estimates.
Do Census Bureau workers still knock on doors to conduct the household survey?
Karen Kosanovich: The household survey data collection is both in person and over the phone. The first contact that we make is a letter that the home receives, and the first contact to collect data is in person. You get better results if there’s an individual that comes to the door that can show you their governmental ID and walk you through a series of questions. At the discretion of the respondents, we’ll collect information over the phone or in person — whichever they prefer.
But the unemployment rate is only one part of the monthly jobs data the Bureau of Labor Statistics releases. Where does the number of jobs added or lost each month come from?
Julie Hatch Maxfield: The establishment survey is exactly what it sounds like: we go after the establishments. Depending on the size of the firm, we will target someone in the payroll or accounting office. And we ask them to provide us the total number of employees that worked or received pay for the pay period that includes the 12th of the month.
We collect data many different ways. A lot of our data collection is done electronically, mostly through the web, and through large data dumps from the firms themselves. And we receive that data every month to make estimates. There are about 145,000 establishments each month that we survey, which represents about one-third of all payroll non-farm jobs.
What are the limitations of the establishment survey data?
Julie Hatch Maxfield: As with any survey, there is error associated with it. For the establishment survey, a change has to be over 95,000 employees, and then you get a month for it to be a truly statistically significant change. So we have to have a change, plus or minus more than that amount, for us to say that there are more jobs added or lost.
What’s the history of the establishment survey?
Julie Hatch Maxfield: The current employment statistic survey, which is the establishment survey, has actually been around since 1915. It started with four manufacturing industries and then since then it’s been evolving, with service-providing industries added, becoming more a representation of what’s happening in the economy. … The data we use in the national estimates is also used in the state and area estimates as well.
And the origins of the household survey?
Karen Kosanovich: The household survey began in March of 1940. Prior to that time, there was no national measure of unemployment, and in fact, there wasn’t real consensus on how you would approach measuring unemployed individuals.
How has the process changed?
Karen Kosanovich: One of the most remarkable things to me is that it actually hasn’t changed greatly since 1940. … We’ve added additional questions to provide more nuance and additional detail in what we can describe. The biggest change in terms of the household data collection was that in 1994 we switched from a paper and pencil instrument to electronic data capture. But the timeline has been the same: we collect data on the week of the 19th, generally, and ask people questions about the 12th of the month. And we report the data just a few weeks later.
Julie Hatch Maxfield: For the establishment survey, the biggest change has been in data collection. It used to be a mail survey exclusively, and now we do not collect any of the data through the mail.
What kinds of security measures do you take to safeguard the numbers ahead of the official release?
Karen Kosanovich: Confidentiality is important to us and it begins even before data collection. There’s a couple of main avenues. We obviously can’t go into a lot of details about the exact steps, but it begins with computer and electronic security from the very beginning of data collection through the publication of estimates. The individual staff members are an important line of defense — information is available only to those who have a work-related reason to know.
And then we use physical security measures. So you may have noticed entering the building that we’re in a secure location — different parts of the office are physically isolated while production’s involved.
And so it seems remarkable to people that economists have to take out their own trash during certain periods of the month while we’re preparing the unemployment news release. Suites are actually locked; there’s no trash collection or recycling that occurs during that time.
You need an identification badge that allows you to be in that space. There are signs on the door that notify people that they are not allowed to enter. You would be accompanied if you do enter the suite.
What was your reaction to that when you first started working here?
Karen Kosanovich: It was an adjustment the first few weeks. It’s really important that you carry your identity badge if you’re going to the rest room during lockdown, or you need a friend to let you back into the suite. But you get very accustomed to it, and it becomes routine and just a part of our work day — just during our production periods. So this is not Pentagon-level security every day.
So maybe not Pentagon-level, but why such tight security?
Karen Kosanovich: Participation in both these surveys is voluntary, so one of the ways we treat our respondents well is protecting the information they give us. It’s very personal when the government is asking where you work, the kinds of things you do at work, how much you earn. For businesses, it’s very important that they be able to protect information about what they pay their employees. So we respect that information and keep it secure.
And then because the data are economically sensitive, this information is classified as a principal federal economic indicator and so, because of the heightened attention that’s paid to it, there are additional procedures involved to be sure that we’re releasing the information uniformly to all of our data users.
What do you make of the fanfare surrounding the Friday morning release of the jobs report?
Karen Kosanovich: The hype that surrounds the minutes leading up to 8:30 when the data are released and the first few minutes afterwards — I’m not sure I understand all of that excitement. … For us it’s a longer term perspective on, not so much what one month’s number is, but what can we know about what’s occurring…the longer term trends.
Julie Hatch Maxfield: I think part of what we are interested in is how does that message get across. Because we take great care on how the message is delivered. We really want to let you know a change is a change. We would not say that the unemployment rate fell one-tenth of a percent because that’s not statistically significant…
Sometimes there’s a level of frustration for us because we take time to craft our message and either different sound bites go out…or it’s a spin that’s not what we would actually say. You know, sometimes we are called to validate things that either were not the truth or not out there. Our job is to report the data as it is and to give factual analysis, not to put any spins on the data.
We know this data is important to people, and we want to make sure that if we lead with something, that it makes economic sense to do so. We want to make sure that if there is an important change, we explain it as much as possible and if something is not statistically significant that we do not draw attention to it. A lot of what we try to do is give context to how the current month fares in comparison to recent trend.