Polling: Sampling Variation and the Margin of Error

By conducting their own experiments, students will discover how variation occurs when pollsters choose random samples. Students learn how this variation is quantified as the margin of error. Finally, students use this knowledge to interpret poll results and draw conclusions about population differences.

I. Objectives
II. Necessary Materials
III. Estimated Time
IV. Background
V. Teaching Procedure
VII. Online Resources
VIII. Relevant National Standards



I. Objectives
  		•  Students will be introduced to the idea of random variation among samples drawn from the same population.
  		• Students will learn how random sampling variation is quantified as the margin of error in political polls.
  		• Students will learn how the margin of error is used to draw conclusions from sample data about population differences.
  		• Students will generate a sampling distribution simulating an actual polling situation and will use their results to understand the characteristics of sampling variation.


    II. Necessary Materials
  		•  access to either a newspaper, a news magazine, or a news Web site, which has a report on a poll
  		• Student Worksheet and Answer Sheet
  		• Clip-on (lavaliere) microphone or boom pole mic
  		• small boxes or containers (roughly the size of a cottage cheese container), 1 for each group of 4 students
  		• small tags or one-inch squares of paper marked with a 'Y'; there should be 48squares for each box
  		• small tags or one-inch squares of paper marked with an 'N', there should be 52 squares for each box

    III. Estimated Time
    2 hours

    IV. Background
    How the Margin of Error is Quantified
    The margin of error is based on a measure of variation called the standard deviation, applied to the percentage data found in polls. Usually the standard deviation measures the variation of individuals from the group mean. But now, rather than measure the variation of individuals within one sample, the standard deviation is used to measure the variation among the results of repeated samples we would take from the same population. This enables us to quantify the slight variation among samples. This measure, called the standard error, is calculated using the following formula, where p is the estimated proportion and n is the size of the sample. (If you study probability you will learn more about how this formula was derived.)

    Margin of error =

    Real-life data often follow the bell curve, or normal distribution. In a normal distribution, 95% of the data are within two standard deviations of the average. If we took many samples and plotted all the sample results (called a sampling distribution), we would see a normal distribution in which the individual sample results averaged around the true population percentage, and 95% of the sample results would be within two standard errors (standard deviations) of this population value. If we calculate two standard errors based on a middle proportion of .50, we would get the margin of error that is used in polls:

    This formula reduces to 1/, which is much simpler and is the form that most professional polltakers use. As this formula shows, a larger sample will reduce the margin of error. When polltakers decide what size sample to use, they balance the margin of error against the added expense of a larger sample.
    For more information, see Survey Says: Margin of Error.


    V. Teaching Procedure
    1.  It is best to begin this lesson with an introductory discussion of the fact that samples vary. www.politics1.com/polls.htm shows comparisons of several current polls taken at the same time and is useful for highlighting the variation among random samples responding to the same question. Students can discuss how variation among samples is inevitable simply because each sample consists of a slightly different group of people.

    2. 
    This discussion may then be followed by the simulation activity. For student directions on how to complete the activity, see the Worksheet. Working in small groups, students then draw repeated random samples from the same population before pooling their results with the rest of the class and plotting the results.
    If the class has studied basic probability and the characteristics of the normal distribution, the mathematical features of the plot can be discussed. If they have not studied probability, the mathematical analysis may be omitted and students can simply observe how the sample results balance out around the true population percentage. It can be pointed out that statisticians measure the amount of variation around the true percentage to form the margin of error.
    See Survey Says!: Margin of Error for information on how to calculate the standard error and margin of error.

    3. 
    Students may be divided into groups of four, and each group given a small container, along with 100 one-inch squares (48 "Y's" and 52 "N's"). This represents a population of 100 people, of whom 48% responded yes to the question and 52% responded no.


    4. 
    Each group should record the number of Y's in each of its samples and turn in this sheet so that the whole class's samples may be combined. The teacher may photocopy these sheets so that each group has a copy of all the data for plotting, or may put them up on the blackboard.
    To do the plot, make a list of all the possible outcomes (0 yes's out of 10, 1 yes, 2 yes's, etc.) and count the frequency with which each one has occurred, then plot this distribution in the form of a dot plot, histogram or polygon. The average number of Y's for all the samples combined should then be computed and marked on the graph. This average will be close to 48% yes. Best results are obtained when the class has collected 200 or more samples.

    5. 
    After performing the simulation, students can read the text (Survey Says: Margin of Error) and apply the margin of error to the interpretation of actual polls. Students may do the activities on the Worksheet to reinforce the application of the margin of error and look for actual examples in the news and think about how to interpret them. Students may use the Web sites listed below in the Online Resources to find current poll results.

    For your convenience answers can be found on this Answer Sheet.


    VI. Extensions / Adaptations Ideas
  		•  If these activities are used in the context of a larger unit or course in statistics, they may lead to a discussion of confidence intervals and statistical inference.
  		• Students may conduct their own polls and calculate the margin of error as they analyze the results. This calculation will be valid only if students have chosen a true random sample.


    VII. Online Resources
    Democracy Project: Analyze a Poll www.pbs.org/democracy/readbetweenthelines/poll.html

    Gallup Poll
    http://www.gallup.com

    Pew Research Center for The People and The Press
    http://www.people-press.org

    PollingReport.com
    http://www.pollingreport.com


    VIII. Relevant National Standards
    This lesson addresses the following national curriculum standards established by the Mid-Continent Regional Laboratory.

    Data Analysis and Probability:
  		• Develop and evaluate inferences and predictions that are based on data
  		• Understand and apply basic concepts of probability

    Communication:
  		• Communicate mathematical thinking coherently and clearly

    Connection:
  		• Understand how mathematical ideas interconnect and build on one another to produce a coherent whole
  		• Recognize and apply mathematics in contexts outside of mathematics