As a university based survey research center, much of our lives revolve around the academic calendar. As I write this, students at Rutgers are taking final exams, and graduation is just over a week away. It’s been a very busy year. We won’t completely go away for the summer, but we definitely slow down operations during that time. We do expect to do one public poll over the summer break, and then to pick things back up in late August as we move into the 2012 general election.
Following is one more post from another one of this year’s undergraduates working with the Poll. Jack West is an Aresty Research Assistant with us this year, and spent much of the year doing research on how we can better present our results graphically. As we move toward a new website (SOON!) we hope to put his research into action. In the meantime, here is a post from Jack looking at the implications of the margin of error that we all report when we report our polls. In particular, he makes an interesting point about a recent Farliegh-Dickinson PublicMind poll showing a majority of New Jerseyans are now saying the state is on the right track.
The Importance of the Margin of Error in Interpreting Results
In March, the Fairleigh Dickinson University PublicMind Poll reported that, for the first time in a decade, a majority (51%) of New Jersey voters believe that the state is heading in the right direction. These findings were reported far and wide by both news sources and the Christie administration, which touted the results as proof that New Jerseyans support their governor and his political actions. Fairleigh Dickinson interviewed 800 registered NJ voters, and asked fair, unbiased questions. Their work is well-done and generally reliable. So why should readers look further before simply accepting this apparent shift in New Jersey’s political support?
The answer is a small, seemingly insignificant number buried at the bottom of news articles or, in the case of the Christie campaign’s coverage, ignored altogether: the margin of error.
In their effort to inform people and report public opinion, pollsters use somewhat complicated sampling techniques and statistical analysis that provide an estimate of public opinion. The number reported – like 51% – is the best estimate given the sample. But as an estimate, there is ambiguity in the sense that the estimate itself could be wrong. How much it could be wrong is reflected as a range of percentage points that the results are 95% likely to fall into when applying the estimate from the sample to the actual population. This is included in all poll releases as the margin of error.
Although Fairleigh Dickinson reported that 51% of NJ voters believe that the state is heading in the right direction, they also reported a +/- 3.5% margin of error. That could take the result as high as 54.5%, but it could also bring it down to 47.5%. An Eagleton Center for Public Interesting Polling survey, conducted two weeks later in March, reported only a 47% plurality of right-direction responses, adding to the possibility that perhaps Fairleigh Dickinson’s outright “majority” was at the high end of the truth. In fact, our poll had a margin of error of +/- 4.3 percentage points, so our numbers could be between 43.7% and 51.3%. Given that our results overlap, the odds are pretty good that the “true” number is between 47% and 51%.
Ultimately, no one can definitively state the exact percentage of New Jersey voters who believe that their state is doing well unless somehow we talked to all of them. Polling gives us an estimate, and it’s always useful to look at more than one. It could be that Fairleigh Dickinson’s poll does indeed reflect current public opinion, and a decade-old trend has been overturned. However, the remarkable press coverage of this question serve as a reminder that no matter the numbers, and no matter the issue, it is crucial to clearly and openly report margin of error. All reputable polls do this, but not all reports of the polls results follow suit. They should.