Following inaccurate results for final pre-election polls in October 2013 (NJ Special Senate) and November 2013 (NJ Governor), the Eagleton Institute of Politics commissioned an outside study by Gary Langer of Langer Research Associates of New York to identify reasons for the outcomes of these polls. Today, The Eagleton Institute of Politics and Eagleton Center for Public Interest Polling are releasing this analysis to the public as part of a commitment to transparency and education.
The Rutgers-Eagleton Poll reported a final pre-election poll for the special Senate election between then-Newark Mayor Cory Booker, a Democrat, and Republican former Bogota Mayor Steve Lonegan in which Booker held a 22-point lead. Booker ultimately won by 11 points. In the final November gubernatorial pre-election poll, Rutgers-Eagleton had Republican Governor Chris Christie ahead of his Democratic challenger state Senator Barbara Buono by 36 points: Christie won by 22.
The Langer report identifies the primary reason for the inaccurate results as the failure to put the “head-to-head” questions, which asked respondents for their vote intention, at or near the beginning of the questionnaire. Because these questions were asked after a series of other questions, it appears that respondents were “primed” to think positively about Governor Chris Christie in the November survey, which then may have led Democrats and independents in particular to over-report their likelihood of voting for the Governor. A similar process occurred with the October Senate poll, where voters were first reminded of how little they knew about Lonegan and how much they liked Booker before being asked the vote question.
Ruth B. Mandel, director of the Eagleton Institute of Politics stated that, “In response to these results, Eagleton chose to contract with an independent, highly respected, outside survey research firm to review its recent work and offer suggestions for improvement.” She added, “The Institute is committed to contributing to political knowledge in New Jersey and nationally with credible, impartial data. When we saw we had a problem, we knew we had to learn why and what to do about it.”
“Gary Langer and his colleagues spent many hours examining multiple aspects of our polling to understand what went wrong,” said David Redlawsk, director of Eagleton’s Center on Public Interest Polling (ECPIP) and professor of political science at Rutgers. “We are grateful for the efforts they put in and the advice they have provided, both in terms of this specific issue and general operations of the Rutgers-Eagleton Poll. The results of this report will make what we do even better.”
The Rutgers-Eagleton Poll has been a valued source of information about the views of New Jersey residents for over 40 years. As an academic-based survey research organization, ECPIP strives to be transparent and accessible. “We have a special obligation to take our educational mission seriously, which includes informing the public as well as learning from our own errors.” Redlawsk notes that survey research results released by the Rutgers-Eagleton Poll, for example, aim to meet the transparency standards set by the American Association of Public Opinion Research (AAPOR). Further, in recent years, the Rutgers-Eagleton Poll has been providing open informal insights and perspectives about survey research from Redlawsk and members of his staff through its blog at https://eagletonpollblog.wordpress.com. And for many years full data from the Poll has been freely available, generally after a one-year period, at http://eagleton.libraries.rutgers.edu/.
Langer’s major finding is that the order in which the head-to-head ballot test questions were asked most likely added inadvertent bias to the results in both the October and November Polls, although the results came out in opposite partisan directions in the two polls. Decisions made by ECPIP to maintain the standard set of questions about political figures including Cory Booker and ratings of Chris Christie at the beginning of the questionnaire worked to particularly prime Democrats in the November poll and Republicans in the October poll to support the candidate from the other party – Christie or Booker.
Redlawsk noted that the cause was a decision to maintain an ongoing four-year series of questions about Governor Christie that have been asked at the very beginning of a Rutgers-Eagleton NJ Poll since the governor’s inauguration. “We made this decision purposefully to maintain the integrity of our time series,” said Redlawsk. “This long-term research has greatly informed our understanding of public opinion about Governor Christie, and we had concerns that moving these questions after a head-to-head vote question would bias those results for the same reason we ended up biasing the vote questions.”
Most pre-election head-to-head polls focus only on the election and do not include long batteries of additional questions. The Rutgers-Eagleton Poll was unable to field separate pre-election surveys and thus combined the head-to-head polls with the regular surveys of New Jersey public opinion. “In retrospect, this was the wrong choice when one goal was to be as accurate as possible with pre-election numbers,” noted Redlawsk. “We should have either fielded a separate poll or just focused on our long-term work, rather than trying to do both at the same time.”
The Langer report on the cause of the pre-election poll mis-estimates is available to the public now on the Rutgers-Eagleton Poll website at http://eagletonpoll.rutgers.edu (PDF).
eagletonpollblog 