Pollsters should seize every opportunity they get to put theories to the test.
San Diego, CA (PRWEB) July 31, 2013
New York mayoral candidate Anthony Weiner may have entered this year’s race with a splash. But as reported by Domenico Montanaro and Alex Moe on NBC, the latest Quinnipiac poll shows that after an early lead, the former Congressman has fallen from first to fourth place among Democratic primary voters in the city.
According to Competitive Edge Research and Communication, a San Diego based public opinion polling firm, it’s understandable – given Weiner’s problematic and highly publicized past – that the media would eagerly report the latest polling results. However, recent conflicting survey results illustrate how reporting can go astray when the media does not dig deeply enough into a story or doesn’t fully understand the nature of polling.
“New York City is clearly the nation’s most prestigious mayoral job,” says John Neinstedt, Sr., president and CEO of Competitive Edge. “So it’s only natural that the media closely follow this infamous ‘selfie’ tweeting character’s quest to run the Big Apple. But when the media doesn’t do its job well, reporting of poll results can often get misinterpreted or distorted.”
For example, a recent poll conducted by the Wall Street Journal/NBC New York/Marist showed Weiner ahead with 25%. City Council speaker Christine Quinn lagged close behind with 20%, followed by City Comptroller William C. Thompson, Jr. with 13%. The next day, Quinnipiac University released a poll showing a virtual tie, with Quinn at 19%, Weiner at 17% and Thompson at 16%. One day Weiner is up; the next he’s down eight percentage points.
According to Neinstedt, the New York Times’ reporting of the conflicting results was based on faulty logic. The Wall Street Journal/NBC New York/Marist poll was conducted from June 17 to 21, while the Quinnipiac Poll was done from June 19 to 25. The United Federation of Teachers, which represents the city’s public-school teachers, endorsed Thompson on June 19, giving him extra media exposure during the full period when the Quinnipiac poll was conducted.
Compared with polls conducted in May, Thompson gained two percentage points – a statistically insignificant change – in the Marist poll, but he jumped by six percentage points in the Quinnipiac poll from a previous Quinnipiac poll. Without actually saying it, the reporter theorized that the union endorsement gave Thompson a boost that showed up in the Quinnipiac poll – a strong attribution for just one endorsement.
Considering the fact that a 3% difference between Thompson’s Wall Street Journal and Quinnipiac numbers is almost certainly not statistically significant, the reporter didn’t need to propose a theory. Instead, Wall Street Journal researchers could have easily broken down their results into pre- and post-endorsement time periods for the reporter. Applying basic statistical tests would then have allowed real conclusions to be drawn.
“As presented, the story does more assuming than explaining of the conflicting poll results, leaving the public to wonder about the validity of the research,” says Neinstedt. “Pollsters should seize every opportunity they get to put theories to the test. As was the case in this instance, the data is often sitting right there in their datasets. All that’s needed is some smart analysis.”
About Competitive Edge
Competitive Edge Research & Communication is a San Diego-based consulting firm specializing in public opinion surveys and analysis, political polling, civic studies, and phone-based campaign services. Since 1987, Competitive Edge has provided clients with accurate survey research and campaign services, including designing sound research plans, collecting and analyzing accurate quantitative and qualitative data, providing clear strategic advice, and communicating with voters to identify their opinions and persuade them to take action. For more information, visit http://cerc.net.