Nothing But Words

Today I came across a couple more articles that cast serious doubt on the quality of the poll results that are currently plasterd all over the media. The first – Why the Race Is a Lot Closer Than People Think – is by political scientist Ruy Teixeira, who I used to read frequently in grad school. His two main points are: 1. Republicans are oversampled in the polls, and most of the major surveys do not take this into account (Zogby is one of the few that does weight the samples appropriately); and 2. the more highly publicized “likely voter” (LV) results tend to be much less meaningful than the “registered voter” (RV) results. At the end of his piece, he reprints a letter he wrote to Gallup, taking them to task for their recent highly publicized historical analysis, which indicated Kerry was in trouble. They used their current numbers for LVs and compared them to the results for RVs in previous election cycles (i.e. they compared apples and oranges). He offered a rewrite of their report’s conclusion:

“In summary, the history of presidential elections since 1952 suggests that in all cases, the type of gap change that would be necessary for Kerry to tie or move ahead of Bush has occurred. If a gap change does occur, the odds are very strong (11 out of 13) that it would be in Kerry’s direction (i.e., a shrinkage rather than an expansion of Bush’s current lead).”

This clearly sounds quite a bit different [than what Gallup actually reported]. And thinking Kerry is behind by 1 point [Gallup's RV numbers], rather than seven points [Gallup's LV numbers], clearly makes a big difference when considering elections like 1960 and 1980, which loom large in your analysis. Kennedy was behind by a point in 1960 among RVs—the same as Kerry—and Reagan was behind by 4 points in 1980—more than Kerry…

The second – Relying on Phonies – casts doubt on the usefulness of phone-based polls in the 21st century. The author’s main point goes beyond the lack of contact with cell phone users that I mentioned in a previous post:

We Americans simply don’t answer our phones like we used to. Entire industries are now devoted to helping us not answer the phone. Voicemail, Caller ID, caller-specific-rings, cell-phones, even email have fundamentally transformed the ways we (don’t) answer the phone when it rings. These and other technological innovations have moved us from a late-20th Century near-pavlovian automatic response of answering the phone when it rang, to new levels of screening or ignoring calls without a sense that we might be missing something important. When pollsters call under these technological conditions they are now increasingly treated as any telemarketer or unknown caller would be, thus the people who pollsters actually get to talk to are becoming increasingly less representative of the general public. There now may be something unusual about people who are willing to answer the phone and talk with strangers, and we should be skeptical about generalizing from the results of these surveys…

[A Pew survey found that:] “More African-Americans than Whites have caller-ID (73% vs 47%) and a higher percentage of Blacks use it for call screening (34% vs 24%). Young people, ages 18-29 are the group most likely to say they always screen calls with caller-ID (41% say this), compared with only 12% those aged 65 or older.” Pew also found that more women than men were found to use features like call blocking (20% vs. 14%). If we can get over the paradoxical fact that this data was collected in phone interviews (and of course the point of this piece is that I’m not sure we can get over that) you can see that those profiled as being most prone to answering phone surveys tend to be: (more) White, (more) older, and (more) male.