Bad enough that press, pundits and pollsters are obsessed with the horse-race aspect of the presidential campaign. But this one seems like it’s being called by track announcers who aren’t watching the same race. George Bush gets a postconvention “bump” in some polls but hardly a dimple in others, including NEWSWEEKS. Polls taken at the same time are just barely consistent given their margins of error, the “plus or minus 3 percentage points” that gives them the aura of scientificness. Political consultants blame the gyrating results on “the volatility of the electorate,” and while lots of voters indeed keep changing their mind, that doesn’t explain it all. Is “polling science” an oxymoron?
To be fair, polls that at first blush look contradictory may not be. A Miami Herald poll last week had George Bush ahead of Bill Clinton by 7 points (4841) in Florida; in a New York Times/CBS News poll, Bush trailed (4248). But since the polls had margins of error of about 4 points, they over-lapped (add and subtract 4 from each number to get the possible range). The margin of error, calculated according to a textbook statistical formula, varies inversely with the sample size. In general, about 500 responses gives a possible error of 5 percent either way; 2,500 responses decreases it to 1 percent.
Besides getting enough people, the polltaker must get enough of the right kind of people-those representative of November’s voters. Calling more in Philadelphia than in Anchorage is just the beginning. Polltakers also “look at the demographic distribution of the sample to make sure it conforms to what the Census tells us about the [voting age] population,” says Andrew Kohut of Princeton Survey Research Associates, which polls for Times Mirror and U.S. News & World Report. This is where the black magic comes in. If the polltakers don’t have enough, say, young white males among those willing to answer, they give extra weight to the ones they do have. Each polltaker does this differently. All the methods inflate the statistical heft of a few respondents, and might explain divergent results.
Some polls track registered voters, others “likely” voters. Neither approach is foolproof People lie, says sociologist Seymour Martin Lipset of George Mason University: some three quarters of adults say they are registered, but only two thirds really are. The lies matter because the young, the poor, the less educated and minorities are less likely to register; these groups are more likely to vote Democratic. So surveys of “registered” voters include many who aren’t, and therefore may exaggerate Clinton’s strength. Of the major polls, only the Time-CNN and Louis Harris surveys screen for “likely” voters, by asking questions such as whether the respondent voted in the last presidential election. People can lie about that, too, but Time-CNN and Harris have consistently shown a smaller Clinton lead: those likely to vote tend to be older, whiter, better off and Republican. No poll can fully compensate, though, for the almost 40 percent of those phoned who refuse to be questioned or happen to be out of the house. Whose side are they on? Will the hang-ups split like the rest of the voters? The answers will come in November.
Ensuring a random sample also requires something beyond statistics. During Ronald Reagan’s 1984 re-election campaign, his internal “tracking” polls showed him well ahead of Walter Mondale except on Friday nights. “They went into a panic every week until they figured it out,” says Frank Luntz, a political-science professor at the University of Pennsylvania who polled for Pat Buchanan and Ross Perot this year. The explanation: registered Republicans, on average more flush than their Democratic counterparts, were more likely to go out on the town Friday nights and not be home to answer the pollster’s phone call.
Although polltakers take steps to avoid such selection effects, slips happen. CBS-New York Times, seeking the widely expected “bump” Bush would get from the GOP convention, polled on the last night of the Houston- gala. Result: CBS-NYT had Bush within 3 points of Clinton. NEWSWEEK, phoning the day after, found virtually no postconvention boost: the candidates were 14 points apart. Obviously, people who answered their phones when CBS-NYT called were home; many were home to watch the Houston speeches; many watched because they are Republicans.
Even subtle differences in how questions are worded, and in which questions precede the “Bush or Clinton?” query, can skew results. Some organizations identify the candidates by name only; most give party affiliation. If Clinton is identified as the Democrat, he might fare better in the cities of the Northeast and in union strongholds, some of the last bastions of party loyalty. And if the horse-race question comes after a series on, say, the economy-asking whether the voter is better or worse off than he was four years ago, for instance Bush fares worse. Since the ballot doesn’t have an “Are you better off?” preface, such polls underestimate Bush’s strength.
When Ross Perot still had his hat in the ring, polltakers were surprised at how his strength varied depending on how they arranged their questions. Some polls asked respondents only to choose between Bush, Clinton and Perot; others asked “Bush or Clinton?” first and only then added Perot. “When the three-way question was asked after the two-way question,” says Larry Hugick, managing editor of The Gallup Organization (which conducts NEWSWEEK’S Polls), “Perot got more support than when the three-way question was asked alone.” He speculates that asking Bush-or-Clinton first made people focus on their dissatisfaction with the choice, and made them more likely to pick Perot when his name was added. Lumping all three together made people evaluate each on his merits. All these effects are subtle, but since voters are so fickle in their affections this year the tiny influences matter. “Very small puffs of wind can move an unanchored ship,” says Everett Ladd, director of the Roper Center for Public Opinion Research at the University of Connecticut.
That’s why campaigns have grown more wary of tracking polls. But what may be bad for the employment prospects of poll-takers may be good for democracy. “It’s gotten so bad that you can see the candidates staring at the polling numbers beneath the pages of their speeches,” says longtime GOP consultant Doug Bailey. Maybe, if pols believe a little less in polls, their speeches and positions will reflect the numbers a little less and their convictions a little more.
