Close-up
Who you gonna call? Tricks of the polling trade

By MILES BENSON
Newhouse News Service

BILL PUGLIANO / GETTY IMAGES Pollster John Zogby says experts may average the results of several polls to calculate how a race is going, sort of "a poll of polls, and that's not bad. In fact the record over the years has been good."

E-mail this article Print this article Search archive Most read articles Most e-mailed articles

Related stories Past polling may be a clue to the outcome

It's hard work teasing the truth out of people to track trends and gauge attitudes in a nation of 200 million potential voters. Pollsters don't always get it exactly right.

But there are a few things they think they know for sure:

• Based on history, only about 100 million of those potential voters will actually vote.

• You need only 1,000 likely voters for a reasonably accurate national poll. Double the sample size and you double the poll's cost, but not its accuracy.

• It's usually a woman who answers the phone. So to get enough men to balance a survey, interviewers must ask to speak to the oldest male in the household, or to the voting-age male who had the most recent birthday. And it's best to interview only one person per household.

• Never, never attempt to poll on Mother's Day.

Call it science, craft or profession, modern public-opinion pollsters have been working to perfect what they do since 1936, when the Gallup organization predicted Democrat Franklin Delano Roosevelt's second-term victory over Republican Alf Landon.

Since then, methods have changed from door-to-door, in-person interviews to telephone interviews using numbers randomly selected by computers.

And the experts have learned it can be tricky business.

Polls have a short shelf life. They are a snapshot at a moment in time, not necessarily a projection of the future. Events like conventions and debates can change opinions almost overnight.
 
Indeed, 8 percent of American voters are likely to change their votes in the last month before an election, and 4 percent are likely to do so on Election Day, according to a study published in this month's issue of Political Science and Politics, a journal of the American Political Science Association.

So as Election Day gets closer, pollsters supplement their technique. They survey 200 voters on each of three nights and aggregate the results. Each new day, an additional 200 are added and the least recent group is dropped.

"This allows us to keep track of night-to-night changes as a back-and-forth really gets going," said Democratic pollster Celinda Lake. "But there is a danger of overreacting to the shift that takes place on one single day, so these polls are often misinterpreted."

Keeping up with changes

Pollsters must hurry to complete each survey within a couple of days, said Andrew Kohut, director of the Pew Research Center for the People and the Press. "Certainly, when opinions are changing, if you take more than four or five days, you are likely to find yourself part of the next trend."

Although many people think polling is unreliable because different polls show wildly differing results, among the most respected national polls, results usually are tightly clustered. They may differ a bit, but not by much.

Look at national polls of likely voters conducted Oct. 1-3, for example. In the presidential matchup, Gallup found the George W. Bush-John Kerry race a dead heat, with each candidate favored by 49 percent of respondents. The Zogby Poll had Republican Bush up 46 to 45 percent over Democrat Kerry. A CBS-New York Times poll showed Bush ahead 48 to 47 percent.

Experts may average the results of several polls to calculate how a race is going, sort of "a poll of polls, and that's not bad. In fact the record over the years has been good," said independent pollster John Zogby.

Some pollsters distrust results they get on certain days.

Polling the Monday before an election is useless, said Republican pollster Bill McInturff: "People clam up, they stop talking to pollsters. We've had very bizarre data on the Monday before election Tuesday. Undecided goes way up."

"We don't poll on Friday nights," said Democratic pollster Mark Penn.

"Or on Halloween," McInturff said. Wednesday, he added, is a bad night in certain Bible Belt areas where large numbers of voters go to church twice a week.

Weekend nights — Friday and Saturday — are bad because younger and more affluent voters are less likely to be home, pollsters believe.

Voters who do answer calls those nights add up to a "more downscale" group, McInturff said — more likely to be Democratic. That discovery is a legacy of the Reagan years.

"Reagan's support would dip in polling on Friday and Saturday nights," McInturff said, "and then on Monday it would be right back where it was."

Lake avoids Fridays too, but for the opposite reason. "We never poll on Friday night because more Democrats are likely to be out," she said. "Friday is bowling night and there are Friday night football games so you get fewer blue-collar people at home.

"Same for Saturday," she said. "Because younger suburban couples, two-earner families, are out doing stuff. Sunday night is a good time to get everybody, but we never call on Sunday morning because religious voters are likely to resent it. You don't poll during the Super Bowl football game if you want men."

Mother's Day is notoriously impossible for all pollsters. "That's like death, the lines are all jammed, we just give up," McInturff said.

Just as the wording of questions affects how people answer, sometimes the interviewer's gender can make a difference.

"There is evidence when you are running a male candidate against a female candidate the gender of the interviewer has consequence" — the number of undecideds is higher when the interviewer and respondent are not the same sex, McInturff said.

"It could well be women are less likely to disclose their preference to a man," he explained, or that a man "may not want to disclose [to a woman] that he's voting for the guy over the woman candidate."

Samp1ing and error margins

So how can a sample of only 1,000 likely voters possibly signal the way 100 million voters will cast their ballots?

The key is something called "equal probability of selection," said Frank Newport of the Gallup Poll. "If every member of the population has an equal probability of being selected in a sample, then that sample will be representative of the population."

"It's pure probability and statistics," said Zogby. "The same theory is involved as when you take a blood test and the clinician draws only a small sample rather than draining all the blood out of your body."

One thousand interviews will produce a poll with a margin of error of about 3 percent; doubling the sample size only shaves that margin to 2 percent.

The margin of error is a statistical assurance measurement. It means if the same poll is done 100 times, with a new sample of voters each time, the results will turn out roughly the same 95 percent of the time. A 3 percent margin of error means the pollster would get the same result, plus or minus three percentage points, 95 out of 100 times.