By JIM SCHUH
of The Gazette
Polls, surveys, questionnaires - there seem to be plenty of them around these days. Most major polls are credible, since they utilize acceptable methodologies in arriving at their conclusions. But there are some others we shouldn't believe.
Many of us have participated in answering polls that come in the mail - from your congressman, local government and magazines. The Wall Street Journal sends one out from time to time along with a crisp, new $1 bill as an incentive to complete the forms. I like that, and since the Journal is a reputable company, I respond. I also get the tabulated results when they're completed and believe them. Likewise, I receive occasional e-mail requests from Business Week magazine to answer questionnaires, and I usually fill them in. My reward: 30 minutes of free long distance time.
You've likely had telephone requests from political surveyors. Some of them are thinly disguised pitches to vote for individual candidates. The way I get rid of them is to say, "I'm a journalist." That's when they hang up.
The folks at the Gallup Organization send me frequent survey results via e-mail, detailing how people view the government, social issues and lifestyle elements. It's fun to look at changing tastes and new trends. In recent weeks, I've seen poll results about romance around Valentine's Day, John Kerry's lead over President Bush, fewer people going to movies in 2003 than in the previous year, a constitutional amendment to define marriage and the (Ralph) Nader factor. Past polls have dealt with sexual abuse by priests, smoking and eating excesses, alcohol and drug abuse, and several on how people feel about terrorism-related items.
Sometime back, a Gallup poll that concerned me dealt with results taken after the New York Times revealed that one of its reporters had faked news stories and plagiarized from other reporters. They showed five-eighths of Americans believe news organizations are often inaccurate in their reporting - with only 36 percent saying media get the facts right.
That's mostly believable stuff, but let's look elsewhere: You're subjected to silly polls nearly every day of the week, but you might not always recognize them. They're the ones on your local TV newscasts, where the stations ask for your opinions on contemporary topics. These polls are nonsense and you shouldn't waste your time responding. Serious pollsters know that asking viewers to log on or call an 800 number measures only those who respond, and they're not a valid representative sample. Stations know these shortcomings, but all they want to accomplish is to make listeners feel "involved" - that's what their news consultants have recommended. It's a sham to present the results as anything legitimate. What's to stop anyone from calling in five or 10 times? If that doesn't turn you against believing local TV newscast polls, think about the idiotic questions the stations pose nightly.
I had an answering machine installed in my radio newsroom in 1967, and we asked listeners to call their answers to a question on local issues. While some responded sensibly, most recordings I sifted through each morning came from kids making raspberry noises into the phone. A few questioned my heritage. It didn't take long to pull the plug, although some of the sounds and comments were quite creative. We never presented the results as anything but entertainment.
While radio ratings surveys aren't perfect, they're more believable. Arbitron chooses a representative sample of people, and asks them to record their listening habits in a booklet for two weeks. Assuming most people to be honest, the firm then publishes a list of how many listeners each station has.
Still, not all broadcast surveys are valid. In my college days, I was involved in conducting a survey on FM radio listenership - well before FM was popular. Each member of our class at Marquette was assigned to call 125 people, using a defined random selection method. Our questions were well-conceived and tested beforehand to guard against potentially erroneous results.
After a brief explanation of what we were doing, we asked our respondents, "Do you own an FM radio?" That seems straightforward enough, but even such queries can produce funny responses. My favorite was from an accented old man, who quickly said, "No, I own a G-E."
Most of us in the class worked hard to try to produce legitimate results on how many people had receivers and listened to FM radio in the Milwaukee area. But toward the end of the survey period, we discovered that two football players in the class fudged their survey forms, making up all the responses.
It was shameful because their actions invalidated the survey. Still, our professor - who did not know of the shenanigans - published it. When something like that comes out on the letterhead of a university, many people accept it as fact.
The experience taught me to be wary of survey methodology and to question the validity of every survey or poll. I recommend that you do the same, because a little skepticism in this area is likely to serve you well.