Roanoke Times
Copyright (c) 1995, Landmark Communications, Inc.
DATE: SUNDAY, June 25, 1995 TAG: 9507030113
SECTION: HORIZON PAGE: F1 EDITION: METRO
SOURCE: JEFFREY KLUGER DISCOVER MAGAZINE
DATELINE: LENGTH: Long
In the gooey world of high-tar cigarettes, Larks were a virtual parking lot. Recognizing their product's status as something less than a health food, the Lark manufacturers sought to capitalize on that shortcoming, providing proof-of-purchase seals that would allow smokers to send away for their own pulmonary embolism after their very first pack.
Given this, you can imagine my surprise when I discovered that despite all the studies linking smoking to disease, ``eminent doctors and research scientists have questioned the claimed significance of these experiments.''
Better still, some cigarettes, it turns out, can even ``protect the delicate tissues in your throat''!
Oh, and there's one other thing I found out - the study was sponsored by the Tobacco Industry Research Committee.
Cigarettes aren't the only products whose reputations have been unexpectedly redeemed. The worlds of commerce and science are rife with this scarcely objective research. There are oat-bran studies sponsored by Quaker Oats, caffeine studies sponsored by coffee manufacturers and nutrition studies sponsored by the food industry.
Now the problem of biased statistics is at last being addressed, thanks to Cynthia Crossen, an editor and writer at the Wall Street Journal and author of the book ``Tainted Truth: The Manipulation of Fact in America'' (Simon & Schuster, 1994).
The book is one of the most comprehensive looks at the national statistics game, revealing the way numbers are increasingly becoming something we can't count on.
``In the last few decades,'' Crossen says, ``we have become steadily inundated with supposedly quantitative information that simply doesn't add up. If the public is going to be truly informed, it needs to know how the studies are conducted and, most important, who's paying for them all in the first place.''
The history of marketing and public relations is a murky one, and from the earliest days of human mercantilism, consumers have been cautioned to take nothing at face value.
``One of the first things we learn when we become consumers,'' Crossen says, ``is that we shouldn't believe what we read or hear about a product. ... There's a tremendous amount of questionable information used in the business of selling, and courts will generally tolerate almost anything, barring extreme and outrageous lies.''
Among the products people trust the least are politicians, and among the ways politicians most earn that distrust are polls. It was in 1824 that public polling became popular in the United States, when it was used to help predict the outcome of the presidential election between Andrew Jackson and John Quincy Adams.
Lacking computers, complex statistical formulas and even Cokie Roberts, early pollsters were forced to rely on informal methods at best, conducting most of their surveys by placing a box in a public square and inviting voters to deposit a slip of paper with the name of their favorite candidate penciled in.
In a frontier culture that was only marginally literate, such techniques yielded somewhat dubious results, usually predicting that any presidential nominees would lose to a large smiley face by a margin of almost six to one.
More sophisticated surveys would have to wait until well into the next century, when pollsters abandoned the straightforward question ``Who are you voting for?'' in favor of more elaborate and more numerous questions.
Though such methods were intended to increase the accuracy of the surveys, as it turned out they introduced a variable that in many cases decreased it.
``One of the things pollsters quickly found,'' Crossen says, ``is that the thing that most influences the answers you get in a poll is the way in which you frame the questions.''
And the same is true today. During the 1992 presidential election, Ross Perot published a questionnaire in TV Guide in which he sought to measure the popularity of the positions he had taken on the issues.
Cynics, of course, would have expected a list like this to include such questions as ``Do you believe a pack of lemurs tried to disrupt my daughter's wedding?''
Perot, however, played it straight - sort of. Among the questions he asked voters was the seemingly direct ``Should the president have the Line Item Veto to eliminate waste?''
To Perot's delight, 97 percent of respondents answered yes, a ringing endorsement of one of his central beliefs.
As Crossen and others observed, however, the last three words in the question left it utterly slanted, in effect asking respondents to vote for or against government profligacy.
When the question was rewritten to a more neutral ``Should the president have the Line Item Veto, or not?'' and asked of a random sample, the total dropped to just 57 percent.
Even seemingly synonymous words in a poll question can elicit very different answers.
Respondents will have different reactions, for example, when a pollster asks about taxes as opposed to revenue enhancers; welfare as opposed to public assistance; NASA as opposed to the Flying Wallendas.
People with a political agenda to push are not the only ones who manipulate polls, of course. People with products to push do, too.
The history of advertising is rich with enthusiastic consumer endorsements based on polls that are conducted questionably at best.
One of the most obvious examples of felicitous phrasing in a product poll came from the makers of Black Flag pesticides.
In a recent survey, Black Flag's manufacturers found that fully 79 percent of all Americans believed that using a roach disk would be an effective way to control this pest.
The problem was, when most of the subjects were first approached, they admitted to never having heard of a roach disk, so it was left to the pollsters to describe one to them. The language they chose was less than objective.
``A roach disk is a type of product that poisons a roach slowly,'' the explanation ran. ``The dying roach returns to the nest and after it dies is eaten by other roaches. In turn, these roaches become poisoned and die. How effective do you think this product would be in killing roaches?''
Though the Black Flag promoters reportedly decided against additional phrasing in which they promised that the few roaches that did survive would be happy to vacuum the house, pick up the dry cleaning and take the kids to soccer practice, the glowing product description they did offer virtually guaranteed a favorable response.
More troubling than marketers who manufacture polling data are marketers who influence scientific data. Among the most notable industries to turn to science for help are the makers of disposable diapers.
As nonbiodegradable waste goes, disposable diapers have a half-life approaching cobalt's. A newborn can go through up to 10 diapers a day, giving the average baby an environmental impact rivaling that of Three Mile Island.
In 1988 the cloth-diaper industry seized on these problems, questioning whether the nation's landfills could tolerate such a potentially nonbiodegradable load and sponsoring research designed to assess the diaper threat.
Not surprisingly, the study came to the conclusion that the United States was facing something approaching a national nappy avalanche.
The solution? Cloth diapers.
In 1990, Procter & Gamble - the manufacturer of both Pampers and Luvs disposable brands - hit back, hiring its own scientific consulting firm to conduct its own research.
These studies found, to no one's surprise, that once you took into consideration the compostability of paper and the waste generated by manufacturing and repeatedly cleaning a single cloth diaper, the disposable brands were actually the more environmentally friendly of the two.
The study stopped short of recommending that the labels ``low fat,'' ``lemon fresh'' or ``absolutely no cholesterol'' be added to Pampers packaging, but the message of the research was nonetheless communicated.
What made both of these studies disturbing was that neither side disputed the essential facts of the controversy - the number of disposable diapers discarded each year; the number of washings cloth diapers require - but both drew precisely opposite conclusions from them.
``Both sides,'' Crossen says, ``relied on what appeared to be objective data but used it subjectively.''
In short, she suggests, it was the conclusions the researchers wanted to reach that led them to the conclusions they did reach.
Crossen believes research of this kind does a lot of harm to the national data pool - and that the trend won't be easily reversed.
``I think that people respect science possibly more than they should,'' she says, ``mostly because they don't understand it.''
So next time you come across some remarkable research compiled by a company, better say to yourself, ``Where'd they get those numbers? And who did the figuring?''
by CNB