![]()
Part I
Learning the Basics
![]()
A Guide to Part I of News & Numbers
In the first five chapters, we cover the basics:
1. Where We Can Do Better
Improving how stories with numbers are reported.
2. The Certainty of Uncertainty
Scientists are always changing their minds.
3. Testing the Evidence
Thinking clearly about scientific studies.
4. What Makes a Good Study?
Separating the wheat from the chaff.
5. Your Questions and Peer Review
What to ask the experts.
![]()
1
Where We Can Do Better
Almost everyone has heard that âfigures donât lie, but liars can figure.â We need statistics, but liars give them a bad name, so to be able to tell the liars from the statisticians is crucial.
Dr. Robert Hook
A team of medical researchers reports that it has developed a promising, even exciting, new treatment. Is the claim justified, or could there be some other explanation for their patientsâ improvement? Are there too few patients to justify any claim?
An environmentalist says that a certain toxic waste will cause many cases of cancer. An industry spokesman denies it. What research has been done? What are the numbers? How valid are they?
We watch the numbers fly in debates ranging from pupil-testing to global warming to the cost of health insurance reforms, and from influenza threats to shocking events such as the catastrophic Deepwater Horizon oil spill in the Gulf of Mexico in April 2010.
Even when we journalists say that we are dealing in facts and ideas, much of what we report is based on numbers. Politics comes down to votes. Dollar-figures dominate business and government news â and stir hot-button issues such as sports stadium proposals. Numbers are at the heart of crime rates, nutritional advice, unemployment reports, weather forecasts, and much more.
But numbers offered by experts sometimes conflict, triggering confusion and controversies. Statistics are used or misused even by people who tell us, âI donât believe in statistics,â then claim that all of us, or most people, or many do such and such. We should not merely repeat such numbers, but interpret them to deliver the best possible picture of reality.
And the really good news: We can do this without any heavy-lifting math. We do need to learn how the best statisticians â the best figurers â think. They can show us how to detect possible biases and bugaboos in numbers. And they can teach us how to consider alternate explanations, so that we wonât be stuck with the obvious when the obvious is wrong.
Clear thinking is more important than any figuring. Thereâs only one math equation in this book: 1 = 200,000. This is our light-hearted way of expressing how one person in a poll can represent the views of up to 200,000 Americans. That is, when the poll is done right. The chapter on polling tells you how to know when things go wrong.
Although News & Numbers is written primarily to aid journalists in ferreting meaning out of numbers, this book can help anyone answer three questions about all sorts of studies and statistical claims:
What can I believe? What does it mean? How can I explain it to others?
The Journalistic Challenges
The very way in which we journalists tell our readers and viewers about a medical, environmental, or other controversy can affect the outcome.
If we ignore a bad situation, the public may suffer. If we write âdanger,â the public may quake. If we write âno danger,â the public may be falsely reassured.
If we paint an experimental medical treatment too brightly, the public is given false hope. If we are overly critical of some drug that lots of people take, people may avoid a treatment that could help them, maybe even save their lives.
Simply using your noggin when you view the numbers can help you travel the middle road.
And whether we journalists will it or not, we have in effect become part of the regulatory apparatus. Dr. Peter Montague at Princeton University tells us: âThe environmental and toxic situation is so complex, we canât possibly have enough officials to monitor it. Reporters help officials decide where to focus their activity.â
And when to kick some responses into high gear. During the early days after the Deepwater Horizon oil well explosion, both company (BP) and federal officials tended to soft-pedal and underestimate the extent of the problem. Aggressive reporters found experts who sharply hiked the estimates of how much oil was pouring into the Gulf. These journalists provided running tallies of the miles of shorelines where gooey globs were coming ashore, the numbers of imperiled birds and wildlife, and the number of cleanup workers feeling ill effects. Nightly news broadcasts showed graphic video of oil gushing out of the ground a mile under the Gulf. National attention was soon galvanized on the crisis; both government and industry action intensified.
Five Areas for Improvement
As we reporters seek to make page one or the six oâclock news:
1. We sometimes overstate and oversimplify. We may report, âA study showed that black is white,â when a study merely suggested there was some evidence that such might be the case. We may slight or omit the fact that a scientist calls a result âpreliminary,â rather than saying that it offers strong and convincing evidence.
Dr. Thomas Vogt, at the Kaiser Permanente Center for Health Research, tells of seeing the headline âHeart Attacks from Lack of âCâ â and then, two months later, âPeople Who Take Vitamin C Increase Their Chances of a Heart Attack.â1 Both stories were based on limited, far-from-conclusive animal studies.
Philip Meyer, veteran reporter and author of Precision Journalism, writes, âJournalists who misinterpret statistical data usually tend to err in the direction of over-interpretation ⌠The reason for this professional bias is self-evident; you usually canât write a snappy lead upholding [the negative]. A story purporting to show that apple pie makes you sterile is more interesting than one that says there is no evidence that apple pie changes your life.â2
Weâve joked that there are only two types of health news stories â New Hope and No Hope. In truth, we must remember that the truth usually lies somewhere in the middle.
2. We work fast, sometimes too fast, with severe limits on the space or airtime we may fill. We find it hard to tell editors or news directors, âI havenât had enough time. I donât have the story yet.â Even a long-term project or special may be hurriedly done. In a newsroom, âlong termâ may mean a few weeks.
A major Southern newspaper had to print a front-page retraction after a series of stories alleged that people who worked at or lived near a plutonium plant suffered in excess numbers from a blood disease. âOur reporters obviously had confused statistics and scientific data,â the editor admitted. âWe did not ask enough questions.â3
3. We too often omit needed cautions and perspective. We tend to rely too much on âauthoritiesâ who are either most quotable or quickly available or both. They may get carried away with their own sketchy, unconfirmed but âexcitingâ data â or have big axes to grind, however lofty their motives. The cautious, unbiased scientist who says, âOur results are inconclusiveâ or âWe donât have enough data yet to make any strong statementâ or âI donât knowâ tends to be omitted or buried deep down in the story.
Some scientists who overstate their results deserve part of the blame. But bad science is no excuse for bad journalism.
We may write too glowingly about some experimental drug to treat a perilous disease, without needed perspective about what hurdles lie ahead. We may over-worry our readers about preliminary evidence of a possible new carcinogen â yet not write often enough about what the U.S. surgeon general calls the ânow undisputableâ evidence that secondhand tobacco smoke âis a serious health hazard.â4
4. Seeking balance in our reporting on controversial issues, we sometimes forget to emphasize where the scientific evidence points.
Study after study has found no evidence that childhood immunizations can cause autism â yet lay promoters (and some doctors) continue to garner ink and airtime on a popular daytime TV show (see Chapter 12).
On the hot-button issue of âglobal warming,â we must not get carried away by occasional super-cold winters. Year-to-year temperatures vary by their very nature. Climate experts had to study decades of weather and interpret data going back thousands of years to detect the slow, yet potentially dangerous, warming of our planet. The bottom line: Most scientists now agree that global warming is real, and is linked to the burning of fossil fuels. The scientific and societal debate continues over details such as how urgent the threat might be â and precisely what to do about it. Another bottom line: Donât write the nay-sayers off as kooks and tools of industry. Sometimes even very minority views turn out to be right (see Chapter 9).
5. We are influenced by intense competition and other pressures to tell the story first and tell it most dramatically. One reporter said, âThe fact is, you are going for the strong [lead and story]. And, while not patently absurd, it may not be the lead you would go for a year later.â5
Or even a few hours later. Witness the competitive rush to declare election-night winners, and the mistakes that sometimes result.
We are also subject to human hope and human fear. A new âcureâ comes along, and we want to believe it. A new alarm is sounded, and we too tremble â and may overstate the risk. Dr. H. Jack Geiger, a respected former science writer who became a professor of medicine, says:
I know I wrote stories in which I explained or interpreted the results wrongly. I wrote stories that didnât have the disclaimers I should have written. I wrote stories under competitive pressure, when it became clear later that I shouldnât have written them. I wrote stories when I hadnât asked â because I didnât know enough to ask â âWas your study capable of getting the answers you wanted? Could it be interpreted to say something else? Did you take into account possible confounding factors?â
How can we learn to do better? How do we separate the wheat from the chaff in all ...
