The Exchange, November 2003

Issue 10(4), November 2003

In this issue

Canada's newspapers get failing grade for coverage of new prescription drugs

by Alan Cassels (alan@alancassels.com)

It's hard to miss the explosion of media stories regarding new pharmaceuticals. News about miraculous new drugs seems to be everywhere. But given that medicine can both help and harm, how reliable is the information that Canadians-including patients and healthcare professionals-receive from our country's mainstream newspapers?

Nobody in Canada had ever researched this question before, so no one really knew how well the media-one of the most trusted and valued sources of information about health care-was doing its job.

To remedy this situation, I set out with a team of researchers to systematically examine the accuracy, completeness and balance of reports in major Canadian newspapers on a sample of five new drugs. But before I tell you what we found, take this simple skill-testing question to test your drug-reporting savvy:

Hmm. The "42%" sounds impressive. If I thought I was at risk for those diseases I would probably take the first drug. But the second? A "3.5%" reduction in risk seems pretty insignificant to me for the trouble of swallowing a pill every day for 5 years.

If you answered 'yes' to the first drug and 'no' to the second-as I did when I first saw those numbers-then you've been bamboozled by statistics. That's because both questions refer to the same study of the same cholesterol-lowering drug simvastatin (trade name: Zocor)-the results were just reported to you in two different ways.

The trick comes in knowing the distinction between a relative difference and an absolute difference. You see, in the clinical trial of this drug, 8.5% of people taking a placebo had a heart attack or stroke, compared to 5% of those taking Zocor. That's an absolute difference of 3.5% but a relative difference of 42% (5 is 42% less than 8.5). Guess which number grabs the spotlight? Voila! Welcome to the world of drug reporting.

I wanted to know how often this kind of statistical manipulation happened. Our team looked at all articles published in 24 major English and French newspapers during the year 2000 on five recently launched drugs. We extracted 193 articles that mentioned one or more health effects relating to Celebrex, for symptoms of arthritis; Lipitor, a cholesterol-lowering drug; Evista, for post-menopausal osteoporosis; Tamiflu, for the flu; and Aricept, to treat Alzheimer's disease.

Knowing how much better the drug works than a placebo is a pretty important piece of information. If a drug supposedly prevents heart attacks, strokes, or broken hips, how many of these events did it prevent? We found, for starters, that only one-quarter of the articles even mentioned how much a drug worked-the articles didn't even tell us whether the drug helped 3% of the people who used it, or 60%.

One in three articles presented benefits using relative statistics-the misleading 42%- rather than more accurate absolute numbers. Overall, 68% of the articles did not mention a single side-effect or harm related to the drug, and when these were mentioned, they were usually placed towards the end the article. Contraindications-information on who shouldn't take a drug-were mentioned in only seven of the articles in our sample (4%).

Most surprising, perhaps, was the overwhelming silence about who funded the studies. Although pharmaceutical manufacturers fund most of the pre-marketing studies of new drugs, our study found that more than 90% of the time, this link was not revealed. Paid spokespeople were often quoted, but financial or other ties to manufacturers were mentioned only 3% of the time.

Why does completeness and accuracy in drug reporting matter? Because drugs are an increasingly important part of our healthcare system. Collectively, we Canadians now spend $15 billion per year on pharmaceuticals. A side-effect of misleading media reporting on new drugs is that it can lead to much wider use of a drug than is warranted. Overly rosy reporting may leave people begging their doctors to prescribe the latest new drugs, or pressuring policymakers into covering these drugs even if they are not any better than last year's batch of "wonder drugs".

In an ideal world, new drug information that we get from the media-an important and trusted source of information-should contribute to informed healthcare decisions. Although honest journalists would never willingly participate in disguised advertising, they need to immunize themselves with a higher dose of 'healthy skepticism' about who performed the research, a better understanding of the statistics that are really important when it comes to reporting on new drugs, and more concern for the downside of pharmaceuticals.

Alan Cassels is the lead author of Drugs in the News: How well do Canadian newspapers report the good, the bad and the ugly of new prescription drugs? The study described in this article was published in the Canadian Medical Association Journal (April 29, 2003) and by the Canadian Centre for Policy Alternatives (www.policyalternatives.ca). Journalists and consumers interested in learning what they should know about new prescription drugs will also find a checklist and a list of independent sources of drug information.

Editorial-How you say it defines how you think about it

by Geoff Hart (ghart@videotron.ca)

As scientific communicators, we all accept the importance of using the right words and of using them correctly. Even a "mostly right" word or phrase can be misleading-sometimes remarkably so. To find the right words, we pay close attention to the way professionals in our field write about their subjects and strive to emulate their word choice. That's a good start, but we should try to go well beyond that. Why? Because experts make many assumptions that shape the way they write, and those assumptions are often so much part of the scenery that they're ignored until someone trips over them.

A common example arises when we write about evolution. In attempting to make a complex concept easier to understand by encapsulating it in a metaphor, many science writers inadvertently anthropomorphize a process that has no conscious intent and that occurs beneath the notice of any organism other than an evolutionist. But the problem is subtler than that and more common than you might think. Whenever we use metaphors or state facts, it's easy to forget that both are nothing more than verbal shorthand that facilitates the act of understanding. If we're unaware of the assumptions that underlie a metaphor or fact, it's easy to be led astray by those assumptions. "Survival of the fittest" is a misleading metaphor because it conceals how fitness is contextual rather than absolute. The robustly fit dinosaurs that were once Earth's dominant life form proved far less fit than the organisms that replaced them when the environment changed drastically (most likely due to a meteorite impact).

How about facts? The statement that "the speed of light is constant and cannot be exceeded" seems unassailable until we remember two key assumptions that underlie the statement: both statements are only true of light observed in a given medium (e.g., a vacuum), and both depend on the the wavelength of the light. Light travelling in glass moves significantly slower than light travelling in a vacuum, and the extent of the speed decrease depends on the light's wavelength-which is why prisms work.

The metaphor of survival of the fittest misleads us by suggesting that fitness somehow represents superiority and that unfitness represents a pejorative term (i.e., inferiority). Such notions have led to social Darwinism and even arguably to Nazism. The fact about the supremacy of the speed of light trivialize a complex and fascinating part of physics by reducing it to a simplistic catchphrase. Believing unquestioningly in the stated "facts" would have prevented us from understanding prisms and from creating the laser traps that let us slow down photons for study.

Both the metaphor and the facts I've used as examples also assume the notion of constancy and universality-in the first place, of the environment, and in the second, of a physical constant. In so doing, they mislead us into believing that "laws of nature" apply identically everywhere and every time. The evidence thus far suggests this is a reasonable assumption, but it's one that hasn't yet been tested as well as you might think. For example, recent theorizing about gravity has revealed that some fundamental assumptions about the constancy and universality of gravity may be flawed. Modeling by Moti Milgrom suggests that the force of gravity seems to change at different scales and under different external conditions (e.g., during acceleration). Scientists who have devoted considerable effort to developing a theory of "cold, dark matter" to explain aspects of the rotation of galaxies that could not be explained by gravitation alone may have missed the boat by not questioning their assumption of constancy everywhere and at all scales. If Milgrom is right, a revolution may be brewing in astrophysics. (See Nailing down gravity in the October 2003 issue of Discover for details.)

This is all very esoteric stuff, yet there's a clear implication for the profession of scientific communication. The examples I've given show how relying on catchphrases (such as the constancy of the speed of light) and forgetting our assumptions (such as the assumption that the laws of nature remain constant under all conditions) can lead to enormous efforts to justify our assumptions rather than challenging them and discovering something new. The Discover article about Milgrom quoted Martin Nieto, a physicist at Los Alamos National Laboratory, on this very topic: "A physicist has to keep very clear in his mind what he knows, what he thinks he knows, what he's suspicious of, and what he wants [to believe]." That's good advice for us too. As writers, we must learn to distinguish clearly between what we know and what we think we know, to identify what we're uncertain of, and to recognize that what we'd like to believe can adversely affect what we're trying to say.

Understanding ourselves to this extent is difficult work, but can have tremendous payoffs. Next time you're writing about something a bit too familiar, shake up your complacency by asking yourself a few questions: What assumptions are you making about the subject, and how do those assumptions shape your conclusions? What errors are being introduced by your mistaken beliefs or by wishful thinking? What paradigm changes might you promote by examining your assumptions, being suspicious about your favorite catchphrases, and eliminating wishful thinking?

I freely concede that I don't ask myself these questions nearly often enough. Do you?

October is the cruelest month

by Geoff Hart (ghart@videotron.ca)

Those of us who work with scientists often think that we have it tough. Of course, toughness is always relative, and though the grass often seems greener on the other side of the fence, sometimes it's not always grass. Pace T.S. Eliot, this time I'll propose that October is really the cruelest month-at least in the minds of the editors of Popular Science.

The October issue of the magazine contains an article entitled The Worst Jobs in Science that should make even the most jaundiced scientific communicator grateful for the relatively minor ills of our jobs. The editors selected their top 18 favorites by rating the jobs based on the following criteria: career-track bait and switch, futility, "inspires reflexive ridicule", inspires hatred, involves digestive product, olfactory overload, physical torture, political quagmire, psychological torture, risk of death, risk of disease, risk of physical violence, and the onomatopoeic "zzzzzzzzzz". Each job is accompanied by delightful icons representing the criteria that earned it the dubious honor of appearing in the list. My favorite? Using Barney the dinosaur on the icon representing psychological torture.

Pick up a copy of the magazine, or visit Popular Science online (www.popsci.com/popsci/science/article/0,12543,484153,00.html).

A December treat: "Bogglers"

The visual-verbal puns presented below are shorthand for a variety of well-known scientific or technological phrases. For example, I'll give away the solution to number 4 to get you started, since this one requires the ability to see the puzzle in color (which won't be the case if you're reading a printout of the PDF). This square features the word "Shift", colored red, which translates into "red shift" (one possible Doppler effect). Try your hand at getting the others, and don't feel too upset if you miss a few; some are real stinkers! Answers are provided later on this Web page.

[This puzzle appeared in the July 2003 issue of Discover magazine (www.discover.com). Reprinted with the kind permission of the publisher.]

Online updates and resources

Compiled by Geoff Hart (ghart@videotron.ca)

CBE updates and errata

Some people have remarked that the style guide many scientific editors try to follow (Scientific Style and Format: The CBE Manual for Authors, Editors, and Publishers. Sixth Edition) has many typos and various other forms of error. I haven't checked this myself, but a partial list of corrections is now available online (www.councilscienceeditors.org/publications/corrections6th.cfm).

While you're visiting, have a look at the update information for the upcoming 7th edition (www.councilscienceeditors.org/publications/ssf_7th.cfm).

Jobs for medical communicators

Medzilla (www.medzilla.com) lists full-time positions for medical writers and editors.

Alerts for Academic Conferences Worldwide

Wondering what's going on in the academic world? Check out Conference Alerts (www.conferencealerts.com). This site provides information on academic conferences around the world and e-mailed updates about the conferences.

The site links to a variety of conference categories, including statistics, physics, and public health. Choose a category and you'll see a listing of events, by date. Each entry generally contains the event's date, location, and title. Clicking on an event provides varying amounts of additional information, including contact information for the organizer and the call for papers.

You can also subscribe to receive updates about conferences. The privacy policy seems reasonable. Worth a look if you're interested in a particular science or looking for networking opportunities.

[Editor's note: The conference resource was provided by a member of the Scientific Communication SIG, but shoddy record-keeping on my part prevents me from providing the correct attribution. Please contact me if you want to be acknowledged for this contribution in the next issue.--Geoff]

Answers to "Bogglers"

1. half life
2. cross-contamination
3. archeology
4. red shift
5. microscope
6. alternating current
7. parallel universes
8. reverse engineering
9. right angle
10. angular momentum
11. vanishing point
12. inverse-square law

Parting thoughts

"Anecdotes do not make a science. Ten anecdotes are no better than one, and a hundred anecdotes no better than ten."—Frank J. Sulloway

"Croire tout découvert est une erreur profonde : c'est prendre l'horizon pour les bornes du monde." (Believing that everything has been discovered is a profound error: it mistakes the horizon for the limits of the universe.)—A. Lemière, translated by Geoff Hart

"The thinking of today's roboticists, like that of their predecessors in early AI efforts, is infected by their vision of the computer itself, the machine as model human. Again, they mistake the tool for its builder. In particular, the error comes from mistaking the current methods of software-writing as a paradigm for human mental organization. In the 1970s, a computer program was a centralized monolithic thing, a small world unto itself, a set of instructions operating upon a set of data. It should be no surprise, then, that researchers at the time saw human intelligence as a... centralized monolithic, logical mind operating upon the data in a 'knowledge base'. By the 1990s that monolithic paradigm of programming had been replaced by something called 'object oriented' methods, in which code was written in discrete, atomic chunks that could be combined in a variety of ways. And—what do you know?—human sentience is now seen as something emerging from the complex interaction of... discrete, atomic chunks. Is cognitive science driving the science of computing, or is it the other way around?"—Ellen Ullman, Programming the post-human

"Many laypeople mistakenly think that science is about discovering the truth about how the universe works. That's not really accurate, because the truth is often unattainable. Much of the time, scientists hope simply to improve our understanding of a small part of the universe."—Keith Devlin, The Wolfram Controversy

"You can never solve a problem on the level on which it was created."—Albert Einstein, physicist, Nobel laureate (1879–1955)

"I do not know what I may appear to the world; but to myself I seem to have been only like a boy playing on the seashore, and diverting myself now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me."—Sir Isaac Newton

"Problems worthy of attack prove their worth by hitting back."—Piet Hein, poet and scientist (1905–1996)

"No, no, you're not thinking, you're just being logical."—Niels Bohr, physicist (1885–1962)

Contact and copyright information

The Exchange is published on behalf of the Scientific Communication special interest group of the Society for Technical Communication. Material in the Exchange can be reprinted without permission if credit is given to the author and a copy of the reprint is sent to the editor. Please send comments, letters, and articles to the editor.

Editor, publisher, and SIG manager:

Geoff Hart, c/o FERIC, 580 boul. St-Jean, Pointe-Claire, Québec, H9R 3J9 Canada (ghart@videotron.ca)

Webmaster:

Geoff Hart, c/o FERIC, 580 boul. St-Jean, Pointe-Claire, Québec, H9R 3J9 Canada (ghart@videotron.ca)

© 2003, Society for Technical Communication (901 North Stuart St., Suite 904, Arlington, Virginia 22203-1822 U.S.A., 703-522-4114, 703-522-2075 fax, www.stc.org)