Wednesday, September 19, 2007

How many epidemiologists does it take to screw up a health policy?

In keeping with the Daily Duck's continuing attempt to throw the cold water of common sense on the best laid plans of experts and technocrats, I bring you this article by Gary Taubes on the sorry and dangerous state of that pseudo-scientific endeavor known as epidemiology:
At the center of the H.R.T. story is the science of epidemiology itself and, in particular, a kind of study known as a prospective or cohort study, of which the Nurses’ Health Study is among the most renowned. In these studies, the investigators monitor disease rates and lifestyle factors (diet, physical activity, prescription drug use, exposure to pollutants, etc.) in or between large populations (the 122,000 nurses of the Nurses’ study, for example). They then try to infer conclusions — i.e., hypotheses — about what caused the disease variations observed. Because these studies can generate an enormous number of speculations about the causes or prevention of chronic diseases, they provide the fodder for much of the health news that appears in the media — from the potential benefits of fish oil, fruits and vegetables to the supposed dangers of sedentary lives, trans fats and electromagnetic fields. Because these studies often provide the only available evidence outside the laboratory on critical issues of our well-being, they have come to play a significant role in generating public-health recommendations as well.

The dangerous game being played here, as David Sackett, a retired Oxford University epidemiologist, has observed, is in the presumption of preventive medicine. The goal of the endeavor is to tell those of us who are otherwise in fine health how to remain healthy longer. But this advice comes with the expectation that any prescription given — whether diet or drug or a change in lifestyle — will indeed prevent disease rather than be the agent of our disability or untimely death. With that presumption, how unambiguous does the evidence have to be before any advice is offered?

The catch with observational studies like the Nurses’ Health Study, no matter how well designed and how many tens of thousands of subjects they might include, is that they have a fundamental limitation. They can distinguish associations between two events — that women who take H.R.T. have less heart disease, for instance, than women who don’t. But they cannot inherently determine causation — the conclusion that one event causes the other; that H.R.T. protects against heart disease. As a result, observational studies can only provide what researchers call hypothesis-generating evidence — what a defense attorney would call circumstantial evidence.

Testing these hypotheses in any definitive way requires a randomized-controlled trial — an experiment, not an observational study — and these clinical trials typically provide the flop to the flip-flop rhythm of medical wisdom. Until August 1998, the faith that H.R.T. prevented heart disease was based primarily on observational evidence, from the Nurses’ Health Study most prominently. Since then, the conventional wisdom has been based on clinical trials — first HERS, which tested H.R.T. against a placebo in 2,700 women with heart disease, and then the Women’s Health Initiative, which tested the therapy against a placebo in 16,500 healthy women. When the Women’s Health Initiative concluded in 2002 that H.R.T. caused far more harm than good, the lesson to be learned, wrote Sackett in The Canadian Medical Association Journal, was about the “disastrous inadequacy of lesser evidence” for shaping medical and public-health policy.


As I argued in Fast Facts Nation, public health policy is more often motivated by preconceived and politically motivated biases than by scientifically proven facts. Epidemiological studies that are not backed up by conclusive clinical trials are worse than useless - they are often responsible for ill-conceived public health initiatives that cause more harm than good.

We need to rethink the role of government as well as the media in the formulation of health policy. The government too often is driven by political motivations to show some benefit to society in the form of policies and programs. The media is likewise driven by self-serving motivations to appear relevant in the lives of the public. Both are too impatient to wait for scientifically proven consensus to appear before declaring for a given policy. But oftentimes the motivation to help others actually acheives the opposite result.

12 Comments:

Blogger Harry Eagar said...

I got as far as danger of electromagnetic fields. Then I quit.

That wasn't an epidemiological study. Its authors may have purported to be, but they weren't epidemiologists.

September 19, 2007 8:32 PM  
Blogger monix said...

Harry is right, this is a misuse of the term epidemiology, which is not a pseudo-science. My late father-in-law was a respected epidemiologist who spent the years between 1946 and 1960 working in what was then Southern Rhodesia. In that time they were able to virtually eradicate small pox, greatly reduce infant mortality rates and generally raise standards of hygiene and health in the bush villages. The causes of disease and how they are passed on seem obvious to us now and we take vaccination for granted, but that knowledge came from sound epidemiological studies. Perhaps we need a new term for media-interpreted studies?

September 20, 2007 3:22 AM  
Blogger Duck said...

Monix,

OK, maybe my language was a little strong. But from how epidemiology is described in this article it is mainly about establishing statistical correlations between populations of diseased individuals and behavioral, environmental or demographic factors. And as we all know, correlation is not causation.

So epidemiology is a thesis generating activity, which produces theories of possible causative factors which clinical studies should follow up with to confirm/deny the causality. However, as with the nurses study, oftentimes the medical profession will promote treatments for patients before that confirmation has been established. That, in my opinion, is pseudoscience. Or malpractice.

But I understand that clinical studies are long and expensive and not always conclusive, and certain health crises demand action before such scientific certainty can be established. I think that relying on epidemiological studies do deal with true epidemics of communicable diseases has been very successful, as you mentioned with small-pox. But it is being used to try to identify treatments for non-epidemic, non-communicable diseases and conditions like heart disease in women. People are encouraged to use treatments which may carry their own negative side effect, and which have not been clinically proven to be safe, because of unproven, correlative data.

September 20, 2007 6:55 AM  
Blogger Bret said...

Duck wrote: "But I understand that clinical studies are long and expensive..."

I think that overseeing clinical trials should be taken away from the FDA and privatized. Then they might be less expensive and long.

September 20, 2007 9:07 AM  
Blogger Oroborous said...

Probably less expensive, but not less long.

There are exceptional cases, but by and large, it takes years to determine if a given drug, nutrient, or behavior is effective and safe. In the future we may be able to short-cut that with computer modeling, but not yet.

And obviously drugs such as fen-phen and Vioxx should have been studied LONGER, not shorter.

September 20, 2007 9:23 AM  
Blogger erp said...

There's very little that can't be improved by taking it out of the hands of the guvmint.

September 20, 2007 9:59 AM  
Blogger Bret said...

oroborous wrote: "And obviously drugs such as fen-phen and Vioxx should have been studied LONGER, not shorter."

Maybe, but not obvious to me. For the relatively few people who died or whose health was damaged by those two drugs, one has to weigh those who die or whose health is damaged waiting for new therapies to be approved.

Hipocrates said "first, do no harm" but that was easy for him to say because he wasn't dying of terminal cancer.

September 20, 2007 10:33 AM  
Blogger erp said...

I can't speak about Vioxx, but taking Bextra off the market for no good reason put me back to a place from which I can't recover. A Bextra a couple of times a week and an Advil once or twice a month kept me in good shape. Riding my bike, walking miles, going to aerobics classes and moving around almost normally.

When they were no longer available, I couldn't move well because of the pain, so I spent a lot of time in a recliner. Lack of movement probably caused the blood blot that nearly killed me and now I'm on a lifetime of blood thinners.

To all you do-gooders -- thanks so much. The drug companies will recover from your machinations and the U.S. will weather your evil intentions to cripple the economy, but We, the People who were benefiting from those drugs may well end of as cripples, but what the hey, your consciences are crystal clear.

September 20, 2007 12:18 PM  
Blogger Oroborous said...

...one has to weigh those who die or whose health is damaged waiting for new therapies to be approved.

Sure, which is why the FDA allows terminal people to be enrolled in experimental studies, and there's an expedited process for approving potential life-savers.

But neither fen-phen nor Vioxx were intended to treat fatal conditions, which rather renders your argument moot. Given that they both KILLED PEOPLE, the score is:

Lives saved - 0
Lives taken - 100s

And it was known before the drugs were approved that no lives would be saved. They were quality-of-life drugs.

...taking Bextra off the market for no good reason...

While I empathize with your frustration, Bextra appeared to double the risk of heart attack and stroke.

The relatives of those who died from taking Bextra would probably say that it was introduced to the market for no good reason.

Further, Bextra can cause deep vein thrombosis and pulmonary embolism, so you might have ended up where you are now anyhow.

September 20, 2007 4:40 PM  
Blogger erp said...

My information was that the people who allegedly died from Bextra were already at risk and those like myself who were healthy otherwise, had little to fear. It's a moot point now since like many others, there's no going back.

September 20, 2007 7:36 PM  
Blogger Harry Eagar said...

How 'bout Bendectin?

Free market forces did that one in.

Now, if you'd wanted to talk about dumb reporters who don't understand risk factors or statistics, I'd have been right on board.

In that case, shooting the messenger is exactly the right prescription.

September 20, 2007 8:40 PM  
Blogger Bret said...

oroborous wrote: "
Lives saved - 0
Lives taken - 100s
"

You left out half the score. Here's the other half:

Quality of life enhanced: tens of thousands
Quality of life reduced: 100s

Being alive and miserable is very little better (if at all) than being dead.

Also, at least with Vioxx, probably lives saved was greater than zero since some of those took other medications which do, in rare circumstances, also kill people.

September 21, 2007 8:39 AM  

Post a Comment

Links to this post:

Create a Link

<< Home