How many epidemiologists does it take to screw up a health policy?

In keeping with the Daily Duck's continuing attempt to throw the cold water of common sense on the best laid plans of experts and technocrats, I bring you this article by Gary Taubes on the sorry and dangerous state of that pseudo-scientific endeavor known as epidemiology:

At the center of the H.R.T. story is the science of epidemiology itself and, in particular, a kind of study known as a prospective or cohort study, of which the Nurses’ Health Study is among the most renowned. In these studies, the investigators monitor disease rates and lifestyle factors (diet, physical activity, prescription drug use, exposure to pollutants, etc.) in or between large populations (the 122,000 nurses of the Nurses’ study, for example). They then try to infer conclusions — i.e., hypotheses — about what caused the disease variations observed. Because these studies can generate an enormous number of speculations about the causes or prevention of chronic diseases, they provide the fodder for much of the health news that appears in the media — from the potential benefits of fish oil, fruits and vegetables to the supposed dangers of sedentary lives, trans fats and electromagnetic fields. Because these studies often provide the only available evidence outside the laboratory on critical issues of our well-being, they have come to play a significant role in generating public-health recommendations as well.

The dangerous game being played here, as David Sackett, a retired Oxford University epidemiologist, has observed, is in the presumption of preventive medicine. The goal of the endeavor is to tell those of us who are otherwise in fine health how to remain healthy longer. But this advice comes with the expectation that any prescription given — whether diet or drug or a change in lifestyle — will indeed prevent disease rather than be the agent of our disability or untimely death. With that presumption, how unambiguous does the evidence have to be before any advice is offered?

The catch with observational studies like the Nurses’ Health Study, no matter how well designed and how many tens of thousands of subjects they might include, is that they have a fundamental limitation. They can distinguish associations between two events — that women who take H.R.T. have less heart disease, for instance, than women who don’t. But they cannot inherently determine causation — the conclusion that one event causes the other; that H.R.T. protects against heart disease. As a result, observational studies can only provide what researchers call hypothesis-generating evidence — what a defense attorney would call circumstantial evidence.

Testing these hypotheses in any definitive way requires a randomized-controlled trial — an experiment, not an observational study — and these clinical trials typically provide the flop to the flip-flop rhythm of medical wisdom. Until August 1998, the faith that H.R.T. prevented heart disease was based primarily on observational evidence, from the Nurses’ Health Study most prominently. Since then, the conventional wisdom has been based on clinical trials — first HERS, which tested H.R.T. against a placebo in 2,700 women with heart disease, and then the Women’s Health Initiative, which tested the therapy against a placebo in 16,500 healthy women. When the Women’s Health Initiative concluded in 2002 that H.R.T. caused far more harm than good, the lesson to be learned, wrote Sackett in The Canadian Medical Association Journal, was about the “disastrous inadequacy of lesser evidence” for shaping medical and public-health policy.

As I argued in Fast Facts Nation, public health policy is more often motivated by preconceived and politically motivated biases than by scientifically proven facts. Epidemiological studies that are not backed up by conclusive clinical trials are worse than useless - they are often responsible for ill-conceived public health initiatives that cause more harm than good.

We need to rethink the role of government as well as the media in the formulation of health policy. The government too often is driven by political motivations to show some benefit to society in the form of policies and programs. The media is likewise driven by self-serving motivations to appear relevant in the lives of the public. Both are too impatient to wait for scientifically proven consensus to appear before declaring for a given policy. But oftentimes the motivation to help others actually acheives the opposite result.