Avoiding biased treatment comparisons

Format: Texts

,

Language/s: English

,

Target Audience: Self-directed learning

|

Undergraduate

|

Short Description:

Biases in tests of treatments are those factors that can lead to conclusions that are systematically different from the truth.

Key Concepts addressed:

• 1-12 Dramatic treatment effects are rare
• 1-7 Beware of conflicting interests
• 2-1 Comparisons are needed to identify treatment effects

Details

What are biases? Biases in tests of treatments are those influences and factors that can lead to conclusions about treatment effects that are systematically different from the truth.

Sometimes treatments have dramatic effects (click here to list relevant records). These may be unintended and specific, for example, when a person has an allergic reaction to an antibiotic drug. Treatments can also have striking beneficial effects, like adrenaline for life-threatening allergic reactions (McLean-Tooke et al. 2003). Such striking effects are rare, however. Usually, treatment effects are more modest, but nevertheless well worth knowing about, for example, using aspirin to reduce risk of heart attack (Elwood 2004).

Aspirin doesn’t prevent all premature deaths after a heart attack, but it does reduce the likelihood of death by about twenty per cent, which is important in such a common condition. If these moderate but important effects of most treatments are to be detected reliably, care must be taken to ensure that biased comparisons don’t lead us to believe that treatments are useful when they are useless or harmful, or useless when they can actually be helpful.

Biases in tests of treatment are those influences and factors that can lead to conclusions about treatment effects that are systematically different from the truth. Although many kinds of biases can distort the results of health research (Sackett 1979), we have considered design bias, allocation bias, co-intervention bias, observer bias, analysis bias, biases in assessing unanticipated effects,reporting bias, biases in systematic reviews, and research biases and fraud.

Usually, the unfair tests of treatment resulting from these biases are not recognised for what they are. However, people with vested interests sometimes exploit these biases so that treatments are presented as if they are better than they really are (Sackett and Oxman 2003).

Whether biases are inadvertent or deliberate, the consequences are the same: unless tests of treatment are fair, some useless or harmful treatments will seem to be useful, while some useful treatments will seem useless or harmful.

The text in these essays may be copied and used for non-commercial purposes on condition that explicit acknowledgement is made to The James Lind Library (www.jameslindlibrary.org).

References

Elwood P (2004). The first randomised trial of aspirin for heart attack and the advent of systematic overviews of trials. The James Lind Library (www.jameslindlibrary.org).

McLean-Tooke APC, Bethune CA, Fay AC, Spickett GP (2003). Adrenaline in the treatment of anaphylaxis: what is the evidence? BMJ 327:1332-1335.

Sackett DL (1979). Bias in analytic research. Journal of Chronic Diseases 32:51-63.

Sackett DL, Oxman AD (2003). HARLOT plc: an amalgamation of the world’s two oldest professions. BMJ 2003;327:1442-1445.

Read more about the evolution of fair comparisons in the James Lind Library.