Home > Read or listen to the book > About > Why do we need fair tests of treatments?

Why do we need fair tests of treatments?

Modern medicine has been hugely successful at reducing the impact of disease and increasing life expectancy. In spite of this, too much medical decision making is based on insufficient evidence. As a result, doctors and other health professionals have sometimes harmed patients instead of helping them. It is essential that our decisions about what treatments to use are based on fair tests of their effects.


Sometimes we get better anyway

James VI

Jamie the Saxt: sorting out myth from reality in the tobacco debate since 1616

But this can lead us to mistakenly attribute our natural recovery to whatever treatment we happened to be on at the time.

James VI of Scotland understood this. In his “Counterblaste to tobacco” [1], he delivered a far-sighted rebuke to those who claimed that tobacco smoking could cure all manner of ailments, including the common cold.

In this argument, there is first a great mistaking, and next a monstrous absurdity…  Because peradventure when a sick man hath his disease at its height, he hath at that instant taken Tobacco, and afterwards his disease having taken the natural course of decline and consequently the Patient recovering his health, O then the Tobacco forsooth, was the worker of that miracle!  Read more

It’s therefore essential that all treatments are subject to fair tests that compare them to alternatives. 

Read more.

Just because a treatment is new doesn’t mean it’s better

Recent medical history is littered with examples of “exciting” new treatments that caused substantial harm to patients because they hadn’t been properly tested.

Thalidomide for nausea in pregnancyVioxx for arthritis painAvandia for type 2 diabetes and certain types of mechanical heart valve are all examples of treatments that harmed patients due to inadequate testing.

Read more.

Untested theory or flawed evidence can be harmful

Often, the benefits we hope to see in treatments fail to materialize.  This can happen if we follow advice from untested theories, anecdotal evidence or evidence from flawed tests.

Examples cited in Testing Treatments include:  Dr Spock’s advice on babies sleeping position and Hormone Replacement Therapy.

Read more.

More is not always better

Just because a treatment works doesn’t mean that more of it is always better. Often, benefits can reach a plateau whilst adverse effects mount up.

We look at history of intensive treatment for breast cancer, and bone marrow transplantation.

Read more in “More is not necessarily better”.

Earlier is not always better

Earlier diagnosis doesn’t always lead to better outcomes. Sometimes it makes matters worse by leading to unnecessary treatment or simply extends disease labelling.

We explore the issues in neuroblastomaprostate cancerbreast cancer, and genetic testing.

Read more in “Earlier is not necessarily better”.

Sometimes we just don’t know

Often, we need fair tests because we simply don’t know which treatment is best.  There may not be a dramatic difference between one treatment and another. Practitioners may disagree about how to help people with health problems.

In these circumstances, it is very important to have fair tests because a moderate benefit or harm can have a huge impact when scaled up to whole populations.

Explore the examples of antibiotics in pre-term labour and some treatments for breast cancer.

Read more about dealing with uncertainty about the effects of treatments.


  • Stuart, James, King of Great Britaine, France and Ireland (1616). A counterblaste to tobacco. In: The workes of the most high and mightie prince, James Published by James, Bishop of Winton, and Deane of his Majesties Chappel Royall. London: printed by Robert Barker and John Bill, printers to the Kings most excellent Majestie, pp 214-222.

Next: What are fair tests of treatments?

  • Bettina Ryll

    And sometimes, new treatments are so much better than old ones- all novel therapies in Melanoma have been checked against DTIC while Melanoma has been reported to be chemo-resistant for eons – that we keep running trials letting patients die on inferior treatment arms to produce ‘evidence-based medicine’. Fair test of treatments above all also need to be fair to patients- as already stated in Art 8 of the WMA Helsinki declaration ‘While the primary purpose of medical research is to generate new knowledge, this goal can never take precedence over the rights and interests of individual research subjects.’ http://www.wma.net/en/30publications/10policies/b3/.
    New treatments are not by definition better than old ones. At the same time, letting patients in desperate situations die ‘safely’ out of paranoia that the new therapy is not better than the existing in-efficient one is an insult to the noble aspiration of evidence-based medicine: to provide patients with what is truly best for them.

    • Douglas_Badenoch

      Thanks for your comment Bettina. I received this reply from Iain Chalmers and am posting it on his behalf.
      “I couldn’t agree with you more that fair tests of treatments need to be, above all, fair to patients. A long time ago I proposed in a letter to the Lancet that there should be a Patient-led Good Controlled Trials Guide (Lancet 2000;356:774). Sadly, my proposal went down like a lead balloon! And in both editions of Testing Treatments we suggested (see Point 8 of the Action Plan: http://www.testingtreatments.org/tt-main-text/research-for-the-right-reasons-blueprint-for-a-better-future/action-plan-things-you-can-do/) that patients should not take part in clinical trials unless certain conditions had been met. I hope you would endorse all of them.

      “We may disagree, however, that one important purpose of research is to protect patients from the adverse effects of our ignorance about the effects of treatments. John Lantos puts the matter very movingly in his testimony shown here: http://www.testingtreatments.org/2014/05/13/non-validated-therapy-often-dangerous-careful-research/. The problem with the Helsinki Declaration is that it restricts its attention to the tiny proportion of patients who receive treatments within research and ignores the impact of its pronouncements on the vast majority of patients. Testing Treatments gives plenty of examples of widespread suffering because of failure to confront uncertainties about the effects of treatments by doing research.

      “In other words, I believe that when there is ignorance about the effects of treatments, providing patients “with what is truly best for them” involves admitting uncertainties to them, and offering them the opportunity to help reduce the uncertainties. That’s why, out of pure self-interest, I have carried a medical card for over two decades indicating that I want to be invited to participate in research to compare different treatments when there is uncertainty about their relative merits (New Eng J Med 1991;325:1514). And that, in turn, reflects the evidence that, on average, new treatments are only very slightly more likely to be better than to be worse than existing treatments (Djulbegovic et al. Nature 2013:500;395-396).”

      Iain Chalmers