7: What’s the best way of telling if the evidence (on the web or elsewhere) is reliable?

What should people look out for? Unfortunately there is no completely reliable simple marker for reliable information.

If you are not going to look at the original research yourself, you are putting your trust in someone else’s assessment. So it is important to assess the likely competence of that person (or organization) and to note whether there is a conflict of interest (or an axe to grind).

If not, then ask yourself whether you trust them to have found and assessed the best research: is it described and referenced? For example, suppose someone wanted to know whether betacarotene (related to Vitamin A) increases or decreases the risk of cancer. A Google search for ‘beta-carotene cancer’ brings over 800,000 results. Looking at the first ten there are four primary research studies and six that are reviews or opinions. Of those six, there are three that have advertisements for vitamins or alternative medicines on the same page: a worrying sign.

One of these poorer websites says:

‘Question: Does beta-carotene prevent cancer? Answer: Studies have shown that beta-carotene can help reduce the risk of cancer. Beta-carotene can be found in yellow, red, and deep green vegetables. It is also found in fruits. It is a common belief that taking a beta-carotene supplement will have the same effect as eating fruits and vegetables that contain it. However, this is not the case. Studies found an increased lung cancer risk among study participants.’

In addition to the advertisements, ‘studies’ are mentioned but with no description of them or references to published studies – that is a warning sign. It is impossible to tell whether or not the writer has searched for and appraised the ‘studies’ or merely stumbled on ones where he or she liked the conclusions. Contrast this with the Wikipedia entry (also in the first ten): ‘A review of all randomized controlled trials in the scientific literature by the Cochrane Collaboration published in JAMA in 2007 found that ß-carotene increased mortality by something between 1 and 8% (Relative Risk 1.05, 95% confidence interval 1.01-1.08).[15] However, this meta-analysis included two large studies of smokers, so it is not clear that the results apply to the general population.[16]’

This entry states the type of evidence (randomized trials), and gives the references (the numbers in the square brackets). So, the fact that there are no advertisements, and there are specific details about the evidence, is reassuring.

Next: 8: Are there reliable sources of information that can be recommended?