Why is this? What is so special about the double-blind, placebo-controlled test? To find the answer, we need to look at the different ways that treatments can be tested and what the pitfalls are.

Open testing.

In historical terms this was the earliest method used. The idea is to simply test a remedy on patients and see if they get better. A good example is that of using vitamin C (in the form of oranges and lemons) to treat scurvy. Many treatments had been tried, but they all failed until fresh fruit was used (see: James Lind’s treatise of the scurvy). However, this is a case of what is called a high effect size treatment, which means that the effect is so large that it is outside the range of normal variation and simple observation confirms the treatment’s effectiveness.

Unfortunately this is not the case for most treatments. Many treatments have a small effect and the subjective views of the researcher can overlap with the subjective views of the patient. This can lead to error.

Confounding factors.

Confounding factors are variables which can lead to false conclusions in studies by showing associations where there are none and vice versa. Confounding factors include:

• Placebo effects.
  
  This is where a perceived improvement is felt by the patient due to expectations and beliefs. (see: placebo effect). The patient cannot tell the difference between the placebo effect and improvement due to treatment: the improvement feels the same.
  
  
• Reinterpretation.
  
  Having been given a treatment, a patient will often report a reduction in symptoms or an improvement in their condition. Someone given a treatment to reduce itching, for example, may report that their itching has indeed reduced; yet if they are monitored, it can be shown that they are scratching just as frequently as before. Reinterpretation is a form of placebo effect.
  
  
• Observer bias.
  
  This is similar to reinterpretation except it is the doctor/researcher who commits the fallacy. If the researcher believes the treatment to be beneficial, they may well interpret results to match their expectations. Although a researcher should be impartial, this does indeed happen.
  
  
• Self-limiting illness.
  
  Many illnesses are fought off naturally by the body. A useless treatment given during the course of an illness could mistakenly be attributed as curing a disease that would have healed naturally.
  
  
• The regressive fallacy.
  
  This is similar to self-limiting illnesses, only it applies to chronic conditions. A treatment given whilst a condition is at its worst, a flare up, will see an improvement whether the treatment works or not as the condition, being at its worst, can only get better. (see: The Regressive Fallacy).
  
  
• The Hawthorne effect.
  
  This effect was first noticed in the workplace (see: The Hawthorne Effect). It is where staff will perform better if some attention is paid to them. Just taking an interest can improve their productivity for example.This effect shows up in medical trials too. A patient who has been chosen to take part in a trial and is monitored will behave better than they would normally. e.g. a diabetic who is given a new pill to try may eat better and take more exercise simply because they are on the trial; any improvement shown could be due to the altered behaviour rather than the treatment.

Confounding factors were not all discovered at the same time, but as they were realised, they had to be eliminated.

The open trial.

In these trials both the researcher and the patient know the details of the test. These trials are open to all of the biases and are not shielded from placebo effects. Open trials are still used today; however, they are restricted to procedures where there is no alternative, such as surgery.

The placebo comparison.

As placebo effects can make treatments appear successful, it was realised that the treatment under investigation would have to be compared to something known not to work: the placebo control. A treatment cannot be considered effective unless it is compared to a placebo (or at least something else that has previously been tested against placebo or is a proven high-effect sized treatment).

A placebo control, however, is of little value if the patient knows that they are taking a placebo.

The single-blind trial.

Researchers adopted the method of blinding their subjects as to whether they were receiving the treatment under review or the placebo control. The idea was to remove any bias the patient may have regarding their treatment. If the treatment was effective, the benefit should show up clearly in the treated group compared to the control group.

It was also realised that not only could patients have a bias; so could the researchers. Observer bias could lead researchers to the results that they were expecting to find.

There was also the problem that the researchers, through their unintentional bias, could influence patients by unconscious signalling, the Clever Hans effect, or by simply treating those they knew had received the treatment in a different manner to those who had not.

The double-blind trial.

In the double-blind trial neither the patient nor the researchers administering the treatment know who is being treated and who is in the placebo control group. Patients are randomly assigned to one of the groups independently of the researchers, and the actual treatment they receive is coded so that no one knows whether it is real or placebo. This ensures that all possible biases that could be introduced to the trial by the researchers or the patients are eliminated.

Only after the trial, and once all data has been gathered are the results decoded and analysed. If the treated group shows significantly better results than the placebo control group, then the treatment has passed the trial.

This is not the end, however. One trial is not enough as it could be an anomaly, or a poorly designed or controlled trial, so the test needs to be repeated again independently. If the results are repeated in subsequent high-quality trials, the treatment is deemed to be effective.

Conclusion.

As with all tests, it is important to ensure that the results can only be produced by that which is under consideration: all potential biases and complications need to be eliminated or reduced to a minimum.

The randomised, placebo-controlled, double-blind trial is the way in which this is achieved in modern, evidence-based medicine.

Stem-cell treatments are in the news again this week. However, it’s for the wrong reasons this time.

The Department of Public Health Sciences and Faculty of Law at the University of Alberta in Edmonton, Canada has carried out some research looking at private clinics offering stem cell treatments for many diseases, particularly neurological conditions such as Alzheimer’s and Parkinson’s disease, spinal cord injuries and Multiple Sclerosis; and more bizarrely, autism, muscular dystrophy and cerebral palsy.

The researchers looked specifically at clinics offering such treatments online via their websites and looked at the sort of treatments being offered, the claims made for them and whether there’s any clinical evidence to support these claims.

In our article on pseudoscience, we said one characteristic of pseudosciences is that they are often aimed directly at the public:

Scientific breakthroughs will normally have been published in science journals, scrutinised by other scientists, and only announced to the public once scientists have agreed that the scientific breakthrough is indeed genuine. The progress of the acceptance of the idea will be documented and anyone can reference this information in the relevant journals.

Pseudoscientific ideas are sometimes driven by cultural or ideological reasons, but very often they’re driven by commercial goals. A company that is trying to sell its products or ideas without having gone through this scientific scrutiny is giving out a telltale sign that their products will not stand up to scientific scrutiny. A new ‘miracle breakthrough’ healing device, for example, that is being sold directly to the public, but which has no science references to support it, probably doesn’t work.

These stem-cell therapies are being marketed directly to the public precisely in this “direct-to-consumer” manner.

Not surprisingly, when the researchers looked for clinical evidence in the PUBMED database to support the therapies these clinics were selling direct to consumers, the particular conditions that are being treated by these clinics are not backed up by clinical evidence.

Not only that, most of the clinics’ websites that were looked at made no mention of any side effects or risk factors involved with these treatments.

Conclusion

There is certainly a feel-good factor and optimism involved with stem-cell treatments because of the potential and hope these therapies may offer in the future; and this is augmented by success stories like the trachea transplant mentioned earlier. This is all the beginning however; the potential to cure many diseases using cell-stem therapy lies in the future.

The clinics currently offering these stem-cell therapies for various neurological disorders are riding on the back of the optimism, hope, and positive press it gets, but they are clearly making false claims for the efficacy of these treatments as the claims are not backed up by clinical evidence.

In summary we have:

• Claims of cures for treatments that are not supported by evidence;
• Omission of the associated side effects or risk factors involved; and
• High cost of treatment to the consumer: the average cost for these treatments is ~ £14,670 ($21,500 or €16,800).

As is usual, these treatments are not illegal. So it is again a case of buyer beware. Just as with many more obvious forms of quackery, these clinics are selling nothing more than false hope – which is particularly repugnant as they are hitting people who are vulnerable, and they surely know what they are doing.

The general advice to people contemplating stem-cell therapy offered by a private clinic is to keep well away. The technology is not yet advanced enough to cure the conditions that these clinics claim it can.