Understanding Medical Research – Part 3 – Interpreting Results

INTERPRETING MEDICAL RESEARCH STATISTICAL RESULTS

How to interpret the more common results found in medical research papers. What they mean, and how to spot problems.

From Benjamin Disraeli we have: “There are three kinds of lies: lies, damn lies, and statistics.”eretr

Unfortunately this applies all too frequently to the statistical results presented in medical research. While statistics does indeed have the power to elucidate, to find the proverbial needle in the haystack, it also has the power to obfuscate and deceive. One should therefore pay very close attention if there is a possible political or financial agenda associated with the research.

ODDS RATIO, HAZARD RATIO, RELATIVE RISK

Most medical research compares two things, such as outcome from taking a pill versus placebo. Let us suppose that each group has 1000 people and the outcome is ‘all cause mortality’ i.e. death from any cause over a five year period. We’ll suppose the pill takers experienced 25 deaths and the placebos, 30.

The results might be expressed as Hazard Ratio, Relative Risk, or Odds Ratio, duly abbreviated HR, RR, and OR.

HR and RR are ratios of probabilities and are basically the same thing. HR is used in the middle of an experiment or in a group that is followed for a long time. RR usually means that the experiment is over. Most reported research is completed experiments.

Odds ratio, or OR, is slightly different. Odds are expressed as two numbers, as in horse racing (Lucky Lucifer is a 2:1 favorite in the third race at Turnem Downs). This means the probability that Lucky Lucifer wins is 2/(2+1) which is 2/3rds or .66667. If, in the same race, Beetlebaum is given odds of winning of 1 in 30, it means the probability that Beetlebaum wins in 1/(1+30) which is .03225.

The Odds Ratio that Lucky Lucifer wins versus Beetlebaum is  (2/1)/(1/30) which is 60 . However, the RR or HR for this case is ratio of the probabilities which is (2/3)/(1/31) which is quite close to 20, and quite different from 60.

However, if the probabilities are low, then OR is about the same as RR and HR. For the medical example above, the odds of dying for the pill group are 25 to 975, and for the placebo 30 to 970. In this case, the OR works out to .829, whereas the RR or HR would be .833.  In this case OR and RR or HR are quite close.

Usually, medical research deals with low probability events. For instance, people taking statins typically get 3 fewer heart events per 1000 people, perhaps 8 statin users out of a 1000 have heart events versus 11 of 1000 for non users. Here HR, RR and OR can be used interchangeably.

Many studies will present results as Risk Reduction. In the above statin example, the probabilities for heart events for users versus non-users are .008 versus .011. the RR is then .008/.011 = .72. This would then be touted as a .28 or 28% risk reduction.  A 28% reduction sounds a lot more impressive than 3 fewer out of a thousand. (Mr. Disraeli would surely agree.)

You will often see 95% confidence intervals. These numbers represent the consistency of different experiments, meaning specifically that 95% of the results fell into the interval defined by the pair of numbers.

A study in the Journal of Nutrition looked at cardiac heart disease versus nut consumption as reported in several trials. They combined results and came up with an RR of .61. This could also be stated as a 39% risk reduction, (which rather leaves statins in the dust). The four studies reported different RRs ranging from 0.47 to 0.89. They then reported this as a ‘95% confidence interval’ [0.47,0.89], meaning that 95% of the result fell in this interval.  Actually 100% fell within the interval, being as how there were only a few experiments looked at.

When looking at RR or OR or HR, pay attention to the confidence interval (the ‘spread’). If it is too much, beware. The above range [.47,.89] is OK, but something like [.30, 1.05] should certainly cast a dark shadow over the results.

HOW CAN RR, HR, AND OR BE ABUSED

Realistically, data can be massaged to yield just about any result. The researcher can choose which experiments get analyzed, which end points get chosen, which participants get excluded, and so on. So the quality of the research depends on both the integrity and competence of the researchers. How is one to determine this? Top journals are usually pickier about the research they publish, but they are also known to occasionally have biases. Pharmaceutical funded research is fairly notorious for bias. Usually a paper will list researcher affiliations or conflict of interests. Look for these. Follow the money. Sometimes a political agenda will turn up, such as a pro-vegan group presenting research showing that animal product is harmful.  Use your common sense.

GUILT BY ASSOCIATION

Association is thrown around loosely and is subject to abuse. That two quantities are associated simply means they are observed occurring together. They do not necessarily mean one causes the other, though this may often be implied (or inferred).

Usually when ‘association’ is used, the author is talking about relative risk, and might identify something as ‘associated’ if the RR is above 1.

Association does not mean causation. For instance, incidence of Alzheimer’s is associated with low cholesterol level.  Does this mean low cholesterol causes Alzheimer’s, or that Alzheimer’s causes low cholesterol, or that some unidentified third item is causing both? We do not know. Association does not mean causation. We repeated this because this it is frequently forgotten among researchers and the general public. It is patently ignored in most media coverage.

Here is the list of UNDERSTANDING MEDICAL RESEARCH posts (past, present, and future):

PART 1 – Warning Signs Of Fishy Medical Research

PART 2 – Types Of Medical Research Studies

PART 3 – Interpreting Statistical Results

PART 4 – Medical Research Problems – Confirmation Bias.

PART 5 – Medical Research Problems – Commercial Subversion

PART 6 – Medical research Problems – Political subversion

PART 7 – The Cart drives the horse – what causes what?

PART 8 – Playing games with the included items, eg. smoking plus red meat

PART 9 – Using Fear, Uncertainty, and Doubt

 

 

  3 comments for “Understanding Medical Research – Part 3 – Interpreting Results

  1. January 13, 2015 at 8:25 am

    Thank you for sharing your info. I truly appreciate your efforts
    and I will be waiting for your next write ups thanks once again.

  2. January 13, 2015 at 8:34 am

    You really make it seem so easy with your presentation but I find
    this topic to be really something that I think I would never understand.

    It seems too complex and very broad for me. I am looking forward
    for your next post, I will try to get the hang of it!

  3. January 13, 2015 at 9:41 am

    Hello my family member! I want to say that this article is
    awesome, great written and include approximately all significant infos.
    I’d like to look extra posts like this .

Leave a Reply

Your email address will not be published. Required fields are marked *