Saturday, February 22, 1997.
This morning I have been thinking about the general media's lack of proper use of statistics. It may sound like a boring topic, but it is very important and a correct understanding of the use of statistics can often mean the difference between life and death. The two examples that come to mind first are air travel safety and anecdotal evidence of cures for diseases and sicknesses.If a plane crashes anywhere in the world, it will be big headlines everywhere in the world for weeks or even months. But if 50 people die in your state this weekend in car wrecks, it will get only a passing mention in local news. This often gives people the impression than air travel is more dangerous than travel by car. This is very far from the truth.
Another deadly misuse of statistics regards anecdotal evidence of cures for diseases and sicknesses. Just because someone took a supposed cure for cancer and got better means nothing. This shows both a lack of understanding of what cancer is and what constitutes evidence of the cause of a cure. Some diseases, such as rabies and AIDS are, for all practical purposes, terminal under all circumstances. Rabies, if not treated by a certain amount of time, is deadly. I have read of only one case where someone has survived after passing this time limit, and they had major malfunction of almost all internal organs, and survived only with massive life support efforts.
Cancer is not like these diseases. Certain types of cancer have a 50 percent survival rate even if nothing at all is done in the way of treatment. This is because the body has an immune system (white blood cells are part of this) which attacks the cancer cells (mutated cells of the person's own body). If the person's immune system is strong enough or has enough assistance through proven medical techniques, they will live. If the cancerous cells overcome these defenses, they will die. AIDS is so deadly because it destroys these defense systems.
Let's say this particular cancer has a 10 percent survival rate without any treatment, but a 50 percent survival rate using proven medical treatment. If someone takes a non-proven "cure", they will still have a 10 percent chance of survival, so they might live. Let's say they take this non-proven treatment and live. They believe this "cure" saved their life. Maybe you are close to this person, and they have a lot of influence on you. Perhaps you will be convinced by them. Have they proven the doctors and the "greedy medical establishment" wrong? No, they have only taken a very big risk and been one of the lucky 10 percent. Now what happens if you get cancer? Will your decisions be influenced by the luck of your friend?
Suppose someone held a ten shot revolver to your head, and gave you the choice of having only one empty chamber or five. Which would you choose? Would you say, "Last time this happened to my friend, they chose one empty chamber, and they lived, so that must be the better choice. That's my choice too." Of course not, that's ridiculous. Exactly!
So what constitutes a valid test of a cure? First, one person's experience proves nothing, and can be misleading and dangerous, as just discussed. Second, there must be a large number of people involved in tests. Why?
Let's suppose you didn't know that a coin tossed had a 50 percent chance of coming up heads. You want to find out what the chances are. So, you toss a coin 100 times and 60 of them are heads. You conclude the chances are 60 percent. Are you right? Of course not. If instead of 100 coin tosses, you toss 2000 times, you are much more likely to come close to 50 percent. It may not seem obvious at first that more testing means more accuracy. Let's reduce the number of tosses to only 2. There are four possible outcomes: heads and tails, tails and heads, heads and heads, or tails and tails. The first 2 outcomes would give you the right answer, 50 percent. The second 2 outcomes would give you wrong answers of 100 percent and 0 percent. You have only a 50 percent chance of getting the right answer and a 50 percent chance of being as far wrong as possible. If you toss the coin 2000 times, you will very likely come up with an answer close to 50 percent, very unlikely to come up with either 100 percent or 0 percent (all 2000 heads or all 2000 tails). The bigger the test, the more accurate the results. Yet many life or death medical treatment decisions are made on the basis of not 2, not 100, not 2000 people tested, but only one. Is that smart?
So even testing on 100 people is not enough to find an accurate percentage of the success rate of a treatment. Not until 2000 people are tested are the results considered medically conclusive. And with good reason. Your life may be at stake. Do you want to gamble foolishly with that?
What else constitutes a valid medical test? The third ingredient is that the test must be what is called double-blind. This means there must be two groups of people tested. One group is given the "cure" and one is given a placebo, such as a sugar pill. Why two groups? So you have two results to compare. No one in the placebo group is told that they are given the placebo. This is to find out the influence of the placebo effect. What is the placebo effect? With many diseases, giving someone a pill that does nothing gives them a better chance of getting better. This is because it gives them hope of getting better. Their brain reacts to hope by releasing chemicals into their body that strengthens their immune system. Their body does a better job of fighting off the disease. This is called the placebo effect.
For example, lets say you know that for a particular type of cancer, the chances of getting better with no treatment is 50 percent. You give 1000 people the new "cure" and 1000 people a placebo. You don't tell anyone that they are getting a placebo (although you might tell everyone in the test that half are getting a placebo and half are getting the "cure" being tested).
Here are the results you obtain. 550 receiving the placebo get better. 700 getting the "cure" get better. This means that the placebo effect, the hope of getting better strengthening the immune system, caused a 5 percent increase in cure rate. The "cure" being tested gave an additional 15 percent on top of that 5 percent. So the "cure" increases chances of survival 15 percent, plus hope boosting the immune system gives an additional 5 percent, for a total of 20 percent.
Another test. This time, the results are 550 getting better for both the placebo and the "cure". Does this mean that the "cure" gives a 5 percent better chance of survival. No, it only means that it could be used as a placebo, since it gave the same results as the placebo. This is why there must be 2 groups tested, not just one. Many "miracle cures" are tested by a promoter with only one group and give a better result than no treatment. These statistics are quoted as proving the effectiveness of the drug, when in fact, they only prove the effectiveness of a placebo.
What does the double-blind part of this test mean? It means that neither the patients being tested nor the doctors giving the treatment are told who gets the placebo and who gets the "cure". If this procedure is not followed, the doctors might act differently with the group getting the placebo than they do with those getting the "cure". Even without thinking about it, they might smile more to those getting the cure than those getting the placebo. This could cause more hope to be given to one group than the other, causing a greater placebo effect in the group getting the "cure". That group could then get better results than the placebo group, but perhaps the better result might be only a bigger placebo effect. The results of the test would not be accurate as a result, although the doctor or drug company could make millions selling the placebo "cure".
I highly recommend "How to Tell the Liars from the Statisticians" for an easy to read book on the practical application of statistics to daily life.